
Problems With Precision and Judgment, but Not Integrity, in Tesla Test - rrreese
http://publiceditor.blogs.nytimes.com/2013/02/18/problems-with-precision-and-judgment-but-not-integrity-in-tesla-test/
======
gizmo
A few thoughts:

1\. so it looks like the editor concedes that the truth is somewhere in
between the original review and Musk's rebuttal. But the original review is up
as is. It's not amended, not corrected in any way.

2\. Elon Musk said his faith in the NYTimes is restored over twitter. This to
me looks like Elon made a deliberate decision to pick a fight with a single
contributor and not with the entire newspaper. Besides, Elon knows other
people will continue fighting on his behalf, so he can now take the high road
and back off. Good tactics here.

3\. The spin by the NYT editor is really clever. By quoting another person who
mostly sides with Tesla and agreeing with that person she comes across as
responding to the conflict with substance. But because the words aren't her
own she can keep her status as a mostly impartial arbiter. And because she
didn't say which parts of the quoted response she agreed with she can deny any
part of it if needed! What she did here is equivalent to composing a list of 7
statements and adding to the bottom "I agree with 4 out of the 7 statements
above". Sneaky sneaky.

4\. Her final conclusion is that "people will go on contesting these points"
but ultimately "few conclusions are unassailable". That's pretty much a
classic "it's all relative" sort of response. The fact that people are in
vocal disagreement doesn't mean you can just wave your hands and say
essentially "well, the situation is just complex" and use that to justify
keeping the original article up unchanged.

~~~
knowtheory
Okay. I'll bite. What corrections would you like them to make?

Seems like a lot of this is a matter of degree and a bit of embellishment. But
I would hardly call the original article a hit piece or anything.

I suppose if i had my druthers it'd be nice for the original piece link to the
public editor's post, or some other compendium of the saga.

~~~
cma
I'd like them to correct "freezing toes"... look at the cabin temperature
graph

~~~
stouset
My toes practically go numb at prolonged exposure to temperatures in the low
sixties.

------
stcredzero
_> Mr. Broder and The Times have maintained that the article was done in good
faith, and that it is an honest account of what happened_

So, in our public discourse, "honesty and good faith," have drifted down to
the level of "deniably but somehow consistently fudged."

In the words of Richard Feynman, there is: "a specific, extra type of
integrity that is not [just not] lying, but bending over backwards to show how
you are maybe wrong." It's this level of integrity that is needed in our
public discourse concerning the complex and technically sophisticated world of
today. If the NYTimes can't manage this in a simple car review, I wonder how
they do with complex political and social issues.

~~~
niggler
You must have your blinders on. Musk is not a god, and you have been eating up
Tesla's story rather than looking at the data.

Even Tesla's data corroborates most of Broder's account (heat was turned off,
battery dropoff overnight, etc), and the only thing that remains is the
question of the call logs. It's interesting that Tesla has the logs yet isn't
revealing them. I suspect the logs will show that Tesla engineers didn't
properly advise Broder on what to do, which is _Tesla's fault_ , not any sort
of fraudulence on NYT's part.

~~~
stcredzero
_> Even Tesla's data corroborates most of Broder's account_

I see you have it all wrong. I'm not in disagreement with that. (And there
goes the basis for your comment.) Either you are willfully misreading my
comment, or you're trying a subtle sleight of hand. The facts _mostly_ fit
Broder's account. I'm calling attention to the fact that all of the
inaccuracies and bad decisions were _pointedly in one direction._ A reporter
should have some facility in curbing his own bias. Apparently, Broder has
insufficient competence in figures and self awareness for this.

This is precisely what Feynmann is talking about in the quote.

~~~
niggler
"I'm calling attention to the fact that all of the inaccuracies and bad
decisions were pointedly in one direction"

They are not, though. From Broder's account (you should read the original
article), most of the bad decisions and inaccuracies can be attributed to
advice that _Tesla gave Broder_. That is the crux of the problem: Broder makes
a claim based on Tesla engineer advice, and Musk denies and attributes that to
malice.

I am arguing against your implicit assertion that NYT is at fault at each
point of divergence between NYT's and Tesla's account. It's still up in the
air, and as I said: "only thing that remains is the question of the call logs.
"

~~~
stcredzero
_> most of the bad decisions and inaccuracies can be attributed to advice that
Tesla gave Broder._

Yeah, if you let him get away with being scientifically illiterate.

 _> you should read the original article_

EDIT: You litter your comments with these off-hand remarks that imply things
about other's positions which aren't true.

 _> I am arguing against your implicit assertion that NYT is at fault at each
point of divergence between NYT's and Tesla's account._

What, is putting words in other's mouths the modus operandi now? I'm not
saying that. I'm saying that Broder isn't sufficiently competent at curbing
his own bias, knowing how to do empirical reporting, and making sensible
technical decisions in the field to do a car review. He _tried_ to get it
right? Making inadequate efforts at getting at the truth isn't an acceptable
level of performance from a national news media outlet. It's basically, "Hey,
we tried. Be satisfied with our truthiness."

~~~
niggler
"I'm saying that Broder isn't sufficiently competent at curbing his own bias,
knowing how to do empirical reporting, and making sensible technical decisions
in the field to do a car review."

Your statement stems from an attack by Musk. What makes you think that a
journalist would be more biased than the CEO of a car companies with billions
of dollars at stake?

Your stalwart defense of Musk belies _your_ inability to think objectively. My
stance is as it was when I first saw the situation: the only way to resolve
this is to reveal the phone logs. It's a he-said-she-said (that's something
else I've pointed out multiple times) and somehow HN deems Musk (not a party
to the conversation) more credible than the journalist asked to test drive
_without asking for the key information that would resolve this matter_

~~~
stcredzero
_> Your statement stems from an attack by Musk._

No, my statement stems from reading his account, then comparing it to Musk's
data.

 _> What makes you think that a journalist would be more biased than the CEO
of a car companies with billions of dollars at stake?_

Again, this sort of sleight of hand. I'm not defending Musk's bias here,
Musk's bias doesn't in any way exonerate the other's, and nice try, if that
was intentional. I'm purely attacking Broder's incompetence.

 _> Your stalwart defense of Musk belies your inability to think objectively._

So it's "objective" to not recognize Broder's bias in his inaccuracies, or at
least to excuse it because, hey Musk was probably biased too!

...Okay! You're entitled to your position, man!

 _> It's a he-said-she-said_

Yes, the rest of us are just avoiding the "he-said-she-said" quagmire. It is a
quagmire, and the likely ROI is low. Really, the outcome of one car review
isn't so important, but the issue brings up plenty of stuff which is. I'll
tell you what: When Musk is the head of a national media outlet "of record,"
and he publishes the same sort of stuff, I'll hold his feet to the fire for
shoddy journalism for his hasty comments and interpretation of data -- which
he is guilty of. It's not a par-performance, objectively a bit of a PR cock-
up, but still understandable from a CEO. Of _course_ he's biased in favor of
his product. The journalist is only human, but he's not only supposed to try
to be objective as a part of his job, he's supposed to show some competence at
it.

The disturbing part of this whole issue for me is the degree to which
carelessness about empirical fact is seen as "okay." Elon Musk has a
substantial public record which shows a notable degree of diligence towards
getting facts right. Broder's article -- best I could say about it is: enh.

------
kintamanimatt
"I could recite chapter and verse of the test drive, the decisions made along
the way, the cabin temperature of the car, the cruise control setting and so
on. I don’t think that’s useful here."

Yes, yes it would be, especially when addressing the integrity of one of their
journalists! She wore kid gloves when she rendered the comments regarding his
"mistakes". Kid gloves are not appropriate.

It appears the sentiment of this editor's blog post is essentially "we'll
defend our journalists even when their articles have been exposed as unfairly
biased." It feels like the journalistic equivalent of the thin blue line.

~~~
wpietri
I was blown away by that as well. She might as well have written, "I have come
to my conclusion, which is that we are fine, but since you people on the
internet are meanies who are just going to argue your preconceived
conclusions, I'm not going to tell you _how_ I got there."

The weird thing is, I think she's probably right. But I think the job of the
public editor is precisely to address controversies of their reporters in
sufficient detail that we don't _have_ to trust her. Ombudsmen have to be seen
as scrupulously fair to outsiders or there's no point in having them.

~~~
stcredzero
Well, to be fair, from the POV of an insider, I could see how it's a gray area
when you get to the point of saying someone thought they were doing a good
job, but it looks like he's incompetent at getting facts right and curbing his
own bias. Sometimes you have to stop short of even being close to saying a
colleague needs to be fired, and let those matters play out quietly later on.

~~~
wpietri
That is understandable in any other employee at the Times, but it's a total
failure in an ombudsman. They don't have to recommend firing; they can just
say what happened and characterize is honestly. But they do have to always
take the public's side, and they have to do it visibly.

This particular position was created after the Jayson Blair scandal, so it's
particularly important that they are frank in calling out bad reporting. Or,
in this case, explaining why they think it wasn't bad reporting.

------
jellicle
> Problems With Precision and Judgment, but Not Integrity, in Tesla Test

Oddly, I would write the exact opposite headline. The NYT review has minor
problems with precision and judgment, but major problems with integrity.

Imagine a gasoline car review: "I'm here at a gas station with an empty tank,
and I have 61 miles to the next gas station. I'm going to put in half a gallon
of gasoline, hang up the nozzle and start driving to the next station. Oh no,
I ran out of gas! Well, this proves the technology is still immature and that
gas cars suck."

Can you imagine this being a publishable review to the NYT editors?

~~~
niggler
The key facts here are:

1) Broder reached out to Tesla on multiple occasions during the trip asking
for advice when oddities happened

2) He behaved in accordance with the advice he was given.

3) The car ran out of charge

Regarding the non-sequitur of charging overnight, Tesla's marketing materials
clearly state that you can keep the car unplugged for extended periods of time
without a significant dropoff: "The Model S battery will not lose a
significant amount of charge when parked for long periods of time. For
example, Model S owners can park at the airport without plugging in." (pulled
from <http://www.teslamotors.com/models/facts> , older discussion
<http://news.ycombinator.com/item?id=5209639>)

The only way to attack the integrity of Broder is to argue that somehow he
wasn't advised to do something. Guess what: there are call logs, and those
should be brought out. It's a he-said-she-said right now and everyone sides
with the car CEO rather than the journalist. That's all fine, but in this case
Tesla has a way of definitively proving that Broder was being malicious: If
the call logs clearly show that Broder wasn't acting in accordance to what he
claims was Tesla's judgments, then Broder deserves all the criticism.
Otherwise it is Tesla's fault.

Until we see the call logs, siding with Tesla is hero worshipping.

~~~
SilasX
>Broder reached out to Tesla on multiple occasions during the trip asking for
advice when oddities happened

That's what struck me the most, relative to how little attention it got. How
many other people would have needed to call into a car company's customer
service (and that many times!) to finish a simple test drive?

"Mommy, should I charge it all the way at the supercharger station?"

"Mommy it only reads 32 miles, will it last twice that?"

There are a hundred people more competent and deserving of such an opportunity
than someone who doesn't even know the current state of vehicle logging tech.

~~~
niggler
"How many other people would have needed to call into a car company's customer
service (and that many times!) to finish a simple test drive?"

If you saw last night that you had a full tank of gas, and woke up this
morning and saw the tank was empty, wouldn't you call the car company? If you
saw any other critical fluid mysteriously drop in level, wouldn't you call the
car company? I've done this multiple times with strange sounds and other
things I never expect to happen.

If I were in Broder's situation and saw strange things happening with the
range indicator, I absolutely would be calling. _Especially_ if it's a test
vehicle -- the courts have decided that journalist drivers are responsible for
the cars: [http://www.digitaltrends.com/cars/you-break-it-you-bought-
it...](http://www.digitaltrends.com/cars/you-break-it-you-bought-it-
journalist-ordered-to-repair-2-million-porsche/)

------
busterarm
Everyone is really ignoring the real impact of this mess here.

The NYT published something that is absolutely impossible to fact-check
outside of an op-ed column. This wasn't even fit for publication to begin
with.

Print journalism is in no way superior to the bloggers they are in competition
with - in fact, most journalists have to keep their own blogs now just to
compete. Margaret's reply basically dances around saying that Broder failed to
do his due diligence as a journalist.

If you need any clearer sign that newspapers are not just declining but dead,
this is it.

~~~
tbrownaw
So newspapers are not permitted to perform original research?

~~~
busterarm
That's not really the job of newspapers.

Newspapers are supposed to accurately and impartially report the news. Op-Ed
work needs to be clearly marked as such.

Now, it would be fair to say that the Automotive section, specifically the
reviews, contains a fair bit more opinion than the rest of the paper, but
there needs to be clear distinction. Going by the headline and the way the
article was written, this clearly isn't the case. It was reported as news...as
in it was about the car and neglected the fine personal detail that a review
would have. It turns out though that the story here is more about Broder and
how he drove the car. The way he drove it was more important than anything he
reported about the car - it's not news.

Broder writes other stories for the NYT that are genuine news. It's shocking
to me that the current editorial format of the NYT allows crap like that to be
published in the same venue.

I'm basically saying that if this weren't written by an NYT staff reporter, it
would have been an Op-Ed (or similarly labeled) if they allowed it for
publication at all. They should not have.

~~~
crazygringo
> _That's not really the job of newspapers. Newspapers are supposed to
> accurately and impartially report the news._

I think you just made that up. According to who?

I read newspapers with movie reviews, play reviews, comics pages, advertising,
news analysis, car reviews, industry analysis, foreign affairs analysis, etc
etc etc.

Newspaper reporters win _Pulitzers_ for their original research, investigation
and analysis.

God forbid if newspapers should ever merely "report the news", it would be a
sad day.

~~~
busterarm
Reviews are always presented as reviews though. There's a personal element in
most of those.

Analysis usually is either a summation of another publication (with citations)
or is an Op-Ed.

Investigative journalism is pure news.

------
ignostic
Here's what seems clear from the whole thing:

1\. Musk jumped to conclusions. For example, he claimed the car was driven in
circles with the intent of running it out of battery. I think the innocent, "I
was looking for the charger" explanation made sense. He concocted his own
narrow explanations without verifying the facts or asking questions.

2\. Broder failed to understand the car and the advice given to him. We can
only speculate on the quality of that advice. What Tesla didn't log were the
conversations. Broder also trusted the "one hour charge" despite failing to
confirm he was good to go, or listen to common sense.

Broder published the article with emotion and without further consulting with
Tesla on the cause of his bad experience. Musk immediately claimed the article
was done in bad faith rather than checking the facts and opening a dialog like
an adult.

The resulting situation is almost poetic. Both Mr. Musk and Mr. Broder's
mistakes provide their own unique insights on the importance of understand
human communication. I know the crowd I'm talking to puts a lot of stock in
the vehicle logs, but we're missing the communication logs we'd need to know
what really happened or who said what to whom.

Maybe next time Tesla should log the conversations, too.

~~~
cube13
>Broder published the article with emotion and without further consulting with
Tesla on the cause of his bad experience. Musk immediately claimed the article
was done in bad faith rather than checking the facts and opening a dialog like
an adult.

Broder claimed in his first response that Musk had reached out to him for
another go at the trip, once more stations are up. Musk has not denied that
that took place, so I don't see any reason to believe that this isn't the
case.

------
gfodor
The problem with this "wrap-up" of the drama is that it fails to leave the
reader with a clearer understanding of if the Tesla Model S is an automobile
that has serious engineering problems. Instead, it dives deeper into the
"controversy" which isn't really the point. Politics and Musk/Broder egos
aside, any competent third party observer can come to the conclusion that the
review was flawed and hence should not should influence the buying decision of
a potential Model S customer one way or the other. This is the entire role of
the NYT in this issue: to inform the consumer. Instead of providing clarity
for the consumer the NYT continues to prefer to dive into meta-talk about
itself and its story (as the media loves to do, report on itself) and fails to
call a spade a spade. Pathetic.

------
tunesmith
Well, that said just about nothing at all.

~~~
fiendsan
absolutely... especially for an editor! she talked a lot and said nothing! she
has no opinion and leaves it absolutely open, aparently there is no fault by
anyone? ... well at least the CEO of Tesla gave facts and not just his opinion
(like the new york time journalist and editor)!

~~~
taligent
So which is it ? Does the editor have an opinion or not have an opinion.

Also Musk gave his interpretation of the facts. One person's deliberate deceit
is another's innocent mistake.

~~~
fnordfnordfnord
The editor has an opinion. She knows that Broder is full of it, and condemns
him in the strongest allowable terms. She can't flog Broder outright (absent
some glaring evidence of fraud or something), so she needs to smooth the
waters. If there had been an angle to attack Musk with, I think there's a good
chance that she'd have wailed on him.

------
tlrobinson
Elon Musk seems satisfied:
<https://twitter.com/elonmusk/statuses/303585941304537090>

~~~
wamatt
Strange reaction from Musk. He seemed awfully quick to jump fully onboard with
a rather pallid response from the NYT's Public Editor[1] (a position somewhat
analogous to the head of PR in a corporate).

While there could be many reasons, two more probable IMHO could be:

1) The "out" he was looking for: absolution for a somewhat strong initial
overreaction.

2) He's simply being naive, and not reading between the lines.

[1] <http://en.wikipedia.org/wiki/Public_editor>

~~~
stdbrouw
Err, a public editor is not at all like a PR person. They may have some
loyalty to the company they work for, but they work independently of the
newsroom and in fact reporters tend to distrust them or consider them a
nuisance. At worst, it's soft PR, but at best, it's a true ombudsman position.

~~~
wamatt
>"They may have some loyalty to the company they work for, but they work
independently of the newsroom and in fact reporters tend to _distrust them or
consider them a nuisance_."

Do you have anything evidence for that claim? Either first hand experience or
perhaps a citation?

I will concede they are not directly PR people, but yet the role of "Public
Editor" was created in the wake of an ethics scandal and PR tsunami[1] at the
NYT. The position is also on the payroll of the NYT.

Occam's Razor would indicate we should follow the incentives.

Keeping an open mind as always, but I don't think there is good cause, a
priori for the belief that the "Public Editor" is working tirelessly for the
good of the people.

_[1][http://en.wikipedia.org/wiki/Jayson_Blair#Plagiarism_and_fab...](http://en.wikipedia.org/wiki/Jayson_Blair#Plagiarism_and_fabrication_scandal*)

~~~
stdbrouw
The fact that they work independently of the newsroom and sometimes even
independently of the company (as a contractor) is simply the nature of the
public editor / ombudsman and how it is set up at pretty much every newspaper
that has one.

Re: how the public editor at the Times is perceived:
<http://www.nytimes.com/2010/06/13/opinion/13pubed.html?_r=0>. The money
quote: "A writer shaken by a conclusion I was reaching told me, if you say
that, I'll have to kill myself." I would hope your company's PR person
wouldn't make you feel like that.

And a broader look at how journalists handle criticism:
[http://www.pbs.org/idealab/2010/06/why-cant-journalists-
hand...](http://www.pbs.org/idealab/2010/06/why-cant-journalists-handle-
public-criticism167.html) but also [http://www.pbs.org/idealab/2011/04/theres-
no-problem-newsroo...](http://www.pbs.org/idealab/2011/04/theres-no-problem-
newsrooms-in-denial-about-rampant-errors115.html)

You can't really avoid having the public editor on the payroll of the
organization he or she works for, unless some wealthy donor is prepared to
step in and pay for one. That means we always have to be sceptical of where
their loyalties lie and their independence, but it doesn't mean they don't
have any.

Also, the New York Times is one of the only papers in the world that tracks
errors internally, so they can easily see which of their journalists are
playing fast and loose, and will let them go if necessary.

Is it perfect? Obviously not. Are journalists and editors generally really bad
at owning up to their mistakes? Yep. But an ombudsman is an attempt to do
something about that, not an extension of that culture.

~~~
wamatt
Kudos. It appears you've made a compelling case and I found those links to be
interesting reading.

Of course we should still keep a close watch on the future to see how true it
remains to the historical record and theory. But in the meantime, I'll be be
updating my model, and nudging the needle, a little further toward the left.
(if 'for the people' is left, and 'for the company' is to the right, on a 1
dimensional axis).

Thanks for taking the time to respond.

------
tokenadult
This statement by the New York Times public editor certainly reads like a
rebuke of reporter John Broder for how thoroughly he (didn't) take notes
during his reporting drive, and suggests that reporters will have a
responsibility to bring along more electronic data-recording devices in future
reporting. The latest blog post by Bruce Schneier

[http://schneier.com/blog/archives/2013/02/automobile_data.ht...](http://schneier.com/blog/archives/2013/02/automobile_data.html)

(just submitted to HN,

<http://news.ycombinator.com/item?id=5241508>

moments ago) suggests that even with a journalist leading a quantified life to
submit quantified stories, there will still be issues of interpretation of the
data that will be very hard to resolves, especially when a corporation wants
to sell cars with a public subsidy while a news organization wants to preserve
a reputation of hard-hitting, non-toadying journalism.

~~~
fnordfnordfnord
>...especially when a corporation wants to sell cars with a public subsidy

Know a lot of domestic car companies who aren't on the dole?

------
headbiznatch
Funny - I feel precision and good judgement are the primary qualities by which
the integrity of a journalist should be measured.

------
teyc
I would have thought that a Tesla's fancy trip computer would be able to
account for, among other things - elevation changes, cold weather performance
and provide smarter advice than what a human support agent would. Instead, the
range estimator seems to project current usage over the remainder of the
entire trip.

------
Osiris
The good thing to come out of this is that the next time someone is given a
test drive for a review that they'll be very aware that everything is being
electronically monitored. At least that should help keep people honest.

------
jwatte
So, Broder had admitted to braking the law by going 85 to "keep up with
traffic." That's the most bizarre omission here -- we don't get to pick and
choose which laws we decide to follow!

------
monsterix
Don't how many of you'd seen this coming, but @homosaur and I'd discussed
exactly this discourse [1] a week ago.

Personally I think that entrepreneurs working on products that solve immensely
challenging problems should always be given a benefit of doubt. Or two or
three or may be even five times. Clearly this did not happen in Tesla's case.

And yet in the original post from OP this part still seems a bit questionable:
_Mr. Broder’s irresponsibility in fuel management was in hope that something
beyond “inconvenience” would happen to make the story more interesting.
(Otherwise, no one, including me, would have paid much attention to his
article.)_

[1] <http://news.ycombinator.com/item?id=5203207>

~~~
jug6ernaut
"Personally, I think that a hard working entrepreneur, particularly those
working on immensely challenging problems, must always be given a benefit of
doubt in early reviews."

Products should be reviewed based on there merit, "given a benefit of a doubt"
is left up to the consumer.

A reviewer should review every product the same, else the reviews are
worthless because there is no continuity to compare the reviews.

Do you want your reviews from some person you have never heard of or from
someone well established? The answer is obvious.

~~~
monsterix
You're right. Reviewer must always be objective and review every product the
same. Or at least be careful about the process used to arrive at the
reportable opinion (which NYT seems to confess having missed completely!
Strange?).

However, here is a beautiful piece from Anton Ego of Ratatouille:

" _In many ways, the work of a critic is easy. We risk very little yet enjoy a
position over those who offer up their work and their selves to our judgment.
We thrive on negative criticism, which is fun to write and to read. But the
bitter truth we critics must face is that, in the grand scheme of things, the
average piece of junk is more meaningful than our criticism designating it so.
But there are times when a critic truly risks something, and that is in the
discovery and defense of the new. Last night, I experienced something new, an
extraordinary meal from a singularly unexpected source. To say that both the
meal and its maker have challenged my preconceptions is a gross
understatement. They have rocked me to my core. In the past, I have made no
secret of my disdain for Chef Gusteau’s famous motto: Anyone can cook. But I
realize that only now do I truly understand what he meant. Not everyone can
become a great artist, but a great artist can come from anywhere. It is
difficult to imagine more humble origins than those of the genius now cooking
at Gusteau’s, who is, in this critic’s opinion, nothing less than the finest
chef in France. I will be returning to Gusteau’s soon, hungry for more._ "

------
OGinparadise
_A little red notebook in the front seat is no match for digitally recorded
driving logs, which Mr. Musk has used, in the most damaging (and sometimes
quite misleading) ways possible, as he defended his vehicle’s reputation._

Personally I agree, and to the suggestion that people read the manual and blah
blah blah...how many people in the real read it in detail? So this is /was
more of real person test drive. Moral of the story: buy an electric car or a
Tesla only for short term drives and if you are willing to go through a series
of hoops to charge and care for it. From the review--and especially from
Musk's attitude--Tesla would not be for me, even if I had the money.

Elon needs a rule, like holding off any comments for 24-48 hours, he needs to
calm down. Driving 0.6 miles is not grounds for "he was trying to run the
battery down on purpose to fake the interview." He is a loose cannon and does
his company no service.

