
A Case Study of Toyota Unintended Acceleration and Software Safety - mzehrer
http://betterembsw.blogspot.com/2014/09/a-case-study-of-toyota-unintended.html?m=1
======
silverpikezero
This is an absolutely fascinating slide set. Thanks submitter very much for
the link. I have written embedded C before, and the following facts just blow
my mind:

1\. The Throttle Angle function in the Toyota code had a McCabe Cyclomatic
Complexity of 146 (over 50 is considered untestable according to slides)
[slide 38]

2\. The main throttle function was 1300 lines long, and had no directed tests.
[slide 38]

3\. I find the static analysis results quite alarming. [slide 37]

4\. 80+% of variables were declared as global. [slide 40]

I find this to be a stunning lapse of quality, especially for a safety-
critical system.

~~~
threeseed
Static analysis does not show you the complete picture. I have worked on
countless horrifically complex, buggy and unmanageable J2EE applications that
had perfect cyclomatic complexity et al scores. I have also written perfectly
reviewed, manageable and well tested piece of code that didn't.

I very much question the experience of anyone who can refer to a codebase as
"spaghetti code" without seeing it and relying solely on static analysis.
Software is not that simple or transparent.

~~~
userbinator
I'm going to guess that the applications you refer to are "simple" at the
level of individual functions, but that's only because the complexity has been
spread out so much that it becomes difficult to understand the whole. That's
exactly why I think cyclomatic complexity and related metrics are of little
benefit, and may even be harmful - "refactoring" to reduce "point complexity"
can result in increasing the complexity of the whole.

I've always wondered what the cyclomatic complexity of TeX is. While Knuth is
a bit of an "edge case", it would be fun to see what static analysers think of
his code... complete with copious use of global variables, goto, and very,
_very_ long functions. Give someone who has never had any experience with TeX
the results and ask them what they think the defect rate would be, then show
them the fact that it's one of the most bug-free pieces of software ever
written.

~~~
sillysaurus3
_it 's one of the most bug-free pieces of software ever written._

Not doubting you, but do you have a source to demonstrate that claim? If I
make the claim to someone else, it'd be easier to provide evidence than for me
to handwave.

TeX version is 3.14159265, so some of those are probably bugfixes.

EDIT: Um. Look, I rarely complain about downvotes, but what's up with the
downvoting on HN lately? Is it me, or what? This is a simple request for more
information about something I don't know about. It's not an easy thing to
Google. It's up to the parent to provide evidence.

[https://www.google.com/search?q=tex+bug+free](https://www.google.com/search?q=tex+bug+free)
shows a lot of evidence that TeX is absent of bugs, but that's not the
question. The question is the total bugs that have been fixed since it was
first written relative to every other major software project. That's not so
easy to answer.
[https://www.google.com/search?q=low+total+bug+count](https://www.google.com/search?q=low+total+bug+count)
brings up nothing relevant. In fact, it could turn out to be entirely false
that TeX had a low total bugcount over its history relative to its size,
especially during its very days. We don't know, because no one has provided
evidence one way or another.

All of this is exceedingly obvious, and it's getting tedious to type out huge
edits like this whenever something straightforward is downvoted.

I'm seriously tempted to create my own community at this point out of
desperation, one that focuses on technical merit and being nice rather than
posturing. I wonder if one already exists? I've heard some pretty good things
about newsgroups, but haven't really looked into any.

~~~
userbinator
From
[http://en.wikipedia.org/wiki/TeX#Development](http://en.wikipedia.org/wiki/TeX#Development)

 _Knuth has kept a very detailed log of all the bugs he has corrected and
changes he has made in the program since 1982; as of 2008, the list contains
427 entries, not including the version modification that should be done after
his death as the final change in TeX._

The file is called "tex82.bug":
[http://mirrors.rit.edu/CTAN/systems/knuth/dist/errata/tex82....](http://mirrors.rit.edu/CTAN/systems/knuth/dist/errata/tex82.bug)

Wikipedia is (slightly) out of date since there's 428 currently in the above
file, but note that not all of these are actual (functionality-breaking) bugs,
just changes; for example, #2 is just a renaming of variables and #425 is an
optimisation.

That would be 428 total changes, in a span of a little over 32 years, with the
majority of them extremely early in TeX's history - #214 was in 1983, #321 in
1985, #400 in 1991, #420 (a "missing goto") in 2007.

~~~
sillysaurus3
Thanks! That's one of the coolest changelogs I've ever seen. It's pretty
amazing to see 9000 lines represent 32 years of changes.

~~~
jacquesm
There are a lot of people that would be very happy to receive a check from
Knuth for reporting a bug (those checks are rarely if ever cashed, for obvious
reasons).

So it's not like people aren't looking for bugs.

~~~
jabagawee
Donald Knuth no longer writes personal checks for finding errors in his books
and code, though he still issues a credit for the correct amount at the
(pretty sure it's fictional) Bank of San Serriffe [0]. The page I linked to
also mentions that he will try to send legal tender to a bug-finder if she or
he really wants it.

[0]: [http://www-cs-faculty.stanford.edu/~uno/news08.html](http://www-cs-
faculty.stanford.edu/~uno/news08.html)

------
erdle
Not sure if it's widely known how they were caught... but it was when they
outsourced their translation of documents and a translator read documents and
blew the whistle.

Which is why the top translation firms now offer... security!

~~~
veidr
Wow, do you have a link for that? I think this is a fascinating story in and
of itself, but one of the tangential things that also interests me about it is
that nobody I know in Japan (where I live) has ever even heard of this story.

~~~
bb_wordsmith
Here's a link to one key article: [http://www.asbpe.org/wp-
content/uploads/2014/07/Toyota-Cover...](http://www.asbpe.org/wp-
content/uploads/2014/07/Toyota-Cover-Story.pdf)

In Japan, some journalists know, but media organizations won't publish the
story.

I am the translator.

------
phugoid
My only criticism is that he's comparing Toyota's code and processes to a
published but unenforced standard and clearly they are nowhere near that
standard. It would be more relevant to compare Toyota to other manufacturers.

It seems the legal argument against Toyota is that they were not following
industry standards - but if no one else was, could you really call it an
industry standard?

~~~
rwmj
So if all car manufacturers are as bad as each other, that's OK?

~~~
phugoid
Not at all. But if it turned out that Toyota was already trying three times
harder than everyone else to get it right, it would be harder to argue they
didn't take "reasonable" precautions.

In the talk, he did mention that the government agency that certifies vehicles
does only basic checks and does not enforce any standards on software. One
could easily argue that they are partially to blame, as they're leaving it up
to the manufacturers.

------
snowwrestler
Also see this report from another experts involved in one of the suits:

[http://www.safetyresearch.net/blog/articles/toyota-
unintende...](http://www.safetyresearch.net/blog/articles/toyota-unintended-
acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code)

And the slides (PDF):

[http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUB...](http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf)

------
cbd1984
I thought the "unintended acceleration" was proven to have been due to older
drivers who got confused?

~~~
bigtones
The example he gave was a young police officer who was a very experienced
driver, and had a passenger call 911 in a panic while he was unable to stop
the car with his foot firmly planted on the brake - and it ended up killing
all four occupants. That negates the 'old people can't drive properly' theory.

~~~
lnanek2
Although in Toyota's defense the officer was too clueless to know he just had
to hold the start button down for three seconds to turn off the ignition.

~~~
jacquesm
That's got to be the worst ever user interface. Press the _START_ button to
turn _OFF_ the ignition? That's not accidental that's malicious, especially in
an emergency situation. Have you ever seen a chunk of industrial machinery?
Those big red mushroom buttons are the emergency shutdowns and they don't take
'three seconds' are not labeled 'start' for extra confusion in times of panic.

~~~
gizmo686
In push to start cars, holding down the start button is the normal way to turn
them off, so it is not unreasonable to attempt to do so for an emergency
shutdown.

Having said that, I would like an emergency stop button in cars (preferably
one that works indepenently of the onboard computer).

~~~
afarrell
It's certainly a reasonable thing to try, but as as your aunt eropple notes,
an emergency shutoff should be something so obvious that a person in fear for
his life would think of it.

~~~
FeeTinesAMady
Such as an ignition switch which you turn to the left to physically cut power
to the ignition system, like we used to have back when cars weren't so
overcomplex.

------
jacquesm
I can see an excellent case here for open sourcing such code.

~~~
nitrogen
I've always wanted to build an open source ECU, but haven't had the time.

------
veidr
I think this is one of the most interesting software stories in recent years,
up there with Stuxnet.

The software development process seems so staggeringly, jaw-droppingly
incompetent and negligent, and it now seems clear that software flaws really
_did_ kill people despite the heavy layer of spin that it was driver error,
floor mats, etc.

I almost certainly couldn't buy a Toyota ever again knowing this. But it also
really makes me wonder: how bad is the QA and testing for the software
components of _other_ carmakers' vehicles?

~~~
tobinfricke
> I almost certainly couldn't buy a Toyota ever again knowing this.

What makes you think any other car company is different/better?

~~~
veidr
That's what I mean; I _don 't_ necessarily think it is (which is scary).

Maybe we only got to know how horribly flawed the software situation is at
Toyota because they had finally had enough people killed that they just
couldn't keep it hidden any longer, and Audi and Honda are just as bad and
just haven't yet had this kind of exposure event.

I would prefer to doubt that, but it does all kind of make me want to buy a
restored 1978 Datsun with carburetors and mechanical everything.

~~~
UrMomReadsHN
Idk... there was an article on here a while ago about how after the
Ford/Firestone tires = death incidents a decade and a half ago actually
resulted in Ford making much better quality cars.

~~~
veidr
If this didn't result in Toyota making better cars, I think that would be an
indication that human society is fundamentally broken.

------
upofadown
If someone ever comes up with a practical method of creating reliable software
then everyone would use it. You can always find a programmer who will claim
that they would of done it better... For all we know whatever Toyota is doing
is way better than what everyone else is doing and this is their only safety
critical bug (assuming that the bug actually exists)...

In the end the software is either correct or it is not.

~~~
tokenrove
The information that has been released so far, especially Michael Barr's
comments on it, suggest that this is far from the only bug.

For example, [http://www.edn.com/design/automotive/4423428/Toyota-s-
killer...](http://www.edn.com/design/automotive/4423428/Toyota-s-killer-
firmware--Bad-design-and-its-consequences) quotes Barr's claims: "Toyota’s
electronic throttle control system (ETCS) source code is of unreasonable
quality." "Toyota’s source code is defective and contains bugs, including bugs
that can cause unintended acceleration (UA)."

I am a little appalled at the number of apologist comments on this story.
There is mounting evidence that this wasn't a "it could happen to anyone" bug,
but rather a serious violation of software engineering ethics. Code must not
kill.

~~~
upofadown
>...software engineering...

Some of us do not believe that such a thing exists in any meaningful form. In
the past every approach to creating reliable software has failed to deliver.
That makes it hard to believe that any particular approach is finally the
answer.

I agree that the Toyota software is poorly written. That in no way means that
conforming to a particular standard or method would of automatically produced
software that was more reliable.

------
EliRivers
I work with some people in the static analysis industry, and I'm told that
Toyota did have static analysis tools that would have identified a number of
these issues; as a particular example, recursive functions, which are verboten
under the MISRA rules they should have been following.

I heard to a less reliable degree that the tools _had_ been used, and results
ignored.

------
aristidb
I'm not really a fan of MISRA-C or V process and stuff like that, but the
quality of the source code and process is truly appalling if these slides are
right... tens of thousands of global variables, no bug tracking and so on and
so on.

I really hope they improved since then, for the sake of anybody driving a
Toyota.

------
codys
Anyone know what the things he "couldn't talk about" were? I was hoping to
hear the open mic bit at the end for a hint but the video is cut off.

~~~
MBCook
He mentioned he's still involved in legal proceedings and expects to be
deposed in further trials. I imagine specific implementation details, things
under NDA, etc. are what he's trying to avoid.

In work like this, you certainly wouldn't want to accidentally cause a
negligent company to get off the hook because of an unimportant public
statement.

------
tek-cyb-org
This happened to my parents in a 2002 civic.

~~~
serf
2002 civic used a mechanical throttle cable and mechanical transmission
linkage. These Toyotas/Lexus(s) did not use any such mechanical links to the
drive train, they were dependent on software wholly.

The engine management on your parents' Honda cannot adjust the throttle
percentage or gearing, only ignition and valve timing along with air/fuel
ratios and gear hold-out timings. This could result in a surge of power, or a
stalled car, but nothing like the experience of wide open throttle.

The Toyota UA is thought to have been rooted mostly in the fact that they
adopted fly-by-wire throttle schemes while failing to compensate for the much
more major software/hardware/management risks that come with such systems.

