
A year of digging through code yields “smoking gun” on VW, Fiat diesel cheats - AndrewDucker
https://arstechnica.com/cars/2017/05/volkswagen-bosch-fiat-diesel-emissions-cheats-cracked-open-in-new-research/
======
mabbo
While I certainly don't support the 'rogue engineer' excuse that VW originally
gave everyone, the idea does bring up an interesting point.

An engineer did write this code. Almost certainly, many of them, working
together. Now, their managers told them it was okay to do this, maybe that
legal had given the go-ahead or that they had to do it to keep their jobs...
But it's still pretty clear that this was not right. They still did it, and
then didn't tell anyone. And that doesn't sit well with me.

In my opinion (and I welcome disagreement and debate) engineers have an
obligation to say 'no' to wiring unethical code.

~~~
Sunset
>In my opinion (and I welcome disagreement and debate) engineers have an
obligation to say 'no' to wiring unethical code.

I respectfully disagree. Engineers have an obligation to do whatever is asked
of them by management. If that means building nuke-hand-grenades so be it.

Being an engineer( be it software or mechanics ) should be a morally neutral
thing. Do the job and keep your morals out of it. This is the only way we can
have some semblance of sanity in the field.

~~~
lmitchell
'Once the rockets go up, who cares where they come down? That's not my
department! says Wernher Von Braun.'
[https://www.youtube.com/watch?v=kTKn1aSOyOs](https://www.youtube.com/watch?v=kTKn1aSOyOs)

I really really _really_ strongly disagree with this. As the ones who are most
knowledgable about the systems they're building, I definitely think engineers
have a moral responsibility to make sure they're at the _very_ least following
applicable laws and regulations - I would argue that we have a moral
responsibility to act ethically even in cases where it's _not_ covered by laws
or regulations, but I'll admit that's a bit more controversial.

~~~
Sunset
The problem with "moral obligations" is a person's moral axioms are
fundamentally arbitrary. Would you still want engineers to be guided by their
moral convictions when you fundamentally disagree with them yourself?

~~~
lmitchell
Inasmuch as I want everyone to be guided by their moral convictions, yes. I
mean, even if I don't agree with them, I'd much rather an engineer's decisions
be guided by 'what they feel is right' rather than guided by nothing at all,
which seems to be what parent was implying.

Also, this is why we have engineering codes of ethics, at least in Canada and
the US (and while I'm not familiar with elsewhere in the world, I would assume
similar things hold in most first-world countries). We don't necessarily have
to agree on everything, but there _is_ a baseline for what we consider
ethical, and engineers _are_ expected to uphold that baseline, otherwise they
are not permitted to practice engineering. Unfortunately the line between
'engineers' and other practicioners isn't as well-defined for software
engineering as it is for most engineering fields - but that doesn't mean we
should ignore it completely.

------
bambax
> _In 2015, regulators realized that diesel Volkswagens and Audis were
> emitting several times the legal limit of nitrogen oxides (NOx) during real-
> world driving tests. But one problem regulators confronted was that they
> couldn’t point to specific code that allowed the cars to do this. They could
> prove the symptom (high emissions on the road), but they didn’t have
> concrete evidence of the cause (code that circumvented US and EU
> standards)._

I don't understand this from a regulator's point of view: as a regulator, all
you have to do is test for symptoms. You don't have to explain root causes.
You drive the vehicle in conditions as close as possible to real ones, measure
emissions, and decide whether or not they're above the norms.

Why would regulators do this in a lab? It's like health inspections that would
ask restaurants to send food to be tested, instead of showing up anytime,
unannounced.

Regulators should pick up real cars from real owners and test them on the
road, at regular intervals.

Or, modern technology should allow to test a car all the time and report
emissions and fuel efficiency, etc. during its lifetime.

People cheat, and if cheating is easy they cheat more. The one thing a
regulator cannot do is trust the industry.

~~~
majewsky
> Why would regulators do this in a lab? It's like health inspections that
> would ask restaurants to send food to be tested, instead of showing up
> anytime, unannounced.

Because the law on how regulators work was written by the auto lobby. (At
least, that's how it is in the EU. Don't know about the US.)

~~~
kadavero
Funny you should mention the lobby. The US auto lobby was the reason the US
NOx diesel emission requirements are more stringent than EU in the first
place, specifically disadvantaging European diesel cars (which are much
cleaner overall).

~~~
dubyah
>The US auto lobby was the reason the US NOx diesel emission requirements are
more stringent than EU in the first place

No, that was a result of the Clean Air Act Amendments of 1990 for the
reduction of acid rain. Though it was targeted towards industrial emissions of
SO2 & NOx, but stricter regulation for vehicles were an additional effect.

>specifically disadvantaging European diesel cars

That's a weird argument given that diesel passenger vehicles in the US are
held to the same standard as gasoline ones, but to a separate standard from
their petrol counterparts in the EU. I mean, one could argue the opposite,
that an EU emissions policy favorable to diesels amounted to an equivalent
13-16% import tariff. [1] Several domestic rather than just foreign diesel
engine manufacturers were also penalized for using defeat devices in 1998.[2]

> (which are much cleaner overall)

That's quite arguable, trading lower CO2 & CO for increased NOx & PM.

[1]: [http://www.eugeniomiravete.com/papers/MMT-
Diesel.pdf](http://www.eugeniomiravete.com/papers/MMT-Diesel.pdf)

[2]:
[http://articles.chicagotribune.com/1998-10-23/news/981023011...](http://articles.chicagotribune.com/1998-10-23/news/9810230110_1_detroit-
diesel-corp-engines-mack-trucks)

~~~
kadavero
>but stricter regulation for vehicles were an additional effect

Surely car manufacturers didn't have a say, which is why US and EU emission
standards look like this [https://longtailpipe.com/wp-
content/uploads/2015/10/us-europ...](https://longtailpipe.com/wp-
content/uploads/2015/10/us-europe-emissions-standards-difference.png) .

>same standard as gasoline ones

Well duh, let's keep diesel cars to petrol standards so that their benefits
don't matter and their disadvantages are prohibitive!

~~~
dubyah
>Surely car manufacturers didn't have a say, which is why US and EU emission
standards look like this

As per the source of the image says, "On the other hand, American regulators
are focused on smog and health impacts of air pollution." Which the graphic
you provided well indicates.

Look, California was probably the first governmental entity to regulate
tailpipe emissions. Such so that it's written in the Clean Air Act by name to
run its own regulatory scheme to enact stricter regulation(with federal
waivers, but that's another issue). The reason being, that LA's unique
geography makes smog worse. Heck, in the 1940s, they had an episode severe
enough they thought they were under chemical attack by the Japanese. As such,
CARB's emission standards were focused on reducing the more directly harmful
pollutants like hydrocarbons, ozone, NOx & PM. So, given California's
influence on the original 1970 Clean Air Act and the 1988 California Clean Air
Act's influence on the subsequent amendment in 1990, I don't see how that
graphic would support your argument. I mean, had they such hypothetical power,
they could have also blocked the banning of leaded gasoline that was in the
same amendment.

>Well duh, let's keep diesel cars to petrol standards so that their benefits
don't matter and their disadvantages are prohibitive!

Emissions vs fuel economy. You're being facetious, but if that argument was
true, why bother importing diesel passenger vehicles into the states? They
didn't even start reintroducing diesels in America until they thought they
could harmonize emissions from Euro 5 with Tier II Bin 5.

------
avar
I may be missing something here, but if you take a step back this just seems
like a tragedy of misaligned economic incentives.

Both the US & EU would have wanted this information from day one, and this
whole fiasco cost VW billions in fines.

There would have been any number of engineers at VW and Bosch that knew
exactly how this worked, but there was nothing in it for them to come clear
about it. They were never going to get charged for writing that code, and they
would have likely been out of a job or destroyed their career at those
companies if they volunteered to authorities how this worked.

So why don't investigators just offer a huge cash prices to engineers at those
companies who can provide details about exactly how this worked, along with
immunity as long as they're forthcoming with information about who instructed
them to implement this?

You'd have an army of engineers overnight willing to spill the beans, and
you'd save millions in investigative costs, and quickly get to the real root
cause of the corruption.

Instead some independent team of investigators is left digging through old
firmware images posted on forums to reverse engineer how the defeat device
worked.

~~~
obstinate
I don't know for certain whether the concept of a bounty was considered, and
if so why it was rejected. But I can think of a few possibly good reasons to
not do this. For one thing, it could be thought of as rewarding the engineers
who wrote the evil code, maybe even creating an incentive for engineers to add
such code in the future. There are precedents for enforcement actions having
the unintended consequence of causing more incidents. Another thing is that
you may actually wish to prosecute the people who wrote the code, to serve as
a disincentive to future engineers considering following management orders on
a similar defeat device.

~~~
d0mine
Why do think the notion of whistleblowers or witness protection exists?

~~~
obstinate
I'm pretty sure whistleblower protections are not a get out of jail free card.
If you blow the whistle on actions that you took part in, you can still be
prosecuted.

------
jancsika
> Firmware images were gleaned from car-tuning forums and from an online
> portal maintained by Volkswagen for car repair shops. Documentation, in the
> form of so-called “function sheets,” was harder to come by. The function
> sheets were necessary to give the binary context, but the sheets are
> copyrighted by Bosch and generally not shared with the public. The research
> team ended up turning to the auto-performance tuning community again. These
> hard-core hobbyists and professionals share leaked function sheets so they
> can make aftermarket modifications to their cars.

This is crazy. And, as we move toward self-driving cars, dangerous.

Can someone point me to a well-regarded source in the industry that makes a
cogent and convincing argument for why all the software running in a car (save
_maybe_ for the entertainment panel) should _not_ be required to be open
source?

To be clear: I'm not asking for speculation, or even an explanation from an
expert based on years of experience. I'm asking for a publicly accessible
reference based on research data that explains the security benefits of not
releasing the source code for life-critical software in the car.

Edit: clarification

~~~
obstinate
I would be very surprised if such a thing exists. Are there any industries,
safety critical or otherwise, where the default is open source? If not, why
would anyone think to justify the decision to not open source code, and
especially why would anyone think to make such a memo or piece of research
public?

~~~
ryukafalz
>Are there any industries, safety critical or otherwise, where the default is
open source?

Cryptography.[0] When trustworthiness is paramount, as in crypto (and IMO in
safety-critical applications like this), being able to inspect the code helps
a lot.

[0] [https://www.schneier.com/crypto-
gram/archives/1999/0915.html...](https://www.schneier.com/crypto-
gram/archives/1999/0915.html#OpenSourceandSecurity)

------
Sniffnoy
> The researchers also say that it’s high-time regulators dispense with the
> kind of lab tests that US and EU governments have required for years.
> Instead, some kind of active scan for illegal code needs to be developed.

Alternatively, combine the lab tests with imprecise real-world tests as a
sanity check, or keep the exact nature of the lab tests a secret and vary them
over time. Really, the first of these seems like a good idea regardless of
what else you do.

~~~
tyingq
The real world sanity checks for cheating make sense. But the nature of the
regular tests can't be secret. Carmakers need to know the bar they are
supposed to hit. Building something on the speculation that it might pass,
maybe, is a whole different business model.

~~~
adrianN
"Build a car that doesn't emit more than X of exhaust Y when a typical driver
drives it a) ten kilometers in the city, b) a hundred kilometers on the
highway."

That seems like a good enough requirement to me. Define "a typical driver" as
"out of a hundred random test drivers, no more than 20 exceed the thresholds,
no more than 5 exceed the thresholds by more than a factor two." That forces
the car manufacturers to have a sufficient safety margin in their emissions.

I don't know how difficult the actual measurement is, but maybe you could pay
a couple thousand people a reasonable amount of money to have some devices
attached to their cars for a month or two and collect data. Or make the car
makers pay for the procedure.

~~~
joncrocks
So this is going to happen once they've built the car + engine, and started
selling it? Is 'car tester' going to become a side-hustle?

I agree that a degree of randomness is likely a good idea to avoid defeat
devices, but one also has to consider that it could have two unintended
consequences: 1) more expensive cars, due to more stringent QA procedures 2)
relaxing of standards to ensure that companies can still practically make cars
that conform to 'standards.'

As ever, it's important to consider that a layman's "seems reasonable to me"
is another experts "that's not how things work."

~~~
adrianN
"The average driver" is unlikely to change their driving style very quickly,
so the tests would be reasonably reproducible from year to year.

------
parennoob
> In 2015, regulators realized that diesel Volkswagens and Audis were emitting
> several times the legal limit of nitrogen oxides (NOx) during real-world
> driving tests.

This means that they can detect emissions levels during real world driving
tests. What's wrong with just making those tests the actual regulatory ones?
So whatever ingenuity automakers can use will be put to minimizing emissions
in the _exact same scenarios_ that will be used in real life.

~~~
nisa
> What's wrong with just making those tests the actual regulatory ones?

Nothing. But Germany has a strong position in the EU and to a large degree
does what VW/BMW/Mercedes want and they wanted to avoid stronger or better
tests. This topic is quite often in the news here and it's clear that there is
no political will to establish real word tests.

There are also a lot of other cheats in this firmware:

\- Below 14°C? Just blast the emissions out. It's not like these cars are
driven in the winter.

\- Autobahn? Go blast out the emissions!

\- and so on...

This should be a far bigger scandal than it is now.

~~~
majewsky
> Below 14°C? [...] in the winter

It doesn't take winter for temperatures to drop below 14 °C (57.2 °F). The
morning commute takes place between 6 and 9 AM. Even in the summer,
temperatures are probably below 14 °C more often than above it at these times.

~~~
nisa
Indeed the Umweltbundesamt PDF (see my other comment) states that it's at
least 50% of the time the case! 50%!

------
dkarapetyan
Weird conclusion. This kind of activity should just be deemed illegal instead
of treating it as an arms race between the manufacturers and regulators. Have
3rd party code auditors look through the code and flag any shenanigans.
Instead of treating the code as an unregulated black box.

------
obstinate
What puzzles me about all this is why the investigators did not compel the
companies to produce the exact mechanisms by which they cheated emissions as
part of the settlements. I guess it doesn't _really_ matter, since what you're
chiefly interested in is the manufacturer ceasing the behavior and not
repeating the offense, so that might be why they didn't.

------
garaetjjte
Why they were analyzing time/distance curves, steering wheel angles,
temperature, etc. instead of just using accelerometer to check if car is
really moving?

~~~
tyingq
I don't think there's any accessible to the software. They are used for
airbags, but I think that's not on the CAN bus. If one were, it might be too
dead a giveaway to poll it, since you already have speed via rotational
sensors.

------
DonHopkins
It smells more like a "smoking tailpipe".

------
edejong
An interesting twist to this would be to change liability. Make the driver
liable (not the manufacturer) for driving a car breaking the emission
standards. This will set in motion the following steps:

\- Car users will be offered insurance against breaking emission standards

\- Insurance companies will hire specialists to lower insurance rates of above
insurance

\- Consumers are disincentivized to modify their ECU in a non-compliant way

\- Consumers are incentivized to buy cars offering more transparency in
firmware (and car manufacturers will offer more transparency)

I'm a firm believer in: you buy it, then you're responsible for ascertaining
its safety. If you can't do this, hire someone to do it for you.

~~~
PhasmaFelis
> _I 'm a firm believer in: you buy it, then you're responsible for
> ascertaining its safety. If you can't do this, hire someone to do it for
> you._

This seems like one of those plans that goes

1\. Deregulate everything

2\. ...

3\. All irresponsible companies go out of business when intelligent consumers
patronize responsible ones instead

Skipping over the "decades of death/disruption" in step 2, and being awfully
optimistic about consumer intelligence in step 3.

~~~
edejong
I don't want to deregulate everything, especially not all security regulations
on cars. What is needed is removing distance between regulators and users.

Would you rather buy a locked down laptop which is factory protected against
viruses (with a known set of them) but no chance to modify nor understand the
virus protection system. Or, choose for a regulating system that punishes
those who help spread viruses because no scanner was installed?

The latter teaches the users more and keeps them closer to the product. It
creates a market where more parties can enter and provide security advice.

Lastly, the 'stepping over death and destruction' is obviously a straw man. We
have seen many deregulations that we in dire need and worked out well. It all
depends on transparency and communication.

