
Volkswagen and the Blame the Engineer Game - jacquesm
http://jacquesmattheij.com/volkswagen-and-the-blame-the-engineer-game
======
PeterStuer
My personal suspicion goes way beyond VW in this case. Look at it this way. If
someone in your domain of expertise would put out a feat, that according to
your experience is just so far beyond the state of the art that it seems to
realy violate the rules of what you think possible, and you work in a highly
competitive field, then your competitive intelligence will be all over it.
Every diesel engine R&D center at any auto maker would have inspected,
measured, tested and dismantled these engines to the very last detail. And
they would all have discovered the simple truth they suspected all along
(benchmark cheating through test cycle detection and software manipulation is
nothing new in the industry. Automakers have been caught en fined for this at
least since 1998). Yet, none of these competitors spoke up about it. Why?

~~~
jacquesm
You could very well be on to something there, see:

[https://news.ycombinator.com/item?id=10717216](https://news.ycombinator.com/item?id=10717216)

9 Vehicles tested under driving conditions for NOx emissions.

~~~
kuschku
Indeed. And it is troubling to see only VW being targetted here, and with
extremely high potential fines shown in the media.

If it turns out that other competitors did the same, VW – and even other
German companies, due to the hit the brand "Made in Germany" got from this –
might be able to sue the US via ISDS for lost sales in hundreds of billions.

~~~
jacquesm
That's not how it works, what will happen instead is that those other
manufacturers will _also_ be hit hard by the US regulatory entities.

~~~
kuschku
That does not matter.

The media focus has been solely on VW, despite others doing the same.

This is enough that VW can show to be unfairly targeted, as it’s standard
market practice.

If VW can show that others did the same, but did not get the same media
resonance, then VW can sue for the lost sales based on an estimation of the
difference in reputation loss.

Hell, even other companies in Germany have reported lost sales due to the
"German engineering = pollution" that US media have pushed recently.

~~~
jacquesm
> The media focus has been solely on VW, despite others doing the same.

That's because there is evidence about VW.

> This is enough that VW can show to be unfairly targeted, as it’s standard
> market practice.

No, it really is not.

> If VW can show that others did the same, but did not get the same media
> resonance, then VW can sue for the lost sales based on an estimation of the
> difference in reputation loss.

It's an interesting theory about how the law could work but unfortunately
somewhat detached from what I know about how the law actually works. VW does
not get to claim any 'lost sales' directly attributable to their own conduct,
regardless of who reported on it and how frequently even if _all_ their
competitors did it too. If VW has evidence that other manufacturers cheat as
well they're probably more than welcome to present it. (And if they had such
evidence they probably would.)

> Hell, even other companies in Germany have reported lost sales due to the
> "German engineering = pollution" that US media have pushed recently.

Yes, and VW is to blame for that. Not the US media.

~~~
kuschku
The original report that caused this whole thing presented evidence for 9
companies.

9.

VW is not the only one in this whole situation.

~~~
takeda
The report shown that results from the tests did not match the actual usage.
It raised questions that those companies most likely are tampering with tests,
but nothing incriminating. It was speculated that possibly that the tests were
not reliable.

In case of VW, in addition to that report an actual device to fool the tests
was discovered.

------
Maarten88
Saying the compliance department exists to make sure the company adheres to
the law is like saying the HR department is there for employees. That is what
they say, but in reality it's the other way around: Compliance acts to protect
the company against the law. This has clearly failed at WV.

Also the post assumes a big organisation operates as a whole. My experience is
that management high up are either competent and evil, or ambitious and
ignorant. That mix lends itself to all kinds of random organisational
behaviour. I know nothing about WV, but I assume many people knew this but did
not think too much of it (everyone cheats), thought it would help their career
(hit that target), or wanted their company become the largest car manufacturer
in the world.

~~~
hoorayimhelping
Jesus, that is a cynical way to look at the people who work in an
organization. Is any non engineer not evil or amoral in your worldview?

~~~
Maarten88
Very few people really think of themselves as evil.

But in my worldview most people (including engineers) cheat a little bit and
then rationalize it so they don't have to feel bad about themselves. Big
corporations and complex economical systems lend themselves very well for
rationalizing your part in something evil away, especially for upper
management.

Watch this video. It changed my view on the world:
[https://www.youtube.com/watch?v=XBmJay_qdNc](https://www.youtube.com/watch?v=XBmJay_qdNc)

p.s. It has been shown that the psychological profile of top-criminals and
CEO's are remarkably alike.

~~~
jacquesm
> But in my worldview most people (including engineers) cheat a little bit and
> then rationalize it so they don't have to feel bad about themselves.

So, if we assume that is 100% accurate then maybe a company the size of VW
should have a department that checks up on these sneaky cheating engineers to
make sure they don't expose the company to liability.

The whole idea is that the company avoids - given the size of the penalties
involved - to be caught cheating, especially not at this scale.

Then there is the matter that the ethical rules for engineers specifically
state that an engineer will _never_ assist or cause a company they work for to
break the law. (For some this sort of an oath may hold more strength than for
others, but it must have been very clear to the people involved they were
doing this to break the law and they would have definitely wanted at a minimum
indemnification from higher ups about the possible consequences of this if
they didn't actually blew the whistle internally.)

This is the sort of thing that Germans tend to be big on and it is one of the
reasons 'made in Germany' was seen as a source of trust and quality.

VW management is throwing all these values under the bus as if 60 years of
brand creation are utterly worthless.

~~~
bardworx
I don't know how VW gets audited but in US, when a company is audited by their
accounting firm, one of the sections is a "Process Audit" \- checking to make
sure that regular business practices are followed.

In a perfect world, audits would catch these things, however, because there is
a lot of pressure on accounting firms, usually, this step isn't given a lot of
consideration.

In the event a company is caught, the accounting firm takes a hit on their
reputation and have to go back, re-audit, and change their designation of
company policy.

Source: my wife works at one of the big 4 accounting firms and recently
explained how an unrelated company had similar issues in failed business
practices etc. etc.

~~~
megablast
Sure, and sometimes these audits are people asking the managers "do you do
this?". That is all they do. They don't sit in and watch people.

~~~
sithadmin
>sometimes these audits are people asking the managers "do you do this?".

I work for one of the large pro services firms that does internal audit,
compliance/risk projects, strategy consulting, etc.

I'd argue that unless there are explicit sets of controls that HAVE to be
tested as part of the project scope (which is usually defined by regulatory
requirements), _most_ audits are just people asking questions and ticking off
boxes on a checklist. Most of the bigger firms even staff these projects with
fresh graduates that often know absolutely nothing about what they're
auditing.

~~~
jackgavigan
Conducting an audit doesn't necessarily mean that you check everything. Often,
auditing is looking for evidence as to whether a particular procedure was
followed or not.

For example, say you're auditing a large company for compliance against a
information security management system under ISO 27001, and you're checking to
see whether the organisation you're auditing actually _does_ keep its
operating systems patched and its anti-virus software up to date.

You'll certainly ask questions about whether, how and how often the updates
are done but you're probably _not_ going to check every single server, desktop
and laptop because that would be too expensive. What you'll probably do is
check a sample, and if all the devices in the sample are all up to date,
you'll tick that box and move on.

If, on the other hand, you find that half the desktops you check _haven 't_
been updated, then that's evidence that the updates policy _isn 't_ being
followed, and would trigger further investigation, and probably a failure on
that specific audit point.

The equivalvent check in the VW case would be to run a test of the engine's
emissions. It's not unreasonable for the methodology used to be based on that
used by the regulatory body, in which case, it would have passed the audit.

~~~
sithadmin
Nothing you're saying is wrong or theoretically unsound. But in practice,
'looking for evidence'-type activities are frequently glossed over, even by
very reputable audit firms.

------
leereeves
Is it fair to characterize Volkswagen's response as "blame the engineer"?

For example, from [http://www.wsj.com/articles/vw-shares-up-ahead-of-
emissions-...](http://www.wsj.com/articles/vw-shares-up-ahead-of-emissions-
findings-1449740759)

> Volkswagen has suspended nine managers suspected of being involved in the
> fraud

> Volkswagen has replaced six executives since the beginning of the year and
> has switched leadership of seven of its dozen brands.

> Volkswagen also disclosed in November that employees had tipped management
> off to another unrelated issue, saying the company had understated
> greenhouse-gas emissions and fuel consumption on up to 800,000 gasoline-
> powered cars. After an internal investigation, Volkswagen lowered that
> figure to fewer than 50,000 cars.

~~~
nerdy
Categorizing it as "blame the engineer" is sadly accurate in my opinion. That
has been VW's stance from the very beginning. VW's US CEO said, while denying
a corporate-level scandal, "software engineers who put this in for whatever
reason," to a congressional panel back in October:
[http://arstechnica.com/tech-policy/2015/10/volkswagen-
pulls-...](http://arstechnica.com/tech-policy/2015/10/volkswagen-
pulls-2016-diesel-lineup-from-us-market/)

Certainly doesn't get any more "blame the engineer" than that, unless they're
supposedly somehow changing their tune now... which is not my impression.

With regards to middle management being laid off, it strikes me more as the
C-level players insulating themselves from the taint.

~~~
URSpider94
At least two of the people fired were C-level engineering managers (chief
engineers), and in fact they are the people often cited with hatching this
plan in the first place.

I'll keep saying it -- there seems to be some belief that anyone sitting in
the boardroom is not an engineer. That may be how you see it, but it's not the
common perception.

------
coldnebo
Yes, but in this case compliance is quite difficult. Why would the compliance
department go out of its way to develop a new mobile testing infrastructure
when that isn't the industry standard? So it seems very plausible in this case
that compliance might not have found the issue. It required a novel method of
testing to discover the cheat.

I have to be honest, when I first heard about this, I suspected a genetic
algorithm as the culprit. It's exactly the kind of result you'd expect when
giving an impossible optimization without sufficient boundary constraint. AIs
can and have been known to cheat, and because the fitness driver would have
been compliance (V&V) the GA could plausibly have settled on methods that were
undetectable (i.e. Not part of the fitness function).

Often the output of GA code is not understood formally by engineers, or
reverse-engineered into a traditional algorithm, so it's plausible that it
could have remained hidden in plain sight.

BUT, if that had truly been the case, the engineers at VW would have quickly
suspected and publicized it. So the fact that VW blinked showed that in fact
they lied and didn't have an honest technical explanation.

However, I think the GA scenario is good to keep aware of as we increasingly
rely on machines to optimize solutions based on testing.

~~~
ArkyBeagle
The "sin" they are accused of it explicitly detecting that certain test
equipment is connected. That's a binary, if-then-else sort of thing. "The boss
is looking; look clean."

A GA (or any machine learning ) within a controller is less about being able
to cheat than it is ...rebasing something that would otherwise be a constant
(from the frame of reference of the algorithm) in a control-feedback
arrangement.

If the GA was part of a test stand and exhibited emergent... dishonest
behavior :) then ... they did it wrong.

~~~
Shivetya
There are valid reasons to write code to determine if your vehicle is being
tested. Modern cars contain all sorts of computerized safety features that
would go active in the typical test scenario thereby preventing the static
tests that the EPA and others do.

So the car detects wheels straight, only drive wheels turning, and so on, so
it disengages the features that would active traction control, reducing engine
power, and more. The key here is that VW engineers went further and used that
as they indicator to run the full emissions process and only then. There is no
viable reason to have any toggle on emissions processes except perhaps with
regards to cold weather starts.

~~~
ArkyBeagle
We _think_ it's "run the full emissions process" but we're guessing. It's a
very good guess, but it's essentially unsupported. It could have been an
interaction with one of the other things that got turned of. This seems
unlikely,. but it's not 0% probability.

------
brazzledazzle
>...without the compliance department...

In my experience compliance departments make sure you're compliant within
reason. They aren't going to go out of their way to determine if you're
cheating unless they've been mandated to due to prior internal or industry
incidents. Even in companies where they have to audit controls because of
regulatory obligations they're only going to audit what they're required to.
If you find a way to outsmart the controls or checks and balances they're not
going to notice until you slip up or you're otherwise caught.

>...and all outside verification labs being in cahoots.

Does this have to be true? They would only need to apply the cheating behavior
for the lab testing the way they did for regulatory testing.

And VW suspended nine middle managers. I think it's less about blaming the
engineers and more about deflecting blame away from senior management.

~~~
kuschku
> And VW suspended nine middle managers. I think it's less about blaming the
> engineers and more about deflecting blame away from senior management.

Well, they also had two CEOs resign over this, one even refusing to take any
golden parachute he might have gotten otherwise.

It’s more the board that’s scared right now...

~~~
jacquesm
I suspect it is a matter of time now before someone leaks an email to the
press indicating that the previous CEO _did_ know about this (they'll be
thrown under the bus), and the board will then try to wash their hands of it.

It's a classic fall-back action, first sacrifice some lieutenants, General
admits ignorance but admits to no wrongdoing and is pensioned off.

If the bad weather blows over that's where the story ends. But if it doesn't
then sooner or later the eyes will focus on the General again, what he knew
and when he knew it and then the inquiry will begin in earnest.

There hasn't been anything surprising in this whole affair to date.

~~~
kuschku
Well, the first CEO, who resigned in April, admitted already to personally
pushing the whole issue.

It was his personal project (he’s been involved with some engineering for
quite some time) to use that specific type of engine, which provides mileage
of up to 270mpg in real world usage, but produces NOx in amounts that Satan
would be jealous of.

So, that’s it.

~~~
jacquesm
Yes, but he did not admit to actual knowledge of the defeat device.

------
csours
Disclaimer: I work for a competitor.

Bottom line: real world use should be tested, but this is expensive and
neither side wants to pay for it.

Regarding compliance department complicity, the biggest factor in my mind is
two-fold: You test for compliance with regulations and you have limited time.
You ensure that the internal test parameters match the external regulatory
test. You don't have funding to do real world tests. You don't have funding to
do static code analysis. You believe the documents you are given, as they have
been signed by responsible adults. You use the tools you are given, because
they have been developed by professionals.

Regarding what competitors knew and when: I haven't seen a discussion of
exactly how much better these engines were than competitors.

Regarding the scandal in general: The public has been demanding better and
better automotive safety and quality represented by government regulations,
voting with dollars, and lawsuits. It takes time for these lessons to be
internalized, and different corporations and internal groups do this at
different speeds. If you are the largest automaker in the world it is kind of
natural to think that everything must be ok! (See also: GM, Toyota)

~~~
jakejake
I can imagine this as not totally different from software dev and QA. Many
times the development team puts in some feature that doesn't have any visible
output for the testers to test - rather it is some internal change or
improvement.

The QA department, then responsible for testing this new "feature" will most
likely go off of a script that was sent to them by the developers to verify
the old and new behavior, going on some metric that perhaps they don't really
understand.

In some cases, the QA may actually have to rely on a tool actually built by
the developers to test the feature. So in some cases the QA, not being
software developers and not responsible for actually reviewing the code, will
have to somewhat go on faith that the scripts and tools provided by the
developers are accurate.

I'm not saying that means the engineers were responsible at VW, but it does
explain how something could slip through the checks-and-balances that have
been set up.

~~~
jacquesm
> The QA department, then responsible for testing this new "feature" will most
> likely go off of a script that was sent to them by the developers to verify
> the old and new behavior, going on some metric that perhaps they don't
> really understand.

That's wrong right there. If you do this properly the people writing the QA
test script work from the same _spec_ the programmers used to implement the
feature but they are actually different people and the test gets written
first.

Obviously in smaller shops that won't work but anything the size of newsworthy
should be set up like that.

~~~
jakejake
I definitely agree that's the way it _should_ be. I've seen the reality inside
a few large corporations and basically nothing would surprise me.

------
jakobegger
The funny thing is that the engineers that wrote the code don't actually work
at VW. If I am not mistaken, their engine control sytems for their Diesel
engines come from Bosch.

Also, if you talk to an engineer you will quickly hear that there is a
connection between fuel efficiency, power, and emmissions. If you drive the
engine with parameters tuned for low emissions, you'll get less efficiency
etc.

But the engineers don't seem to care that the engine stats claimed by VW are
not actually possible...

~~~
scotchmi_st
> If you drive the engine with parameters tuned for low emissions, you'll get
> less efficiency etc.

Can you explain that further? Given a specific power output, lower emission
levels would to me indicate that the engine is doing more with a smaller
amount of fuel, indicating better efficiency.

~~~
Hermel
Your argument mainly applies to CO2. Generally, Diesel engine reach better
efficiency at higher temperatures, but higher temperatures also lead to more
NOX being created. So in the case of NOX, it really is a trade-off.

~~~
logfromblammo
Unless you use ammonia (from heated urea solution) in a catalyzed reaction, to
react with the NOx to produce N2 and H2O. In that case, the trade-off is that
you need to add a separate urea tank to your vehicle, and distribution
infrastructure at all diesel fueling stations.

------
URSpider94
I think there's a perception problem here. What the press and VW management
has said is that this project was knowingly commissioned by VW's chief
engineer. Yes, this guy's title is "chief engineer," and he has the bona fides
to prove it, but you should focus on the "Chief" part -- he's a C-level
executive whose day to day activities and demeanor would be indistinguishable
from a CFO or CEO.

There's this reflexive reaction on Hacker News that has come up again and
again of, "don't blame the engineers, they are just the nerds in the basement
hacking the code, some evil management type put them up to it."

What we know so far is that the evil management type who put them up to it --
was an engineer. The chief engineer in fact. So, in fact, it's highly likely
that "the engineers" designed and implemented this entire scheme without
telling anyone else. In my mind, that's an entirely plausible explanation.

I'm not an expert on VW corporate structure, but it's entirely possible that
the Compliance/Quality departments report into the Chief Engineer. They do at
other companies where I have worked. If that's the case, then they could have
been in on the job and it wouldn't have changed the fact that "Engineering"
was responsible for this implementation.

~~~
dclowd9901
I don't think a blame group size of 10 (the number thrown around by VW execs)
denotes a systematic problem, which you would expect with a C-level executive
being called out. That's a ridiculously small chain for a company of VW's
size.

------
fallingfrog
Here's the question that you need to ask yourself: The cheating that VW did
allowed them get an emissions score that is an order of magnitude lower than
the truth. Now, if everyone else was being honest, either A) their performance
would be an order of magnitude lower than VW's, or B) everyone else has
discovered magical technology that VW lacks. Neither of those things is
happening. The conclusion is obvious: _everybody is cheating, and they always
have been._ I mean, a hack of this magnitude wouldn't be performed without
management approval from top to bottom, and let's face it, the only way
management would have the balls to do something like this is if it were
industry standard practice.

~~~
jacquesm
That's not the whole menu of choices. The other option on the menu is: VW
found that cheating software was cheaper than actually doing something about
emissions. The other manufacturers are not that much better than VW at
engineering they simply threw more money at the problem. VW tried to do an
end-run around both solving the problem _and_ passing the tests.

Another reason why VW has the spotlight on them is because they are the
largest player. If VW were given a free pass and a lesser manufacturer would
be taken to the cleaners it would be a different situation altogether. I think
there will be more manufacturers that will have to own up to 'gaming the
tests', maybe even some that actually have a similar device but the majority
of the cars are not nearly within the range of test:regular that that
particular engine is. 40:1 is a bit more than an oversight or a small error.

~~~
fallingfrog
You're right, that is possible. But given that VW is the largest player, why
would they be the one to throw less money at the problem than anyone else? But
VW being the largest is a good reason to make an example of them.

~~~
jacquesm
I suspect because of timing. If they had not been fully invested in that
particular engine at the time they would have had a way out. But given that
they already had made a commitment the choice was to drop the US market for
their small diesels or to cheat and they - unfortunately - chose the latter.
It was both cheaper and the only solution that worked under time pressure.
Someone probably thought they were very clever when they thought of this.

------
jkot
I would blame an engineer, but those who designed and ran emission tests.

US suddenly discovers that a few million imported cars do not meet their
requirements, despite passing all tests and paperwork. Good way to collect a
few billions on fines and damage competition.

If EU would enforce their laws on US products similar way, there would be an
economical war.

~~~
mikeash
The regulations require certain levels of emissions to be met under all
conditions. The tests exist to verify compliance, but the regulations don't
merely require passing the tests. When you optimize solely for the test and
lie on your paperwork, you shouldn't be surprised when you get smacked down. I
strongly doubt that if a US company did this and got smacked down by the EU
that any "economical war" would ensue.

------
maurits
I am not fully convinced by the compliance argument.

People from other industries, for example the financial, have written very
candidly on how easy it is to side step and circumvent the rules leaving their
compliance departments in the wishful dark.

~~~
jessaustin
Yes an in-house compliance department, like a private auditing firm, leads
inexorably to agency problems. They don't work for the general public or for
investors; they work for the management of the firm. Generally it's best to
just live with these agency problems, since we don't want to shut down
corporations entirely, but in cases like this the government might need to do
more.

------
ExploitsforFun
I don't buy this from VW for a second. I am not sure what the work environment
is in Europe but in all the corporate places I have written code for in US if
I pulled a stunt like this with no backing from managers the best I could hope
for is getting yelled at. Most places would have fired me on the spot. The
possibility of this being a rogue engineer is extremely remote. This screams
management cover-up.

------
bambax
> _compliance department_...

Right. You know who else has compliance departments? Financial institutions.
Did any of those prevent or foresee or had any clue at all about the financial
crisis? Nope.

You know why? Because compliance departments don't have teeth.

In an ideal world, compliance departments should be staffed with asshole anal
geniuses.

In reality, they are staffed with dopes and yes-men.

They shuffle papers and cover themselves, asking questions such as "are you
following rule A and B as stated above"? and all you need to do to get them of
your back is answer "yes".

They never have any further question because they're not sure about what the
regulation actually says and they fear you'll find out if they start to dig
deeper.

~~~
masklinn
> You know why? Because compliance departments don't have teeth.

Unless they do of course.

> In an ideal world, compliance departments should be staffed with asshole
> anal geniuses.

That's the (legendary) IBM Black Team option. That's not the only one though,
the (almost as legendary) "on-board shuttle group" did that with friendly
rivalry and an absolutely anal _process_ (which, importantly, is _solely_
blamed and amended whenever problems occur).

------
mchupa
I spent a large part of my career in the global automotive arena supplying
products to VW. Martin Winterkorn was a frequently in attendance at meetings
where technical proposals were presented for consideration. He typically led
the way in asking tough questions related to form, fit, and function. VW
invested billions and bet the future of the Company on clean diesel technology
and performance. Martin Winterkorn is hands- on to the point of being a
control freak, and it's impossible to believe that he didn't exert influence
to bend the rules.

~~~
chmaynard
Interesting. At this point, I think the VW executives are trying to avoid
going to jail. They will say almost anything to bend and distort the truth.

~~~
jacquesm
Yes, that is my impression as well. That's why they've decided VW as a brand
can be thrown under the bus as well as long as they get away with this. But it
is getting so large now that I doubt this strategy is going to work they are
clearly reaching for straws to be able to claim plausible deniability.

------
iolothebard
I work at a place exactly like this.

They give you 2-3x the work you can complete, then tell you "you get paid the
big bucks" (which we don't) so you need to work as many hours as necessary to
get it done. Oh, and don't get behind on any of your other work as well.

It's why my last day is coming in the next week or so.

------
PeterStuer
Having worked in highly regulated environments, I would characterize the role
of the compliance department more as enabling business operation and
development while keeping the regulatory risks acceptable. Often times I found
out technical staff had a far more black & white view on rules and regulation
than was warranted by compliance.

~~~
HeyLaughingBoy
Ditto. I would add that Compliance only really works when they are given
accurate information.

If the engineers asks "is it OK if we do X" and Compliance says "No" but
engineering goes ahead anyway, that will likely be the end of it unless
something bad happens. The reality is that a lot of questions & answers end up
not being documented so even an internal audit later won't discover them.

------
_jp__
What bothers me is they fail the reguntaltions for such a high margin that i
cant belive all other auto makers complay with them. I mean, can X be 400%
better than VW. Are their engineers 400% better? This is atonishing at least,
hardly belivable. There is no such a difference with the state of the art in
any industry.

~~~
masklinn
Other manufacturers don't have 400% better engineers, the difference is _other
manufacturers didn 't globally bet on diesel_.

The US has very, very stringent NOx regulations, much more so than Europe
(VW's cheating was in fact discovered through european clean diesel advocates
pushing for tighter US-style regulations): EPA Tier 2 / Bin 5 has a 0.043g/km
limit whereas EURO5 is 0.18g/km and the new (2014) EURO6 standard lowers that
to 0.08g/km (still twice EPA)[0].

Diesel has plenty of advantages but is a big producer of NOx (compared to
gasoline) because NOx is formed at high pressures and temperatures, and
gasoline engines have much, much lower pressure and temperatures than
diesel[1]. Also importantly, increasing diesel engine efficiency runs them
hotter, which increases NOx production.

VW bet on a all-diesel strategy worldwide, and that included a "clean diesel"
push in the US.

[0] for what it's worth, VW engines don't pass EURO5 either (in fact they
don't even pass EURO3, the first EURO standard including NOx) which is why
they're _also_ in the shit EU-side, just to a way lower extent than in the US
(their engines exceed EURO5 limits by a single-digit factor versus double-
digit for EPA limits)

[1] EURO standards reflect this (though whether they should is debatable),
EURO5 has a 0.18g/km NOx limit on diesel engines but 0.060 on gasoline

~~~
_jp__
Thanks for your explanation. But i think the problem here is not the strategy.
We are talking about individual car performance. This is not about the NOx
produced by the whole VW cars. Every other single manufacturer car is at least
4000% better ? Incredible. This is the question.

~~~
masklinn
> But i think the problem here is not the strategy.

The problem _is_ the strategy.

> We are talking about individual car performance. This is not about the NOx
> produced by the whole VW cars.

We are talking about the NOx emissions of diesel engines in general, and of
VW's decision to base their US and global strategy around diesel.

> Every other single manufacturer car is at least 4000% better ? Incredible.
> This is the question.

What's the point of praising my comment if you didn't fucking read it? Other
manufacturers don't have 4000% better cars, in the US other manufacturers
_sell gasoline cars which intrinsically produce significantly less NOx_.

~~~
_jp__
No other auto maker sells diesel in us? Is VW the lonely seller? I read your
comment, it seems you not. Or VW is the lonely seller and you are right :)
Anyway drawk has a point on this

------
tonylemesmer
If the design requirements state: "engine must meet emissions during test" and
the tests show this to be the case, how do the compliance department argue
otherwise?

~~~
kabouseng
The compliance departement is supposed to test independantly for exactly the
same reason you software testers should actually test the product and not just
believe the software engineer when he says: "it works".

------
bargl
I just want to say, as an Engineer this is why you don't willingly break the
law for a company. I had a job where I stayed for less then 3 months because
they wanted me to use the community version of Visual Studio. Except it broke
the terms of service and I could be liable if they tried the "blame the
engineer game." Otherwise it was a great company that was doing innovative and
fun things. They just wanted me to break the law, and I said no.

P.S. This was innocent on the companies part, they thought it was OK and
didn't read the TOS, when I brought this to their attention they entered
"talks" with Microsoft to get a new set of licenses, I just wasn't willing to
accommodate them in the short term. I also didn't know what the talks were so
I had to cover my butt and get out.

~~~
UK-AL
It's ok to use to the Community edition if your company is under a certain
size, revenue etc

~~~
bargl
Part of what I did was point that they didn't fall into that category out to
them. And that because of that they couldn't use Community edition for anyone.

------
jmedwards
Why are so few people questioning the regulators, too? I cannot conceive that
it is that easy to fool them.

~~~
jacquesm
The testing capabilities of government labs are handily outstripped by
industry capability to create differentiated products. The cost burden of
regulation testing is to some extent pushed back to the manufacturers with a
big 'comply or else' warning to make sure they take it serious. The 'or else'
bit translates into a fine per product sold which means if you get caught
cheating that will eat up a very large chunk of your turnover for that market
making getting caught something you will want to avoid so much that compliance
is the easier way out. And if you can't be compliant then the better idea is
probably not to try to sell your products into that market.

~~~
URSpider94
This. Almost all regulatory bodies, whether CARB, OSHA, EPA, USDA have been
pushed into a mode of "self-certification", where the regulated effectively
police themselves. Compliance bodies rarely have the staff or the budget to
audit more than a scant fraction of the test results that are submitted to
them.

For this to work, the punishment, when caught, has to overwhelm the profit
from the deception by not just a little, but by a large multiple. Otherwise,
companies will realize that they can just take the risk and will only have to
pay up in the unlikely case that they are unmasked.

------
joesmo
It doesn't even matter if it was the engineers, the fact that management is
not taking the blame itself is utterly despicable. I'm certain the claims of
this article are true, but let's pretend it was a bunch of rogue engineers'
and compliance people's conspiracy. It is the responsibility of the company to
take responsibility for ALL actions of ALL employees, whether they were known
at the time or not. That's the whole point of being an executive. That's the
whole point also for people who are not executives. Most engineers could start
their own firms but choose to work for others for a number of reasons, not the
least of which is minimizing risk. One of the major risks that one minimizes
through employment is that of being sued or holding blame in _exactly a
situation like this._

VW knows this, yet they're still trying to make it seem like this wasn't the
fault of their upper management and executives. They're still trying to shirk
the blame when they should be firing their whole upper management employees
and executives. There is no excuse for not doing that, and the CEO quitting is
not enough in this case. To blame it on engineer and compliance workers is
absolutely despicable and irresponsible.

~~~
URSpider94
What do you mean that "management" is not taking the blame? The CEO resigned.
The two chief engineers who reportedly hatched the scheme (who are as upper
management as you get, they reported to the CEO) were fired. The head of US
operations was also fired.

What more are you expecting? And how does this make it sound like it's not the
fault of upper management and executives?

------
jrjarrett
This is exactly what happened with Enron and the backlash against that level
of cheating -- the Sarbanes-Oxley Act.

I refuse any argument that a line-level IT staff member cooked the books there
on their own, yet the response - draconian controls placed on access to
production data - does nothing to stop that from happening again.

All it did was make maintaining IT systems far, far more difficult to the pool
on call schlep who has to delete a bad file at 3 in the morning when they have
to involve 6 different people and levels and levels of approval (which takes
until 10am to do). SOX and its ilk does nothing to stop unscrupulous top-level
execs from hiding things.

------
w_t_payne
The Automotive engineering culture is very much an optimising one - constantly
squeezing to reduce R&D lead times and engineering costs to somewhere just
below the bare minimum. This works fine when it is balanced by objective
measures of quality, so you know when the job is done and when you are being
lied to, but as complexity increases, so the risk of operational ignorance and
plain-old self-deception proliferate.

The more complex the system, the greater the reliance upon the integrity and
professional ethics of the engineers working on it, and on the correct
functioning and sophistication of the checks-and-balances within the
organisation.

------
cognivore
"The only alternative explanation (and one that is even more worrisome) is
that the entire compliance department and the entire management of VW were so
dangerously incompetent that they allowed a situation like this to come into
being, exist for several years and required an outside party to bring to
light."

Hanlon's Razor: "Never attribute to malice that which is adequately explained
by stupidity."

That being said, it really doesn't matter if it was on purpose or on accident,
both indicate a company which should not be able to sell cars in the U.S.
There are plenty of other cars to choose from.

------
markbnj
Of course, they could have fooled the compliance department using the same
techniques used to fool regulators in the U.S., i.e. the exact same capability
they'd built into the software. And they could probably have managed to trick
engineering into creating that capability in the software by labeling it with
a different and seemingly legitimate reason for existing. So I think you could
plausibly get it down to just upper management, but I agree that there is no
way engineers knocked this off by themselves.

------
gwbas1c
Never attribute to conspiracy what you can attribute to incompetence.

I work on a software product that's very hard to test. IMO, it's harder to
test the product that I work on than to implement the product.

In VW's case, I think it's safer to assume that emissions testing is harder
than we realize. The engineers might need to make assumptions that, in
practice, require a very complicated test situation to verify.

Therefore, the real problem is playing the blame game.

~~~
jacquesm
VW is actually stating there is an in-house conspiracy but the top level
management didn't know about it.

------
tbyehl
I'm still waiting for the smoking gun of actual code that detects a testing
cycle and changes the operational parameters in a way different from the way
parameters are adjusted normally for different operating ranges.

Optimizing for test cases is rampant in the car industry. Nobody who has owned
a GM car in the OBD II era has ever said "Wow, I get better MPGs than the EPA
rating!" Heck, I'd be thrilled if any GM I'd ever owned came close on the
highway doing 55 with the cruise control on. One of my GM's had something
called Active Fuel Management. Cylinder deactivation. In normal driving it
rarely comes on, but when it did there would be an annoying vibration and hum
from the exhaust that would make you give the car more throttle in order to
make it stop. My real-world MPGs sucked but apparently that anti-feature
boosted GM's numbers for MPG and CAFE purposes.

Another GM car I currently own with a manual transmission forces a 1 -> 4
shift if I attempt to shift from 1st below certain throttle position and RPM
thresholds. Out on an actual road, even with a giant V8, lugging around in 4th
gear at 1,000 RPM is an undesirable condition. Naturally I do my best not to
let that happen by applying my foot to the throttle, but somehow, again, this
anti-feature improves the government numbers while I average 18MPGs on a
mostly highway cruising commute and much less in stop-and-go city driving.

How are these GM anti-features any different from what VW has admitted? In the
GM case the cars very obviously are detecting conditions within a test cycle
and causing the car to operate in a manner undesirable to the driver, and
which will cause the driver to alter their behavior to avoid them.

Is what VW did so blatantly obvious to a driver? What's the legal distinction?
Where's the smoking gun code piece of code that makes their actions a "Defeat
Device"?

On the premise of this article, the whole automotive testing industry is
optimized around testing to the government standards. Just like No Child Left
Behind has our teachers "Teaching to the Test", the automotive industry is
"Testing to the Test." Nobody has any motivation to care about, or even look
at, what happens outside of the test parameters. No grand conspiracy is
necessary for this to have gone undetected or ignored. If the government
considers this a problem then the solution is to create more tests that better
simulate real-world driving conditions.

~~~
jacquesm
The legal distinction is that the code in the VW ECU actively detects the
testing regime and then switches the engine to a mode of operation it would
_never_ use during real world conditions.

That's no longer optimizing for a test, that's simply fraud. Other
manufacturers performance deviates gradually the further away you get from the
test parameters. The VW would _never_ have the performance it has during a
test in any real world conditions, not even those matching the test (because
the vehicle is moving and it can tell the difference).

~~~
tbyehl
I see speculation that there exists code that detects the test and activates a
special mapping mode. I haven't seen that anyone has actually found the code
or map. The map ought to be easy to find, VW ECUs appear to have as many
freely-available DIY tuning tools as other platforms.

I've also seen it speculated that the code to detecting testing is based on
determining that the car is on a dyno, and that all manufacturers perform that
detection in order to disable the stability control systems. If it were as
straight-forward as VW switching to a different map on a dyno I'd have
expected that VW tuners would have blown the lid on this thing long ago.
Tuners use dynos.

------
mironathetin
If an engine does not meet the requirements and because of this, cannot be
sold on a market as large as the US or Europe, this is a big thing.

Unthinkable, that this has been measured and not been reported, but instead
quickly and silently hacked. Indeed, even with a new head, there is still
something wrong at VW.

~~~
jacquesm
Even more unthinkable that there now is a 'quick fix' that costs very little
money per vehicle and will magically make the car meet emissions standards
_and_ will not lead to loss of power or increased emissions. If it was that
simple you'd think those bad boy engineers would have implemented that right
from the get-go rather than to go this somewhat harder route to the goal.

~~~
mironathetin
By the way, as a reaction to this issue, a german motor magazine has measured
real life compliance to the Euro-6 norm for a couple of popular cars: VW Golf,
Opel, Audi, Mercedes c-class, Fiat 500 etc. All, and I mean ALL of them emit
2-8 times as much as allowed by Euro 6. The heavier the car, the worse the
result. All SUVs like the Audi Q3 or BMW Xx have terrible measurements. The
best - was the VW Golf with the 2 litre engine. The worst was the Audi Q3,
which emits 5 times as much as the Euro-5 norm (which it was adapted to) and
thus 10 times as much as Euro-6. Shame.

~~~
jacquesm
Link?

~~~
mironathetin
I was about to write: "there is something like printed journals" (I read it at
the physiotherapist while waiting for my treatment). But it is online:
[http://www.auto-motor-und-sport.de/testbericht/real-
abgastes...](http://www.auto-motor-und-sport.de/testbericht/real-abgastest-
wie-sauber-sind-diesel-wirklich-vw-abgasskandal-10254994.html) Test your
german.

I just saw, that the Fiat 500 is even worse that all the others. Shocking
results.

~~~
thescriptkiddie
If I understand correctly, _all_ of the cars tested (including the petrol one)
exceeded their NOx ratings by at least 60%. Of these, the VW was actually the
second best after the petrol one. The older Audi model they tested didn't fair
as well, but it still did better than the BMW, Opel, Volvo, or Fiat. Going off
of that, it would seem that VWAG was unfairly singled out.

~~~
mironathetin
"it would seem that VWAG was unfairly singled out"

If you look at the true numbers, yes. But VW has been targeted for
intentionally cheating. And don't forget, some Audi engines are also inherited
from VW.

------
draw_down
It's ridiculous, how they've tried to shift blame. But forget compliance
departments and everything else, the situation is actually very simple: the
company made money on the cars they sold, so the company is responsible for
how the cars were built.

------
PeterStuer
Another pertinent question for the techies here: how come we have secure boot
loaders and signed drivers for tablets locking them down, yet chipping or
remapping the cars ECU to negate any 'environmental restrictions' seems to be
unrestricted?

------
lucio
Does the engines comply with EU standards? Why US Standards are different?

~~~
sokoloff
Why is US money different? Why are US plugs and voltage standards different?

Sovereign countries have the right to set their own regulations and
bureaucrats don't have a strong track record of international collaboration.

~~~
mahouse
I would say he was asking for the technical details...

------
randyrand
Everyone even the engineers deserve blame.

------
mathieubordere
"zum kotzen"

