Hacker News new | past | comments | ask | show | jobs | submit login
E.P.A. Finds More VW Cheating Software, Including in Porsches (nytimes.com)
257 points by peterkrieg on Nov 2, 2015 | hide | past | web | favorite | 153 comments

Clearly it's the fault of those pesky engineers again, how else could you explain the same type of cheating across multiple divisions of the company. They only other explanation would involve Management, and obviously that's not it.

Though the fact that none of the other manufacturers spoke up about the violation of emissions standards when when they must have done their own tests to see how VW managed to get emissions so low, seems to point to the fact they they were all doing it to some degree. Perhaps not as brazenly as VW, but still no one wants to rock the boat they are in.

While I certainly don't buy the "pesky engineers" argument, it's actually somewhat plausible Porsche didn't know about the cheating (beyond what everyone knew, which was that the stated mileage and emissions were virtually impossible to achieve by any known mechanism). Since they dropped a VW manufactured engine (3.0 TDI) into a shared VW chassis (Touraeg) and built the Cayenne around that, there's a good chance the engine and engine management were treated as a black box and simply integrated at face value.

Surly Porsche has their own quality assurance and compliance engineers though? And they're proper Teutonic Engineers, right? I mean, they're Porsche engineers... They'd just take an engine from their parent company and not see that it lives up to the Porsche marquee?

Maybe it's possible, but if it is, Porsche is really not the Porsche I think it is or want it to be.

The A4, Q5, Q7, Touraeg, Phaeton, and a lot of other VW vehicles which used the same 3.0TDI have the exact same rated power, which suggests that the engine calibration probably was shared as well. So I think it's plausible that Porsche dropped the engine in, went through their usual drive testing and tuning, and never thought about the emissions since they were already handled on the VW side.

Porsche have a bit more go juice than a lot of the other stuff.

If they did a ECU remap (likely) coupled with some hw changes to the engine (maybe, maybe not), they would likely have at least suspected something.

Like, I can tune my Lancer Evo, with stock parts, to around 360 chp / 290 whp from 291 chp / 220 whp. This will put more strain on the parts, to nuke the little gas mileage, up the emissions (no CARB), and void the warranty on the drivetrain. In order to do that the ECU needs to be flash (by say cobbs tuning) and if there was a cheating thing they will need to be aware of it or else it'd be likely gone in the flash.

If I was to put on new parts and make it go even more... Then for sure you need a new tune to make it actually work properly.

The rated horsepower on the 3.0 TDI in the Cayenne is the exact same as in all of the other applications I listed - do you have any source that it had a different map installed? I'm kind of basing my whole theory on the concept that it didn't because the rated output is identical.

The thing has a 150 kW to 191 kW rating, granted on closer look it seems the Porsche one is shared across a lot of trims so maybe they didn't see it.


guess audi is the one that could be in deeper trouble.

I wouldn't be surprised if Porsche customized the throttle mapping and/or transmission programming. ECU remapping would be a bigger job though

I have said this before and i'll say it again.

The legend of the prowess of German engineering is one of the longest-standing and most successful corporate marketing campaigns pulled of, right up there with DeBeers 3 month salary rule. Watch any Volkswagen ad, and you can see it in action.

Well, they're Porsche. I imagine "seeing if it lives up to the Porsche marquee" primarily involves flogging it to make sure it drives well.

Do you associate the Porsche brand with a dedication to fuel economy or low emissions?

The vehicles have to independently pass the emissions tests. Even if they have a a crate engine from some other supplier. All of the compliance engineers throughout all of VAG only test emissions to the EPA test rig? Even the ones at Porsche?

Nobody is going with Porsche for fuel economy but nobody is expecting to have their Porsche lose power due to a recall either. It's a premium badge, I expect them to dot more of the I's and cross more of the T's when I pay that extra $20-$40kk for a Porsche branded Touareg.

Sometimes the magic of branding is that they can get that extra cash for it for not a whole lot of actual extra engineering in return. Thus, profit!

Remember, we're talking about the diesels. The "Teutonic Porsche engineers" are working on the sports cars.

While I'm certainly not making excuses for VW cheating, I think that maybe the EPA needs to revisit its diesel emission standards. The EPA standard is the strictest in the world, and they have successfully ruined many diesel powered devices. Examples:

1) I was recently looking at compact tractors. To meet the standards, new tractors are fitted with Diesel Particulate Filters. Even if they don't flat-out fail (which happens too often, and costs thousands of dollars to fix), these filters require "regeneration", which is code for "a light comes on on the dashboard, your tractor stops moving, and runs wide-open throttle for 20 minutes while you stare in bewilderment". Google Kubota B3350 Regeneration if you want lots of fun horror stories.

2) Diesel commercial trucks. My uncle drives a large cube van over the road. When his truck goes into regen mode, it doesn't require wide-open throttle like a compact tractor, but it does lose power and tops out at 40-50MPH. If you are on a freeway where the flow of traffice is at 80MPH, this is dangerous and a major problem.

3) Diesel pickup trucks: Many trucks, in an effort to avoid additional hardware above and beyond the DPF, have taken to injecting fuel during the exhaust stroke. The idea is to have the unburned fuel get pumped through the engine and on to the DPF, where it burns and increases the temperature to the point that the carbon burns out of the DPF. Problem is, when you inject fuel into an engine during the exhaust stroke, it dilutes the oil in the cylinder, reduces lubrication, and causes premature wear/engine failure.

In the two cases I am familiar with, diesel motorhomes and diesel compact tractors, the value of used vehicles has increased quite a bit in response to the fact that all of the new ones suck. My Dad owns a gravel pit, and he (along with everyone else in the industry) buys and re-builds existing vehicles from the frame-up to avoid having to purchase new products, which are unreliable, sometimes dangerous, and uneconomical to maintain. Google "glider truck kits" -- it's basically the idea of purchasing all the parts of a new vehicle (except the powertrain), and then manually taking the powertrain from a worn-out truck, re-building it, and installing it into the glider kit. This is often done at an expense that is similar to that of purchasing an entirely new vehicle.

Many equally horrible solutions to this problem are employed. Some designs incorporate a fuel burner directly into the exhaust design. Others require a separate tank of catalytic fluid that must be filled regularly. One thing that all of the designs have in common is poor performance and piss-poor fuel economy. Many vehicles from multiple manufacturers have had trouble with catching on fire due to emissions control systems.

If the EPA wants to make tough rules that will force manufacturers to produce products with the best available technology and encourage development of new technologies, that's one thing. When the EPA makes rules that result in dangerous and/or non-functional product designs, I think it's time to back off just a bit until the technology has come closer to being able to reasonably achieve the regulated cleanliness levels. It's completely insane to make a truck that will force you to slow to 40mph while you're driving it, or to make a tractor that will force a farmer to take an unplanned break in the middle of the day (remember -- farmers work all day long, and daylight is a limited, precious commodity to them).

EDIT: To be completely clear, if you are doing work with the tractor that can be done at full RPM, you don't necessarily need to park it. However, if you're doing something that requires lower RPM (using a PTO attachment that needs low RPM like a post-hole digger, or a spreader, etc ... or doing low-speed work on a tractor with a geared transmission), then it's time to go get some coffee. Personally, the two major things I want to do with a tractor are run a large snowblower and maintain our horse pasture. For the horse pasture it's not a big deal, but if I need to clear the driveway before leaving for work in the morning, an unplanned 20 minute wait is completely unacceptable.

The EPA standard is the strictest in the world, and they have successfully ruined many diesel powered devices.

One thought, then, is that diesels are not suited to the task. If that particular type of engine can't move things down the road without filling the air with pollution, perhaps it's time we moved on to something else. I mean, what tasks require a diesel? Commercial vehicles, okay. Consumer cars? Not if they spew crap into the air just to save a few MPG. Small farm equipment? See below, totally unnecessary, IMO. Class A motorhomes? Please. The fact that it's designation has "recreational" in the name tells you that kittens won't die if we don't have anymore of them.

An extreme stance? I dunno, maybe. Just like getting rid of lead in our fuel was probably an extreme proposition at one time. But we knew lead was bad, we knew it for decades. But profits uber alles, eh? I just don't know how one would justify to future generations, "sorry 'bout the smog, but Daddy needed to run his garden tractor on diesel, and wanted to save a few bucks on fuel."

I could see an exception for commercial vehicles, but I can't see a good argument for consumer diesels if we can't produce ones that don't choke the air with particulates and NOx.

Personally, the two major things I want to do with a tractor are run a large snowblower and maintain our horse pasture.

Not to pick on you personally, but those tasks don't require a diesel. PTO, plowing a small field, running a bush hog or deck mower, pick your poison, it can all be done just fine with a gas engine. And the gas engine will start when it's -20F and that driveway needs plowed. I grew up in farm country, and the only ones running diesel where the ones with farm equipment the size of your house. I refuse to accept that we'll back off on EPA regulations because some dude feels like he needs a diesel to maintain his five acre play farm. (Again, not directed at you; I have no idea how many acres you have.)

Prior to the current (Tier 4) standards, the Tier 3 standards had already reduced NOx and particulate emissions to the the point that it's misleading to characterize those engines as "filling the air with pollution". Now that CO2 has been declared as pollution, the improved thermal efficiency of diesels over gasoline engines probably means that they were already "better" than gas under some operating conditions. Even ignoring their superior thermal efficiency, Tier 3 diesels achieved similar results to gasoline engines when both were operated at high load (Say, running down the highway at 80mph). This was accomplished with technologies that, while they added complexity and cost to the engine (such as direct injection, EGR, DOC, VVT, advanced FIE, improved combustion chamber design, increased compression ratio, advances in turbochargers), did not compromise the utility and safety of the device using the engine.

As you have suggested, many manufacturers will probably move back to gasoline engines in some of the smaller applications ... but they will moved to air-cooled V-Twins for many of these, which fall into the same power range as the diesels they replace (~20-40HP). Air-cooled gasoline engines are not clean beasts, and this move will probably actually end up being a net environmental loss.

Have a look at the chart on page 2 of this: http://www.mtu-online.com/uploads/tx_templavoila/WhitePaper_...

The additional NOx reductions achieved by tier 4 are extremely small compared to the cost & trouble caused by the technologies used to achieve them ... the previous tech (up to Tier 3) was fairly reliable & had already achieved most of that was had in the overall journey from unregulated -> Tier 4.

Also, almost none of the garden tractors sold are diesel, because in that application it doesn't make sense - those engines aren't run at high load for long periods of time. However, for a compact tractor that may well be run all day long in high heat with a demanding attachment, a diesel actually does make a lot of sense - there is a large environmental cost to dumping a machine with 5-600 hours on it because the engine gave out prematurely (which happens to entirely too many of the garbage lawn & garden tractors sold at box stores, despite their relatively easy lives).

FWIW, I let my lawn die in the summer and probably mow it fewer times in an entire season than many folks do in a month ... living in the sticks with nobody to judge me has its benefits :) You did get reasonably close on the size, though. My dad and I live close enough together to share a tractor, and we have 26 acres, a goat, five horses, and sundry other critters between the two of us (and our wives, of course).

> One thing that all of the designs have in common is poor performance and piss-poor fuel economy.

At what point does it become economical to just run gasoline instead of diesel then (that is, manufacturers selling gasoline models)? If we are talking about diesel still being economical instead of gasoline, but just not as economical as when it was allowed to be more polluting, I'm not sure I can muster much sympathy (but I'm fairly uneducated on the subject, so it may just a matter of not knowing enough).

> When the EPA makes rules that result in dangerous and/or non-functional product designs

Did the EPA make rules that required that, or did manufacturers mess up in their attempt to offer the same features as before? Did the EPA dictate this solution, or did the market respond with a poor solution to an EPA requirement on emissions?

> At what point does it become economical to just run gasoline instead of diesel then (that is, manufacturers selling gasoline models)? If we are talking about diesel still being economical instead of gasoline, but just not as economical as when it was allowed to be more polluting, I'm not sure I can muster much sympathy (but I'm fairly uneducated on the subject, so it may just a matter of not knowing enough).

People run diesels due to durability. They run at lower RPM than similarly designed gas engines, and last longer between rebuilds. They also make more low-end torque, for the same reasons. Both of these factors are extremely important to the types of equipment that you see diesels in, probably even moreso than the problems caused by diesels. For many years, Diesel fuel was more expensive than gas, but people kept running them anyway in the applications for which they were most appropriate. For cars, yeah, we'll probably just see the death of diesel in that space.

> Did the EPA make rules that required that, or did manufacturers mess up in their attempt to offer the same features as before? Did the EPA dictate this solution, or did the market respond with a poor solution to an EPA requirement on emissions?

The market responded with the only solutions it could come up with. The EPA made standards that are impossible to meet using any available technology without causing some or all of the problems I mentioned.

> The EPA made standards that are impossible to meet using any available technology without causing some or all of the problems I mentioned.

I think you're being hyperbolic here in saying that the standards are impossible to meet. What about urea injection? That works pretty well, right?

But even ignoring that, it's not like there's some god-given right to run diesel engines. Either the emissions standards are reasonable or they aren't. That's the only consideration here. If they are reasonable, and most diesels can't meet them, well then too bad, diesel is just an inferior technology. There's alternatives that don't find it impossible to meet environmental/health standards.

I don't think replacing "inferior technology" with inferior technology is a good idea.

I don't follow. What "inferior technology" (in scare quotes) is being replaced by what actually inferior technology? Are you really trying to claim that gasoline engines are inferior to diesel engines for passenger vehicles?

I don't think people / companies really have a 'right' to pollute, so as long as the standards are possible to meet at reasonable costs I don't see a problem.

PS: Low end torque is really just a transmission problem. X(1) power at Y(1) RPM > X(2) power at Y(2) RPM.

> People run diesels due to durability. They run at lower RPM than similarly designed gas engines, and last longer between rebuilds.

This makes plenty of sense for heavy machinery and trucks. This makes no sense for passenger cars, and little sense for most consumer SUVs and light-duty trucks. On any decent modern car a gasoline engine with proper maintenance is the about the last thing to go. How often does a typical car owner rebuild their engine? By the time you get to 200K, a car is typically a money hole for reasons having little to do with the engine. This goes double in northern climates with real winters. By the time the engine needs a rebuild you have a rust-bucket clunker that should be replaced with a more contemporary model.

>> People run diesels due to durability. They run at lower RPM than similarly designed gas engines, and last longer between rebuilds.

Reliability is one of these stubborn diesel myths that refuses to die. The lifespan of modern gasonline engines -when driven normally- is several multiples of what the average owner will drive the car for before it is replaced. You can easily put 300K kilometers on a gasoline engine before it needs a 'rebuild' (which typically involves not much more than changing the piston rings and rod bearings). Around 150K you may need a mild cylinder head revision (valve seals, valve seats, tappets) if you torture the engine enough. You won't have to break the bank for either of those. I don't know a single person except myself (driving a 25 year old car with a high-performance engine) who ever needed an engine rebuild.

Meanwhile, regular servicing for diesel engines is guaranteed to be more expensive compared to a gasoline engine, so in the long run you will not save any money on maintenance unless you drive your car without any care for reliability.

Of course, gasoline cars went through similar pains in the 70's, and eventually came out better than ever. Emissions controls were really what drove electronic fuel injection [0], which was unreliable at first but far superior to carburetors today.

[0]: See motorcycles, which stuck with carburetors as long as possible, until they were forced to implement electronic fuel injection around 2005 due to tightening emissions standards.

Very true - regulation undeniably accelerates technological advancement for emissions technology ... I think that the key is just striking the right balance. I was just trying to express that in the case of Diesels, the EPA may have done too much too fast.

As an aside, even when the technology matures and goes from "sucks"->"unquestionably better", all of this is not without cost. New car prices have significantly outpaced inflation in the past few decades. For instance, 50 years ago the base model mustang would have cost ~$18k in today's dollars, and the 2015 base model mustang is ~$24k. To many people a forced 33% increase in the cost of a base model vehicle is not trivial -- this is something that disproportionately effects low-income folks. Since the USA is so spread out, in many areas where population density is low it's basically impossible to get by without a car. (I chose the Mustang because somebody else already did the legwork - source: http://fortune.com/2014/10/17/ford-mustang-cost/ )

In my home state of Michigan, we don't have mandatory emissions testing, which I personally see as a good thing. Cars don't last much longer than ~15, maybe 20 years here anyway due to the extensive use of salt on the roads during the winter, so they get replaced quickly enough with newer/better/cleaner models by virtue of the fact that they rust away. I see a lot of people with cars who couldn't afford it if they weren't allowed to drive beaters. As you may have guessed from my post on shopping for tractors, I happen to live in an area where public transport isn't practical for much of the suburban/metro area (Grand Rapids/Holland), although we do have it (to some extent) in the more densely populated sections of town.

Interestingly, with the revitalization of downtown that's been happening around here, many poor people are being pushed into areas where public transportation is either inadequate, not available, or completely and totally unfeasible (if the houses are 200-500+ feet apart, or the small town has a population of 1,000, it just doesn't make sense).

Mustangs got more expensive, but that's only one data point :)


Heavily salted states certainly have an easier time justifying no emissions testing. You start out with a "known good" (regulated by the EPA) and count on cars to expire quickly. Compare to California, which thanks to gentle climate still has a huge fleet of 1980's cars on the road.

That's about the only good thing about the salt (aside from not dying on an ice patch :) ). My wife's 2005 Pontiac G6 with 120k on it is already starting to rust. I dumped my last car at ~9 years old with ~130k because it was starting to rust.

OTOH, I have a '96 Camaro that I only drive during the summer that doesn't have a speck of rust on it (driving an RWD car in a northern state during the winter is a good way to end up in an accident). It's amazing how big a difference the salt makes ... and dirt roads -- I think they kill cars even faster than salt. We are lucky enough to live on a paved road, so I don't have to worry about that particular problem.

Yep. I have a '97 Honda Del Sol. I did drive it in the winter some for a while but haven't for a number of years; I live in Massachusetts. Now, whenever it's in at the dealer, I invariably have a mechanic or service manager commenting on what great shape it's in. (It's not in mint shape by any means but it doesn't have serious rust other than the exhaust replacements that it's inevitably had.)

My current 2013 Ford Focus sticker price matched up exactly with inflation on the sticker price of the 97 Escort it replaced.

I was kind of amazed given all the safety improvements.

I did get 250,000 miles out of the escort before the salt rusted out too much.

On the other hand, if you've ridden a bike with a diesel vehicle in traffic nearby, you'd swear that the EPA was being far too lenient about emissions.

you're making pretty much the same argument that GM did with electric cars some years ago.

>When the EPA makes rules that result in dangerous and/or non-functional product designs, I think it's time to back off just a bit until the technology has come closer to being able to reasonably achieve the regulated cleanliness levels. It's completely insane to make a truck that will force you to slow to 40mph while you're driving it, or to make a tractor that will force a farmer to take an unplanned break in the middle of the day (remember -- farmers work all day long, and daylight is a limited, precious commodity to them).

it is not EPA at fault here, it is truck and car manufacturers who need to get off their a&& and start manufacturing good products. History shows that only tightening of regulation can make them do it. Without tightening of regulations, the manufacturers will be mining the same technology/product again and again...

I wonder if it would be even remotely feasible to have an electric tractor for farming use that is powered by a power line attached to a tall pole on the tractor (with a winding/unwinding mechanism as the tractor moves).

I thought that DEF wasn't a big performance killer? (limp mode with low levels notwithstanding)

The parent poster is talking about DPF (Diesel Particulate Filters), which remove soot from diesel exhaust and require manual cleaning and/or the regeneration process mentioned (where the soot is essentially burnt out of the catalyst).

DEF is a fluid which is injected across a catalyst in a process called SCR (Selective Catalytic Reduction) which is designed to remove NOx from the exhaust stream. SCR (and the use of DEF therein) replaces extensive EGR (Exhaust Gas Recirculation), not the use of a DPF. Catalytic reduction instead of EGR reduces the load on the DPF and increases economy, since dirty exhaust is not being recirculated and combustion is more efficient. However, they're complimentary systems, not replacements for one another.

Yes, I own a VW TDI, I know the difference between the two. AFAIK, EGR adjusts the combustion temperature so as to avoid an overly NOx-producing lean burn. I was responding to the blanket characterization he made in this statement quoted below.

>Many equally horrible solutions to this problem are employed...Others require a separate tank of catalytic fluid that must be filled regularly. _One thing that all of the designs have in common_ (emphasis mine) is poor performance and piss-poor fuel economy. Many vehicles from multiple manufacturers have had trouble with catching on fire due to emissions control systems.

I doubt it. Manufacturers, and especially ones like Porsche, routinely test the quality and performance of their subcomponents. If they didn't they'd be negligent.

The only plausible answer is that they used the same broken metrics that the engine had been programmed to defeat.

Why would a set of engineers make such a decision? This is clearly a decision which should be taken only by the SEO? What could their motivations be?

I think someone else said it in this thread. You either make it work, or you will be replaced by someone that can.

So... the engineers comply.

I once had an engineer come to me saying that he was being victimised by other engineers about a technical solution. I took my most trusted engineer and asked him to independently review the code. He came back and said the code was crap and needed to be rolled back. We rolled it back, a code walkthrough was performed on this code and we went into testing. When that function failed, I approached my most trusted engineer. My translation of what he said it "I didn't like the guy"... So, I've learned that even great engineers can be compromised, despite many claims I see on the Internet.

Regarding Porsche's knowledge, I agree with another poster in this thread who said that Porsche may have tested the unit in the conditions that didn't show the emission problems. Without further information, i'd give Porsche the benefit of the doubt.

I wouldn't be surprised either way, but why would the Porche engineers, getting the engine control software from the common engine control group, spot check the emissions on the test track rather than in the globally approved testing setup?

If you presume they are doing their job and not cheating, why would you cross check that your emissions were valid in both the dyno test and on the track? It requires that you assume your fellow workers are cheating, which is hard to do at most places.

> Clearly it's the fault of those pesky engineers again

Well, in any case, it shows that ethical codes will probably not work for engineering professions (or any profession for that matter).

My impression is that the process was structured in a way to give higher levels deniability. I highly doubt direct orders were given from the top to cheat; instead it was made clear that the performance numbers would come out right AND the emissions standards would be met, or heads would roll. And lo and behold! The numbers and sales targets were met. And management can say "well, we don't know how they did it. We never asked!"

It is the fault of those pesky engineers. Of course, these particular engineers were promoted to top management, so...

if those pesky engineers are looking for a job after getting fired from VW, I heard there is a young intern position available with the Donald Trump campaign.

It's all just a play to get us to accept always-on emissions checkers on every car complete with location logging, you know, for verification.

Sure. But that doesn't pardon VW/Porsche of cheating when other vendors did not.

Unless this is the tip of the iceberg and everyone is cheating but hasn't been caught yet...

> Unless this is the tip of the iceberg and everyone is cheating but hasn't been caught yet...

Perhaps it's time to examine these standards. Perhaps the standard is what's unreasonable, not the vehicle's emissions.

If these vehicles achieved greater horsepower and greater fuel economy, perhaps the long-term net pollution caused by these vehicles is equal-to-or-less-than the standard allows.

Spoiler alert...

It's worth noting that the newly replaced CEO of the VW group was the CEO of Porsche.

The CEO of VW was just fired for allowing this scandal on his watch, and he was replaced by a guy guilty of (at the very least) the same exact thing.

Background: I used to be an automotive software engineer. I speak for myself only here.

Whenever the topic of automotive software comes up on HN there are comments alongs the lines of "global variables bad", but not much construtive feedback.

I want to explain some of the tradeoffs that lead to the architecture used in automotive software to get a better discussion going with HN readers about that architecture.

tl;dr Given the hardware restrictions, real-time requirements and measurement capabilities required in automotive software, shared global variables without locks is a fast and safe way to share state between different software components as long as each variable is only written in one place in the program.

The microprocessor has to run for 10+ years in a wide range of temperatures and be dirt cheap, so you end up with specs like 180 MHz, 4 MB of flash and 128 KB of RAM.

The program must run deterministicly with respect to memory. There is no malloc/new in the code. All variables are statically allocated.

Because the physical world doesn't pause, no code is allowed to block while waiting for resources, especially synchronization primitives like mutexes.

The software architecture is in 2 main parts: basic software containing the real-time OS and hardware drivers, and the application layer which has the domain-specific code for controlling the engine, brakes, etc.

The basic software is implemented using usual C programming techniques. It has an API provided by function calls and structs to hide implementation details of each microcontroller.

The application software is where the programming model is different.

To understand why, you need to know where automotive software comes from and what it is trying to acheive.

Originally all controllers were mechanical: a valve opens proportionally to the vacuum in a part of the system. Then some controllers were implemented in analog electronics: take multiple voltages, feed them through an op-amp and use the output to control a valve.

So automotive software reproduces this: get some inputs, compute the same physical equations at a regular rate and generate outputs.

This is dataflow programming. Blocks of code have inputs and outputs. They are executed at a fixed rate that depends on the physical phenomena (air flow changes fast, temperature changes slowly). Different blocks are conneceted together in a hierachical way to form subsystems. Encapsulation is acheived by viewing these blocks as black boxes: you don't need to care how the block works if you are only interested in knowing which inputs it uses and outputs it produces.

Here's an example component to control a gizmo.

It might be implemented in a visual environment like Simulink by MathWorks, or it implemented by hand from a spec.

  #include "GizmoController_data.h"
  void GizmoController_100ms() {
    Gizmo_Gain = interpolate2d(Gizmo_Gain_MAP, EngineSpeed, CoolantTemp);
  void GizmoController_10ms() {
    Gizmo_Error = Gizmo_PositionDesired - Gizmo_Position;
    Gizmo_DutyCycle = limit(Gizmo_Gain * Gizmo_Error + Gizmo_Offset_VAL, 0, 100);
It takes some inputs (EngineSpeed, CoolantTemp, Gizmo_PositionDesired, Gizmo_Position), has some intermediate values (Gizmo_Error), and outputs (Gizmo_DutyCycle). Those are implemented as global variables. It also uses some constants (Gizmo_Gain_MAP, Gizmo_Offset_VAL). It has 2 processes, running every 100ms and 10ms. All this information would be specified in an XML file.

The header GizmoController_data.h is auto-generated at compile time by a tool from the XML file mentioned above. It will contain global variable definitions for the inputs, intermediates and outputs with the appropriate volatile, const, static and extern storage classes/type qualifiers. This ensures that the compiler will enforce that inputs can't be written to, intermediate values are private to the component and outputs can be read by other modules.

Note that no explicit synchronization is needed to access inter-process variables like Gizmo_Gain or inter-component variables like Gizmo_Position. It's shared memory between 2 processes scheduled in OS tasks that can potentially interrupt each other, but since the write is atomic and happens only in one place, there is no data race. This is huge! Concurrent programming, without locks, with the best efficiency possible, using a simple technique anybody can understand: only one place in the program is allowed to write to any global memory location.

Calibration is another aspect of automotive software. In most software the constants either never change or can be set in some kind of configuration file. For an automotive controller, the value of constants (gains, offsets, limits, etc) depend on the vehicle so they must be configurable at run time during development. This is implemented in the C code by putting all constants in a memory area that is ROM in production units, but RAM in development units. The compiler enforces that application software cannot change constants, but the basic software includes code so that constants can be changed from the outside in development. This process is called calibration and is done by calibration engineers who are usually not the ones who wrote the software. Note that calibration can drastically affect the behavior of the software. What would happen if Gizmo_Gain_MAP is set to all zeros?

Measurement of variables is essential to understanding what's going on inside the embedded controller. Having all that state available in global variables makes it possible for the calibration tool request the value of any variable in the software at a fixed rate and display it in a virtual oscilloscope.

The measurement and calibration tool needs to know how to access the variables and constants. It uses a file that maps from names to addresses for a particular version of software. That file can easily be generated a compile time since all allocations are static.

Going back to the architecture of the application software, let's look at where our gizmo controller fits. It is not the only component needed to make the gizmo work. You also need components to calculate the gizmo position from some external signal (let's say an analog voltage), to route the output signal to the powerstage driver on the PCB, to determine which position the gizmo should currently occupy. These would form the gizmo subsystem package.

When the supplier releases gizmo 2.0 (TM) they upgrade the input signal to be a PWM input instead of an analog input. Modularity in the software allows the software team to simply replace the gizmo position component with one that reads a PWM instead of an analog voltage and keep the rest of the gizmo subsystem the same. In the future, projects that use gizmo 1.0 use one version of the gizmo subsystem and projects that use 2.0 use another.

This is true at any level in the hierarchy: as long as the inputs and outputs are the same, a component or subystem package can be replaced by another.

Version control in automotive software reflects this. Instead of having one tree of versions and releases like a typical software project, each component, subsystem package and software project has its own tree of versions. Each software project will reference the subsystem packages required for their engine type, vehicle platform, sensors and actuators, etc. This is how code reuse is acheived.

Testing is a mix of simulation (the sensor/actuator is simulated in Simulink and connected to Simulink block diagram of the software component), hardware-in-the-loop (a computer simulates the vehicle, but the real electronic control unit is used) and vehicle testing.

Thanks for reading. I hope this improves your understanding of how automotive software is structured.

I'm hoping the discussion will bring examples from other fields like robotics, drones and aeronautics that have similar real-time requirements on how they architect their software.

So you can use global variables and the like, but is there any progress in going higher level than C in industry? In which case you use global variables and whatnot, but you have static assertions about things like "this is only written to in one spot"

For example, I know there's an embedded systems company that writes everything in Haskell, and running their Haskell program actually generates "safe" C code that they compile down to their chips.

My impression from the oft-cited Toyota report was not only that there was a lot of global variable stuff, but that these "write-once" principles weren't super respected.

Without going into specifics, there was a static analysis tool that was part of the toolchain that caught multiple writes in different modules or in tasks of different priorities. It was a build error unless allowed on a case-by-case basis.

Testing also included very expensive (SW license cost, now HW), cycle accurate simulators at Delphi. ASIC specs were implemented into C and hooked into the simulator. Prototype firmware was compiled and run on the CPU simulator and "board" (netlist of simulated ASICs). This helped discover bugs/spec deficiencies before ASIC tape-out (save $$$). Of course, it was all wrapped in a Tk "dashboard/test unit" GUI for the other teams to consume. Tk was actually quite a pleasure to use, especially when interfacing with C!

EDIT: It basically provided at least two implementations for an ASIC by independent teams. The ASIC guys would implement the real deal in an HDL and test it. However, full system tests where the HDL simulation is hooked up with CPU simulation were too slow. By implementing the same spec in C, you gain simulation performance and get a second implementation and set of eyes that can help find bugs.

Quality post. Thanks for sharing.

If some of these timers are implemented as separate OS tasks that can interrupt each others, what's to stop one from taking a bit too much time and throwing off the timing of another?

You're welcome.

There is only one OS task for each rate (10ms, 100ms, etc). The order of the function calls in each task is determined at compile time while respecting constraints (A_10ms must run before B_10ms because B_10ms uses values computed in A_10ms).

In addition each OS task has an overrun monitor that triggers if a task is scheduled to run but the previous run has not finished yet. This type of error typically sets a "control module performance" diagnostic trouble code.

I'm not an embedded developer, so I don't have much value to add re: architecture; however, this caught my eye:

> The microprocessor has to run for 10+ years in a wide range of temperatures and be dirt cheap, so you end up with specs like 180 MHz, 4 MB of flash and 128 KB of RAM.

If I'm paying tens of thousands of dollars for a car, how come they're using the cheapest possible components? If tinkerers can ship the Raspberry Pi for $30 per board, inc. a 900Mhz quad-core chip and 1GB RAM, you'd think GM could get components at least that modern for an insignificant cost relative to the car they are controlling.

This is because the parts are orders of magnitude more expensive when being put in harsh environments

These "really slow" parts are actually tested and built for much more extreme conditions. For microchips that go into satelites, you even have hand-checked chips that go through a very long (and costly compared to something like a Pi) testing process. Multiply this by all the eletrical components, and you got yourself a lot of things to check.

Put your Raspberry Pi next to a car motor, and it's pretty likely(1) a part will fail in the heat and grime conditions.

(1) actually, I'm not sure about the likelihood, but there's no assurance that it will be fine

> Put your Raspberry Pi next to a car motor, and it's pretty likely() a part will fail in the heat and grime conditions.

Depending on how hot your motor goes, there's a good chance that unshielded Pi will simply de-solder itself into pieces.

Fun harsh environment: I reviewed software a few weeks ago that ran on one of our boards while it was irradiated by a neutron source (I work in aerospace). Radiation has interesting effects on electronics :).

$30 Raspberry Pi (2) reboots itself when exposed to strong light. The RPi line is also totally unreliable, it tends to bug up and/or crash after running for too long.

The car components are made to different spec than consumer electronics (for starters, there are several types of component, depending on what "range of temperatures" you need to operate them in; e.g. the stuff that goes into the traffic lights in northern parts of Europe is not something you buy off Farnell for your hobby project), and they also need to be reliable and tested. Add, of course, the usual graft and overhead.

Another thing is that a engine controller doesn't need a near-GHz quad-core chip and shit ton of RAM. It's not meant to stream HD videos or run Python, it has to run a bunch of feedback loops fast enough. Less powerful chips tend to be cheaper and more reliable.

All good points.

The CPU, Flash and RAM share the silicon die with lots of specialized peripherals like CAN controllers and hardware timer units. There's a lot more on the chip than just a CPU.

Also, cost is king in automotive...

If I'm paying tens of thousands of dollars for a car, how come they're using the cheapest possible components?

Save a dollar on the part, and you make a million cars, how much does that add up to? Or conversely, why spend more than you have to if the quoted specs do the job? What the hell am I going to do with four cores if I follow the well-written description above?

And I'd give your Raspberry Pi a lifespan of about a month if you were to strap it to the firewall of your car while you drive it. Less if you actually powered it up.

You'd do it because the million dollars in parts is peanuts compared to the combined salaries of the engineers who are paid to have the specialized knowledge to do this, and the time it to bring up a new application/car. I know that a million dollars is lowball, but this same principle can be applied to other components and scaled up (to a point).

Potting electronics in epoxy isn't terrifically difficult, and would greatly increase the durability. I'm willing to bet that a potted Beaglebone Black (for its real-time subsystem) would do just fine in my engine bay for a very long time. The hard part would be getting a single large-pin-count plug coming out of it to handle all the inputs.

This point is actually kind of moot because for a long time the ECUs of cars were located in the passenger cabin with wires fed in through a grommet in the firewall.

I'd wager that the cost of replacing this system with some standard real-time platform is so monumental, it won't ever be done. Good luck getting the automotive equivalent of BSPs (board-support packages) running on any system other than this.

Where this gets interesting is when the people who actually know this system die start retiring/exiting the market. There will be a pretty strong incentive to 1) keep reusing the same system with no new development, 2) accept the lead time in training students/etc, or 3) start paying crazy salaries for people to go out of their way to learn it.

Cars are life-critical systems. If GM half-assed it by putting...

> a potted Beaglebone Black (for its real-time subsystem)

...into an actual production car, what happens when the thing breaks down and kills someone? Death, recalls, lawsuits, and massive brand damage.

> This point is actually kind of moot because for a long time the ECUs of cars were located in the passenger cabin with wires fed in through a grommet in the firewall.

Even in the passenger compartment, cars aren't exactly friendly environments for electronics. They have large temperature swings (from a controlled climate to very hot in summer or very cold in winter). So putting it in the passenger compartment doesn't necessarily moot concerns over withstanding extreme environments.

The first point was meant to illustrate that it's not good enough to say that it's okay to not innovate because there's no better hardware available. I'm not suggesting that GM would seriously put a potted development board inside a vehicle. TI can simply produce automotive-grade equivalents for them.

What I'm suggesting is that things like higher sampling rates of input sensors and more composability and reusability can be realized by a newer, more standardized and open approach to hardware.

I would also wager that the outcome of failure would be the same as with the traditional systems. GM shifts the blame onto the ECU development shop that it contracted out to make the hardware or software, or the QE shop that failed to pick up the failure.

Vehicle electronics are typically rated from -40C to 125C, although it's seen that they routinely fail even inside these margins. The BBB for example can have an operating temperature from -40C to 85C. There are also plenty of industrial boards that meet or exceed the requirements of automotive operating environments. Companies such as TI are (in part) releasing development boards (such as the BBB) to try to inspire innovation in these environments. The dev boards themselves aren't rated for it, but the platform can be.

There's a few things at play here.

Economy of scales is huge of course. A lot of the parts are actually shared between many different cars at different price points. There can be on the order of 50+ micro-controller in that car. Do you want to pay 10$ extra for each of them just because they have cooler specs?

The various parts are designed at various times so that car might have controllers for the anti-lock braking system that were designed 5 years ago.

Hard real-time applications (brakes, ECU, etc.) don't magically need more gigahertz, that engine with direct injection only needs to calculate fuel volume maybe 20000 times per seconds (once per cylinder per 2 rpm) and that would probably be a ridiculous upper bound.

In-car navigation and entertainment packages are completely separate from the hardware actually controlling the car.

You'd be surprised at how much "good" software can accomplish. Just think of an 80mhz embedded cpu. You have 80,000,000 calculations per second right there in your hand.

In a side project I was involved in. We were using 60mhz chips to modulate data at 1khz. I kept worrying about running out of cpu cycles, I mean after all, I was doing a few thousand things per second. But in reality, I still had a ton of head room to go :)

Thanks for the post. Very good read.

When coming to work from university (1989-90), I made a somewhat similar though more primitive design for the embedded software of a redundancy controller of a satellite modem. The various tasks (control of RF and baseband electronics, display and buttons UI, network management interface, etc) would run according to a scheduler, each in turn. No dynamic memory allocation, internal communication via global variables that are written only in one place. There was no mechanism to actually enforce this, it was just the principle in code, and the enforcement was possible because it was a one-man software project so I had no one else to blame.

This was written in PL/M (which looks like Pascal but is more similar to a primitive variant of C) on a 8031 microcontroller, from scratch (no 3rd party SW components at all - that was a long ago).

Later on, I used the same platform for a design we did on a telephone/data cable modem we designed for domestic use (We saw a need for internet enabled cable modem in 1992 and started work on a thing that provided POTS and 128 kbit/s over SLIP). That was also running a 8031, but as we wanted to make it dirt cheap, the board did not have dual-port RAM. The hardware wizards made it run with a hack on the regular RAM access so that the SW could only access RAM if it read any memory location twice: first read would just place the address bits on the bus, and doing another read would actually fetch the content. This enabled us to save several dollars per device.

(This was possible because the 8031 has an architecture which is not strictly von Neumann design: the code and address spaces are different memory spaces and you use different CPU instructions to access the different memories, so your program memory on EPROM works normally but your data memory in external RAM does not. There's also a small internal RAM in yet another overlapping address space which was not impacted by address bus weirdness.)

This is just to show what kind of strange limitations you sometimes have on the hardware in embedded devices. Unlimited virtual memory, large non-volatile storage instead of a puny i2C bus EEPROM... you just dreamed of those. Modern controllers with 4G flash are really huge compared to that, but they are still far below what people have on desktop computers, or even an Android phone.

The limitations in embedded devices indeed come from cost per device as well as heat, vibration, radiation, humidity and similar environmental factors.

Thanks for the insight, I often wonder what automotive software and its development looks like.

I found this extremely demystifying.

I'm working with embedded on a similar industry, and also really enjoy web dev. I don't like the tools and edit-compile-cycle when working with small microcontrollers...

May I ask why you left the industry?

The "closed by default" attitude drove me nuts. Teams develop tools to improve their work and refuse to share the source code, even internally, even when there is a business reason (I'll improve the tool for you and me). Forget about cross-company collaboration on open source like you see in web dev.

the release itself: http://yosemite.epa.gov/opa/admpress.nsf/bd4379a92ceceeac852...

As alleged in the NOV, VW manufactured and installed software in the electronic control module of these vehicles that senses when the vehicle is being tested for compliance with EPA emissions standards. When the vehicle senses that it is undergoing a federal emissions test procedure, it operates in a low NOx “temperature conditioning” mode. Under that mode, the vehicle meets emission standards. At exactly one second after the completion of the initial phases of the standard test procedure, the vehicle immediately changes a number of operating parameters that increase NOx emissions and indicates in the software that it is transitioning to “normal mode,” where emissions of NOx increase up to nine times the EPA standard, depending on the vehicle and type of driving conditions. In other tests where the vehicle does not experience driving conditions similar to the start of the federal test procedure, the emissions are higher from the start, consistent with “normal mode.”

I have not read a substantive response from any part of VW to today's allegations.

There was a good This American Life episode recently [1] that walked through a few different alternatives for how VW could respond to this whole scandal. The host solicited feedback from 3 different marketing firms:

One firm, run by the guy who saved Jack In The Box with the ubiquitous sphere-head Jack commercials after an E. Coli outbreak killed 4 kids, said that VW should run an ad of a new executive blowing up the boardroom to symbolize changing the old guard.

Another firm suggested they crowdsource answers to how they can fix the problem/make it up to the public.

The last firm said they should just shut up and not draw attention to themselves. They're not in a good position to say anything right now, and they should just let the collective public forget about it and move onto the next scandal.

I'm assuming the third options is what VW's retainer-ed marketing firm has opted for.

[1] http://www.thisamericanlife.org/radio-archives/episode/569/p...

Mercedes-Benz was in a similar position in 1997 when the newly introduced A-Class flunked the moose test. After a short period of denial they hired a specialist (Armin Töpfer), halted production for several months and retooled all cars with a different suspension and ESP. The ESP wasn't even necessary but put competitors under pressure to include it in their compact class cars as well. The crisis was eventually overcome and the car sold very well in Europe (1.1 million produced in 7 years). There's a fascinating book (sadly in German only) on this called "Die A-Klasse: Elchtest, Krisenmanagement, Kommunikationsstrategie." (http://www.amazon.com/dp/3472037997)

Notice the different 3.0 litre engine affected.

Yes and this new NOV as well as the excerpt I highlighted from yesterday applies to those engines. Essentially EPA has evidence of a defeat device on the 3L now too.

Since Porsche do not use small diesels in their cars this looks to me like the scandal is a lot bigger than first thought. It isn't just the team working on the 1.8 litre diesel engine that fiddled the software, there is something more endemic going on as the 3 litre upwards size engines are implicated.

The funny thing is this:

Porsche could sue over £25 a day congestion charge (2008)


> The mayor's office responded saying the threatened legal action was "a double attack on Londoners".

> "First Porsche are trying to deprive Londoners of their democratic right to decide in the mayoral election on 1 May whether they want gas guzzling and polluting cars to drive in London when there is absolutely no need for them to do so. Second they are trying to impose on all Londoners unnecessary levels of pollution and greenhouse gases by a tiny minority," said a spokesman for the mayor.

> "No one is allowed to throw their rubbish in the street and Porsche should not be allowed to impose gas guzzling polluting cars on Londoners who do not want them."

I hope Porsche get taken to the cleaners.


That article is from 2008.

Since when is history not allowed?

The point of that article is to show that Porsche are a company with total disregard for emissions, pollution and noise. It is in their DNA, it is what they do. Hence it is great that they finally get their comeuppance.

VW emissions cheating is also from 2008, incidentally.

> new tests that were conducted on all diesel car models in the United States by E.P.A., the California Air Resources Board and the regulatory group Environment Canada.

Environment Canada, a "regulatory group"? It's a department of the Government of Canada, yo.

Which is how they have to authority to regulate things.

Meanwhile there seems to be no enforcement of large diesel pickups "rolling coal" with modified emissions.

Wut? What does this have to do with vehicle manufacturers programming their ECUs to defeat emissions tests? There isn't a single diesel truck that 'rolls coal' from the factory.

Also, diesel trucks that emit black smoke do fail emissions test - in counties that actually have emissions tests.

Apparently a lot of people who make these sorts of modifications can pass emission tests. They hook up a settings swapper to the ECU, and when they need it to be clean, they put it into factory default mode. It is essentially the same sort of cheating as VW.

On a scale that isn't even remotely as large as what VW has done. The scale matters, a lot.

You're talking about something affecting maybe tens of thousands of trucks, versus 11+ million vehicles including Porsche. It doesn't make sense to make the comparison.

I was comparing methods, not scale.

The scale of an impact of one really bad car when it comes to certain emissions shouldn't be understated; it is important to remember that VW was simply seeking pollution they could get away with. Emissions tests are done on individual cars rather than a bulk standard by make/model for that exact reason.

VW is the second largest auto-maker in the world...few comparisons are going to hold up if that's how you think.

Scale matters a lot, though.

The EPA's goal is to regulate the overall quality of the atmosphere. To the extent that they regulate individual machines, it's just because of their contribution to the total.

When something is done by a few idiot hobbyists, it's just not worth trying to enforce. The effort would be better used elsewhere.

Individual cars are tested for emissions not so much to catch illegal modifications (although I'm sure they're happy to do so when they can) but to detect failures due to age or broken equipment.

"Rolling coal" is obnoxious and ought to stop, but it shouldn't be surprising nor upsetting that there isn't much enforcement effort against it.

If the EPA had to tackle the problem standalone, then, sure, the scale wouldn't make it sensible. It could end up influencing whatever regulation goes into cars to prevent this sort of dishonesty in the future. Perhaps cars will need to start logging to a black box of sorts to track engine related settings, or emissions will need active monitoring and logging.

It is also worth noting that the hobbyist solution has the potential of becoming widespread. If VW merely does a software fix, more than a few owners with suddenly sluggish cars might look into the devices. There is nothing specific about 'rolling coal' by trucks...most uses of the devices are purely for performance tuning.

The difference is that the customers are fully informed of – and paying for – the cheat.

It's a small enough phenomenon that existing enforcement is generally considered a reasonable compromise between combating "rolling coal" and excessively onerous emissions enforcement.

It's also very much a local issue that varies widely across the country, as emissions testing programs are generally administered at a county level.

I'd love to get my car tested... Anyone know how I could do that?

I have a 2015 Q5 TDI which while not specifically mentioned has the same size engine as several of the ones that are mentioned (3.0-liter). I would be shocked if they just started using the defeat device in the 2016 model year for the Q5 considering they were already getting pushback from the EPA before the 2016 models even got announced.

I bought the diesel specifically because of the high mileage and supposedly clean emissions. What a crock.

From the EPA news release [0], your 2015 Q5 TDI seems to be safe: "on September 25th EPA initiated testing of all 2015 and 2016 light duty diesel models available in the U.S using updated testing procedures specifically designed to detect potential defeat devices".

It's also possible that EPA just hasn't finished testing all of the vehicles yet and your particular car may get added to the list. Sorry about that. I actually test drove the Touraeg TDI that shares the same engine and almost bought one a couple of years ago...

0 - http://yosemite.epa.gov/opa/admpress.nsf/0/4A45A5661216E66C8...

That's why I'd like to get it tested. I think they may have gone with the 2016 since that's what is currently on the lot, but I am very skeptical it is new to the 2016 model year when the required standards and engine did not change.

Why would you want to get it tested? Why not just wait for the EPA to test another 2015? I'm not sure what information benefit you'd get from commissioning your own test that would offset the time and money it would take to execute such a test.

The original tests that broke the scandal open were some grad students who borrowed cars to test them. I'd be happy to help in such an effort.

I'd also like to know if I purchased a fraudulent vehicle ASAP so that I can figure out next steps.

I would guess curiosity in one of the biggest (the biggest?) automotive scandals in a decade could be one reason.

You bought a Q5 3.0 Diesel for clean emmisions? Wait... What?

I'm surely not going to defend VW or any of the associated companies for cheating. But it should be kind of obvious that a car with such an engine does all but clean emmisions.

Yes. It's a 4500lb vehicle that averages 30mpg and supposedly passes the strictest emissions standards in the world... If I was in the market for a smaller car I would of course looked at models with a smaller engine (though I could have easily found myself in the same situation with a 2.0l TDI).

I don't know much about the auto market, but it seems that until this year, "clean diesel" was something people said with a straight face.

Meta: Earlier in the scandal, I recall reading that the engine management software is from Bosch. I should read further, looking for confirmation of this.

If that is the case, then this is an example of the maxim: When you provide a feature, someone will use it to their own ends -- meaning, "misuse" it.

Another example to wave in front of all those politicians and people advocating for encryption "back doors".

If you put the feature in there, people will use it any way that suits them. Including and especially ways that you did not intend nor want.

If it wasn't Bosch, perhaps there was nonetheless some "legitimate" argument within VW for adding this functionality. Enough to get the software folks -- particularly those not making big bucks off of the deal -- to implement this.

But an observant engineer might nonetheless ask themself, 'what might hypothetically be done with this?' And the engineer with a little more real world experience ("the cynic" ;-) might assume that someone will do it, sooner or later.

That was part of my reputation, for a while: Thinking of what was possible, and assuming -- or insisting -- it needed to be addressed. A year or two later, having done so would prove to have been of benefit. It would sometimes piss Management off, in the short term. But eventually, they came around.

Anyway, looking from the outside or the inside: If it's there and can be used "that way", someone's going to get around to doing so.

I also read about Bosch's involvement but this story was based on a report by the Bild am Sonntag newspaper which is not trustworthy. [1] is an English language source that relays this report. The gist of the story was that Bosch supplied the software for R&D purposes and Volkswagen engineers decided to put it into production cars too.

However, this was later rejected by Bosch who claim that writing additional code is required by the car manufacturer to get the emissions test detection feature [2].

[1] http://blog.caranddriver.com/report-bosch-warned-vw-about-di... [2] http://www.reuters.com/article/2015/10/07/volkswagen-emissio...

I'm conflicted. What do people think about the ethics of buying (non-diesel) Volkswagen cars at this point? Obviously new cars pad VW's profits and reward bad behavior.

But what about used ones? Seems like prices will be plummeting. And if you like the cars, seems like a great time to buy.

I have one of the cars that will be impacted by the recall. Performance and fuel economy are both top notch. The cabin fit and finish are on par with vehicles three times the price. I'll certainly be holding on to mine. If the used vehicle price plummets I'd definitely consider getting a 2nd one for my wife.

Will you comply with the recall?

I would like to but it depends. I intend to wait a few weeks and see what impact other owners report before I decide. I would be willing to trade some performance or economy for better emissions but if the software/hardware changes turn these cars into either a Geo Metro or an F150 (shitty performance and economy, respectively) I'm going to opt out, then take the car down to the local tune shop and have a stage 1 kit put in.

Performance and fuel economy are both top notch.

Considering it's doing the automotive equivalent of leaving a turd on the sidewalk everywhere it goes, I'd be really disappointed if its perf and mileage were merely average.

You're not the first to express this sentiment. It feels similar to having a random in a bar bitch at me about my carbon footprint when they hear I burn coal while blacksmithing on the weekend. I appreciate environmental awareness but are you sure you're maximizing your efforts? http://i.dailymail.co.uk/i/pix/2014/07/07/article-2683735-1F...

What do people think about the ethics of buying (non-diesel) Volkswagen cars at this point?

Hey, if you want to buy a shit car, I don't think ethics really factor into it.

All kidding aside, despite being in absolutely no danger of ever buying another VW, I would think of it like I think of buying from (to pick a recent popular whipping boy) Amazon: does giving money to the company vote for the world you want to live in? I like workers to be treated fairly and with respect, so no more Amazon for me. I like clean air and products that win because they're great products that don't rely on cheating to make them great, so no more VWs for me.

As for used ones (assuming non-diesel), I'll use another metaphor. I'm a vegetarian. What if my veggie pizza shows up with pepperoni (and assume I otherwise like pepperoni)? I eat it. Cow's already dead, I didn't request that it be killed for my meal, and they're just going to throw it away if I send it back. Might as well eat it. Used VWs? Money's already in the pocket of a crooked company, might as well buy one if it suits your needs. Just take it somewhere other than the dealer when it needs work. :-)

What comparably priced cars would you consider competitive with the Golf on interior quality? I want to love the Mazda 3, but until the 2014 model year it looks like garbage to me. I'd be much happier with a 3-series, but that's 3x as much money as a 2012-2014 Golf.

I don't mean to be a smartass, but does my post strike you as coming from a guy who pays much attention to VW these days? :-) VW had their chance several times, so they are now on my "don't even bother to look" list, right next to Fiat.

Besides, I spend my real money on motorcycles, so I'm truly not the guy to ask about compact cars regardless of brand. That, and the interior is low on my list of priorities when selecting a vehicle.

My point is VW fills a niche that's not otherwise well served. If you like the German sportscar aesthetic and can't afford a BMW, it's kind of your only choice.

Well, there is always the argument to be made that people wouldn't buy a new VW if they knew getting rid of it in 3 years is going to be really difficult. But that's not going to happen.

Was looking at a new diesel but will maybe go for the GTE hybrid instead. The Passat GTE the only full size hybrid wagon that I know of(?) and it's very reasonably priced at around $5-10k over the non-hybrid.

I firmly believe in punishing companies that are unethical by personally boycotting them. VW doesn't even sell the best gasoline-powered cars for the price, so there's no way I'd be tempted.

May I suggest that the government raid the ECU engine control unit supplier Bosch and examine their email servers which of their customers has cheated.

"But even so, Bosch still supplied the “defeat” code EDC 16 engine management system at the heart of the #Dieselgate scandal"

From http://www3.epa.gov/otaq/cert/documents/vw-nov-2015-11-02.pd...

"the high temperatures heat the selective catalytic reduction system ("catalyst") and improves the catalyst's ability to reduce tailpipe NOx emissions"

There has been recently http://www.thestar.com/autos/2015/09/30/canadian-trios-exhau...

I wonder if this tech would work for SCR systems or not.

VW sales are plummeting in Europe. You cannot trick the users all the time.

Pot, meet kettle:


They were experimenting on human subjects without informed consent and using the data to overstate results that favored the regulations they wanted to impose.

These regulations set the particulate limits diesel powered vehicles must meet.

Page 5 of the report: "The subject provided informed consent"

The real problem is drawing any conclusion from a sample size of 1.

Explains why one of the first heads to roll was that of Wolfgang Hatz, head of engine R&D at Porsche AG.

Why doesn't the EPA simply ban the sale of any car manufactured by VW until they start passing the new emissions tests? Even better, begin requiring previously purchased VW cars pass these emissions tests. If the cars don't pass the new emissions tests then VW is required to reimburse the blue book value of the car for false advertising, fraud, and acting in bad faith.

They have prevented the sale of all current model year diesel VWs and that is sensible.

I want VW to be held accountable, but I don't want them to decimate the company.

While I want clean air, and believe in global warming; I'm afraid that the EPA will use VW as a reason to tighten emmission standards to to point where we will need to buy new vechicles in order to drive. I can't afford, nor really want a new vechicle.

I'm at the point now that I pray my vechicles passes smog checks. I'm not working as a mechanic now, but I have been to automotive school. I take those smog results, I get every two years, and tune my vechicles appropriately. My vechicles are tuned properly. I use a secondary 02 sensor attached to secondary bung holes before, and after the catalytic converter. All, so I can get the engines in perfect stoichiometric values. I clean out my erg valves. I change the oil. I double check some emmission sensors with a DVOM. I look for that lazy sensor and replace it if it's not in spec.

Even with all that, it seems like it's a crap shoot on wether my vechicles pass.

(I can't blame the EPA, or CARB totally. I have found gross errors in Motor Emission publications. Off subject, but smog stations are only required to have one emmission's reference on site. Most shops usually just have the current copy of Motor's Emmisson Manual. It's cheap, and that's what they show you if your vechicle fails the visual. The publication is filled with errors. If you failed the visual on your smog test, double check the information with Mitchell Manuals. I have never found an error in a Mitchell manual. Off subject--yes, but when your arguing with a smog tech. you might remember this post.)

There is little or no history of emissions standards forcing existing vehicles off the road.

There is some chance that better testing will result in lower emissions standards, if the better tests reveal the existing standards to be unrealistic.

Blue book value that recently dropped because of the emissions scandal? ;)

Requiring VW reimburse purchase price would make more sense, but still would not account for time spent having to look for new car etc.

On the other hand, there are numerous reports of owners not wanting to return their cars for software updates out of fear that such fixes would change performance characteristics of their vehicle for the worse.

In terms of banning sales, that would be an effective strategy to 'press' VW into acting faster but is likely to severely impact the co financially - something Germany simply can't allow due to huge numbers of people employed/at stake.

The harm is against society as a whole, not really against the owners.

How do we go about measuring this "harm"?

"Premature deaths" is not a useful measurement because it does not take into account how premature the death was. You could say everybody who dies of a respiratory problem dies prematurely, because pollution made them die a few hours/minutes/seconds faster than they would have breathing perfectly clean air. The correct unit for measuring the harm caused is the disability-adjusted life year:


I'm not sure. You could make several arguments. I think the idea that environmental regulations are reasonable presupposes that harm to society comes from environmental damage (negative externalities of other behaviors).

A few such arguments might be that there were harms to:

- peoples' lungs, wildlife, etc.

- profits lost by firms that played by the rules

- more cars sold than would have been (hence more environmental damage caused than with fewer overall cars on the road).

Good points. I sort of touched on this in another comment, but perhaps the emission standard on these vehicles is what's unreasonable.

We have a more-or-less arbitrarily set maximum allowed tailpipe emissions for a given class of vehicles. Is it fair to apply this arbitrary standard uniformly across all vehicles in the same class?

If vehicle A and vehicle B both meet the same emissions standards, but vehicle B is capable of better fuel economy, isn't it reasonable to suspect vehicle B's net pollution is less over the vehicle's lifetime?

In that light, perhaps these standards are unfair to the companies who are trying to make this class of vehicle. In the case of VW, the vehicles will have to lose fuel economy in order to adhere to this standard, which means more fuel is being consumed over the vehicle's lifetime, which may very well equate to a net gain of pollution (not just for the vehicle's tailpipe, but also the entire system from the oil well through the pump).

None of this excuses blatant falsification of tests - however, perhaps it's time we took a better look at what exactly pollution is, and how tweaking our vehicles may impact the output.

That is a good point. You may very well be correct.

I think that from an economic perspective, reduced fuel economy is also beneficial to the environment because the driver is less likely to spend money traveling (keeping the vehicle's engine running, producing pollution) when the cost is higher.

However if the vehicles in question were part of a fleet whose behavior would not change much based on fuel costs, there would seem to be nothing but downside to diminished fuel economy.

What's interesting to me about this is that it seems that a few other manufacturers were able to engineer diesel engines with power, fuel economy, and emissions characteristics that met the guidelines without cheating. It would seem that this is quite a feat, if VW had to resort to gaming the system.

On a tangential note, I also find it interesting that we accept certain tradeoffs about noise pollution vs air pollution, since if we simply removed mufflers from cars, fuel economy would increase.

It is unlikely that a reduction in fuel economy of X% would incent a reduction in driven miles of more than X%. Fuel cost is a minority component of total lifetime cost of the vehicle.

Most people have a (perceived) fixed distance of driving they need to drive. A 10% reduction in mileage won't have drivers biking 1 day every other week.

Much as I'd like to see it happen, bankrupting a multi-billion dollar company isn't something that regulators do on a lark.

A muli-year global emissions cheating campaign isn't a lark.

If X person(s) modified their X car(s) in such a way, they'd be forced to take it/them off the road.

The value of X should not matter.

'Should' and 'does' aren't always the same thing in real life. You're talking about upending the lives of many thousands of people that depend on their cars to get to work, and lost of people who depend on VW for their pay checks. I'm sure this will all get worked out, but it would be insane to just pull the cars off the road.

> You're talking about upending the lives of many thousands of people that depend on their cars to get to work, and lost of people who depend on VW for their pay checks. I'm sure this will all get worked out, but it would be insane to just pull the cars off the road.

While pulling the existing cars off the road in an instant seems to be an expensive and bad idea, I don't like the idea of suddenly caring about the livelihoods of VW employees. We're talking about a Big Co here, the kind that axes thousands of people because sales said they can't meet their quarterly goals.

How am I, as a citizen, ever to trust the government can hold corporations in check, if not just them, but we ourselves hesitate to take action because of collateral damage. No surprise VW is pulling off the shit they are - they know perfectly well they'll get away with it even if it goes public.

I think it does matter.

Think of every VW, Audi, and Porsche dealership and manufacturing plant in the US being mothballed overnight. Not even mentioning the secondary effects, that's a lot of unemployed Americans that are paying for mistakes made by people they've never met.

> that's a lot of unemployed Americans that are paying for mistakes made by people they've never met

And? So the company structures its mischief so that any correcting action will damage innocent bystanders, therefore holding everyone in check. What happened to "America doesn't negotiate with terrorists"?

On the other hand, VW has out aside $6.8bn for the scandal and has $3bn loss this trimester, after putting the cost of the scandal. Will the next trimester be positive? NB. $6.8bn is 3% of the revenue. Sometimes when there is too much corruption, those companies need to be dismantled and rebuilt from scratch. Enron.

>Why doesn't the EPA simply ban the sale of any car manufactured by VW

do they have the legal authority to do so? If they did, I doubt the republican led congress would stand by idly.

They do and have. They won't certify any 2016 VW Diesels. That's the whole reason this blew up now instead of last year when researchers first discovered it.

Are they testing other cars beside VW?

There are at least going to be long term changes to how the testing is done.

In Europe the Commission (likely also pushed by the Council/Germany/France/Spain/Italy) shamefully decided to raise the NOx limits by 2x until 2019 and 50% until 2021. It's the first time since they launched the emissions "euro" standard that they actually raised the limit.

I guess they saw everyone cheats by at least 2x, as some reports have also said.

Blame it on the programmers! /sarcasm

More shoes dropping.

I definitely think management was involved here. However, its entirely possible that some management was not aware of the details. I doubt that more than some couple tens of people at VW group know intimate details of the VW diesel ECU. With badge engineering/platform sharing, its possible that Porsche/Audi/VW or whoever was not directly responsible for delivering the code could be in the dark about it. Im sure they even want it that way so there is plausible deniability. There are plenty of things that go on in the company I work for that I have no knowledge of. There are plenty of things in the software I customize/implement that I do not know about.

Bottom line- VW group cheated, was caught, and Im sure has a few dozen scapegoats lined up in accordance with automotive scandal rules and regulations. People will use this as a platform to get their names in the spotlight as a crusader for the people, the managers that made the decision will go largely unpunished, and the world will move on. I think its crappy behavior, but in the scheme of things they did not directly murder anyone and people willingly participate in much more dangerous activities than breathing excessive NOX fumes... I think that they should just fine them make VW say "We are truly deeply heartfeltly sorry" and then we can worry about the bigger problems in life.

NOX fumes kill tens to hundreds of thousands of people every year, there isn't many "bigger problems in life" than things like that.

Pollution has very diffuse, but very real human and ecological consequences. One of the reasons this fight is so hard is because they are so hard to reason about or even witness. It took systematic study of events like https://en.wikipedia.org/wiki/Great_Smog_of_1952 to realize how many people stuff like NOX emissions kill and disable.

What if doing something took a day off of the life of every single person in the world. In terms of "days of life destroyed" you would have an event unparalleled by any genocide in history, but yet how many people would even care?

> people willingly participate in much more dangerous activities than breathing excessive NOX fumes

What does that have to do with anything? If Alice chooses to wrestle sharks in her spare time, that's completely irrelevant to Bob being forced to breathe toxic fumes.

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact