

Air traffic system failure caused by computer memory shortage - hudibras
http://www.reuters.com/article/2014/05/12/us-airtraffic-bug-exclusive-idUSBREA4B02320140512

======
sehugg
This is part of a system the FAA has been trying to replace (unsuccessfully)
since at least 1990: [http://www.ainonline.com/aviation-news/aviation-
internationa...](http://www.ainonline.com/aviation-news/aviation-
international-news/2013-10-02/eram-development-reminiscent-failed-aas-program)

(The AAS often appears in software engineering classes as an example of a
classic "train wreck" project)

------
damian2000
Sounds more like a bug rather than a lack of memory to me ...

 _The flight plan did not contain an altitude for the flight, one of the
sources said. While a controller entered the usual altitude for a U-2 plane -
about 60,000 feet - the system began to consider all altitudes between ground
level and infinity.

The conflict generated error messages and caused the system to begin cycling
through restarts._

~~~
hudibras
They didn't have enough memory to cope with an infinite number of inputs.

Obviously a hardware problem...

~~~
gonzo
is there a plane that flies at infinite altitude?

Is there an aircraft of concern to the FAA that flies above FL1000? One or two
historic flights have been that high, so raise the ceiling to 150,000'

still not infinite.

~~~
cstuder
Yeah, let's hardcode a maximum altitude somewhere deep in the code and be
completely surprised when, a couple of years in the future, newer, higher-
flying planes/shuttles disappear from the system.

~~~
hatbert
Well, considering that altitudes above FL1000 have been, for the last 60-odd
years, considered to be outside the FAA's jurisdiction... what is the problem
exactly?

~~~
makomk
One presumes they still need to know the location of those flights to deal
with them descending into the range of altitudes used by commercial aircraft.

~~~
cstross
No commercial aircraft operate at such altitudes. No, seriously: you run out
of air for air-breathing engines above FL600 (the SR-71 flew at FL800 _very
fast indeed_ partly because it needed to go like a bat out of hell to keep
enough airflow through its engines: and because it was able to go that fast
due to there not being much air up there to produce drag), and the turbofans
used on commercial airliners rapidly run out of wheeze above FL400.

The only civil aircraft that ever flew above FL500 were Concorde and the
Tu-144, neither of which are in service. But they didn't fly at supersonic
speed and high altitude in crowded airspace -- simply because getting an SST
up to altitude and speed takes several minutes, a couple of hundred miles, and
a prodigious amount of fuel. To say nothing of the sonic boom issue, which
basically limited supercruise to over-ocean flight.

While there's been a lot of hot air about supersonic bizjets over the past two
decades, so far nothing's escaped from the CAD package much less gone into
flight testing.

So what you're asking for is for an FAA system to be retrofitted to handle
_exotic_ military requirements, future space shuttles, and a class of
commercial aircraft that would not operate in that flight regime within a
couple of hundred miles of any airport.

~~~
Daviey
Funnily enough, I understood UK ATC service was provided to FIR FL245, with
basic service UIR FL660.

------
colanderman
A perfect example of why embedded or safety-critical systems should never
dynamically allocate memory from a common pool.

Such systems should always preallocate fixed-size heapsfor each subsystem,
each of which can then dynamically parcel out pieces of their own heaps if
necessary. This helps limits the damage of any memory leaks in the system.

------
atlantic
So much talk about Snowden, and then a newspaper reveals how to trigger a
software bug than can shut down the entire air traffic control system for a
large segment of the US. This is a genuinely damaging revelation, even if some
critical details are lacking. Curious that the government is not up in arms
about it.

------
awjr
Having worked on ATC, this really does sound like a software bug. Fixing such
a bug on a delivered system would require an immense amount of costly testing.
Adding memory is the cheap option.

The other thing to consider is that some of these systems are rather old and
could easily have less than a gig of ram.

~~~
andreasvc
That sounds very worrying. The fact that this issue occurs shows that they
didn't prepare for OOM conditions, and another issue could easily cause a
different OOM condition they didn't prepare for. It's not inconceivable that
there could be an issue which gobbles up an amount of memory you throw at it.
It is a tragic irony that the fact that testing is costly may be the very
reason this condition was not tested for.

~~~
andreasvc
I can no longer edit. "an amount" should be "any amount".

------
strictfp
Lack of constraint leads to infinite search space. Sounds like constraint
programming.

------
kijin
_said Dan Kaminsky, co-founder of the White Ops security firm and an expert in
attacks based on over-filling areas of computer memory._

That sounds more like buffer overrun than OOM. One is a Heartbleed-class
vulnerability, the other is a pretty straightforward DoS situation. But of
course why would we expect Reuters to be able to distinguish between the
two...

~~~
andreasvc
That's irrelevant to the rest of the article, it's only how his expertise is
characterized.

------
huhtenberg
I would guess that it was some form of an numeric overflow, signed/unsigned
issue or out of bound array access, because the only abnormal thing here is
U2's altitude.

~~~
andreasvc
Have you read the article? The cause is explained quite explicitly, and it was
the very absence of the altitude not any of the things you mention.

~~~
huhtenberg
> very absence of the altitude

And? Consider what this means exactly and how come the system saw the plane
but didn't record the altitude.

------
jmnicolas
What's the point of flying the most stealthy plane on earth if you have to
register a flight plan ?

I thought such a plane would be 100% autonomous and be able to "see" and avoid
other planes.

~~~
ars
> flying the most stealthy plane

A U2 is not at all stealthy. You must be thinking of some other plane.

> if you have to register a flight plan

It's not a military operation in a foreign country, it's a routine flight in
the home country.

> I thought such a plane would be 100% autonomous

The U2 was made long before such computers existed. They do have drones now.

> and be able to "see" and avoid other planes.

It wasn't the plane with the problem, it was ground control.

This must be a world record: A faulty assumption in every single sentence!

~~~
joezydeco
_" It's not a military operation in a foreign country, it's a routine flight
in the home country"_

Is it worth asking, at all, _why_ we're flying a military reconnaissance
aircraft back-and-forth over Los Angeles?

~~~
a3n
Atmospheric studies? Training in commercial environments? To see what happens?
Because they can? Sekrit stuff?

If there's an actual military mission for any military plane that requires it
to fly over or through a commercial area, wouldn't you prefer that pilots and
ground operators already know how to do that?

