
Some American Airlines planes are grounded because the pilots’ iPads crashed - prostoalex
http://qz.com/393909/american-airlines-planes-are-grounded-because-their-pilots-ipads-have-crashed/
======
0x0
Just a wild guess: Could the enterprise in-house code signing certificate have
expired? That would lock out the app. Imagine that happening mid-flight!

(The iOS enterprise in-house code signing certificates, which only Apple can
issue, always seem to come with 1-2 year hard expiration dates)

~~~
ceejayoz
> Imagine that happening mid-flight!

Not a problem, really. FAA regulations require all sorts of backups and
emergency procedures. Even my dad's little four-seat general aviation plane
has backups for every critical instrument.

This issue prevents you from taking off (out of caution), but if it happened
in-flight everyone'd know what to do.

~~~
wodenokoto
More over, isn't this just a manual? I mean, for most normal situations, the
pilots know what to do, so even if the backup also died, chances are pretty
good that they would land without problems, I imagine.

~~~
VLM
Maps too, from what I hear. The problem is scalability. If no one carried
maps, everyone would have to talk verbally and take notes with ATC for five
minutes for every landing to make sure nothing had changed.

If only a couple planes pester ATC then the system will keep up. It would be
safe enough. What are the odds of an inflight emergency requiring a diversion
at any instant? Pretty low. So if it happened to one plane they'd figure it
out verbally. However requiring all ATC and all pilots to talk over all maps
at all landings isn't scalable.

Something similar happens with manuals. One plane loses a system and its ipad,
you call home on the radio or satphone or contact a fellow pilot on 123.45 MHz
or call ATC for help. This doesn't scale to every plane in the air at the same
time for every mostly minor maintenance writeup. Maybe you can ignore error
messages this flight, after all most measurable things are not important, just
like in business. Or maybe engine code 3141Z.kjdf.subsection2 means the engine
is about to explode on the wing, but you've got no manuals...

Something often overlooked is megacorp analysis paralysis. You'll be formally
written up if you don't file an engine operation temperature profile thingy
form or some meaningless catering receipts or whatever according to boring
corporate policy, and it will be brought up at your annual performance review,
affect your promotion chances, etc. This scales pretty well when one ipad
breaks and you use the copilots ipad or call your boss and argue and he issues
you an indulgence or the boss takes dictation of the form for you or you
refuse to fly (its only one flight after all). This doesn't scale well when
every ipad in a company stops working.

Personally, if I were in a plane over water and I lost all my papers out the
window, no fuel estimates, no weight and balance, no maps, no flight plan, no
list of obstacles, no nav plan, no notes, nothing at all, I'd probably panic
and head for the closest coastal airport (and where is that, without any
docs?) If one plane does this, it scales VERY well. ATC calms you down,
vectors you in, gives you some geographical advice, and you land. If every
plane in the air does this simultaneously, you'll overload ATC and probably
get someone killed.

This applies everywhere to all industries not just planes.

~~~
ceejayoz
> If every plane in the air does this simultaneously, you'll overload ATC and
> probably get someone killed.

We've already had a test run for this, and it worked fine. 9/11 involved
exactly that scenario - every flight in or heading to the US had to make for
the nearest airport and land immediately.

~~~
scott00
But those planes all had access to their charts and weren't reliant on ATC for
navigation.

------
lo_fye
"The issue was centered around an update that was pushed 4 days ago" \- that's
an app bug, not an iPad bug.

~~~
SixSigma
> The pilot came on and said that his first mate’s iPad powered down
> unexpectedly, and his had too, and that the entire 737 fleet on American had
> experienced the same behavior.

user mode programs causing power down is not an app bug

~~~
draugadrotten
> that the entire 737 fleet on American had experienced the same behavior.

That seems to be inaccurate. American Airlines confirms "several dozen
flights" which is probably less.

[http://www.theguardian.com/technology/2015/apr/29/apple-
ipad...](http://www.theguardian.com/technology/2015/apr/29/apple-ipad-fail-
grounds-few-dozen-american-airline-flights)

And the fix was available through nearest Wi-Fi. Leaving cockpit a few
minutes. The horror!

"In some cases, the flight has had to return to the gate to access a Wi-Fi
connection to fix the issue. "
[http://www.theverge.com/2015/4/28/8511993/ipad-issue-
grounds...](http://www.theverge.com/2015/4/28/8511993/ipad-issue-grounds-
american-airlines-737s)

~~~
duncanawoods
>> Leaving cockpit a few minutes. The horror!

For an airliner, yes, the horror.

Why would you want to downplay the dangers of pilots losing access to their
flight manuals?

~~~
CHY872
because there aren't any such dangers.

Nothing safety critical would have been entrusted to the iPad. The pilots
would be able to deal with the problem with either ATC support (anything the
pilots can read to each other, ATC can read to them) or backup papers.

Yes, if you're on the ground, you won't be taking off without the stuff, but
that's obvious when it can be fixed by the pilot getting out for a few
minutes.

People don't screw around with plane safety. The failure mode of 'our iPads
turned off' will have been considered as a prelude to their being introduced.

~~~
msandford
> Nothing safety critical would have been entrusted to the iPad.

I'm afraid I must disagree here. The iPad is a replacement for a bunch of
paper manuals and maps and charts which are the backups to the ones contained
in the flight computers. So if your iPad doesn't work any more, you don't have
a backup anymore and that is safety critical.

[http://www.cnet.com/news/american-airlines-pilots-to-use-
ipa...](http://www.cnet.com/news/american-airlines-pilots-to-use-ipads-for-
flight-manuals/)

~~~
jsmthrowaway
It is not safety critical. In the unlikely event of _both_ flight computers
failing (many aircraft carry a backup FMC/FMS, but even a single computer is
far more reliable than anything made on HN), _and_ the consumer iPad which is
barely FAA approved as a backup not working, any pilot can land the aircraft
with their eyes closed. You think reference materials for the aircraft are the
only thing keeping you alive and everything is hopeless if the pilot loses his
weight tables? They're specifically trained when they get type rated and have
all of the important figures memorized. Not ideal, sure, but "not safe" is
another ballgame.

EFBs have a long history in the cockpit. Charts are required to be carried by
FAR, but honestly, a scheduled commercial flight genuinely needing to consult
one in anger is very rare. That was the impetus to get the FAA to approve
iPads for EFB use in the first place, I think, was pilots making the case that
they're needed so infrequently that an iPad is fine.

Threads like this make me wish there was more general awareness of what goes
on in the cockpit, because I think it would go a long way toward assuaging
unnecessary fear. Aviation is one of those rare regulation success stories
where pretty much everything has been thought of.

------
apalmer
Kind of weird, there historically have been similar 'do it the old school way
because its more reliable' fallbacks for other things in air transport. What
happened to the planes that were mid flight when this bug occurred?

~~~
ithingsnoway
It is outrageous that flights could be stopped because of a reliance of shiny
iThings and the software running on them. Paper versions do not run out of
juice.

There is a place for shiny tech, but the flight deck is not it. Paper worked,
and we should go back to it.

~~~
imglorp
Paper comprises an entire suitcase. Charts, airport docs, aircraft manuals,
maint logs, regulations, etc etc. Updates are frequent and also paper. The
fuel savings alone from not carrying the paper is substantial. As long as
there's a backup tablet, it should be okay. If they get lost they can always
call in for vectors.

ref:
[http://news.delta.com/index.php?s=20295&item=124327](http://news.delta.com/index.php?s=20295&item=124327)

~~~
ithingsnoway
Didn't the backup tablet also experience issues in this case?

~~~
CHY872
in which case they can call in for vectors

------
CaptG
If you're curious, check out
[https://www.foreflight.com/](https://www.foreflight.com/), it's available
from the app store and is marketed towards GA pilots. I've used it with some
success on a domestic US flight with the inflight WIFI.

EDIT: I just checked ForeFlight had a version update on 4/27\. I doubt
American is using the same software on their tablets, but then again you never
know.

~~~
fathom108
From that link: "All backed by Fanatical Pilot Support™"

Strikes me as a poor choice of words for their market.

~~~
rpmcb
Can you elaborate? Our customers tell us it's one of the best things we offer.

~~~
praneshp
I think (s)he means the word "fanatic" has negative connotations, post 9/11.

------
bvrlt
Thanks for the better title on HN. If you look at most reports on the issue,
the title mentions "An iPad glitch" making it sound the iPads are at fault
rather than a software issue.

[http://www.theguardian.com/technology/2015/apr/29/apple-
ipad...](http://www.theguardian.com/technology/2015/apr/29/apple-ipad-fail-
grounds-few-dozen-american-airline-flights) [http://qz.com/393909/american-
airlines-planes-are-grounded-b...](http://qz.com/393909/american-airlines-
planes-are-grounded-because-their-pilots-ipads-have-crashed/)
[http://money.cnn.com/2015/04/29/technology/american-
airlines...](http://money.cnn.com/2015/04/29/technology/american-airlines-
ipad/index.html)

~~~
_broody
Yes, the app crashed. That was no doubt an app bug. The iPads also crashed.
That's eyebrow-raising, and definitely an iPad glitch.

The airline will fix the bug and it won't happen again. But relying on a
closed, unauditable platform to keep any sort of safety-critical data in these
conditions seems absolutely unconvincing to me.

~~~
ifewalter32
I'm interested in a better solution/idea if you have any.

~~~
arfar
Picking an open source platform you can audit, like Linux (Android) for
example.

------
pmontra
Can anybody here estimate the money lost due to late flights, crews, etc? I
don't know where to start from but I'd love to compare it with the costs of
keep using paper.

~~~
humanrebar
I'd like the estimate to include money and time lost by passengers, but that's
probably hoping for too much.

~~~
capnrefsmmat
The DOT actually has estimates of the value of time lost by passengers, used
for economic analyses of delays. See this PDF, page 19:

[http://www.dot.gov/sites/dot.dev/files/docs/vot_guidance_092...](http://www.dot.gov/sites/dot.dev/files/docs/vot_guidance_092811c.pdf)

Your time is worth $45-57 an hour, depending on if your travel is personal or
business-related.

~~~
notahacker
Seems a tad generous considering a back-of-the-envelope calculation puts the
average amount contributed to GDP per US citizen per hour at around $6.

From an airline point of view, if their schedule is sufficiently flexible to
accommodate the delays without flight cancellations and the delays are
insufficiently long to require passengers to be compensated, the biggest costs
will be additional fuel if the delays involve the aircraft being put in a
holding pattern at the other end and additional time or penalty charges levied
by the airports. Staffing isn't that big a deal even if they're on overtime.

~~~
jackweirdy
Though if you’re travelling for business - you’re probably not doing "normal"
work. Travelling to sign big deals, and so on. Even a few large deals would
skew that number upwards.

------
adim86
Is it me or is this headline misleading. As in AA for one reason or the other
has a faulty app that refuses to launch or crashes the iPad. How is this
Apple's fault if you want to be fair. This is like saying 7 bakerys had to
close down for the week because their HP desktop crashed , when really they
did not upgrade their software. What happened to investigative reporting
rather than clickbait?

------
danans
Perhaps they chose too complex a platform, given the use case. Paper, while it
has its own failure modes, is pretty failsafe, especially for information that
doesn't change during the course of the flight.

I wonder if there is an in-between solution that would provide the benefits of
an electronic display without introducing as many failure modes as a device
like an iPad which is designed primarily for consumer use.

------
nomercy400
Don't forget about the savings for American Airlines. No more paper
maintenance manuals, saving lots of weight, on every flight. The ipad powering
down will probably result in a claim to the supplier of the ipads. Of course
it is embarrassing, but think of the environmental benefits of saving all that
weight.

~~~
stefantalpalaru
> all that weight

A few hundred grams? A few Kg?

~~~
userbinator
According to the link in the article:

[http://hub.aa.com/en/nr/pressrelease/american-airlines-
compl...](http://hub.aa.com/en/nr/pressrelease/american-airlines-completes-
electronic-flight-bag-implementation)

 _35 pounds_ , or around 16kg. That is rather heavy...

I'm thinking that they should not all carry iPads, but have one iPad and one
other device from a different manufacturer running different software but with
the same information, for redundancy purposes. The weight of carrying an iPad
and backup device is still tiny compared to what it replaced.

~~~
pmelendez
> , but have one iPad and one other device from a different manufacturer
> running different software

Exactly my thoughts. I am surprised they didn't have any contingency plan as
backup.

~~~
draugadrotten
> they didn't have any contingency plan as backup.

Do you have any Source of your claim that there was no contingency plan?

FAA regulations seems to indicate that there are plans:

"(5) Procedural means. g. Procedural Mitigations. If one or more onboard EFBs
fail, resulting in loss of function or the presentation of false or
hazardously misleading information, a contingency plan or process will need to
be in place to provide the required information."

Page 23,
[http://www.faa.gov/documentlibrary/media/advisory_circular/a...](http://www.faa.gov/documentlibrary/media/advisory_circular/ac%20120-76b.pdf)

~~~
pmelendez
The same articles says: "The pilot told us when they were getting ready to
take off, the iPad screens went blank, both for the captain and copilot, __so
they didn’t have the flight plan __,” " (emphasis mine)

What I meant is taht they didn't have a second electronic contingency plan.
Probably they had to switch to paper on those flights (otherwise the impact
would have been minimal)

------
ericcumbee
While I am sure there are plenty of advantages to a full on app. I kind of
wonder what the downside of a lot of that content being just PDFs is. Seems
like there are less things to go wrong with that approach.

------
birk5437
One of the first things I was taught in a freshman CIT class was "Don't forget
about paper! It has its place!" Not saying pilots should have to carry around
heavy paper charts...but yeah.

------
arikrak
I think I remember reading here that an airport was able to function even
after there computers went down since they kept backup printouts of entire
flight schedules. Seems like a pretty big contrast.

------
xbryanx
Those flight plans sure look like XKCD comics from a distance.

------
Mikeb85
Why on earth would you entrust anything critical to a proprietary, consumer-
grade product? Seems like a bad idea from the get-go...

~~~
mikhailt
Because it is not mission-critical in the first place. If you think the flight
plans on the iPad is the sole critical factor in the overall flight operation,
you're mistaken.

There are fallbacks, there are fallbacks even for the fallbacks and finally,
the pilots themselves are trained in the event of total failures on all ends.

In addition, it was a software glitch. We don't know what kind of glitch it
was but if it was, it doesn't matter if they were using proper-graded billion
dollar worth of products, software glitches exist everywhere.

------
learnstats2
I understood it was illegal to depend on electronic equipment in the air?

~~~
jagermo
I think that is for passengers during the critical phases (takeoff/landing) so
that they are not distracted.

A few airlines have switched over to digital maps for airports, since they
offer some substantial advantages (less weight, more up to date, etc).

If you want to have a look at one of these apps, its on the public app store
([https://itunes.apple.com/de/app/lido-iroutemanual-
aeronautic...](https://itunes.apple.com/de/app/lido-iroutemanual-
aeronautical/id413078249?mt=8)) - though I could not find a source that AA is
using the LH Systems app (and if it was the source of the problem, I guess
there would be several more stories about it.

edit: Source: did some work for LH Systems, but not affiliated with them.

~~~
manarth
FAA permitted "PED" use (Personal Electronic Devices) for passengers from late
2013:
[http://www.faa.gov/news/press_releases/news_story.cfm?cid=TW...](http://www.faa.gov/news/press_releases/news_story.cfm?cid=TW189&newsId=15254)

Other countries are slowly following suit.

------
acd
My iPAD and iPhone also crashes frequently since iOS 8.2+ They totally hang
and then you need to reboot it by holding power and home.

I'm switching to Android.

~~~
Kurtz79
My anecdotal evidence against yours, I haven't experienced a single crash on
my iPad(s) in three years.

~~~
pmelendez
My anecdotal evidence supports OP's and againts yours. My Ipad crashes more
frequently with each iOS upgrade. My wife's Ipod touch is even more prone to
crash that the Ipad. I just gave up with both iOS and Android (I gave the Ipad
to my son). It turns out I only needed a phone and a kindle paperwhite after
all.

~~~
Kurtz79
That's great!

We already have three data points.

A couple of thousand more and our anecdotal evidence could turn into a factual
one.

------
gremlinsinc
Better their ipad, than their airplanes ;) (Sorry for the reddit-like comment,
couldn't resist!)

