Hacker News new | past | comments | ask | show | jobs | submit login
Embraer Phenom 300 yaw damper fail due to loss of GPS signal [pdf] (faa.gov)
80 points by a-no-n on Sept 17, 2016 | hide | past | favorite | 45 comments

What a mess. We botched GPS/AHRS integration in our 2005 Grand Challenge vehicle. Seen this type of problem.

Here's the problem. You have an Attitude and Heading Reference System (AHRS) with three rate gyros and three accelerometers, plus a magnetometer as a compass. From this information, you want to get aircraft orientation and heading. Integrating the rate gyros gives orientation, but because you're integrating a rate, there's cumulative error over time. The accelerometers give you a down reference. Not always a good one. Some AHRS units will lose their down reference if you fly in a circle for a while, and the accelerometers see a consistent but wrong "down" direction from centrifugal force.

So AHRS systems are prone to cumulative error accumulation. You also have a GPS system, which is prone to short-term noise but does not accumulate cumulative error. GPS is position only; there's no orientation info from GPS. (There are some multiple-antenna GPS systems that get orientation, but those are rare) So AHRS/GPS systems try to combine the two using various filters. Those filters embody assumptions about the error properties of each system. Fusion of the two sources provides position information, with the GPS fixes being augmented with short-term info from the AHRS. However, it's possible for GPS position to "jump" due to radio propagation problems. You see this on smartphones all the time. It's not as bad for aircraft, which usually have a clear view of the sky. (It's much harder for ground vehicles, which can't always see enough GPS satellites. We had a lot of trouble with this in 2005.)

The important outputs are attitude (pitch, roll, and yaw) which drive the artificial horizon ball (or, today, a graphic which looks like one). There's also heading, which drives the compass. The attitude outputs also drive the aircraft's stability control systems. The AHRS data can be used raw, or combined with GPS data to get not only attitude but position.

Usually, the AHRS outputs are used without GPS info to drive the aircraft systems that just need attitude. But the fused AHRS/GPS data is usually better, and dealing with inconstencies between the unaugmented AHRS info and the fused data is a pain. So there's a temptation to use the fused data for everything. Embraer apparently did this.

There's an argument for source integration. Air France 447 crashed because they lost airspeed and altitude data when the pitot tubes and static ports froze up. An integrated AHRS/GPS system would have given them accurate info on vertical speed and altitude even without air sensors. So integration isn't fundamentally a bad idea. If you integrate sources, though, you have to deal with sensor conflicts. That's a hard problem. Filters alone will not do it.

The integration used by Embraer (who did the avionics, by the way?) apparently doesn't handle GPS failures properly. The AHRS is still there, and that's all you need for aircraft stability. But if the fused outputs are used for stability augmentation, stall warning, and such, apparently they can fail when the GPS data does.

> who did the avionics, by the way?


This is a known issue - and old issue.

These jets cannot fly without GPS.

When they (the military) knocked out GPS intentionally around China Lake NAS a few months back (for testing aircraft in GPS denied environments) -- all Embraers were told to avoid the area:



> These jets cannot fly without GPS.

No need for FUD. Loss of Yaw Damper is not something that would endanger an aircraft

tried to edit it. but why would they mention it so prominently in the NOTAM?

Because when this bug happens, the pilot is going to face a "christmas tree" of warning lights and cautions, most likely provoking an immediate PAN or Mayday and a possible diversion to the nearest suitable airport.

Loss of attitude info is a serious business, in the worst case the pilot may be left to fly only "raw data" and using the backup "steam gauge" instruments, because the displays and automation will stop working. Such event is no cake on a big jet, even during daylight, perfect weather and with the crew perfectly current and capable to fly like that.

Murphy will have it that it happens during night, in a storm and most crews today are trained to rely on the automation and rarely getting to hand fly, even less raw data, without any of the computers. The result will be another AF443 ...

See the other links around, there's a Garmin manual showing the failure modes regarding loss of GPS data (even some people saying loss of GPS only should not cause YD failure)

And it's AF447

Because it's still unexpected behavior, and suddenly losing aircraft systems mid-flight is a concerning event for a pilot. Especially ones related to flight control.

That doesn't mean the plane can't fly. But it does mean that the pilot is going to have to fly it differently to avoid dutch roll, possibly flying at a lower altitude.

Keep in mind that this is aviation, which is very risk averse (by necessity). Even minor issues are treated as significant.

I'd say because aviation is risk averse and if a condition will impact a certain type or model of aircraft it is not going to be omitted

(the fact that it's a smaller aircraft might imply in having some certification differences as well)

Do you have any links to the reasoning that would let this kind of system be allowed? It seems rather fragile - GPS can be messed with rather easily.

I have none -- I don't claim to know much more than that about Embraer (or any other type). I found it to be ridiculous as well.

My interest was more to the weapons system testing itself.

How does an aircraft gets certified with out an INS for stability control?



Can someone provide context?

It appears that the Embraer Phenom 300 has GPS so deeply integrated into its "flight control computer" that the loss of GPS results in the loss of a number of important systems, including the system that prevents a common aerodynamic instability, known as "Dutch Roll".

Modern planes aren't flown directly by people. They're flown by computers, with people using the controls in the cockpit to tell the computer what they want the plane to do (there is no direct link between the stick/yoke and the flight surfaces). The computer integrates all the information at its disposal and determines what movements of the flight control systems are necessary to meet the pilot's commands (this is known as fly by wire). In this case, it seems that computer is _really_ dependant on GPS.

> (there is no direct link between the stick/yoke and the flight surfaces)

Almost true and oft-repeated. There are still means to somewhat control most FBW aircraft in the unlikely event all electrical control is lost; A320 can pitch/yaw with mechanical linkage to stabilizer trim and the rudder, for example (but, to your point, neither involve the sidestick). The flight is very likely headed for an unpleasant end but the intent is to buy time to restore computer control. Landing with trim, rudder, and differential thrust alone is possible (some have landed with less), but it's a big ask for a pilot accustomed to automation.

That scenario aside, when a whole lot of things fail an Airbus can also go into a mode called Direct Law, which basically tells the computer to take a hike and puts the pilot in direct control of surfaces proportional to airspeed (and puts structural integrity in jeopardy, because the computer isn't helping to obey design limits). The aircraft is extremely sensitive in this configuration.

(If you saw Sully recently, one of the reasons he quickly started the APU was to prevent the aircraft from leaving the relative safety of Normal Law in the absence of electrical power. This was described in the report, if I recall, and was one of the more critical decisions made.)

Boeing is entirely different in FBW philosophy from Airbus. A coarse summary might be that Airbus drivers are negotiating with a computer while Boeing drivers are generally flying the airplane. Both have merits and drawbacks and vocal camps.

> Almost true and oft-repeated. There are still means to somewhat control most FBW aircraft in the unlikely event all electrical control is lost; A320 can pitch/yaw with mechanical linkage to stabilizer trim and the rudder, for example (but, to your point, neither involve the sidestick).

Yeah, that's why I was specific about the stick/yoke connection. Though, after some research, it appears I was wrong about that as well (as least in the case of Embraer). In the 190, the ailerons are, effectively, directly controlled (there is a hydraulic system in line, but it is directly actuated by cables from the yoke). I was unable to find any conclusive details about the Phenom 300 after a cursory search, but it appears it is "fully" FBW.

> Boeing is entirely different in FBW philosophy from Airbus. A coarse summary might be that Airbus drivers are negotiating with a computer while Boeing drivers are generally flying the airplane. Both have merits and drawbacks and vocal camps.

Yeah, and both sides can point to major incidents that wouldn't have happened in the other cockpit.

The Phenom 300 does not use FBW. The Stability Augmentation System (generic term) is digital, but there is a mechanical link to the aileron and elevator.

My understanding is the GPS and Inertial Measuring Unit (IMU) are combined. If there is a loss of GPS, the Autopilot & Yaw Dampener considers the IMU from that channel unavailable.

Only the newer Embraer 450 & Embraer 500 models are FBW aircraft. They also have side-stick controls.

I know of Air France 447 in the "fly the airplane" camp. (An inexperienced copilot stalled the aircraft all the way from cruise altitude to impact, and because the sidesticks aren't linked, no one else could tell that was what was wrong.)

What's a good example in the "negotiate with the computer" camp? Perhaps it's merely a tendency on my part, but I don't think I have run across any accident which militates as strongly in favor of Airbus's design as AF 447 seems to militate against it.

AF447 is more complicated than just that sidestick issue, if you look at the accident report closely.

The contributing factors were loss of outside air temperature and airspeed sensors because of icing, a dark moonless night above the ocean, and flying/falling outside of the operating range of the stall horn (when they pitched the nose down, they started recovering from the stall, airspeed went up, and the stall horn activated. when they pitched up, they went back outside of the operating range, and the horn muted).

A big contributing factor as well was that in an airbus, in normal operating conditions, pulling the stick all the way back will configure the aircraft for the best rate of climb (which can be different from just putting the elevator in full climb conditions). Due to the pitot tubes being iced up, the flight computers were in Direct mode (where movement of the sidestick is directly translated in movement of the flight controls, without computer intervention), which didn't help.

> the flight computers were in Direct mode

Almost that, they were in Alternate Law

>I don't think I have run across any accident which militates as strongly in favor of Airbus's design as AF 447 seems to militate against it.

The references I've seen to advantages tend to be more subtle than the AF447 tragedy. Things like the Hudson river landing where in that strange situation where you have no power the workload was diminished by having the computer keep the airplane flying most efficiently and thus be able to reach the river instead of crashing by losing airspeed.

It also makes sense the advantages are less obvious as the flight envelope protection actively prohibits dangerous situations and you can't report on non-events. The stats also show Boeing and Airbus airplanes pretty evenly matched on safety.

Not a Boeing but the Airbus A300 predates fly-by-wire and the pilot stamping on the rudder pedals snapped the tail off of AA Flight 587 [1] - you might reasonably expect FBW to have intervened.

1: https://en.wikipedia.org/wiki/American_Airlines_Flight_587

AA 587, which ironically happened on an early model Airbus before they went FBW.

Your perspective will influence how you classify the accidents. Proponents of full FBW will argue AF 447 happened exactly because part of the FBW system was disabled.

The Boeing 737 is still controllable with no electric or hydraulic operation, since the yoke and rudder are linked with cables to the control surfaces. From what I understand the hydraulics also provide power assist, so it might not be too dissimilar to driving a car without power steering or brakes when that happens.

Scenes in movies of pilots physically struggling with the plane are not as unrealistic as they may seem.

They're not, and you're right. You're literally pulling or pushing a lot of control surface against a lot of fast-moving air, and it doesn't want to go there. There were many inaccuracies in the opening drama of Flight, but the yokes beating the shit out of the crew was quite believable given the scenario.

For those who want to know a bit more about a few of these topics, I found this article via HN discussing the role computers, fly by wire, and a lack of training had in the AirFrance 447 crash. Found the article quite interesting.


> A coarse summary might be that Airbus drivers are negotiating with a computer while Boeing drivers are generally flying the airplane.

Would you mind taking a moment to spell out what these statements mean to a non-pilot?

(For example, I expect that Airbus pilots still operate controls which affect the movement of the airplane; they aren't typing parameters into a computer. And I expect Boeing pilot's actions are still mediated by a computer.)


On all FBW systems there's a computer in between the pilot's appendages and the wiggly parts built into the airplane that control its orientation (roll, pitch, yaw, what have you).

Airbus systems position the pilot as a suggestive force. At any given time the computer has its idea of what's happening and the pilot influences this calculus within safety tolerances for various bad things that happen to airplanes. One example is a stall, which results from a variety of factors including excessive pitch of the wing against the oncoming air (called angle of attack -- think about tilting your hand back when holding it out of the car and it dropping back on you. that's a stall). Airbus computers typically prevent the pilot from introducing the situation by dampening, modifying, or outright ignoring control input to avoid critical angles of attack. That's one example.

Basically, without going alternate law, the Airbus computer will work to make it very hard (not impossible, obviously) to apply control input that would place the flight in jeopardy.

Boeing would allow the input and mostly permit the pilot to introduce the situation. That's the difference. They treat the pilot as the end of authority in command. That has compelling merits. So does Airbus. As JshWright said, both sides have incidents on the other they can point to as supporting data.

There was a cancelled, very large proposed GPS outage (NOTAM) centered on NAWS China Lake which specifically warned that the EMB-505 Phenom 300 business jet specifically should avoid operating in the Western US during this mysterious GPS outage/jamming test event.[0] Pressure from the AOPA and others likely contributed to the scrubbing of this would-be very large outage.[1] It turns out that the EMB-505 300 lateral stability at high speed is extremely dependent on GPS.

Folks on the forums suspect the root cause is related to the Garmin GRS 77/GMU 44 AHRS part of the Garmin Prodigy G3000 package.[2,3]

0. https://www.faasafety.gov/files/notices/2016/Jun/CHLK_16-08_...

1. http://www.ainonline.com/aviation-news/business-aviation/201...

2. http://static.garmin.com/pumac/778_InstallationManual.pdf

3. http://static.garmin.com/pumac/1152_GRS77_GMU44Troubleshooti...

What would be the purpose of this outage?

"China Lake is the United States Navy's largest single landholding, representing 85 percent of the Navy’s land for weapons and armaments research, development, acquisition, testing and evaluation (RDAT&E) use and 38 percent of the Navy’s land holdings worldwide. In total, its two ranges and main site cover more than 1,100,000 acres (4,500 km2), an area larger than the state of Rhode Island."

Possibly to ensure that weapons and weapon systems that use GPS will still function in it's absence.

It's classified, but they may have been testing GPS jamming equipment or other rf tech that has that as a side effect.

Anyone have a sense of what the primary failure is here? Is there some Kalman filter that gets out of whack when an error term can't be calculated, or what?

Translation: we never tested flight with the GPS off.

Planes are not built by "stuff.js" hackers I'm afraid

And it does work without GPS, as this link (which was posted in this page http://forums.jetcareers.com/threads/emb-300-phenom-yaw-damp... ) shows

I have a commercial rating. The HN title is wrong.

What this is saying:

1) If you don't have a yaw damper, then you'll have a rough ride

2) because the autopilot will induce oscillation (dutch roll) after loss of GPS

The HN title is literally the subject of the notice, very slightly paraphrased (abbreviations expanded, etc), so you may want to let the FAA know they got it wrong...

I also think you misread the notice. The report is stating the loss of GPS apparently resulted in the subsequent loss of a number of other systems, including the yaw damper.

I think you're right. The wording about "no yaw damper" would have been clearer as "failed yaw damper" - I guess that's what was meant.

How much oscillation? On a scale from "bumpy ride" to "holy shit my wing's off".

Dutch roll is characterized by a low frequency oscillation, with a period in the order of seconds. So it wouldn't be a "bumpy" ride but a slooow yet continuous, very nauseating oscillation (try to slowly rock forward and backward in your chair).

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact