
Boeing's software fix for the 737 Max overwhelms the plane's computer - sra77
https://www.moonofalabama.org/2019/06/boeings-software-fix-for-the-737-max-problem-overwhelms-the-planes-computer.html
======
dba7dba
From the article

\-----------

 _Boeing says that it can again fix the software to avoid the problem the FAA
just found. It is doubtful that this will be possible. The software load is
already right at the border, if not above the physical capabilities of the
current flight control computers. The optimization potential of the software
is likely minimal.

MCAS was a band aid. Due to the new engine position the 737 MAX version had
changed its behavior compared to the older 737 types even though it still used
the older types' certification. MCAS was supposed to correct that. The
software fix for MCAS is another band aid on top of it. The fix for the
software fix that Boeing now promises to solve the problem the FAA pilot
found, is the third band aid over the same wound. It is doubtful that it will
stop the bleeding.

The flight control computers the 737 MAX and NG use were developed in the
early to mid 1990s. There are no off-the-shelf solutions for higher
performance.

Boeing's latest announced time frame for bringing the grounded 737 MAX planes
back into the air is "mid December". In view of this new problem one is
inclined to ask "which year?"_

\-----------

Ouch...

Introducing a new CPU into airplane like 737 will require another whole series
of test/certifications.

And right about now, Boeing management is thinking _we could use some of those
senior engineers we laid off because we thought they were not needed as our
products are now mature._

~~~
raxxorrax
I know military jets to have unstable flight characteristics to get advantages
in maneuverability that need an operating flight computer.

But I have problems accepting this fact for civilian planes, especially if the
manufacturer just wanted to save development costs. Because you could simply
design a more stable plane. Not trivial, but also not impossible.

~~~
mgsouth
Then you probably don't want to fly in a T-tail jet. [0] [1] Or a helicopter
[2]. Or in a jetliner at cruise. [3] [4]

The fact is, every single airplane ever built is unstable in the wrong
circumstances. They stall, they spin, they lose lift, they suffer mechanical
failures. Aeronautical engineering, rigorous maintenance, and thorough pilot
training have resulted in the safest transportation system that has ever
existed, but the only completely stable aircraft is one that's sitting in a
hanger.

[0]
[https://en.wikipedia.org/wiki/T-tail](https://en.wikipedia.org/wiki/T-tail)

[1] [https://aviation.stackexchange.com/questions/1400/how-do-
con...](https://aviation.stackexchange.com/questions/1400/how-do-conventional-
and-t-tails-differ)

[2] [https://aviation.stackexchange.com/questions/35764/are-
helic...](https://aviation.stackexchange.com/questions/35764/are-helicopters-
aerodynamically-stable)

[3]
[https://en.wikipedia.org/wiki/Coffin_corner_(aerodynamics)](https://en.wikipedia.org/wiki/Coffin_corner_\(aerodynamics\))

[4]
[https://www.faa.gov/regulations_policies/advisory_circulars/...](https://www.faa.gov/regulations_policies/advisory_circulars/index.cfm/go/document.information/documentid/1020859)

~~~
raxxorrax
By stable I meant gliding properties. If the center of mass is moved too close
to the center of lift, planes tend to pitch up and then are in danger of
stalling. Some people suggest Ed the heavy engines might be responsible for
this, but I believe this was misinformation.

------
nercury
Does this mean that one of the reasons MCAS relied on a single sensor could be
the already saturated CPU? Mindblowing.

~~~
laythea
I thought the extra sensor was an optional upgrade?...

~~~
pritambaral
No, that was the AoA indicator in the cockpit. The second AoA sensor was
always there, MCAS just didn't consult it.

------
bcaa7f3a8bbc
Simple question: Why doesn't Boeing upgrade to i386 for their new planes? I
heard it's available for aerospace applications.

~~~
0815test
Forget about upgrading - they could just press the Turbo button on that 286.
That's what it's there for!

~~~
rbanffy
On my boxes, pressing the Turbo button donwclocked them to make Zaxxon
playable.

------
aneutron
This smells like more lives are about to be lost if Boeing doesn't get their
act together

~~~
bradknowles
No.

A few executives might shave a fraction of a penny off their multi-million
dollar annual bonuses. And a lot of lower level workers are going to lose
their jobs.

But no more lives will be lost if Boeing fails to consolidate their defecation
on this matter.

------
zxcb1
Could this be considered unacknowledged technical debt?

~~~
mbar84
Maybe we're just ignorant of the available chips for this use case. Otherwise
just reads like they've been kicking the can of system redesign/refactoring
down the road for decades. It appears that short term thinking abounds in
every industry.

~~~
fit2rule
I'm a developer with extensive experience in SIL-4 programming, specifically
for life-critical systems (rail transport). I've built software subsystems
that are used in 38 countries around the world to keep the trains safe.

The issue is the extensive costs of changing from one CPU type to another.
Certification for these systems is a multi-year process and can cost millions
of dollars even before any kind of success is guaranteed.

There are still very old CPU's out there, functioning just fine for decades.
Safety programming doesn't just swap out parts like consumer computing does -
it takes a lot of work to change CPU's.

I don't think Boeing has been kicking the can down the road on the upgrade. I
do think they've been trying to cut costs and exploit their customers by
offering extra safety features as upgrades, rather than making them standard.
Its interesting to note that some of their customers don't require such
stringent safety features in the regions they operate - i.e. this is as much
of a legislative issue as anything else. It could very well be that Ethiopia
doesn't have the same safety requirements encoded in its laws governing flight
as France does, so Boeing offers different features not just according to
budget but also legislation - although we are sure to see that change rapidly
now.

------
tanakachen
As has been discussed on HN, this whole 737 MAX flaw is not a software
problem. Boeing wants you to believe it is. The flaw is actually a physical
design problem of the aircraft. Even if the software were to work perfectly,
the plane is still flawed. The only real fix is not to fly the 737 MAX.

~~~
kuzehanka
This is entirely false. Stop spreading disinformation.

Myth: the 737 MAX 8 is not inherently stable, has relaxed stability, etc.
Fact: it's very much inherently stable.

Myth: the 737 MAX 8 is easier to stall than other planes. Fact: no it isn't.

The pitch-up characteristic of the MAX 8 is less strong than of e.g. the 757
and that plane flies just fine.

The actual problem with the MAX 8 is that Boeing added MCAS to allow it to
share a type rating with the rest of the 737 family (allowing existing 737
pilots to fly the MAX 8 without additional training), and they fucked up MCAS.
There's a number of solutions on the table, including removing rather than
fixing MCAS and giving up the 737 type rating.

I am continuously astounded that even on HN people are focusing on news cycle
bullshit about inherent instability instead of the actual issues with
Boeing/FAA that caused this situation.

Evidence so far suggests that MCAS was originally a non-critical system that
was found to be too weak during flight testing, and given significantly more
pitch authority. For whatever reasons, this didn't trigger the
reclassification of MCAS as a critical system and it all went downhill from
there.

Here's a pair of sources slightly more credible than the bullshit news cycle:

[https://www.youtube.com/channel/UCphqjYZxxzjNbONVmY-0J7Q](https://www.youtube.com/channel/UCphqjYZxxzjNbONVmY-0J7Q)

[https://www.youtube.com/channel/UCwpHKudUkP5tNgmMdexB3ow](https://www.youtube.com/channel/UCwpHKudUkP5tNgmMdexB3ow)

~~~
mgsouth
Ironically, I believe that your information is also inaccurate. MCAS is
required _in order to meet federal airworthiness requirements_. Without it, in
certain flight conditions the back-pressure on the control yoke gets less as
the yoke is pulled further back. It's like over-steer in a car, and is simply
not allowed. Yes, the plane is dynamically stable; if you leave the yoke in
one position it won't pitch up even more. However, the forces must be
corrected somehow, regardless of type rating concerns.

 _" The 737 MAX was a bit too easy to pull into a stall when flying with high
AoA and making abrupt maneuvers. The larger engines for the MAX hung further
forward from the wing, added a destabilizing aerodynamic area ahead of the
center of gravity, destabilizing the pitch moment curve at high AoA.

Boeing and the certification authority, FAA, decided added margins was called
for. Boeing added a pitch augmentation at high AoA called Maneuvering
Characteristics Augmentation System, MCAS.

The aircraft should trim nose down to increase the stick force needed once it
passed into the light grey area where the base aircraft had a region of less
stability. Before the augmentation, the pilot felt if the aircraft wanted to
fly into the stall, it got easier to increase the AoA after 12°AoA. With the
augmentation the felt extra force was the same for the first and last part of
the curve before the maximum lift was achieved at stall (and stall warning
kicked in)."_ [0]

The _manner_ of the fix (MCAS transparently pushing the nose down) was
designed to avoid pilot retraining and thus keep the same type rating.

Edit: The fact that the 737-Max needs a handling tweak is _not_ a failure.
Modern planes have all kinds of these tweaks, whether aerodynamic (such as
strakes), mechanical (stick shakers) or enabled in software. As the cited
article continues: _" So far so good. It's common an aircraft’s flight control
system has fixes to stability margin changes in different parts of the flight
envelope."_ The problem is that Boeing had a pretty severe collapse of its
systems engineering regime.

 _" The implementation for the 737 MAX had two problems, however:

\- The fault checking of the triggering AoA signal was not rigorous enough.
This problem has been discussed a lot. No need to add anything.

\- The judgment the pilots would identify a problem with the augmentation as a
trim runaway and shut the trim off was wrong. Why the pilots didn’t see MCAS
rouge actions as a trim runaway is poorly understood."_

(The article was published in February. Since then lots of information has
come to light about how MCAS determinedly fought correction, and the huge
mental and physical loads imposed on the pilots.)

Edit 2: FAA regulation mandating increasing elevator forces for _all_
transport aircraft: FAR §25.253 High-speed characteristics, (a) Speed increase
and recovery characteristics, (3):

 _With the airplane trimmed at any speed up to VMO /MMO [maximum operating
airspeed], there must be no reversal of the response to control input about
any axis at any speed up to VDF/MDF [maximum airspeed demonstrated in
testing]. Any tendency to pitch, roll, or yaw must be mild and readily
controllable, using normal piloting techniques. When the airplane is trimmed
at VMO/MMO, the slope of the elevator control force versus speed curve need
not be stable at speeds greater than VFC/MFC [maximum control airspeed], but
there must be a push force at all speeds up to VDF/MDF and there must be no
sudden or excessive reduction of elevator control force as VDF/MDF is
reached._ [1]

[0] [https://leehamnews.com/2019/02/08/bjorns-corner-pitch-
stabil...](https://leehamnews.com/2019/02/08/bjorns-corner-pitch-stability-
part-9/)

[1] [https://www.ecfr.gov/cgi-bin/text-
idx?node=14:1.0.1.3.11#se1...](https://www.ecfr.gov/cgi-bin/text-
idx?node=14:1.0.1.3.11#se14.1.25_1203)

~~~
kuzehanka
"MCAS is required in order to meet federal airworthiness requirements"

Cite this. Specifically this. The rest of your comment agrees with mine
without this being true, and I have not seen any evidence of this being true.

~~~
mgsouth
Hmm. OK, better wording would be "Handling mitigation such as MCAS is required
in order...". I've seen several news articles, such as the one cited, which
reported that the yoke forces decreased near stall AoA in certain flight
regimes. Here's one from the NYT: [0] Originally MCAS was implemented for
rather extreme maneuvers, and required both the AoA sensor and a G-force
sensor to agree. Later, it was discovered that low-speed stalls also had yoke-
force problems, and the control authority was increased, and the G-force
requirement dropped.

[0]
[https://www.nytimes.com/2019/06/01/business/boeing-737-max-c...](https://www.nytimes.com/2019/06/01/business/boeing-737-max-
crash.html)

~~~
kuzehanka
And as stated in countless well cited articles, this was done in order to
maintain the 737 type rating and make the MAX 8 handle like other 737. Not to
meet regulations that were otherwise unmet.

------
mongol
Is the 80286 still in production? Or is it old stock on a shelf somewhere that
is used?

~~~
ChuckNorris89
AFAIK, Airbus has stockpiled huge amounts of legacy CPUs in climate controlled
conditions. Boeing must have done the same

~~~
pratap103
Was really curious as to what Airbus was using, thanks for this! Why not just
use the latest CPUs though? I can't imagine the cost would be relatively
significant in the greater scheme of things. Are the legacy CPU's really
better for this use-case?

~~~
ChuckNorris89
Aero industry moves at a very slow pace. The state of the art F-22 Raptor uses
intel 386 cpu and the F-35 lightning upgraded to more "modern" PowerPC CPUs,
similar to the ones on legacy Macs.

Planes don't need insane multitasking processing power like our smartphones or
PCs. They mostly do signal processing and sensor fusion in a tight loop which
is quite trivial even for legacy CPUs as it's basic flight math equations
which results in highly optimized code.

In terms of aero chips, basic is always better as you want a silicon that's
tried and tested for decades to have a deep understanding of it's quirks and
bugs so you know the code execution is reliable.

------
stunt
They may find a physical fix for it later. Like adding weight where it can
help. Even then, perhaps nobody wants to fly with it plus it can't offer the
same performance.

------
laythea
It amazes me how Pilots are required mainly for the purposes of being able to
act as a last resort against a computer which has gone haywire, and the public
feel better thinking that the pilot can "take control" and then at the same
time aircraft manufacturers are removing the ability for the said Pilots to
override the computer. That's progress!

------
majewsky
Mods: Can you remove the #more from the submission URL? The fragment makes the
browser scroll down which skips the first half of the article.

~~~
dang
Sure.

