
Why Free Software is a Matter of Life and Death  - strawberryshake
http://www.computerworlduk.com/community/blogs/index.cfm?blogid=14&entryid=3090&utm_source=ycombinator&utm_medium=sb&utm_content=anguyen&utm_campaign=sb
======
pascal_cuoq
It saddens me to see the linked article taken seriously and its political
agenda repeated as if the existence of an article advocating Open Source in an
Open Source conference gave credence to the idea of mandatory Open Source for
life-critical software.

The article "Killed by Code: Software Transparency in Implantable Medical
Devices" is weak. It mixes safety and security, which are subtly different
beasts. Specifically, it justifies the idea of mandatory Open Source for
safety-critical software with security examples. Unfortunately, while there
are independent security researchers, I have yet to meet my first independent
safety researcher, who looks for flaws in life-critical equipment for fame or
money or ego…

The article also completely ignores the fact that DO-178B certified code, as
found embedded in civil aircrafts, has a perfect safety track record (although
some incidents have, no loss of human life has yet been attributed to software
malfunction in a civil aircraft).

I work on static analysis tools for critical software. In this domain, the
unavailability of examples is a big pain in the neck. My colleagues and I
would love to see more examples of critical embedded code available to try our
tools on. But in spite of this, I have to say that I find this article highly
unscientific and so tainted by politics that it becomes distasteful.

(oh, and our analysis framework is Open Source. By choice, not by constraints
based on unsound reasoning)

NOTE: I merely claim that the article is unscientific, not that it makes a
particular unscientific claim such "that opening up software will solve all of
it problems". But since apparently I need to be more specific, here is a
paragraph straight from the article:

Other public sector agencies, such as the U.S. Navy, the Federal Aviation
Administration, the U.S. Census Bureau and the U.S. Patent and Trademark
Office have been identified as recognizing the security benefits of publicly
auditable source code.20

There is only one reference for the four sources. Let's take a look at that
reference:

20 FAQs, Open Source for America, <http://opensourceforamerica.org/faq> (last
visited July 16, 2010).

How does that page justify the above paragraph? You tell me if you find it.
This is not how references are supposed to work in a scientific article. The
last time I saw this kind of "he said she said" reference, it was in a text
trying to justify homeopathic claims.

~~~
aquila
"I have yet to meet my first independent safety researcher, who looks for
flaws in life-critical equipment for fame or money or ego…"

You have now, this is what I do for a living.

"The article also completely ignores the fact that DO-178B certified code, as
found embedded in civil aircrafts, has a perfect safety track record"

DO-178B is a process, and not a "certification". It's utilization has not
produced a "perfect track record" in safety critical systems.

You should know better than to make such grossly incorrect statements. Experts
in this industry know for a fact that testing only proves the lack of a
"tested for" failure condition, and cannot be relied upon to assure safety.

DO-178B is but one of several criteria by which the Aircraft certification
authorities judge an avionics system to be safe.

Software is too complex to be assured by testing alone.

Static analysis is merely a tool, and is not a solution to this problem. You
should be well aware of the limitations of Static analysis.

Regulators no longer are provided with access to design and testing data
sufficient to fully review a product for safety, and they routinely rely
heavily on the manufacturers, thru delegated authority, to regulate
themselves.

Design defects can only be eliminated in the application of a transparent
review process. Complexity issues in "limited human cognition" and "human
bias" are the key causes of life threatening defects in safety critical
systems.

Without external review and oversight by "outside consultants" such as myself,
life threatening safety defects routinely go undetected.

All safety critical systems will ultimately be "public domain" for this very
reason. Just as proposed by the Software Freedom Law Center.

The FAA does not absolve aircraft manufacturers for liability, as the case
with the FDA and Medtronic. (see Varig v. United States) 28 USC 2680(a)
Discretionary function exception

DO-178B software does not "in fact" have a "perfect safety track record".

Turkish Airlines Flight 1951 was a passenger flight which crashed during
landing to Amsterdam Schiphol Airport, Netherlands, on 25 February 2009,
killing nine passengers and crew including all three pilots.

software defect utilized "invalid altitude data" as valid.

"Tests showed that the Rockwell Collins Enhanced Digital Flight Control System
(EDFCS) uses radio altitude values that are characterised as ‘non computed’
(unusable,) whereas this characterisation should have prevented this."

<http://en.wikipedia.org/wiki/Qantas_Flight_72>

ADIRU software defect utilized "invalid data" as valid.

This device was certified to DO178-B Level A, the very highest level of
certification, where failure can result in catastrophic loss of aircraft.

1 August 2005, Boeing 777-200 registered 9M-MRG, from Perth to Kuala Lumpur,
Malaysia.

use of invalid data as valid

This device was certified to DO178-B Level A, the very highest level of
certification, where failure can result in catastrophic loss of aircraft.

~~~
pascal_cuoq
> You have now, this is what I do for a living.

Perhaps there is a misunderstanding. I was speaking of security researchers
who look for flaws without having been prompted to do so (the "money" in
"fame, money and ego" was the money that you can get when running a well-
organized botnet, not the money you get for a honest job well done. Sorry for
the confusion). Is this what you do, in the case of safety? If so, this is
interesting and I would like to know more. What do you do with your results,
publish them? What is your affiliation? Where can I find a list of such
publications? Or do you have a contract with someone who is paying you to
assess the code under NDA, in which case, where does Open Source come into
this model? Would you do your job for free if the code was open? Under a
deadline?

> DO-178B is a process, and not a "certification"

I used the words "DO-178B certified". As a Google search will confirm, I am
not the first one to use this idiom. Since respectable actors are using it and
have gone as far as buying it as Google AdWords, I stand by my use of these
words.

> Static analysis is merely a tool, and is not a solution to this problem. You
> should be well aware of the limitations of Static analysis.

I didn't make any claims one way or the other. It is not unusual to meet
people who have strong ideas on the subject, and I avoid this sterile
discussion whenever I can.

I made clear in which sense I meant "perfect safety record". There were no
casualties in Qantas' Flight 72. The report at
[http://www.onderzoeksraad.nl/docs/rapporten/Rapport_TA_ENG_w...](http://www.onderzoeksraad.nl/docs/rapporten/Rapport_TA_ENG_web.pdf)
is new to me. Thanks for the reference.

What always strikes me, when I read reports such as this, is the self-
correcting attitude displayed by the relevant authorities. The question, when
analyzing an incident, is always, "how do we make sure the same mistake is not
made again?". The reason why I think public domain critical code will not
happen is that there is no answer to this question (you can't make it more
public domain the next time).

Again, I wish it was different. I just don't think it will happen in the
foreseeable future.

> All safety critical systems will ultimately be "public domain" for this very
> reason.

Do you want to bet this practice has not been adopted in a single domain where
it could apply by 2020?

It seems to me, but forgive me if I misunderstood your arguments, that you are
saying "Testing is not perfect. Static analysis is not perfect. Therefore,
critical code has to be made public domain". I'm not sure it follows. Whatever
solution you are rooting for does not necessarily win by default because the
other solutions are not perfect. Open source could win if it had something
tangible to bring to the table. It wouldn't be automatic, and I am not even
sure that the tangible advantages exist. Plane designers have been making
software more and more complex because they could. If it turns out they can't
complexify it any more, they can also just stop complexifying it. I can assure
you that a company such as Boeing or Airbus would rather scale back on the new
features used to promote new models than expose their designs to their
competitors three years in advance (the code has to be verified before the
first flight, right?).

At the risk of repeating myself, I am only describing the situation as I see
it, not the way it would be convenient to me for it to be.

Pascal

