
Executing Software Engineers for Bugs - hlieberman
https://blog.setec.io/2015/11/01/ethics.html
======
Eridrus
When people bring up a code of ethics, actuaries seem to come up a lot because
they can potentially lose their license for mis-evaluating risks, but
anecdotally what I've heard is that in reality if your boss asks you to sign
off on something, and you refuse they'll find someone else to sign off on it
and just label you as not a team player. At which point you've harmed your
career and had no real impact.

I bring this up because when people propose a code of ethics, they assume it
will help, but I'm not really convinced that we will really get to a point
where codes of ethics are widespread and taken seriously enough that they will
do anything but harm the altruists among us.

~~~
hlieberman
That's true. Is there a value to refusing to do work, even though it might not
be able to stop something from being built in spite of it?

If you know, a priori, refusing to work on something you consider to be
unethical will have /zero/ effect on the overall project, but negatively
affect you, is it then ethical to do the work, or are you required to fall on
your sword?

~~~
femto
I'd contend that you don't know that it will have zero effect until you have
tried, and how you go about refusing assent is of critical importance. If you
simply say "no", your boss will move onto the next person for assent, and you
will be left vulnerable. Make it a formal process, it it may well have an
effect.

1) Listen to all competing arguments. Make sure you are making a sound
decision and your opposition is genuine and not based on a personal bias.

2) Make it explicit that you are are saying no in your role as a professional,
and that this is not just a casual "no". Ethics require engineers to make
decisions only in areas of competency, so make it clear that you are competent
to make this decision, and your decision derives directly from your
competency.

3) Write it down. The document should cover 2) above, and set down the
defensible reasons why assent is being refused, addressing counter arguments.
Consider that the arguments you write down could well be tested in court, and
you will be cross examined on them. The document/report should be of the same
level as if you were a consultant to your employer.

4) Sign and submit the document (keeping a copy for yourself). You need to be
open about your opposition. Keep it at a professional level. Hopefully you
will get some respect, as your document should make it clear that you have
legitimate concerns and are not just being difficult.

5) Be prepared for the consequences. Your opposition is now in writing and
impossible to ignore. No sane boss is going to ignore it, as the legal
consequences of doing so are too great if you turn out to be right. You will
have heat applied to you, as the alternatives are to either address your
concerns or to get you to retract, and the latter option is cheaper.

6) If the final decision is against you, either live with the decision or
resign. If the issue is of sufficient gravity your only option may be to
resign, whether that be for ethical reasons or your own legal protection. (Yes
your Honour, my work killed 100 people, and I was fully aware of it as shown
by this document that I wrote.)

If you end up resigning, then you're probably making the right decision
anyway, as the events leading up to your resignation will have shown your
employer to be a dud.

Granted, reality won't be as cut and dried as above, and will involve a lot of
anguish.

~~~
hlieberman
Very true. From a philosophical standpoint, I still believe considering the
zero effect case is interesting, but from a practical standpoint, I will
gladly cede the point to you. This process seems an impressively good balance
for a very hard, miserable situation.

------
tsotha
>But making the leap from that to "my code can't harm people" is a bridge too
far.

Meh. My application is an internal app for a large company. It's basically
scheduling software for a part of our business process. To even start to hack
it you'd first have to break into the corporate network, and in the end you'd
have data you didn't care about. Hell, I'm not even sure the people who use it
care.

Worst case, a subtle bug (and it would have to be subtle for my users to miss
it) might cost my employer a few thousand bucks.

Again, meh. There are a whole lot of internal applications that fall into this
category.

~~~
nickcano
That's not the right way to think about it. Just because you cannot imagine a
way your software can be used to attack your company or harm the users,
doesn't mean such a thing is impossible or unlikely.

------
WildUtah
"Executing Software Engineers for Bugs"

Never have I been prouder to be a plain old programmer. Those software
engineers, architects, systems analysts, and data scientists can keep the
capital punishment for themselves.

~~~
rootlocus
You don't need a title to take responsibility.

~~~
calinet6
The idea that "taking responsibility" is how we prevent bugs is what's wrong
in the first place.

Quality is systemic, and has its roots across the organization.

------
linuxkerneldev
This author's first sentence is: "There is an apocryphal tale I've heard many
times about how, in Ancient Rome (or, in some tellings, Greece), the engineers
responsible for the construction of an arch were required to stand underneath
it as the final wooden supports were taken out."

This story is derived from Law #229 of the Code of Hammurabi (of Babylon).
Babylon was not part of Greece, but it was very briefly part of the Roman
empire.

" 229 If a builder build a house for some one, and does not construct it
properly, and the house which he built fall in and kill its owner, then that
builder shall be put to death. "

[http://avalon.law.yale.edu/ancient/hamframe.asp](http://avalon.law.yale.edu/ancient/hamframe.asp)

~~~
JoeAltmaier
My Dad told the tale of a farmer coming to the repair shop with a tractor gas
tank needing repair. The mechanic refused. The farmer insisted he'd cleaned
the tank; no danger of explosion while it was being welded. The mechanic
replied "Ok, I'll weld it, if you hold it while I'm doing that." The farmer
left.

------
lambdapie
I'm not a big fan of professional bodies or mandatory qualifications. I think
these tend towards rent seeking, with these bodies existing mainly to justify
their own existence. In the case of software engineering, I'm especially
concerned about academics with little real world experience using valid
security and privacy issues as an excuse to force their own view of how
software engineering should be done on everyone else.

That said, my employer has standards for security and privacy that go well
beyond industry norms, so if I was working elsewhere maybe I would feel the
need for better standards across the industry.

In my experience, software engineers tend to be conscientious. Caring about
the big picture is a big part of open source, hacker and nerd culture. But
knowledge is hard to come by. I learnt from the experts, but I doubt most
engineers would be able to build a simple CRUD app form scratch without major
security holes.

It would be nice to see some best practices around security and privacy emerge
without forcing everyone to write Ada or Coq or completely change their
approach to writing software.

------
kabdib
We _can_ write software like this.

The reason why we don't has been explained many times: It's too expensive, by
many orders of magnitude.

~~~
calinet6
This is simply false. True systemic quality will lower costs, speed
production, and increase value all at once.

The reason is that, beyond a certain organizational complexity, we fail at
creating the systems necessary to create the positive feedback loops necessary
for quality, low cost, and fast execution to flourish naturally. Instead, we
live in dysfunctional organizations where the tradeoffs prevail, and we choose
speed and cost over quality in nearly all cases. Most of the time, we don't
understand how to even begin to think about quality at an organizational
level.

It's fully possible. Quality pioneer W. Edwards Deming explained the full
system for creating an organization that is capable of quality in the first
place. It mainly involves systems thinking, knowledge of variation,
understanding psychology deeply, and creating systems of effective
learning—and management and leadership versed in those new leadership
competencies.

Instead our instincts are to blame, to punish, to hold accountable,
effectively to leave each engineer as an island in charge of his or her own
level of quality—when an organization and its production is so far removed
from individual control it's not even funny.

We absolutely _can_ write software like this—and we can do it quicker, and for
lower cost. We need to work on the systems that produce and influence that
quality/speed/cost relationship—not each individual part in isolation. There
is a positive cycle to be achieved.

[https://en.wikipedia.org/wiki/W._Edwards_Deming](https://en.wikipedia.org/wiki/W._Edwards_Deming)

~~~
eloff
You've stated that it's not more expensive. But everyone in the industry knows
otherwise. Unless you can produce some evidence to the contrary, I'm going to
insist it's you who is incorrect here.

~~~
calinet6
That's a highly simplistic and incorrect viewpoint—and the correctness of a
model is not dependent on popular vote.

When looking at increasing quality (essentially, scope) in isolation of
systems and the environment in which that quality is created, then the only
way to increase quality is to increase cost. That much is true, but it's also
incomplete.

When looking at and eventually optimizing the whole system—from the customer
and their needs, to the leadership of the company, to the clarity of purpose,
to values and culture, to the intrinsic motivation of your employees, to
methods of management, to your learning and education processes, to your
processes of production, to your communication links, to your output and how
it's measured, and how each is controlled and understood in great detail—you
can and will improve quality much more than you could by simply increasing
cost in isolation—and at the same time, you'll decrease your overall costs in
the long term, you'll speed up production, and you'll increase sales and
customer satisfaction. The end result is a better product, for lower cost,
with more predictability, proud happy employees, and faster to boot. That's
what Quality really means.

Looking at quality in terms of cost alone is the naive mistake. Look at the
whole system.

And for some evidence, look at Japan circa 1948. These exact methods of
systemic quality, "total quality control" and statistical quality control
taught to Japan by W. Edwards Deming turned them from the ruins of World War
II into the second largest economy in the world, to the point of such success
that Japanese companies were putting American ones to shame by the 70's. They
did this by following exactly the model I've described above, decreasing total
costs, increasing quality, and improving time to market, all by focusing on
their total systems across the organization. The proof is in the success of an
entire country's economy, I don't know how it can get much clearer than that.

It is not my fault that everyone in our industry has forgotten these lessons
on running companies, but the prevailing methods remain incorrect, as does
your original statement.

~~~
kjksf
Either you're wrong or literally every software company on the planet is
wrong.

I'm betting it's you.

Please do prove us all wrong by starting up a software company that produces
bug free software faster and cheaper than every other software company on the
planet. There's a big pot of gold at the end of that journey. Go and claim it.
If everyone else is as incompetent as you claim, it's going to be a walk in
the park.

~~~
calinet6
Not only software companies, but most companies in the US are indeed wrong
about the prevailing methods and styles of management and leadership. Is this
really that surprising to you? Do companies you work with appear chaotic yet
surprisingly successful? Do the same things keep going wrong? Do they get in
cycles of cultural upheaval? And do we accept these things as "normal
business" and go on with out lives, powerless to control them?

And I'm not saying it's easy. Not at all. It might be the hardest problem in
all of business—combining the true systems of work into a cohesive philosophy:
systems, statistics, people, and knowledge. I'm just saying (and I'm just
repeating the words of some great thinkers in the quality space, mind you)
that if we change the way we think about quality and companies in general,
we'll get a lot of gigantic unrealized gains in productivity.

This happens naturally in many companies in small pockets, but human nature
and psychology tends to break it down eventually.

If I'm wrong, then a great many other people are also wrong, primarily W.
Edwards Deming, who, as I noted, single-handedly turned around the _entire
economy of Japan_ using these exact methods and ideas.

Here's a great place to start: [http://www.amazon.com/Leaders-Handbook-Making-
Things-Getting...](http://www.amazon.com/Leaders-Handbook-Making-Things-
Getting/dp/0070580286)

------
joe_the_user
OK,

I think we all can agree that programmers almost never begin a project with
the intention of causing harm. All the examples cited involved situations
where immense pressure applied by higher-up impel a programmer to knuckle-
under and agree to such approaches.

The solution that is offered seems very specific to now - we take the fall guy
who caved in to one sort of incentive and we put an opposite incentive on him
to force a different behavior (btw without removing the first incentive). How
fucked-up is that?

Obviously, the better, saner solution is removing the existing perverse
incentives, giving software engineers more leverage in decisions, punishing
high-ups if they don't give software engineers autonomy to make decisions. And
heck, execute the CEOs when the bridges collapse. The bucks should stop with
them, right?

~~~
hlieberman
The problem becomes... what if the CEO is unaware of it? Volkswagen, for
example. It is highly unlikely in my mind that some engineer somewhere in the
trenches decided, in desperation, to add in a few lines of code to make sure
the engine passed emissions tests. But, do I think the CEO of Volkswagen
signed off on adding the code? No.

There's a diffusion of responsibility that makes it hard to point to any given
person after the fact. How can we arm people - software engineers, but product
designers, project managers, QA engineers, CEOs, and the welders on the
assembly line can say, "Hold on, stop, this isn't right!"?

~~~
joe_the_user
I think that shows the limits of the punishment after the fact approach. The
point is to demand that companies create an atmosphere where professionals
have some autonomy to make decisions based on their expertise rather than the
good of the company. Unfortunately, the trend is things going the other way
for virtually all professionals, not simply almost-professionals like
programmers.

------
exmicrosoldier
Until it is the CEO..CTO..and vc that is standing under the bridge...nothing
will change.

~~~
roel_v
How would they know whether the bridge is designed badly? What's the reasoning
behind your underlying suggestion?

~~~
lazyjones
> _How would they know whether the bridge is designed badly?_

It's their responsibility. If it's not possible for them to comprehend the
design and execution thoroughly enough, they'd better make sure they had
someone check it whom they can trust to understand it. Like ancient builders,
who didn't lay every stone themselves, but were still responsible for the
outcome.

Perhaps our standards for responsibility of leaders are a little low these
days.

------
hlieberman
Those of you with a membership in ACM or IEEE, I'm interested in hearing your
thoughts on why you do.

Those of you who don't, would you consider pledging yourself to a code of
ethics if they met your requirements? What would those requirements be?

~~~
trentmb
If it meant having a job or not, sure I guess.

~~~
hlieberman
What if sticking to it meant that you'd have to resign? A code of ethics that
you have to sign for work is one thing. A code of ethics that you hold above
employment is quite another.

~~~
slavik81
Programmers are well-paid and have excellent employment prospects. In the off
chance I did encounter an unresolvable ethical issue, it would not be very
hard to resign and find new employment elsewhere. Given how much better off I
am than most other workers in this country, I think it's a small sacrifice.

To address the thread topic, I'm a P.Eng, ACM member, and IEEE member.

~~~
hlieberman
Are you a P.Eng in software, or something else? (I know there are only a few
places in the United States where you can even challenge the PE exam as a
software engineer).

~~~
slavik81
I'm a Canadian. My degree is electrical, but my field of practice is software
engineering.

------
powera
No! This is the equivalent of saying that kids shouldn't have chemistry sets
or be able to play on jungle gyms. Saying that critical infrastructure needs
to be "bug-free" is one thing, saying that dating site apps need to be is
ridiculous by comparison.

And even in the case of bridges and the like, they do have a tendency to
collapse to earthquakes, wars, and the like. Given enough time, most
everything has bugs.

~~~
mannykannot
> Saying that critical infrastructure needs to be "bug-free" is one thing,
> saying that dating site apps need to be is ridiculous by comparison.

I think you are right, up to the point of security. The problem comes when the
handling of customers' credit cards is treated as if it were just a dating
app.

------
kickingvegas
An old observation, but still relevant: We really have no liability in our
profession, compared to other licensed professions. Why does software quality
suck? Because our customers almost never sue us if the work we do does not
meet expectation.

~~~
SolarNet
I think this is the problem, solving people caring about software would also
make them more likely to prefer more open (e.g. open source) designs.

------
studentrob
> Unlike our brothers and sisters in the other engineering disciplines,
> though, software engineering does not have the same history of mandatory
> formal education.

Was there a time when other engineering disciplines were not formally
schooled? Did they develop into having formal education or were they born that
way? Is computer programming moving in the same direction via organizations
like the ACM?

Can we learn from other fields who have in fact experienced similar issues on
some level? Do we need to reinvent the wheel of creating an effective culture
that rewards all the values humans desire? What are those? Success, ability to
express oneself, ... ?

~~~
Kalium
If anything, with the advent of bootcamps, software engineering is moving away
from formal education and into trade schooling. Personally, I think this is a
step backwards for the discipline.

~~~
TheSpiceIsLife
I resent a little bit.

I'm a boiler-maker / welder by trade (metal fabrication). My trade certificate
actually says "Engineering Tradesperson". I operate a laser cutter, and have
written scripts to automate some of my computer related tasks, and have a
rudimentary understanding of Python. I've also worked at an ISP in NetOps
doing physical security and infrastructure.

I live in Australia so I can only speak for the system we have here: trade
school is _formal education_ , the on-campus component of trade education is
delivered throughout the nation by TAFE[1] campuses. We are formally educated
to build things that don't kill people, and can certainly be held responsible
if our actions or omission of action leads to injury. We are expected to
escalate anything we see on workshop drawings that could be problematic. Each
of the tradesmen in our workshops are required to perform a weld test every
six months, which is examined using ultrasound techniques, to ensure our work
meets or exceeds AS1554[2]

While us tradespeople aren't the ones engineering the bridges, we are the ones
building them.

Comparatively, the software development discipline is still in it's infancy,
whereas the construction trades have been around for millennia. Structural
Engineers and tradespeople build physical structures to withstand Category 5
cyclones, but my bank (in the top 10 globally by market cap) can hardly have a
month go by where some software system doesn't fail.

The advent of bootcamps is probably a side effect of 'coding' becoming
relatively easy compared to it's more esoteric history, with the advent of
high-level languages like Python and Ruby. Or maybe that stuff about the
average IQ increasing over time is correct, and more people have the capacity
to understand writing code. Probably both.

Another comment here spoke about TDD - Test Driven Development, that's
probably closer to the physical engineering disciplines with destructive
testing of random batches of building materials.

1\.
[https://en.wikipedia.org/wiki/Technical_and_further_educatio...](https://en.wikipedia.org/wiki/Technical_and_further_education)

2\.
[https://infostore.saiglobal.com/store/PreviewDoc.aspx?saleIt...](https://infostore.saiglobal.com/store/PreviewDoc.aspx?saleItemID=2247480)

~~~
Kalium
Please accept my apologies.

With the current state of bootcamps in the US, I feel it's a stretch to call
them formal education. They are at best short training programs. There's no
accreditation, no accrediting body, and no real quality control beyond the
press. That the field has few widely accepted standards does not help matters.

Personally, I suspect that the advent of bootcamps in the US is a fad driven
by the explosion of the tech sector and the amount of time it takes to train a
computer scientist. I suspect that in ten years, they will be much less
popular as people who only know Ruby or PHP drop out of the field. It's hard
to have a multi-decade long career in computing if you don't actually
understand computers.

------
zellyn
There's no such thing as bug free code. So yes, I've deployed coffee I knew
had bugs. And if I took twice as long, and spent the extra time checking, it
would still have bugs...

------
rumcajz
Effective code of conduct requires stable environment, people knowing each
other and so on. No way it can be implemented in profession that's booming at
the rate programming is.

------
incanter1
Whoever claims that he never saw his software deployed to production with bugs
has never written anything of any complexity. Adding "... he knew about" to
that statement softens it a little, but not too much.

Problem comparing software engineers to the civil engineers lays in the
ability of the latter to constrain their users, e. g. stating the weight limit
for the bridge. Try to do that with software and enforce it.

------
studentrob
> If we were required to "stand under the arch", as it were, how many of us
> would have pushed back against things that our employers asked us to do?

We can't achieve this consistently with privately owned software. Software
must be public and transparent to have consistent accountability.

In the case of Grindr, for example, I assume we don't know exactly who wrote
the original code. And the company would probably protect that information.

I believe those who contribute to open source software can and do subscribe to
the author's desired level of accountability specifically because they know
everything they write is available for public scrutiny for eternity.

------
cronjobber
Regarding the various calls that it should really be the CEO who gets
executed, I'd wager those ancient bridge builders probably _were_ the "CEO" of
the relevant bridge building operation. Specifically, they probably managed
substantial funds obtained from some Royal and had plenty opportunity to
trade-off between personal profit and structural integrity.

Calls to transfer the idea to today's lowly programmer drone would be akin to
ancient bridge builder absconding with substantial extra profit after
convincing his Royal that it really should be the bricklaying crew who should
line up under the bridge.

------
mirimir
Some Mexican drug gangs _do_ execute engineers for fucking up. And they
recruit by kidnapping family. Hardball capitalism, for sure :(

------
exabrial
140ce? What the hell does that mean?

~~~
satori99
[https://en.wikipedia.org/wiki/Common_Era](https://en.wikipedia.org/wiki/Common_Era)

