
Why Software Engineering Isn’t Engineering - josep2
http://blog.iancackett.com/2015/06/07/software-engineering/
======
kgrin
"Software engineering estimates and plans often fail to live up to the reality
that follows. It seems to be the only engineering discipline in which this is
regularly the case."

Is it though? I live in Boston, home of the notorious* Big Dig[1]. While
particularly egregious, it's far from the only large-scale civil engineering
project that's gone off the rails. In fact, I'd argue that until fairly
recently, many more public works projects shared the "surprise factor" of
software projects. I'd recommend Caro's "The Power Broker"[2] for a
fascinating history of NY-area public works (among other things - great book
all around), including how much of that process was about adapting the plan to
new things the builders were learning along the way ("oh, turns out that soil
is _completely different_ than we planned...")

That's not to say that there aren't particular features that make software
engineering its own special snowflake - as there are meaningful differences
between how civil, structural, mechanical, etc. engineers operate. But spend
some time in another engineering organization and you'll find it's different,
but not as different as you think it is.

(And FWIW, even civil engineers sometimes follow "agile" concepts - a company
I once worked for was contracted to design a highway, and even after the
construction started, engineers were "embedded" with the builders to make on-
the-fly adjustments based on the environmental factors they discovered
throughout the process... I wish I could find their project write-up, but it
was a while ago and the company has long since been gobbled up by a bigger
company).

* As a (subjective) kicker, I'd add that the Big Dig, over-time and over-budget as it was, was ultimately quite worth it... much like many software projects!

[1]
[http://en.wikipedia.org/wiki/Big_Dig](http://en.wikipedia.org/wiki/Big_Dig)

[2] [http://www.amazon.com/The-Power-Broker-Robert-
Moses/dp/03947...](http://www.amazon.com/The-Power-Broker-Robert-
Moses/dp/0394720245)

~~~
iancackett
Yep, you're right... I'm forgetting that physical engineering also suffers
from some of the same problems. The thing that it adds though is a level of
rigour, and ability to reason about the project, that we're still struggling
with somewhat in the software world. Believe me, in 20 years since I was
taught it's not strict "engineering", I've yet to see that change much, other
than in safety-critical and cleanroom projects.

~~~
StavrosK
Engineering is much more straightforward, you have the plans and know that you
need X amount of parts and Y amount of labour. Software is more like research,
where you don't even know if you can solve the problem acceptably, most of the
time.

~~~
vonmoltke
If you have the plans, it's manufacturing, not engineering.

The engineering in the design is in developing the prototypes and proving that
theoretical concepts can be implemented practically. There is nothing
straightforward about that, unless it is an incremental improvement to an
existing implementation.

The engineering in manufacturing is about deriving ways to manufacture parts
with greater yields and greater efficiency. In some cases, it is about
developing manufacturing techniques that were hitherto impossible. Again, not
much straightforward about this.

All of the above is _rigorous_ , but that is not the same thing. This rigor
can be applied to software development just as easily as any other product
development, and when it is it becomes software _engineering_.

------
jasode
_> projects delivered precisely what they said they would, and in the
timeframe they originally promised._

Those factors are " _project management_ " which include concepts like
"scope", "budget", and "critical path".

It's arguable if those are "engineering". They certainly _affect_ engineering.
(And likewise, some engineering constraints can feed back into project
planning and feasibility.) In any case, project planning is a separate area of
study. Using the author's strange definition of " _engineering_ ", it means
the NASA Space Shuttle is "not really engineering" because it didn't deliver
to the specifications of 50 launches per year at low cost.

A more reasonable definition of engineering that most could agree on would be,
" _the study (or art) of balancing technical tradeoffs against real-world
constraints_ ".

~~~
iancackett
As I'm commenting elsewhere, this is true... I didn't acknowledge that other
physical engineering projects do indeed suffer from some of the main problems.
What's missing in software engineering is the rigour and ability to reason so
precisely about what's being built. I was originally taught (on a Software
Engineering degree course) that it doesn't formally fit the definition of
engineering, and I've not seen that change much, but I'd agree the definition
is certainly up for grabs, and perhaps extension to include modern software
practices, such as Agile.

~~~
mason55
> _What 's missing in software engineering is the rigour and ability to reason
> so precisely about what's being built_

That's only true if you choose for it to be. There are ways to formally prove
that your software is correct but they require a large time tradeoff. For
example, the software in the chip in your car has gone through as much
engineering rigor as a bridge. Alternatively, look at a cheap toy produced in
a shitty factory in an undeveloped country. It will have parts in it that were
designed by a mechanical engineer but they choose to be less rigorous to keep
cost savings low and as a result you get a toy with "bugs" in it.

~~~
EliRivers
_For example, the software in the chip in your car has gone through as much
engineering rigor as a bridge._

We'd like to think so. I currently do some work for a static analysis company
that has many, many customers in the automotive industry. The MISRA rules are
part of the standard package bought by these customers, and an awful lot of
MISRA violations will be caught by the analysis.

Then you get the example of Toyota, who I am pretty sure are one of the
customers of my current employer (and some of their subcontractors/suppliers
are also customers). They definitely had the opportunity to identify the
failings in their software (some of which would definitely have been caught;
recursion, for example) and either ignored the results, or just plain didn't
bother.

------
falcolas
You see this sentiment come up about every few months here. We're not rigorous
enough. Or we can't estimate our build time well. Or we don't have any
liability. Or we don't have a codified set of morals that govern us.

BS.

We're still exploring the space, and most of the time the things we build are
good enough to work - just like the first wooden planks across streams were
good enough.

We also do have the capability of designing and building the software
equivalent of suspension bridges across the bay, using tools and processes
like Ada, Coq, formal software proofs, etc.

Most engineering projects don't live up to your high standards, either. I knew
a Blacksmith who would make circular staircases, and would regularly have to
work around floors which were 4-6" higher or lower than specced. Or bridges
which were put into earthquake zones, but not designed to withstand
earthquakes. Or highways built with inappropriate foundations that buck like a
bronco with frost heaves after the first winter passes. Or cities built under
the freaking water table.

Ease up here folks. Our tooling and processes are still undergoing growth, as
we take in the incredible scope of what we are capable of. Rome wasn't built
in a day, and the understanding of stoneworking, woodworking, and city
planning which enabled the building of Rome certainly took more than the < 100
years we've had.

~~~
lambdaelite
> You see this sentiment come up about every few months here. We're not
> rigorous enough. Or we can't estimate our build time well. Or we don't have
> any liability. Or we don't have a codified set of morals that govern us. BS.

I don't see the BS. Coming from a traditional engineering discipline, I'd
agree with each of those. The scariest to me is on professional standards and
ethics: it seems to be an almost point of pride among some HN commenters that
they operate outside of any formal standard. That baffles me.

~~~
falcolas
> The scariest to me is on professional standards and ethics

OK. What standards and ethics would you recommend we adhere to?

First, do no harm? One, define harm. If I'm depriving a corporation of money
by automating something for consumers, is that harm? How about if I write a
clone of netcat, am I liable for its use in nefarious scanning of ports?

Thou shalt not write bugs? That would be awesome, were it possible. I can't
think of many developers who wouldn't want to get to this point, assuming of
course it didn't result in a 1 month project plan to write a sorting
algorithm.

[http://googleresearch.blogspot.com/2006/06/extra-extra-
read-...](http://googleresearch.blogspot.com/2006/06/extra-extra-read-all-
about-it-nearly.html)

~~~
lambdaelite
The IEEE-CS/ACM jointly publish one:
[http://www.computer.org/web/education/code-of-
ethics](http://www.computer.org/web/education/code-of-ethics)

It doesn't seem that different from other professional codes.

------
JoeAltmaier
Building a bridge of a million bricks - most of them do the same thing. A
software project may have a million moving parts, all of them doing different
things. The comparison is silly.

I'd use the original moonshot as a better example. It was hugely complex,
unpredictable, late, had a large number of last-minute problems. They
essentially used Kanban to solve it - hanging a drawing of the rocket on a
large conference room wall, and taping notes to each place where there was a
problem; reviewing each note at a 'scrum' meeting each morning.

So Engineering has been working this way for at least 50 years. There's
nothing to see here folks; move along.

------
iancackett
Wow, so, Hacker News... didn't see that coming. Thanks for the comments, good
and bad, folks.

One thing I certainly agree with: Regular physical engineering isn't entirely
free of many of the problems I mention with software engineering, but the
ability to understand and plan more effectively does appear to be there. The
complexity in physical engineering does seem to be a little more tangible. I
know a few civil and mechanical engineers, so I'm not plucking this from my
behind ;-)

In terms of the definition of engineering, and the argument that software
engineering isn't "engineering", I was taught this way back at university
(City Uni, London), in their Centre for Software Reliability, and it's
something I've largely agreed with. However, I think it was mainly used as a
warning mechanism for newbie software "engineers" like me, to let us know that
what we do is significantly different from other forms of engineering, in
terms of the rigour. Perhaps it still deserves the title, but I guess that's a
longer discussion.

------
penprogg
So I decided to search the definition of engineering on the internet because
I've heard this whole "software engineering is not engineering" thing before:

> Engineering (from Latin ingenium, meaning "cleverness" and ingeniare,
> meaning "to contrive, devise") is the application of scientific, economic,
> social, and practical knowledge in order to invent, design, build, maintain,
> research, and improve structures, machines, devices, systems, materials and
> processes.

So, software "engineering" definitely involves the use of scientific,
economic, social and practical knowledge to build, maintain, research, and
improve devices, systems, and processes. So that, in my lowly opinion, makes
it engineering "in the strictest sense of the term".

Arguing about semantics is extremely stupid if you are not a PHD in
linguistics.

~~~
orblivion
I don't think it uses science, other than scientifically studied development
processes, and how many start ups use thast? If you count computer science,
I'd kick the can down the road and ask if computer science is a real science?
Does it use the scientific method? It seems more as applied mathematics.

~~~
kdoherty
Science involves the application of the scientific method. Take academic
research, for example. Engineering may involve the scientific method, but it
isn't required to validate that scientific knowledge was used. Scientific
knowledge could be an algorithm for autonomous environment mapping.
Implementing that in an autonomous car and ensuring system functionality in
the many possible environments it might end up in, that is certainly a feat of
engineering.

------
nickpsecurity
I like that he included the counterpoint to his position early on in the
article (Cleanroom etc). The article is correct that our type of engineering
is a bit different with all the interactions between components and the fact
that they're often different. Yet, there's a subset of the field that counters
these problems through choice of development style (eg Cleanroom) with well-
understood components with consistent interfaces. The style is fairly easy to
pick up but the good components are built up incrementally over time.

Truth is, manager's and developer's choices are why most of this isn't the
case. Management might push use of a problematic technology, give too little
time to assure quality, and so on. Developers might care little about making
things maintainable, prefer a style which impedes it, or use fad technologies
with tons of unknowns. The combination led to pervasive issues in the industry
the article references.

The solution is for quality-centric IT shops to differentiate by going in the
opposite direction. The same for FOSS projects. Fortunately, we see a little
bit of this here and there. Altran-Praxis was delivering engineered software
via Correct by Construction methodology. One student's combination of Python
and Cleanroom was promising. Ada and Eiffel communities are using Design-by-
Contract to aid predictability/maintenance. So, we see pieces of how vanilla
IT shops & FOSS can raise the bar closer to engineering. We just need
courageous groups to jump and grab it.

------
rb2k_
I don't have a particular dog in this fight, but I thought this is
interesting:

The last O'Reilly Software Architecture Conference had a talk by Glenn
Vanderburg titled "why software development is an engineering discipline":

[https://www.youtube.com/watch?v=zDEpeWQHtFU](https://www.youtube.com/watch?v=zDEpeWQHtFU)

I think the author of the blogpost does a lot of the things discussed in the
talk (e.g. comparing it mostly to civil engineering as opposed to all the
other engineering disciplines).

~~~
iancackett
Yes, very true. And I am also comparing the startup flavour of software
development (which I am involved in), rather than perhaps other flavours, such
as finance, health care, etc.

------
makeitsuckless
> "Software engineering estimates and plans often fail to live up to the
> reality that follows. It seems to be the only engineering discipline in
> which this is regularly the case."

Yeah... nearly stopped reading after that opening statement. There are many
reasons why software engineering is different from other forms of engineering.
And maybe those reasons are enough to not call it engineering. Personally, I
don't think that's particularly relevant.

But wildly exceeding estimates is definitely not something that distinguishes
"real" engineering from software.

Real world engineering often has the advantage of working almost exclusively
with known quantities. But add a variable, like digging a tunnel through an
unpredictable underground, and sometimes projects go off schedule by _years_.

No engineering discipline can predict the unpredictable. Some disciplines are
just younger than others, and therefor have more unpredictable elements.

The big difference with software engineering is that the costs of failure are
so relatively low that we prefer to go full steam ahead instead of trying to
find ways to reduce uncertainty.

This makes software engineering an outlier. Everything else is just semantics.

~~~
lambdaelite
> The big difference with software engineering is that the costs of failure
> are so relatively low that we prefer to go full steam ahead instead of
> trying to find ways to reduce uncertainty.

I think this is an excellent point. Software is treated with a different level
of rigor and respect on safety-critical projects. Same for embedded systems
where the cost of updating a system is prohibitive.

I do think that there is an engineering discipline in software, just that most
people aren't practicing it.

------
j_m_b
I think "Software Engineer" is just a title given by a place of employment.
However, for those who genuinely believe themselves "Software Engineers" I
would ask: What was the outside institute that certified your software
engineering program? What professional exams did you take in order to be
licensed as a software engineer?

~~~
intortus
Sorry, I meant to say software _architect_.

~~~
iancackett
That's a great point, and one of the reasons I was originally taught it isn't
"engineering", strictly. I think we do a lot of what counts as engineering.
But, you're right, we aren't formally certified as an engineer. Perhaps that
(and the required rigour) will change.

~~~
walshemj
It depends you can jump trough the hoops to get Ceng status but the one of the
problems is the BCS/IEEE keep changing the goal posts for those going the
industry experience route

The other bigger elephant is what in it for me what have the BCS / IEEE
actually done anything to improve the status of engineers.

In the UK Jeremy Clarkson's 1 hour documentary on Brunel has probably done
more for the profile of engineers than the IEEE has done in the last 25 years.

------
graeham
I think the biggest difference is tradition engineering systems are mostly
linear, while software rarely is. Even non-linear mechanical or electrical
systems can typically be modelled.

Modelling is a key activity of engineering - it allows predicting the
behaviour of a final system. Linear systems are handy because they are
scalable. A bridge to hold 10 people can be scaled to hold 100, 1000, or
10,000 people (usually) quite easily. A software processing scaling from
10->10,000 will often fail in all sorts of interesting ways in the process.

Software often fails silently. In mechanical systems, there is often noises,
vibrations, or yielding to give warning and insight to where problems are
happening. Software often doesn't have this inherently, but can be overcome
with debugging and testing.

I think a large part of this is the newness of the field. People were building
bridges, houses, and carts long before Newton. But after we had the modelling
tools and theory, we were able to go much further.

(I'm a mechanical engineer, who tends to do a lot of software for controls and
models, and increasing web development)

~~~
iancackett
Great point. Modelling and understanding what's being built are certainly
important aspects of engineering.

------
taylodl
We've been talking about this for over forty years now and it just hit me -
engineers are categorized by what they _build_ whereas software engineers are
categorized by what they _build with_. If you asked an engineer who's an
expert at building bridges to then build a skyscraper, they're likely to have
estimates that are off-base. Meanwhile in the software world we say we're Java
developers or Ruby developers or Haskell developers are whatever. The field is
only now starting to really mature to the point where we can start
categorizing different kinds of software projects and specializing in
different practices.

~~~
iancackett
Great point. Perhaps we'll be able to classify certain software practices as
"engineering". Maybe even stricter forms of Agile. It would be great to head
in the direction of it becoming engineering, rather than in the opposite
direction.

------
akgerber
A couple counterexamples from civil engineering:
[http://www.businessinsider.com/why-the-7-line-extension-
stil...](http://www.businessinsider.com/why-the-7-line-extension-still-isnt-
open-2015-4)
[http://en.wikipedia.org/wiki/Line_C_%28Rome_Metro%29](http://en.wikipedia.org/wiki/Line_C_%28Rome_Metro%29)

------
websitescenes
Software engineering is not engineering, yet you call yourself a software
engineer?

------
sebnap
I think this is one reason why you don't get an engineering degree in Germany,
after completing computer science studies. It is more seen as math or physics,
just a natural science.

------
shogun21
His argument against calling it "engineering" is because we don't properly
estimate our time?

To me, engineering is a way of thinking, it's all about problem solving.

~~~
lambdaelite
> To me, engineering is a way of thinking, it's all about problem solving.

Interesting. I would say it's all about the problem solving _process_.

------
tsunamifury
This guy seems to be stating a made up tautology due to the fact that he is
redefining eng.

According to this guy we do mostly design and specification. Of course then
you could say the Bay Bridge, the Space Shuttle, the 787 and every product
that requires a long design and exploration pathway is not engineering. I
don't know of anyone at Google who considers dev the same as mechanical
engineering. It's more of an exercise in managing complexity, mathematical
optimization, social science and design theory. But we sure as heck aren't
designing for the knowns required to do what this guy defines as engineering.

But so what?

~~~
iancackett
I'm simply going with the definition of true "engineering" taught to me as a
software engineering major at university... and specifically that we don't do
that formal definition of engineering. Of-course, other fields (mechanical,
civil, etc) don't always follow all the formalities either, but there are ways
of modelling that simply aren't available or aren't used by software
engineers.

------
wai1234
This is possibly the silliest post I've ever read. Apparently you don't know
the definition of engineering and, most astonishing, claim building projects
are always perfectly estimated. I doubt ANY building project ever finished on
time or under budget. If everything were tidy and well understood, it wouldn't
be engineering. Stick to software, Ian.

~~~
iancackett
Having studied Software Engineering as a degree, I can assure you they teach
software engineers the definition of "engineering", and stress that we are
practicing something else or, at a stretch, an extension of engineering
somewhat different from civil or mechanical. I realise that building projects
probably suffer from many of the same weaknesses, but the degree of rigour,
and ability to reason about the project, is definitely way higher than on
software projects. I knew a few civil, mechanical and aeronautical engineers,
and they tend to agree. That said, definitions like this can be hard to pin
down!

------
mikekchar
In my opinion, software, if it is a form of engineering, is a very strange
form. A mistake in one part of the code can potentially affect any other part
of the code. Not only that, but the effect it has is practically unbounded. If
we were making a car, then it's a bit like putting in a cup holder causing the
fuel tank to explode.

We can't efficiently reason about our designs because we can't efficiently
build from reusable parts that are known to work. Sure that cup holder works
in every body else's car, but it makes our fuel tank explode. There is no
equivalent to things like "tensile strength" that we can calculate to say,
"This will be good enough for this project, but for a different project we
should use something else". We just shut our eyes and pick our components
based on what the most annoying person in our team wants to use.

Many organisations make the mistake of thinking of software development as if
it were like building a bridge. We think that we can choose a framework and it
will _reduce_ the amount of thinking that we have to do. Then half way through
the project we are madly trying to hire Rails experts because nothing works
right and it has something to do with the internals of Rails (which we were
trying to avoid understanding).

Putting aside the gargantuan task of gathering requirements for a second,
building software sometimes has the feel of building something physical.
Because software is seen as a thing that people utilise, they think that we
will eventually use the same techniques that we use for building physical
object/systems. The reality is that software does not follow laws in the same
way that a bridge or a car must follow physical laws. You make up your own
laws in software and the only thing that is important is internal consistency.

It's a bit like creating a universe for your bridge to live in. The success of
your bridge depends on whether or not you chose sane rules that other people
could understand. If we were to think of it as engineering, we would need
"engineers" who studied every software system so that they could understand
the "laws of physics" (which explains why Rails developers get paid so much!).

BTW, I'm picking on Rails for no good reason. You can insert whatever
framework/library/language/system you currently hate because it really doesn't
matter all that much. Each one encapsulates it's own universe and requires us
to study it to understand how it works. Our ability to extrapolate from one
system to another is dependent upon whether or not the developers actually
chose to imitate each other or not.

On the other hand, we have some advantages over engineering. Our universe is
made up. If we choose we can limit the rules to only things that we understand
very well. We also get to see the source code (unless you work with
proprietary systems, in which case you have my undying pity). We don't have to
discover the laws of physics by experimentation (hmm... it doesn't stop some
programmers, though...). The laws might change from day to day, but we can
even write so-called "tests" to alert us when some idiot has inadvertently
changed the gravitational constant and caused the universe to implode.

Personally, I think the differences between programming and engineering are
big enough that we lose a lot by hoping that it will become engineering. For
some reason there seems to be a desire to call programmers "engineers". I hope
this trend reverses and we embrace a new discipline that is more suited for
our needs.

------
mpdehaan2
Estimates aren't so much of a concern for me as quality.

While bridges can fail after 50 years, the general state of many software
applications (particularly in enterprise software) is seemingly less good.

I primary issue is lack of standardization and rigor in the field that allows
for quantifying allowable defects, test coverage requirements, and
understanding of failure scenarios. Simply put, so much is played very fast
and loose. Fragile systems are sometimes built in haste on fragile
foundations.

In say, electrical and building codes for houses (not engineering so much, but
applicable), there are established standards for protection of consumers and
standardization of work. In software, usually these don't exist - or they
exist in small legal areas like PCI or HIPAA, and don't really explain how the
software is built either, but only some of the properties. Not only do
building codes exist, but the similar codes and standards exist for the parts
the technicians install.

Rather than codify the practices of the craft that firm things up, everybody's
still trying to figure out what those practices are. And maybe that's ok.
Bridge building has been done for thousands of years, and software has been
done for far less than a century.

We are still figuring a lot out. Yet, at the same time, I don't think we
remember much from history, software that has "nailed it", and analyze what
works and doesn't.

Software is also kind of not engineering because it's an expressive medium to
some, kind of an art, hence the application of "craftsmanship" frequently
applied. We are sometimes inventors, sometimes engineers, sometimes sometimes
carpenters, sometimes plumbers.

These are all valid fields, but I do want for greater engineering rigor over
the long haul. I think it would make things less stressful. But right now, we
(as the people in the industry) are doing all these breadth first forays to
figure out how software is built - sometimes getting stuck in local minima and
maxima until we can upset an ideology enough to try something new - and that's
partly why it feels like it does. We have a lot of people with different
experience levels and different specializations, and often conflicting
opinions, where sometimes none are clearly right or wrong.

On the other end, business also needs to change. A bridge is never "sold
quickly", it is sold and then takes as long as it takes. Electronics can be
pitched heavily, but the cost of field replacement is so remarkably large it
must be gotten right the first time. However, software can quickly be replaced
on the fly. More so a problem if you are working on a .com than on firmware,
the need for rigor is reduced and engineering deliverable is more controllable
by the desires of the business side.

Ultimately, it's still a remarkably new industry, and it itself is able to
evolve quickly, as we are not realing dealing with the rules of chemistry, but
rather conceptual ideas and logic. Logic and ideas are fuzzy complicated
beasts.

~~~
iancackett
Some great points, thanks!

I think "we're still figuring a lot out" sums it up for me. Perhaps it will
become more rigorous over the next few decades. Perhaps we'll start to see
more parallels with physical engineering.

You're right that quality is the key.

