
E. W. Dijkstra Archive: “Why is software so expensive?” (2009) - tomaskazemekas
https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD648.html
======
markbnj
It's a good piece, and Dijkstra's views on this remain relevant, even if the
whole thing is ultimately an exercise in trying to build an ornate analogy
that will somehow survive translation to other realms. Anyone who has been in
software long enough to have tried to explain it to someone who isn't in
software has had to go through the same basic process. That in itself ought to
be somewhat enlightening. In any event, the article is worth the read if only
for this:

>> How do we convince people that in programming simplicity and clarity —in
short: what mathematicians call "elegance"— are not a dispensable luxury, but
a crucial matter that decides between success and failure?

Amen, brother. Great question.

~~~
crdb
I disagree. There's a realm of fields to which software engineering applies.
There's fields where correctness, elegance and formal methods definitely
matter; those involve complex products like airplanes (RIP, A400 victims...)
or nuclear plant control systems. Or academia, which cannot progress with
imprecise thinking. You can't run Facebook's operations the way you'd run a
bakery's website's.

But then there are heaps more fields where correctness absolutely does not
matter, where a shiny cover over an awful spaghetti ball is enough, where you
can lose half the client's data and say "sorry, crash" and he'll just nod and
shrug, and where this is a lot cheaper and faster than hiring a professional
and doing things properly, where the determinant of success is not the
engineer, but the management team, their connections, and the sheer amounts of
money that investors are willing to pump into the company to win the market
(money they will mostly give only to the first competent person that pitches
them the idea, giving the pretty-"MVP"-in-a-week team the edge over the
formals).

It's a trade-off between time and cost to market, and product quality; it
won't change until the second way beats the first on the first two dimensions
as well as the last.

~~~
AnimalMuppet
And what's ironic is that EWD's preferred approach actually increases the cost
of (at least the first version of) many pieces of software. And his article
was "Why Software Is So Expensive".

I mean, his approach would have prevented Microsoft from ever existing. Or
Linux. Or Apple. Which means we never would have had the PC revolution. We
would just have a software industry that produced software _right_ , and code
that didn't crash, but we'd have maybe 1/100th of the software industry that
we have. We'd miss 99% of what computers have brought us. I'm not willing to
make that trade-off to get software that never breaks, even though I hate it
when software breaks.

~~~
crdb
I'm not sure one could describe Microsoft as "fast and dirty". Gates' BASIC
interpreter is a famous example of technical ability...

~~~
AnimalMuppet
But _Dijkstra_ would oppose Gates' BASIC interpreter, because it wasn't built
using Dijkstra's approach of mathematical reasoning, and therefore couldn't be
trustworthy.

------
emilburzo
>> One of the problems caused by the non-understanding software manager is
that he thinks that his subordinates have to produce code: they have to solve
problems, and in order to do so, they have to use code. To this very day we
have organizations that measure "programmer productivity" by the "number of
lines of code produced per month"; this number can, indeed, be counted, but
they are booking it on the wrong side of the ledger, for we should talk about
"the number of lines of code spent".

Amazing how we still haven't figured this one out yet.

>> I know of one —very successful— software firm in which it is a rule of the
house that for a one-year project coding is not allowed to start before the
ninth month! In this organization they know that the eventual code is no more
than the deposit of your understanding.

It's so hard for me to imagine that such a software firm could actually exist.

Even though I've noticed the advantages of thinking first and coding later on
a personal project done in a more relaxed period, I guess it's just too hard
to justify your cost without having deliverables (code commits, in this case).

~~~
HenryTheHorse
> for a one-year project coding is not allowed to start before the ninth
> month!

And here we are, asking team leads why development cannot begin till week 2 of
a 24-week project.

It is a really strange and funny industry we are in.

~~~
psykovsky
I suspect managers and programmers might be counting the weeks from different
points in time...

------
noblethrasher
Why is the title labeled "2009"? I presume it's because that's when the
_transcription_ was last revised, but the actual essay was published in 1982;
anything written by Dijksra cannot be more recent than 2002.

------
stephengoodwin
Minor correction: This was publish in 1982[1] and was transcribed from a PDF
in 2009[2].

[1]
[https://www.cs.utexas.edu/users/EWD/ewd06xx/EWD648.PDF](https://www.cs.utexas.edu/users/EWD/ewd06xx/EWD648.PDF)

[2]
[https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E...](https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD648.html)

------
aidenn0
This was written in the late '70s, not 2009:

[https://www.cs.utexas.edu/users/EWD/indexChron.html](https://www.cs.utexas.edu/users/EWD/indexChron.html)

------
thewarrior
While Dijkstra sits in his ivory tower , patiently figuring out the most
elegant algorithm , in the startup world as Zuck says is all about moving fast
, breaking things in order to find out whether we even need the algorithm in
the first place.

When my boss comes to me with the next story , I can't tell him that , "Hey ,
I think I need two weeks for this as the system is becoming overly complex. I
think I need some time to refactor the system into something simpler and more
reliable .. "

I've seen an example of this first hand. We had an image editor that allowed
users to manipulate and rotate layers using like you have in Photoshop.
Instead of using straightforward affine transforms , they came up with some
weird , extremely complex implementation I'm yet to get my head around .It
isn't even documented as to how it works.

Finally when the time came to make it behave differently , I reimplemented
only the new behaviour using affine transforms and simple trig and routed the
flow through the old code or the new one using flags.

Not the most elegant solution but that's how it is in real world projects.
It's only the output that matters. The internal complexity is not something
the customers gives a shit about. They want their feature and they want it
now. And they're not even sure what they want.

So the need for rapid iteration to discover what you need to build in the
first place , and the quest for the most ideal and mathematically elegant
solution will always be in conflict with each other.

~~~
jacquesm
Dijkstra did a lot more for moving the programming needle than Mark
Zuckerberg. Sure, Mark made more money, but that's not the only yardstick by
which we measure achievement (fortunately!).

Without a bunch of people doing basic research _none_ of us would have jobs.

~~~
thewarrior
I have great appreciation for people like Dijkstra and Knuth and the work that
they do. They write operating systems , compilers and databases.

But coming down from the ivory tower of academia , do you really need to go to
the trouble of writing your custom CMS in Haskell after proving it's
correctness in Coq ?

As a further example , the succesful company KissMetrics built the first
version of it's product in a month and shipped using SQLite (!!) as a DB.

In their own words , they optimized 100 % for shipping speed.

Software doesn't exist in a vacuum , it plugs into a business.

The amount of rigor and research that must go into a piece of software depends
on the domain and business requirements. To insist that all software be
written to such a high standard is unrealistic and will never happen.

I'm yet another software engineer who (most of the time) writes CRUD apps as
quickly as possible so my boss pays me on time. I have no pretensions of being
a 10x or being the hero who drags the world out of the dark pits of poorly
specified and mathematically inelegant software with his superior intellect.

~~~
vezzy-fnord
_But coming down from the ivory tower of academia , do you really need to go
to the trouble of writing your custom CMS in Haskell after proving it 's
correctness in Coq ?_

Dijkstra was nothing like this. In fact, he was probably one of the more
practical academics (if that even makes sense to some people). He advocated
top-down, logic-based methods for proving correctness that were based on
incrementally transforming a high-level program specification into a final
product fit for implementation. He likened it to composing music.

He also deeply studied paradigms that were firmly imperative in nature.
Besides his work on semaphores for concurrency control, his deductive system
of predicate transformer semantics and the Guarded Command Language built on
top are some of the most comprehensive notations for imperative languages. He
was also on the team that developed ALGOL 60, so it's quite likely you owe
most languages you've used partially to him.

That and your conception of an academic using the analogy of the "Haskell CMS
verified by Coq" is largely a straw man, though I can understand why you used
it. HN is pretty crazy about these things, so it's easy to get caught up in
the thought bubble thinking it's the norm.

------
ExpiredLink
The software 'industry' moved in the opposite direction.

