
The Most Important Software Innovations (2001-2017) - charlysl
https://dwheeler.com/innovation/innovation.html
======
dwheeler
Author here... thanks so much for the interest!

Clearly there are other things that need to be put on that list. One challenge
I had was finding clear citations to the very first example. It is often very
hard to find where some idea comes from. In addition, sometimes there is no
one single place, because there's this gradual improvement.

~~~
chubot
FWIW I recently read "The Dream Machine" by Mitchell Waldorf, and it has
interesting details that bear on the "1945 Stored Program" entry in your list.

Basically it said that von Neumann was reporting the work of others in
"draft", and he didn't include the people who actually did the work as authors
because it was a draft. In other words, it was prematurely circulated without
proper attribution.

Ironically, I don't remember their names. It may have been Eckert and Mauchly
as you list on your page. If anyone wants to know more, e-mail me (address in
profile) I will look it up.

There was also a patent dispute between von Neumann and the other party, IIRC,
and that is why there is no patent.

So basically the "von Neumann machine" is probably not a great name. You don't
call it that in your page, which is good, but that's how I always thought of
the stored program! I believe I was taught that in computer architecture class
in college.

\----

EDIT: Yes now that I googled, the way the Dream Machine portrayed it, Eckert
and Mauchly had a stronger claim to inventing the stored program than is
portrayed on your page.

You did say von Neumann "described" the stored program rather than invented
it, and that's the issue that is discussed at some length in the book. Some
snippets here:

[https://books.google.com/books?id=7HpQAAAAMAAJ&dq=dream+mach...](https://books.google.com/books?id=7HpQAAAAMAAJ&dq=dream+machine+waldorf&focus=searchwithinvolume&q=von+neumann+patent)

~~~
aasasd
Wikipedia has a paragraph on the topic:
[https://en.wikipedia.org/wiki/First_Draft_of_a_Report_on_the...](https://en.wikipedia.org/wiki/First_Draft_of_a_Report_on_the_EDVAC#Controversy)

Apparently there was a whole team of people participating in the design:

> some on the EDVAC design team contended that the stored-program concept had
> evolved out of meetings at the University of Pennsylvania's Moore School of
> Electrical Engineering predating von Neumann's activity as a consultant
> there, and that much of the work represented in the First Draft was no more
> than a translation of the discussed concepts into the language of formal
> logic in which von Neumann was fluent.

The elegiac Stigler's law of eponymy is mentioned there: “no scientific
discovery is named after its original discoverer.”

------
jondubois
Interesting that "Big O / Complexity theory" is only considered and is not yet
classified as an innovation. It's hugely important and non-obvious. Also
"Hashes" are extremely important. I don't think that Google's 'PakeRank'
algorithm is that special however; it's just applied graph theory; not so
different from Facebook's social graph algorithms.

Also, I don't think that 'Design Patterns' should make the list. Design
patterns depend heavily on the kinds of technologies being used. It's more
like philosophy than science.

~~~
codeulike
Hashing is there as Content Based Addressing.

Design Patterns should definitely be on there. It was very important at the
time. These days we take them for granted because they are baked into the
numerous frameworks that most things are now built on top of.

~~~
bunderbunder
Content Based Addressing is a late, distributed, and rather distant relative
of hash tables. Hashing itself is item #5 under "Appendix: Software
Innovations Being Considered".

------
i_feel_great
I'd personally add bittorrent, distributed ledger, just-in-time compiling,
garbage-collection, homoiconicity, proper tail calls, db transactions, REPL,
FFI.

~~~
nwhm
Well he mentioned LISP and if we include all of it's innovation then garbage-
collection, homoiconicity, tail calls, and REPLs are covered.

------
GuB-42
> Note that there are few software innovation identified in recent times. I
> believe that part of the reason is that over the last number of years some
> key software markets have been controlled by monopolies. Monopolies
> typically inhibit innovation; a monopoly has a strong financial incentive to
> keep things more or less the way they are.

I disagree. It is hard to talk about software innovation without mentioning
Xerox PARC. The PARC existed because Xerox was a monopoly with too much money
on their hands. Kind of like Google X today.

Monopolies in the software industry are actually quite fragile. Today's
monopolies started small and took over the monopolies of the past in a
relatively short amount of time. And the same fate will happen to them unless
they keep innovating.

Tech is not like railways, where you can't build tracks anywhere you want
because of things like land ownership and city planning. It is not like heavy
industry where capital investments are huge and margins are low. Even their
trove of personal data doesn't mean much: these get outdated rather quickly
and machine learning rely less and less on huge datasets (see AlphaZero).
Unsurprisingly, AlphaZero is a Google initiative, that's the kind of
innovation that could be disruptive in someone else's hands, so they invest in
it in order to keep their position.

~~~
TeMPOraL
> _I disagree. It is hard to talk about software innovation without mentioning
> Xerox PARC. The PARC existed because Xerox was a monopoly with too much
> money on their hands. Kind of like Google X today._

Xerox PARC can be understood as putting a bunch of smart people in one place
and giving them essentially basic income. This seems to be a great way to do
research work; beyond corporate monopolies, another successful source of basic
income for researchers was DOD grants during the Cold War. A software monopoly
that actually tries to monetize the research output directly interferes with
this scheme a bit.

> _Tech is not like railways, where you can 't build tracks anywhere you want
> because of things like land ownership and city planning._

We don't have that, but we now have software patents, which are even worse in
the way - not only you have to pay a lot of money to just be able to work,
with software patents it's easy for existing players to just lock out whole
areas of development by speculatively applying for, or buying, patents.

------
gitgud
At a time where there's more software engineers, better communication and
faster development than ever before... Why does it feel like less innovation
is occurring?

Are people less creative when everyone's watching, learning and copying from
everyone else? ...

~~~
bunderbunder
Two thoughts:

First, the low-hanging fruit has already been picked.

Second, it can take time - possibly decades - for an idea to bear enough fruit
that we can see its importance. Look how long the idea of hypertext had been
around before it became a household name.

~~~
megy
Also, those innovations are worth a lot of money these days, so might be kept
secret.

------
systemBuilder
It's embarrassing that many of these supposed innovations are based on the
messenger, not the message. For example my officemate (Andrew Grimshaw, 1990)
published a PhD on a practical dataflow distributed system infrastructure a
decade before mapreduce implemented a subset of his PhD but hyped it to death
from Google, a very hype-driven company in general. These days Apache airflow
is Grimshaw's work on steroids.

~~~
convolvatron
i dont know if its hype. its seems like a very deliberate form of technical
marketing. when I was being programmed to be a Googler, they stood up in front
of us and listed all the foundational contributions the Google made out of
whole cloth, and could only keep repeating 'but, but, but..' to myself. (which
isn't to say there isn't alot of new technical meat behind them, but
collectively they sure do rush to claim exclusive credit)

thanks for the thesis pointer, i went down the distributed dataflow path later
(but also before Google), and I'd love to read it.

------
tannhaeuser
Nice. Didn't know the text although I should have. Would have helped me a lot
in ongoing discussions about design fads with junior devs, though it's not
much fun to be the _nihil sub sole novum_ guy.

Maybe ACID, CAP, and blockchain deserve a honorable mention as well?

------
willvarfar
Nothing new on the list since 2004!?

I'm wracking my brains for something in the NN or AI or something space, but
yet... yeah, maybe innovation is tailing off?

~~~
charlysl
The author aludes to this and observes that as time goes by more software
ideas are getting patented, which was barely done in the old days, and which
he believes slows down the innovation rate.

I guess that maybe it also has something to do with (relatively) low hanging
fruit, as many of the core problems got eventually solved.

As for AI and NN, he has included backpropagation, which is the innovation
that made NN practical. Also, the key innovation that is behind the current
deep learning craze, GPUs, is a hardware innovation, so it's excluded.

EDIT: The author says at the end: _Note that there are few software innovation
identified in recent times. I believe that part of the reason is that over the
last number of years some key software markets have been controlled by
monopolies. Monopolies typically inhibit innovation; a monopoly has a strong
financial incentive to keep things more or less the way they are._

EDIT: From the article: _it’s difficult to identify the “most important”
innovations within the last few years. Usually what is most important is not
clear until years after its development. Software technology, like many other
areas, is subject to fads. Most “exciting new technologies” are simply
fashions that will turn out to be impractical (or only useful in a narrow
niche), or are simply rehashes of old ideas with new names._

~~~
hyperpallium
The way to make backpropagation work properly (for "Deep Learning") was only
worked out a few years ago. However, this article dates it from the first
idea.

Perhaps similar to how relarional databases weren't practical until SQL and
weren't workable until B-trees, but it's dated to Codd (which is fair).

~~~
p1esk
Backpropagation has not changed since the 1986 paper.

~~~
makapuf
The backpropagation principle itaeld is the same. But gradient computations
and techniques to accelerate it or stabilize its convergence have progressed
much

------
darkerside
Very cool list. However, I was surprised not to see lambda calculus by Alonzo
Church on there. LISP seemed to be invented out of thin air!

~~~
dwheeler
I certainly thought about lambda calculus. My current thinking is that is
really a mathematical construct, not software. LISP was where it seriously got
implemented, and I already have that listed.

------
sleavey
"1967: Separating Text Content from Format" \- funny how that innovation is
still frequently ignored!

~~~
TeMPOraL
Also funny how the web people taken it to absurdity. In many cases, format
_is_ part of the content.

------
fergie
Pagerank is (rightly) excluded, but surely tf-idf and graph theory should make
the list? It was essentially the marriage of these two concepts with map-
reduce that allowed high volume search engines to actually work and become the
most important piece of infrastructure on the internet.

~~~
systemBuilder
First of all, Google doesn't use Pagerank any more, it has become a marketing
term for the nearest seeds algorithm they are using.

Second, Pagerank isn't an invention it would be immediately declared as a
pliagarism of academic influence measure by USPTO - that stat has been
measured and reported by the index of academic citations for DECADES.

------
georgeecollins
What about the first algorithm for back-propagating neural networks? I can't
remember who did that but I am sure someone on HN will remember. When Minsky's
"Perceptrons" was written there were all these problems with neural networks
that back-propagation solved. That seems like kind of a big deal. It also
seems like a lot deep learning is built off of or related to that work.*

* I am not that knowledgeable about deep learning so please correct me if I am misinformed.

~~~
stabbles
Out of genuine curiosity, what exactly is novel about this? Isn't back-
propagation just the chain rule (i.e. as old as calculus). And I figure
'gradient descent' was discovered not long after?

~~~
georgeecollins
I am not an expert, but Marvin Minsky wrote a whole book about the kinds of
problems that Neural Networks can't solve, that didn't take into account back-
propagation. No one refuted it, as far as I know for a long time. This may all
seem obvious now but I don't think anyone actually did it until the mid-
eighties.

------
continuations
I'd add a few more:

\- Gesture-based UI: this forms the basis of the smartphone industry

\- sync communications like IRC, chat, voice, video chat (since email which is
async is included)

\- Consensus algo like Paxos and Raft

\- Computer game: Spacewar

------
systemBuilder
Personally, I think C deserves mention, not because it became THE portable
language after 8 x n-bit implementations became standard, but rather, it was
the first neutral, agnostic language about control structures (for, while, and
do .. until) and argument-passing methods (call by name, value, reference,
...), and block-terminating methds (begin .. end? Begin ... End.? no, {}=;)
and it buried all the idiotic academic flame wars claiming one of each choice
was THE ONLY "right" way to build a programming language. C was also the first
language to popularize access to nearly the entire instruction set of a
machine with it's rich set of operators. C++ DOES NOT deserve mention because
it once again tried to "force" programmers to follow "the one true way" to
write object oriented programs, by outlawing a number of useful programming
constructs that Stroustroup judged unworthy (such as consts without storage in
base classes, which could not be subclassed so he nuked them even though they
are useful, e.g. for the gravitational constant etc.)

------
diego_moita
This list makes me think about the biggest hypes, innovations that would be
important but never fulfilled the hopes people placed on them (e.g.: expert
systems, digital assistants, perhaps cryptocurrencies)

~~~
aasasd
Pretty sure digital assistants are on the rise right now. Afaik expert systems
are used in medicine, and I suppose some chat bots qualify as them too, though
I think people will try to drop manually programming the rules at some point.
Since we're finally marching straight into a scifiey cyberpunkish future, I'm
sure these two things will be around.

------
kaptain
I'm surprised ASCII/Unicode or some sort of encoding isn't on the list.

~~~
zellyn
Did you not search the document for "ASCII"?

------
rozim
May I suggest...

Log-structured file systems (and append-only I/O)

Alpha-beta search

TF-IDF

------
hyperpallium
The rate of progress increased exponentially, peaking somewhere in the
60's-70's. The singularity is over.

------
kstenerud
diff

Cryptocurrencies

Distributed file transfer (such as bittorrent)

Tor

~~~
fnord123
Certainly not cryptocurrencies. They're still a solution in search of a
problem.

~~~
fouc
the blockchain is essentially a solution to the Byzantine Generals' Problem

~~~
singularity2001
PS: The Two Generals Problem was the first computer communication problem to
be proved to be unsolvable.

------
revskill
I would add ReactJS to the list.

~~~
tuukkah
Perhaps virtual DOM could be on the list as an innovation used in React and
Vue: [https://reactjs.org/docs/faq-
internals.html](https://reactjs.org/docs/faq-internals.html)

~~~
makapuf
While an important concept and useful technique , I find it nowhere near as
fundamental as compilation, the mouse or datagram network protocols. Maybe a
more fundamental principle is used by virtual dom that can be the real
innovation?

~~~
tuukkah
I totally agree. Purely functional components are more fundamental but even
that might not warrant a place on a list like this although it's significant
in the JavaScript UI realm today.

