
HPC is dying, and MPI is killing it - ljdursi
http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/
======
dalke
I came into the essay with suspicion. A map-reduce system like Hadoop isn't a
good fit for HPC problems, and I thought it would argue that MPI is old =>
it's stuck in the past.

Instead, and to my joy, it was a well-reasoned essay with good, solid points.

My only quibble is that Charm++ is not "a framework for particle simulation
methods". While the molecular dynamics program NAMD has been using it for 20
years, which is why I know of Charm++, it wasn't designed specifically for
particle simulation methods, nor is restricted to that topic. Quoting from
[http://charm.cs.illinois.edu/newPapers/08-09/paper.pdf](http://charm.cs.illinois.edu/newPapers/08-09/paper.pdf)
:

> NAMD, from our oldest collaboration, is a program for biomolecular modeling
> [2]; OpenAtom is a Car-Parinello MD program used for simulation of
> electronic structure (in nanomaterials, as well as biophysics) [3]; ChaNGa,
> an astronomy code [13]; and RocStar, a code for simulating solid-propellant
> rockets, such as those in the space shuttle solid rocket booster

~~~
ljdursi
Thanks! I'm probably mischaracterizing Charm++ a bit, because I'm most
familiar with it in particle context (OpenAtom, ChaNGa, NAMD). I guess it's
probably particularly used in that context just because it's so good with very
fine-grained distributions of work units. I'll edit that line in the article.

~~~
semi-extrinsic
Another nitpick: 128 cores equals ~ 4 nodes only if your problem is not bound
by memory bandwidth. If it is, 128 cores equals ~ 16 cores, and then the
interconnect matters a lot.

Great writeup though. I do think we need to get more people into the mindset
that MPI won't be the standard in 10 years, otherwise it will still be the
standard in 10 years.

~~~
ljdursi
"won't be the standard in 10 years, otherwise it will still be the standard in
10 years." \-- nicely said.

------
ToastyMallows
MPI is Message Passing Interface, for those who don't know. Author never
explicitly says what MPI stands for.

[https://en.wikipedia.org/wiki/Message_Passing_Interface](https://en.wikipedia.org/wiki/Message_Passing_Interface)

~~~
kenko
"MPI, the Message Passing Interface, began as a needed standardization above a
dizzying array of..."

The capitalization and apposition makes it pretty explicit.

~~~
ToastyMallows
I triple checked before posting this, looks like someone edited the blog ;)

------
acadien
I've heard there is a new effort being led by Torsten Hoefler to modernize MPI
and address a number of the issues mentioned in this article.

[http://htor.inf.ethz.ch/](http://htor.inf.ethz.ch/)

I was at a talk of his last year and there are a number of fault tolerant MP
algorithms being drawn in. MPI hasn't been updated in ages, I don't think that
necessarily means we need to ditch it, it just means the standard needs to be
modernized. I don't feel very strongly about this since working with MPI is a
huge pain in the ass and it seems like the challenge of modernizing it is just
gargantuan.

Also I'm not familiar with spark, but isn't Chapel a decade old at this point
and barely works at all? I tried their compiler last summer and it took 5
minutes to compile hello world, hopefully its improving.

~~~
ljdursi
Certainly Prof Hoefler has done a lot of work driving the design of updated
remote-memory access for of MPI-3, and any further progress would be welcomed;
but I don't think any modernizing of MPI can fix the basic problem. At the end
of the day, it's just too low level for application developers, while being
too high-level for tool developers. There are parts of MPI which don't share
this problem so much - the collective operations, and especially MPI-IO; but
the disconnect between what people need to build either tools or scientific
applications and what MPI provides just seems too great.

For Chapel, it depends on what you count; it very heavily borrows from ZPL,
which is much older, but Chapel itself was only released in 2009. It is
already competitive with MPI in performance in simple cases, while operating
at a much higher level of abstraction. Whether Chapel, or Spark, are the right
answers in the long term, I don't know; but there's a tonne of other options
out there that are worth exploring.

~~~
acadien
Again I'm not sure if I agree or disagree with this. My hatred of MPI is only
outweighed by the fact that I can use it... and my code works.

I think a large part of the inertia behind MPI is legacy code. Often the most
complex part of HPC scientific codes is the parallel portion and the
abstractions required to perform them (halo decomposition etc). I can't
imagine there are too many grad students out there who are eager to re-write a
scientific code in a new language that is unproven and requires developing a
skill set that is not yet useful in industry (who in industry has ever heard
of Chapel or Spark??). Not to mention that re-writing legacy codes means
you're delaying from getting results. Its just a terrible situation to be in.

~~~
ljdursi
Oh, sure. I don't think anyone should start rewriting old codes; but as new
projects start, I think we have a lot more options out there than we did 10
years ago, and it's worth looking closely at them before starting, rather than
defaulting to something. _Especially_ since, once you start, you're probably
pretty much locked into whatever you chose for a decade or so.

~~~
sampo
So, say you wanted to write a weather model, or engineering fluid mechanics
model. Which options (besides MPI) you would look at?

~~~
ljdursi
Chapel has been used for incompressible moving-grid fluid dynamics, so it's
certainly feasible. For that problem the result was ~33% the lines of code of
the MPI version. There is a performance hit, but the issues are largely
understood; if (say) a meteorological centre were to put its weight behind it,
a lot of things could get done.

It's also pretty easy to see how UPC or co-array fortran (which is part of the
standard now, so isn't going anywhere any time soon) would work. They'd fall
closer to MPI in complexity and performance.

You couldn't plausibly do big 3d simulations in Spark today; that's way
outside of what it was designed for. Now _analysing_ the results, esp of a
suite of runs, that might be interesting.

------
shepardrtc
The Fortran Standards Committee is attempting to make HPC easier through the
use of coarrays, which are essentially massive abstractions over MPI.

I really wish people would give Fortran a second chance. It has come a long
way from the ancient, all-caps days.

~~~
na85
I spent a few months learning (modern) Fortran a year or two ago. My chief
obstacle was the difficulty involved in finding modern tutorials. I don't want
to have to read tutorials written in 1994 whose focus is getting people used
to F77 up to speed. I've yet to find a tutorial that approached teaching F08
as if it was a new language, which is what I feel is needed.

Even in F08 there's a lot of backwards-compatibility cruft still left in the
language, too. The IO model still provides very little abstraction and is
based on tape drives. You can/have to "rewind" files. There are obscure "unit
descriptors" that manifest themselves as integer literals in most code posted
online which makes it a chore to learn from. As far as I can tell there is no
functionality that approximates the behaviour of C++'s streams.

It's fast as hell, and the GNU compiler is mature and well-developed, but
Fortran remains a horrid language for doing any sort of interactive
programming. It's best used if you just give it some arguments, let it run
free, and then have it return some object or value that a more sane language
can then interpret and present to the user for a decision.

There is little reason to learn a language where the only sane choice for
doing input/output involves calling your Fortran module from a python script
and letting the python handle i/o.

~~~
markhahn
"horrid for interactive" is ironic, since interactive and HPC are pretty much
disjoint. (well, viz...) From an HPC perspective, Fortran IO should be
performed by HDF5...

~~~
na85
I think you're reinforcing my point.

Fortran might be used more widely, like C++ is, if it wasn't so awful for
doing things other than shitting out numbers at insane speeds.

------
Animats
Yes, "high performance computing" is dying. There's no commercial market for
it. Check the list of the top 500 supercomputers in the world.[1] The top 10
are all Government operations. In the top 25, there are a few oil companies,
mostly running big arrays of Intel Xeons.

CPU clock speeds maxed out between 3-4GHz a decade ago. Nobody develops
special supercomputing CPUs any more. The market is tiny. Old supercomputer
guys reminisce about the glory days when IBM, Cray, Control Data, and UNIVAC
devoted their best R&D efforts to supercomputers. That ended 30 years ago.

Supercomputers have poor price-performance. Grosch's Law [2] stopped working a
long time ago. Maximum price/performance today is achieved with racks of
midrange CPUs, which is why that's what every commercial data center has. Now
everybody has to deal with clusters of machines. So cluster interconnection
has become mainstream, not the province of supercomputing.

[1] [http://www.top500.org/list/2014/11/](http://www.top500.org/list/2014/11/)
[2]
[http://en.wikipedia.org/wiki/Grosch%27s_law](http://en.wikipedia.org/wiki/Grosch%27s_law)

~~~
wffurr
Did you RTFA? It's about MPI:

"MPI is a language-independent communications protocol used to program
parallel computers."

Runs fine on commodity clusters.

~~~
spott
>Runs fine on commodity clusters.

Kind of.... For simple, low communication jobs this is true. But when you
start trying to find the eigenvectors of a large sparse matrix, communication
becomes your bottleneck, at which point MPI on commodity clusters (those
without a really fancy interconnect) "works", but not fast enough to be
useful.

~~~
vonmoltke
I don't think "really fancy interconnects" makes a cluster not commodity,
since the original post in this thread is about supercomputer processors. You
can put Infiniband in any system with a PCI-e 3.0 bus.

------
sampo
in High Performance Computing, there is (1) 3-dimensional simulations
(weather, fluid dynamics, structural mechanics, all kinds of physics
simulations, like magnetic storms in space or nuclear reactors etc.) and then
there is (2) everything else, like data mining, machine learning, genomics
etc.

Some of the sparse matrix computations in structural mechanics and in some
machine learning algorithms have some overlap. But mostly, group 2 has little
reason to be interested in what group 1 is doing.

Now, group 2 obviously has more modern tools than the 3d-simulation community,
because machine learning came to common use much later that numerical fluid
mechanics.

But do 3d-simulation people also have much reason to be interested in what the
machine learning people are doing?

The "machine learning / big data" people are probably not doing anything that
makes a weather prediction model to run faster? Or are they?

~~~
sgt101
They are doing it (interesting things) for lower capex and lower development
costs. On opex, good for operations, bad for power consumption (relatively).

In terms of absolute performance HPC is absolutely faster. In terms of bangs
for bucks, Big Data is hands down faster. Also in terms of accessibility Big
Data is hugely easier - I can build you a 100 core big data system for $300k

~~~
sampo
But your Big Data system is only good for Big Data. If I want to run a weather
prediction model, it is not going to help.

My point is, the big data and the physics simulation people probably do not
have a lot of common interests - besides using large amounts of computing
power.

~~~
acadien
There are big data people running on supercomputers too. I know there are
people writing custom asynch job managers to handle big data type problems
because the top supercomputers have low memory latency.

Also I think the dichotomy you're looking for is IO bound vs CPU bound
problems. Although certainly there are a plethora of different kinds of IO
bound problems (asynch vs synch or disk bound vs memory bound vs cache bound).

------
rz2k
The article briefly mentions Erlang with its focus on message passing, and the
focus on being fault tolerant should be another benefit. I could find a few
mentions of Erlang used for simulations, such as here[1], but I am curious
whether there is much actual usage in scientific computing, or whether there
are some problems in practice.

[1]
[https://books.google.com/books?id=p0h9vAb1m7IC&pg=PA365#v=on...](https://books.google.com/books?id=p0h9vAb1m7IC&pg=PA365#v=onepage&q&f=false)

~~~
shepardrtc
Erlang itself is not good for the type of numerical computation typical done
in HPC. It is amazing as a backend, and there are several ways it can call
code written in other languages through the use of ports or NIFS, but if you
try to do massive number-crunching using its own libraries then you're going
to be unhappy with the results.

~~~
rz2k
How about in simulations where a large number of cellular automata are
interacting with each other, but individually only carrying out simple
computations?

~~~
shepardrtc
You certainly can do that. In fact, I believe I saw an example of that in a
tutorial somewhere talking about what Erlang/Elixir would be good for (I
include Elixir because its the same VM controlling processes underneath, but a
"nicer" syntax on top). Now, would it be the best language for that? Depends.
If you're looking just for speed and massive computations, then no. In the
end, C++ pretty much rules everything in that regard (except for maybe Fortran
in some instances). But, I would say that it might be more fun to set it up in
Erlang/Elixir. And you could pretty easily expand the whole thing just by
adding more processors and simply telling it to spawn more processes (the
Erlang VM is pretty awesome). I would almost say that an Erlang version of it
would feel more life-like. You could probably experiment with it on the fly
more easily, too. Kill off a process here or there and see what happens, etc.

~~~
rz2k
I actually started going through _Programming Elixir_ this past week because I
was thinking it would eventually be a fun way to explore models through ad hoc
substitution of rules and exogenous conditions. There are a lot of models
available in NetLogo, but it doesn't look like it would be very powerful in
terms of running larger scale simulations, and the scope of the language's
usefulness is pretty limited compared to that of a general purpose language.

While I have been using C++/Rcpp to extend R as an occasional time saver (for
analysis rather than simulations), it's only been little snippets written
badly.

Anyway, since its so easy to come up with a too long list of technologies to
learn, then only scratch the surface, hearing that Erlang/Elixir isn't the
completely wrong tool is helpful.

~~~
shepardrtc
As an example, here's the game of life written in Elixir:

[https://github.com/spikegrobstein/elixir-
conway](https://github.com/spikegrobstein/elixir-conway)

How much better that is than a C++ version, I don't know. But if you want to
get better at C++ and give it a try, forget what you know and read this book:
[http://www.amazon.com/Programming-Principles-Practice-
Using-...](http://www.amazon.com/Programming-Principles-Practice-
Using-2nd/dp/0321992784/)

------
east2west
I am trying to get into distributed computing so this article is particularly
interesting to me. I may be mistaken so please excuse my naivety if my points
are off marks. I thought MPI was mainly geared towards communication-heavy
tasks where the underlying network is specialized, for example infiniband or
bus between CPUs. One use of MPI is to manage distributed memory tasks between
different physical CPUs while threads run on multiple cores of same CPU.
Spark, I believe, doesn't handle cases like this well because JVM hides low
level details. I have read papers that propose to layer MPI over RDMA rather
than expose a flat memory model, which came as a surprise to me but it shows
the flexibility of MPI. One thing unclear to me what performance we can expect
from MPI when we use commodity network gears, and how it compares to Spark.
The article is absolutely correct MPI leaves robustness to user and that is
today an oversight.

Modern Hadoop ecosystem is designed for different workload from MPI's. It
emphasizes co-localizing date and computation, seamless robustness,and trades
off raw power for simple programmingmodels. MapReduce turns out too simple, so
Spark implements graph execution, which is nothing new to HPC. As far I know
Spark's authors don't believe it is ready for distributed numerical linear
algebra yet. But a counterpoint is that I am seeing machine learning libraries
using Spark, so perhaps things are improving.

One thing I have learnt today is that MPI isn't gaining popularity. I just
have a hard time picturing a JVM language in overall control in HPC where
precise control of memory is paramount to performance.

~~~
ljdursi
I agree that languages that rely on tracing GC seem like they're fundamentally
at a disadvantage when it comes to pushing the envelope of single-node
performance; the best article I've read arguing this was actually in the
context of mobile games, rather than HPC, but I can't for the life of me find
the article now.

I don't know if Spark itself is the right way forward; but it's an example of
a very productive high-level language for certain forms of distributed memory
computing. And some of these issues - like the JVM - aren't fundamental to
Spark's approach; there's no inherent reason why something similar couldn't be
built based on C++ or the like.

~~~
CHY872
I'd be very wary of criticising Spark based on the technologies it's built on.

In particular, a decent garbage collector will give you performance dependent
on the number of live objects (typically low) and not on the number of
allocations and deallocations, as you might see in a non-garbage collected
language. This gives great allocation performance and reduces overheads.

The disadvantages can be (potentially long) GC pauses and higher overall
memory requirements, but in practice this isn't usually a problem for non-
interactive systems.

Of course, if you do have a device with low memory and low tolerance for GC
pauses (i.e. mobile gaming) there might be a problem.

The main disadvantage seems to be less predictable performance; which could be
a problem in domains which require good timing performance, but that's not
really Spark's problem.

A GC'd language is also generally easier to program in, since one doesn't have
to (in general) worry about memory management, so it's generally a lot easier
to program very large systems with lots of moving parts.

------
bjwbell
Too bad he didn't talk about GPGPU killing MPI too or not. I don't know enough
to say.

I'm not familiar with the HPC space but I thought a lot of new work, at least
in machine learning, was migrating to GPGPU instead of traditional CPUs. The
compute per $ or per watt payoff is too large to ignore.

~~~
stephen272
I wouldn't say GPUs are killing off MPI. You still need some way to pass data
between nodes/GPUs (most of these datasets can't fit within a single GPU).
What you are seeing though, is less and less use of the CPU. If code
developers decide to use GPUs, they try and move their data onto the GPU and
keep it there for as long as possible (data movement across PCIe is a killer
for performance). ORNL's new machine Sumiit [1] will have 1/4 the nodes of
their current machine, Titan, but multiple GPUs per node. Thus emphasizing the
move away from CPUs and onto GPUs. Again though, there are still 3K nodes and
you need some efficient way to pass data between those nodes.

[1] [https://www.olcf.ornl.gov/summit/](https://www.olcf.ornl.gov/summit/)

------
pjmlp
This made me feel old, as I remember the days when we got to learn PVM at
distributed computing classes and MPI was presented as something that some
people were kind of working on.

~~~
dalke
If it makes you feel better, I started with PVM 2.

------
acomjean
I used MPI (Message Passing Interface...) back in the day.

But it was a pain, especially since our code was a mix of c (which was easy to
mpi) and ada (not so easy). Its pretty low level stuff (I think we used Open
MPI). All the nodes need to have MPI set up and configured, fine if you have a
team willing to do it but these days....

mpiexec -n 10 myprocess

I think we liked it because the processes would be put to sleep by the mpi
daemon until a message arrived. You can sleep and wait for a message with
sockets now I think. Its been a while since I've used the unix IPC
(Interprocess communication).

I don't think I'll miss it.

------
wglb
This is a good article.

I built a tiny cluster in my basement (a prototype) and looked at MPI and
decided that it was way to complicated, so I just built something that pushes
the essential bits, pretty much without abstraction, to the nodes and was
done. The cluster is a very specific solution, so I felt justified in not
looking at MPI. And now the decision feels even more justified.

------
siegela
John: You make some solid points. Most though seem to more support the idea
that more research investment/emphasis is badly needed for HPC programming
models. From an application perspective, one sees such a dominant reliance on
MPI+X primarily because typically the value proposition just isn't there yet
for alternatives (at least in the areas where I work, where we have undertaken
new developments recently and done fairly exhaustive evaluations of available
options). Though the coding can be somewhat tedious and rigid, in the end
these shortcomings have been outweighed by the typical practical
considerations -- low risk, extremely portable, tight control over locality
and thus performance, etc. It's obviously not all or nothing - as you say we
could choose something even lower level and possibly get better performance,
but when seen from the perspective of the near and mid-term application goals,
it's hard to make a different choice unless explicitly tasked with doing so.

------
Xcelerate
Thank goodness someone said it. I get tired of using ancient software that's
older than me for relatively simply tasks that should have long ago been coded
into a higher level of abstraction. I programmed a pair-correlation function
calculation routine using MPI once -- yech.

Fortran, MPI, even C to an extent -- can we please move on? I don't understand
why the scientific community is so reluctant to embrace change. It seriously
doesn't take that long to learn a new language or a platform like Github
(yeah, that's still considered "new" in the scientific community), and the
time investment more than pays itself back many times over.

~~~
pron
> I don't understand why the scientific community is so reluctant to embrace
> change.

Let's assume the opposite were true, and it was _fast_ to embrace change. How
much time would be spent on this change -- relearning, rewriting, refighting
old bugs -- vs. actual work done?

 _Change is overhead_. You do as little of it as necessary, and only when _not
changing_ starts costing a lot. Which means you change, but slowly.

As to Fortran, it will go away when something better comes along, and then it
will do so slowly, for the aforementioned reasons.

------
kxyvr
The lion in the room is that the DOE National Laboratories have a huge amount
of code tied up in MPI and they continue to spend millions of dollars both on
hardware and software to support this infrastructure. If you look at the top
500 list:

[http://www.top500.org/lists/2014/11/](http://www.top500.org/lists/2014/11/)

Four out the ten computers are owned by DOE. That's a pretty significant
investment, so they're going to be reluctant to change over to a different
system. And, to be clear, a different software setup could be used on these
systems, but they were almost certainly purchased with the idea that their
existing MPI codes would work well on them. Hell, MPICH was partially authored
by Argonne:

[http://www.mcs.anl.gov/project/mpich-high-performance-
portab...](http://www.mcs.anl.gov/project/mpich-high-performance-portable-
implementation-mpi)

so they've a vested interest in seeing this community stay consistent.

Now, on the technical merits, is it possible to do better? Of course. That
being said, part of the reason that DOE invested so heavily in this
infrastructure is that they often solve physics based problems based on PDE
formulations. Here, we're basically using either a finite element, finite
difference, or finite volume based method and it turns out that there's quite
a bit of experience writing these codes with MPI. Certainly, GPUs have made a
big impact on things like finite difference codes, but you still have to
distribute data for these problems across a cluster of computers because they
require too much memory to store locally. Right now, this can be done in a
moderately straight forward way with MPI. Well, more specifically, people end
up using DOE libraries like PETSc or Trilinos to do this for them and they're
based on MPI. It's not perfect, but it works and scales well. Thus far, I've
not seen anything that improves upon this enough to convince these teams to
abandon their MPI infrastructure.

Again, this is not to say that this setup is perfect. I also believe that this
setup has caused a certain amount of stagnation (read huge amount) in the HPC
community and that's bad. However, in order to convince DOE that there's
something better than MPI, someone has to put together some scalable codes
that vastly outperform (or are vastly easier to use, code, or maintain) the
problems that they care about. Very specifically, these are PDE
discretizations of continuum mechanics based problems using either finite
different, finite element, or finite volume methods in 3D. The 1-D diffusion
problem in the article is nice, but 3-D is a pain in the ass, everyone knows
it, and you can not get even a casual glance shy of 3-D problems. That sucks
and is not fair, but that's the reality of the community.

By the way, the oil industry basically mirrors the sentiment of DOE as well.
They're huge consumers of the same technology and the same sort of problems.
If someone is curious, check out reverse time migration or full wave
inversion. There are billions of dollars tied up in these two problems and
they have a huge amount of MPI code. If someone can solve these problems
better using a new technology, there's a huge amount of money in it. So far,
no one has done it because that's a huge investment and hard.

------
lego12
The sad thing is that MPI-3 doesn't have wide acceptance yet. A lot of
organizations mainly use MPI-1/2\. While using RHEL 6.x. Shoutout to red hat
for maintaining ancient software

------
Dylan16807
W-why are they using that picture at the start when their source is an essay
on exactly the way they're misusing it?

------
MichaelCrawford
This happened with Smalltalk.

I was a Smalltalk coder. I thought it was the best thing since sliced bread.
It has always been clear to me that Smalltalk is far superior to Java.

I left the company after a little while, to do C++ graphics. I later heard
that my former employer rewrote their Smalltalk application in Java.

Now no one uses Smalltalk anymore. While Objective-C is based on Smalltalk,
Smalltalk was far easier to use, however lots of people use Objective-C. No
one uses Smalltalk.

How could it have been different? My friend Kurt Thames once said that
"Smalltalk is the way object-oriented programming SHOULD be done." I have
always agreed with that.

But when new methods (!) of OOP arose, all the Smalltalk crowd did was gripe
about how Smalltalk was far better than Java or Objective-C.

~~~
twic
What "new methods" are you thinking of?

What let Java get ahead of Smalltalk for me personally, as someone getting
into programming in 1996, was that i could write it in the text editor i
already had, compile it with a compiler i could get for free, and then post
the source code on Geocities (actually, Xoom - remember that?) to share with
others.

Whereas when i tried to get into Smalltalk, the first thing i had to do was
learn my way around this wacky environment with its strange class browser and
ultra-retro window manager, and get my head around the fact that my source
code wasn't anywhere particular, and yet was everywhere, and that if i wanted
to share your code, i had to somehow "file out", and then hope that my
internet friends could successfully "file in" to their own potentially
modified images. Once i'd got hold of the tools at all, that is.

Which is not to say that the Smalltalk environment was not better than
Notepad/DOS box/javac, because of course it was. It just didn't lend itself to
adoption and spread nearly as well. It was a tool for masters, with affordance
for apprentices.

Also, Java had pretty good networking right in the standard library, and
networking was really exciting in 1996.

~~~
MichaelCrawford
I'd say that your ability to write Java source in any text editor at all was,
all by itself, a new method of doing Object-Oriented Programming.

We used Visual Smalltalk Enterprise. It had the cool feature that, at the end
of the workday, I could make what amounted to a core dump, then the following
morning I would load my core dump into a running program, and there would be
all my open windows with the cursors in the right places in the source
documents and so on.

That was quite cool I really enjoyed it, however that environment was
profoundly non-portable. I expect that much of the success of Java as opposed
to Smalltalk was the simple ability one had to post a tarball full of source
code on one's FTP site.

~~~
twic
That wasn't new, though. C++ had that. Eiffel had that. Every programming
language that isn't Smalltalk has that.

~~~
MichaelCrawford
I'm not dead certain but I think smalltalk may have come before C++. For sure
it was in widespread use before C++ was in widespread use.

~~~
dalke
Smalltalk started in '71 and was used for research purposes in '72\. Smalltalk
'80 was the definitive version for most. This means that Smalltalk slightly
predates C. By comparison, C++ didn't start until 1983.

You wrote "your ability to write Java source in any text editor at all was,
all by itself, a new method of doing Object-Oriented Programming".

Simula, considered the first OO language, is text based in the same way that
Java is, and not image based like Smalltalk. Since Smalltalk is inspired by
Simula, I do not believe one can say using any text editor is a new method of
doing OO programming.

