

Ask HN: Why can't we prove the Blub Hypothesis? - tel

In a current HN thread, trapper asked the interesting question<p>"Surely if lisp makes a programmer N times more efficient then it would be easy to study scientifically. Have there been any studies comparing productivity?"<p>(http://news.ycombinator.com/item?id=1016583)<p>I've never personally seen one and can almost immediately rattle off a thousand reasons why it would be impossible, but that doesn't mean it is.<p>How would you structure a study to make it practical to gather real, objective, and interpretable data on programming efficiency? It'll be necessary to consider controls, sample size (Mech. Turk, perhaps), and, largely, the definition of "programming efficiency", but with all the talk about how Java is Blub I'd love to see it actively, quantitatively hamper someone's work.
======
mnemonik
Peter Norvig, Lisp As An Alternative To Java: <http://norvig.com/java-
lisp.html>

Peter Norvig compares Java, C/C++, and Lisp between 38 programmers and
himself. All were asked to write the same program, record how many hours it
took, how many SLOC, and then Norvig analyzes the results.

EDIT:

Norvig also links to a couple other "studies" and anecdotes done by others:

\+ Tcl, Python, Perl, and Rexx:
<http://wwwipd.ira.uka.de/~prechelt/Biblio/jccpprtTR.pdf>

\+ More Lisp/Java: <http://www.flownet.com/gat/papers/lisp-java.pdf>

~~~
tel
Running through these papers, the Prechelt (Tcl/Python/Perl/Rexx) paper
probably comes closest to producing the kind of analysis desired, but at the
end of the day they're all begging for more controls.

In particular to some of the Prechelt paper's most interesting results (LOC,
comments, time spent) are marred by the collection methods ("script" languages
were not sampled under controlled settings).

------
cubicle67
I'd interpreted Blub not as any specific language, but rather any language has
the capability to become Blub when a developer falls into a particular
mindset.

To me, a language becomes Blub when the developer using it loses all
inclination to look beyond it; to them there's no other language that could
offer anything more. They can see language A that is obviously inferior as it
doesn't offer feature X, yet at the same time there's no point in learning
language B because they see no use in feature Y. After all, they've managed
fine for years without it.

~~~
tel
That's how I interpret it as well, but in order for Blub to be an interesting
thing experiment it makes the assumption that languages form some sort of
continuum of power/ease/abstraction/usefulness. Generally, the only one of
those four people can agree upon (sometimes) is abstraction and that's not
even necessarily the most important of the four.

I personally think one of the most interesting dimensions to analyze a
language would be in how quickly one could prototype a problem. Lisp often is
said to score well here; the Cs are thought to fall short; Arc was more or
less designed to optimize this technique in online apps; and Haskell somehow
manages to perform here despite a lot of criticisms in its design. So there's
a hypothetical contiuum. Prove to me it exists and then you can pick your
Blub.

~~~
silentbicycle
People seem to assume that there's a _linear_ continuum with a "best"
language, rather than thinking that certain languages' designs make them
better suited to handling certain kinds of problems.

For example, sometimes lazy evaluation greatly simplifies problems, and
sometimes it makes them vastly more difficult to reason about. The language is
just a tool to help clarify and solve your problem, and a language that
doesn't fit can make it worse.

~~~
tel
Within some problem spaces might it not be true the some languages are
somewhat definitively better than others on average?

Knowing many languages definitely operates as a toolbelt for different
problems, but there's a reason languages die out. Could that same reason not
still apply, at least slightly, in the languages still alive?

~~~
silentbicycle
Oh, of course. Within specific problem spaces, there are many languages that
are laughably inappropriate.

The constraints over languages have also changed over time, though. For
example, in _Coders at Work_, a couple people mention logic programming as an
idea worth digging into again now that computers have gotten orders of
magnitude faster. I think they're onto something.

I just take issue with how some particularly vocal Lispers have been sitting
on the sidelines going, "Our language is THE BEST LANGUAGE" for decades.
("Sick of being a blub programmer and working in Blub? Try our Language For
Smart People.") They're so used to being smug because their language was
thirty years ahead of its time, fifty years ago.

~~~
tel
There is undoubtedly specialization, but I also believe in a linear continuum
across efficiency. Any programming problem, at some level, ends up including
solving a constraint problem: how do you fit your solution elegantly into an
initially unknown problem space? Greater expressive freedom and abstraction
are often heralded as wins in programming languages perhaps because they both
promote though toward elegant solutions and then provide the structure
necessary to easily build them.

I think that this kind of test is not so different from testing something like
Sapir-Whorf. Does using language X _by itself_ help someone to solve problems
more quickly/elegantly/efficiently/effectively?

So, instead of considering specialization to be multidimensional and therefore
impossible to quantify, I'm thinking of it as something you'd "just" need to
control for. Randomization across a large number of programming problems could
possibly get a handle on that control.

------
ezy
My guess here is that "programmer productivity" is probably the wrong measure.
And that by trying to be reductive about that, you will probably miss somethng
valuable...

My idea: what if we came up with an app idea and set two teams to it, with the
only restriction being the language they could use (one Lisp, one "x"). The
"winner" is the one who makes the most compelling app. Which one is most
compelling can be decided in some arbitrary fashion -- My opinion is that
it'll be fairly obvious (although I have _no_ idea who would win out! Just to
make that clear :-))

This doesn't sound so terribly scientific, does it? But I think that's about
the best you're going to get if you want to measure the __effectiveness __of a
programming language for a chosen task.

~~~
tel
That would undoubtedly be qualitative, though you're right that it's probably
the closest thing to programmer productivity you can measure.

I think you're wrong that reductionism can't find an interesting gem buried
below. Productivity is something difficult to measure because it takes a long
time and has a lot of influencing factors, but if you can split it apart just
a little bit and lose some of the confounders you might be able to give at
least a little quantitative foundation to claims shown anecdotally.

------
asciilifeform
This is why:

<http://www.loper-os.org/?p=85>

Racing computer languages the way one might race horses is a fundamentally
misguided thing to do. The race is to the Blubulous simply by virtue of being
a race.

------
shalmanese
The ICFP Programming Competition:
<http://en.wikipedia.org/wiki/ICFP_Programming_Contest>

~~~
silentbicycle
Historically, it's been dominated by OCaml, Haskell, and C++ (!) people.

Is it just me, or _has Lisp not placed once in the eleven years of the
contest_?

In a frigging _academic contest for functional programmers?_

Where they were beat by _assembly_ and _TeX_???

Ouch!

(In all fairness, they still have _Lisp in Small Pieces_, _Paradigms of
Artificial Intelligence Programming_, and _SICP_. Even if nobody uses the
language for real programs, it's still in the running.)

~~~
neilk
Team SmartASS (a group of Googlers) wins or places high in that competition
regularly, and they use Java.

------
scythe
[http://stackoverflow.com/questions/1894453/development-
time-...](http://stackoverflow.com/questions/1894453/development-time-in-
various-languages/) should be helpful.

SPOILER: Ada and Haskell get high marks.

