

The Python Paradox is now the Scala Paradox - martinkl
http://www.yes-no-cancel.co.uk/2009/09/18/the-python-paradox-is-now-the-scala-paradox/

======
econoplas
I disagree somewhat about the "not very productive" part, at least in the
particular case of scala.

I've developed several large projects in Python (specifically CPython 2.x) for
$$ for about 6 years and I am learning scala right now. So I have a somewhat
interesting perspective on both.

Although I suppose I could be termed a Pythonista since I "do python" for a
living, I consider myself more of a true polyglot... I also use Perl, Ruby,
C++, Java, Lisp, C#, whatever language I can get my hands on that fits for the
project at hand. I don't subscribe to the Golden Hammer theory.

So far, scala does feel like it has that 5x productivity improvement as
compared to using straight Java, on par with what I experienced when I started
porting significant portions of the code base of a large application from C++
to CPython (embedding CPython in C++, or vice-versa as needed).

I personally see scala being somewhat in a different category from CPython for
2 main reasons: JVM (stability, maturity) & Java interoperability (vast
library resources).

1\. The JVM is arguably the most widely used, stable, and mature virtual
machine ever developed. Not surprising since Sun and IBM have both invested
millions (billions?) in JVM development over the last 15+ years. The latest
generation JVMs sport multi-threaded garbage collection, profiler support,
remote debugging, remote method invocation, compile-and-reload while
debugging, and very advanced JIT. JVM runs in more places than any other VM to
date. 'Nuf said. Yes CLR is probably on similar footing nowadays. By
comparison Python, Ruby, Perl, scheme, or any other language that has its own
home-grown VM is always going to be behind that curve... for example, I was
just reading about how most of the Common Lisps (many pre-dating the JVM) are
still working on adding multi-threaded garbage collection.

2\. The other (main) reason I'm looking at scala is the HUGE advantage of JVM
interoperability and the vast libraries available.

From my Python & Java & other language experience, I have learned that:

=> cool-language-features-productivity-multiplier = 5x to 10x productivity
improvement, such as what Lisp and most of the dynamic interpreted languages
(CPython, Ruby, Perl) claim and pretty much deliver

=> existing-library-productivity-multiplier = 100x or more, since I don't have
to re-invent every wheel every time for file format X reader/writer, GUI
widget, or other library du-jour. Let's face it, we have to interoperate and
communicate with a lot of different gizmos & whatzits nowadays, right?

We all know that we as programmers are vastly more productive when we can get
almost any library imaginable (usually open source) to do almost anything we
want... GUIs, Charting, FFTs/Numerics, networking, 3D graphics, Neural Nets, ,
and can even leverage Eclipse environment and toolkits for programming in the
large for GUI building, writing rich-client platforms (building entire IDE's),
Web server frameworks, web services & SOA development, DSLs & modeling,
reporting, DB persistence, and then write our glue logic in scala or other JVM
language of choice.

We can even intermix them with other JVM languages in the same project
(Groovy, Jython, Jruby, clojure, PNut, etc).

The good news is... if I pick scala, and it doesn't work out... I can always
selectively replace out the pieces I don't want to keep one-at-a-time in Java
or any other JVM language as time permits. But if it works, I'll just leave it
alone. Nobody even needs to know my JAR is written in scala except that they
have to put a couple of scala JARs on the classpath. Not so for CPython where
the installation & deployment overhead is much steeper in my experience. Not
everyone has CPython 2.6 installed on every machine for example, but most
everyone has a JVM 1.5 or greater whether they know it or not :)

Wow.

------
gizmo
Esoteric languages are a lot of fun. But they're not very productive, exactly
for the reasons mentioned: lousy IDE support, buggy compilers, few libraries,
few bindings to existing libraries, and so on. When you work in a stable
language, you can assume that when something goes wrong -you- messed up. In a
new and cool language you don't have that luxury.

You'll be less productive, but you'll have more fun when you get to program in
language. A good trade off? I don't think so. A few years down the road you'll
have to maintain the same project. The honeymoon period is over, and the code
base isn't quite as good as you'd like it to be (you designed the project back
when you didn't know the language very well, and it shows). You know you could
spend a few weeks refactoring, but that would probably introduce bugs and
you'll drive the other people in your team crazy if you start renaming
methods, and move stuff around just to make the code "nicer". And besides,
you'd much rather be programming in the next-nifty-language, because it lets
you extend the syntax at runtime and it has a debugger that will let you go
back in time.

The bottom line is that when you pick a language that is on the cutting edge
of cool right now will no longer be cool a few years from now, but it still
won't have any libraries and other essentials. Languages too have to cross the
chasm, and they all have to go through a period where they're neither new and
exciting nor stable.

If you're serious about software, concentrate on writing something that is
reliable as hell and easy to maintain and install. Want a challenge? Write
your app in such a way that power can be lost at any point in time without
losing any user data. Or (for desktop apps) make sure that the UI -never-
hangs, even when the hard drive has to spin up. Software is never done, there
are always things you can make better, more fault tolerant, faster. It's not
as if we've run out of challenges.

Messing around with unstable languages is a lot of fun. It's educational too.
But don't build your business around it. Python/Django and Ruby/Rails are only
barely stable enough to work with. Thinking that using Haskell, Clean, IO or
Lisp will be a net benefit to productivity is foolish.

(Of course there are obvious exceptions, and that Clojure and Scala use the
JVM does make the whole library thing less of an issue. But it's still a big
deal.)

~~~
gaius
I'm not sure how true this is. I was into Java in '95 when it had "lousy IDE
support, buggy compilers, few libraries, few bindings to existing libraries,
and so on". Java fell far short of its potential, and the ambitions Sun had
for it, but still, that wasn't a bad call. People who got into Python early on
made a similarly good call.

And also, some esoteric languages, like Haskell, the ML family, Erlang and so
on have deep roots, they may not be mainstream but they aren't going away
anytime soon. And sometimes something funny happens; anyone who has ML
experience is going to feel right at home on F# and suddenly the library
problem is solved, the IDE problem is solved, the bindings problem is
solved...

~~~
masklinn
> some esoteric languages, like Haskell, the ML family, Erlang and so on have
> deep roots

None of Haskell, ML or Erlang is an esoteric language.

~~~
jimbokun
I think you have a fairly high bar for esoteric :).

~~~
masklinn
No. Esoteric languages are a fairly clear category of languages created to
test the boundaries of language design.

Esoteric languages are not created for real-world usage (or even for usage at
all), examples of such languages are INTERCAL, Thue, Unlambda, Moo, Befunge or
Java2K. They can be turing tarpits (languages with extremely low number of
commands), funges (multi-dimensional programs), nondeterministic (from a given
global state A and a current instruction f, the next state `f A` of the system
can't be predicted)…

There's nothing esoteric about a language such as Erlang created very
specifically to solve industrial problems, or even about languages originally
created for research (ML or Haskell), especially when they've now leaked into
practical, real-world programming outside of academia.

------
boskone
Scala is _not_ some esoteric MIT, CMU language to make a point exercise. Its a
pragmatic and practical attempt to "fix" Java so it has all the capabilities
of Java with the proper foundational coherence which allows for additional,
some yet to be defined or added, capabilities. Java has long ago painted
itself into a corner. It has reached its evolutionary dead end.

As someone who lead a team which wrote one of the largest retail websites in
the earlier days of Java, and as someone who has spent the last year rewriting
legacy business logic in Scala, there is no comparison.

Scala wins not because its the new hipster language, it's distinctly superior
across the board. You can write better, clearer, far more robust, far more
scalable and far more easier to maintain and modify business logic at higher
levels of productivity. That and its ability to work with legacy Java code
makes it ideal. This is from direct and in-depth experience.

My concern with gizmo's post is its complete lack of claimed experience. No
context of based on my experience where ... so we ... and then did a POC
comparison with ... and found .... and concluded ...

In fact it appears very much to be a direct quote of from the preamble from
the "Enterprise IT Managers Survival Handbook For Those Without Experience Or
Knowledge Of Software Development Fundamentals"

Privately I've played with all of them. I love the shear outrageous power and
beauty of Scheme (PLTs impl) and Haskell, the base competency of SML, and I've
even used them in the enterprise in isolated in very few situations where the
situation demanded it.

However, never once did it cross my mind to even "think" to introduce any of
them into the enterprise (well a brief fling with Python for our operations
group, it was found to be too hard).

Scala deserves to win. It is decidedly superior, however, lots of deservedly
good things, in fact most, never see the light of day in IT.

~~~
gizmo
Note that I primarily object to the "let's do a startup with language FOO,
because then everybody can see how cool we are and every smart programmer will
want to join us because the language is so awesome!" attitude.

It's fine to take a calculated risk (Scala) if the direct benefits are
measurable and significant. I'm not claiming that people should dogmatically
stick to languages from the 90ies because they're safe. I'm saying that you
need a very good reason to use a language with an uncertain future.

Also note that I (deliberately) didn't put Scala in the list of foolishness
that contains Clean and IO.

~~~
plinkplonk
"Note that I primarily object to the "let's do a startup with language FOO,
because then everybody can see how cool we are and every smart programmer will
want to join us because the language is so awesome!" attitude."

(a) no one ever diplayed such an attitude on this discussion or put that
argument forward. Strawman.

(b) This is what you said.

"Messing around with unstable languages is a lot of fun. It's educational too.
But don't build your business around it. Python/Django and Ruby/Rails are only
barely stable enough to work with. "

which is different from reacting to the non existent "everybody can see how
cool we are if use language foo" argument.

"I also note that I (deliberately) didn't put Scala in the list of foolishness
that contains Clean and IO."

But you did say " Thinking that using Haskell, Clean, IO or Lisp will be a net
benefit to productivity is foolish."

You never really showed where the "foolishness" is.

Also, from your original list that still leaves Haskell and Lisp as languages
that don't "provide a net productivity gain", which is a very strange (and as
pointed out by boskone above, unsubstantiated) claim. There are profitable
businesses built on both these languages (lisp is a family of languages but
still .. ), and very capable programmers being massively productive using
both.

You said something stupid with nothing to back it up, got hauled over the
coals and are now backtracking trying to make a less objectionable claim.

You then claimed Perl didn't exist in 1995 (!) and that's why PG used CL for
ViaWeb (!).

quoting boskone "In fact it appears very much to be a direct quote of from the
preamble from the "Enterprise IT Managers Survival Handbook For Those Without
Experience Or Knowledge Of Software Development Fundamentals"

Indeed.

~~~
gizmo
It's only a slight exaggeration. In the OP he states "my heart just said yes,
it wasn't' a rational choice but an instinctive emotional one". The original
article also states/quotes that picking an esoteric language is a good idea
because that way you attract people who learn languages for fun. The "let's
use a cool language" attitude is very visible here on HN.

I stand by what I said in (b). That you shouldn't pick a language because it's
fun -- this I find almost self-evident. When you want to build a business you
need to be more pragmatic and you need to consider other factors. Django and
Rails are now probably the most popular frameworks for web 2.0 shops. I claim
they are not very stable (and I provide supporting arguments in other posts).
So I argue that using Django/Rails is adventurous enough. Barely stable enough
obviously doesn't mean that it's unusable or worthless. The success of many
Django/Rails projects is evidence of that.

The onus is on those who use non-mainstream languages to provide evidence
there is an advantage. I claim there is no advantage, and I point out several
downsides. If I claim that using Assembly for web programming is foolish you
wouldn't expect me to show you why either.

"But of course there are obvious exceptions" covers the few businesses that
have become successful. Here too, the onus is not on me to prove that Lisp did
not contribute to their success, the onus is on those who are successful to
show that it is Lisp that made it so. The reddit people drank the cool-aid,
and it didn't work out for them. A single data point, I know.

I'm not taking back anything I said. I stand by it. I find that some of you
are looking for things to disagree with, and you read my statements in the
most objectionable way. You're saying you know what I meant better than I did
when I wrote it. That's quite presumptuous. To attack my character by implying
I'm trying to weasel out of anything is most uncalled for.

My claim that Perl didn't exist when viaweb was started was indeed stupid. Had
I known viaweb started in 1995 I would never have said that. I (mis)remembered
something pg said some time back, along the lines of "if I had to start viaweb
today, I would use Python or Ruby". Maybe pg never said that, I don't know.
Either way, I was completely wrong there.

edit: found the reference <http://news.ycombinator.com/item?id=36905> (I was
wrong)

~~~
plinkplonk
"The "let's use a cool language" attitude is very visible here on HN."

Some evidence would be nice. I don't see it at all in the sense you claim i.e
"we'll use a cool language because it makes us cool". No one I know on HN ever
makes that claim or thinks that way. And yet to you it is "very visible".
Links please?

"The reddit people drank the cool-aid, and it didn't work out for them. A
single data point, I know"

But you don't accept "single data points" in the other direction do you?
ViaWeb, Jane Street, Orbitz, FlightCaster?

"single data point I know" ;-)

"The onus is on those who use non-mainstream languages to provide evidence
there is an advantage."

You are the one who makes outrageous claims that there is no productivity gain
_without_ ever actually using "esoteric" languages in a business context, so
you could have some experience backing t your claim that there is no
productivity increase. Your troubles upgrading Python (or whatever, your
arguments on this are very vague and unconvincing) are not sufficient. Sorry.

When people who actually do use powerful languages to run their businesses
report increased productivity, you refuse to accept it or consider it a
possibility.

<Shug> Flip Bit. Move on.

"You're saying you know what I meant better than I did when I wrote it."

I don't know you in person. I have to judge your argument by what you actually
write not by what you thought in your innermost heart when you wrote it. In
your _writing_ you come across as uninformed wrt your claims, and constantly
shifting positions when confronted with counter claims, citations or evidence.

That said, I am done discussing this with you. Your argument has nothing
backing it but "I think thus and thus with no experience or data or evidence
and I refuse to consider anything contrary to my stated position".

Good Bye, Have a nice day.

------
petercooper
_Because it's down now, a vague summary from what I saw._

Python and Ruby are now basically "mainstream" languages with little of the
all new sparkle they had several years ago - little early adopter aura around
them. What languages have that 2004 shine in 2009? Author decides to try Scala
for this reason and likes it. Author thinks Scala is the new Python because
people are excited by it.

------
wheels
I have to say that, even as someone fascinated with Scala this week, I find
the argumentation here a little disingenuous. There are a lot of ways to
filter developers and I don't think that the ability to learn a new language
sets the bar high enough to actually select for the good ones and potentially
means selecting the wrong tool for the job.

There are a large group of people that are effectively programming language
hobbyists, frequently learning new programming languages for kicks, and while
that set intersects the set of good developers, it is not a subset of it.

~~~
gaius
The initial hump in hiring is always the shortlist - you might get a hundred
(or a thousand!) CVs and you can maybe first-interview 10 and second-interview
2-3. So the priority is always to get down to a manageable list by rejecting
candidates who are obviously unsuitable.

So you are correct, but you could probably even trust your HR department armed
with a suitable list of esoteric languages keywords (instead of their usual
"Java" etc) to build you a pretty good first-interview shortlist, even if you
were after Java developers.

------
lsb
The Haskell Paradox, too.

~~~
davidw
He mentions that. And Erlang, too.

~~~
gaius
He forgot OCaml ;-)

~~~
tomjen2
I am not sure OCaml has gained any traction lately. The only paradox of OCaml
is why people use it~

~~~
lsb
Jane St likes it because of the rock-solid type system, but they've been
eyeing Haskell compiled with -Wall for the same reasons.

~~~
paulgb
I think these guys use O'Caml as well: <http://skydeck.com/>

------
DrewHintz
Site down. Cancel.

~~~
barrkel
Google cache has it though:

[http://www.google.com/search?q=cache:http://www.yes-no-
cance...](http://www.google.com/search?q=cache:http://www.yes-no-
cancel.co.uk/2009/09/18/the-python-paradox-is-now-the-scala-paradox/)

~~~
steamboiler
Cache is not working for me :(

