
Would You Bet $100M on Your Pet Programming Language? (2007) - jacquesm
http://prog21.dadgum.com/13.html?
======
Ixiaus
Caveat before reading my comment: this article is from 2007 and the scene 8
years later is pretty different but the spirit still holds, I think. The
author also specifically placed "pet programming language" in the article's
title which makes my reaction to it a bit unsound since, for example, Haskell
is not a "pet programming language".

WhatsApp built a 19B product on-top of Erlang. I'm building our startup's
product on a mix of Haskell and Erlang; so far it's going well.

There's pessimism in this article that neglects to acknowledge adaptability. I
agree _to a degree_ with the thought that if I chose to build our product on-
top of Idris I would be spending more time building tooling and dealing with
immature parts of the language than I would actually building the product.

This is not the case with Haskell and other innovative but mature languages /
platforms. Once at a certain size we may need to train programmers or develop
some tooling internally but I see that happening even with Java and C++.

If you can boot up the product with the language / platform effectively and
quickly, it's good enough to tackle a 100M product and adaptability can take
care of the long-tail.

~~~
sheetjs
It's funny you mention WhatsApp and Erlang, because the article actually
addresses that point:

> You're more dependent on the decisions made by the language implementers
> than you think ... some odd case that didn't matter at all for the problem
> domain the language was created for

The language implementers (employee of Ericsson) made design decisions based
on telecom constraints, exactly the same constraints faced by WhatsApp and
other messaging systems. In the parlance of the article, WA is in Erlang's
problem domain, so a strict interpretation of the author's words would have
him agreeing with WhatsApp here

~~~
MetaCosm
Exactly, it is silly to consider Erlang a "pet language", it is an industry
hardened language designed for the a near cousin to problem domain of
WhatsApp. 20+ years routing billions of calls a day under soft real-time, high
reliability requirements... because you simply expect your phone to work, even
phone calls from one end of the world to the other.

------
mathattack
Massive jobs like this tend to require a lot of inter-team dependency. The
best language is also one that everyone on the current team understands, and
that everyone supporting it will understand. And that will grow in the
direction of such a massive project.

My 2 cents... Companies have to selectively pick where they are on the
bleeding edge. Extra large projects aren't the best spot to play with the
bleeding edge of new programming languages. (This is why COBOL stuck on so
long for billing systems)

The place for experimenting with new technologies are smaller self-contained
projects where the risk is less, and the benefit can be more clearly
explained.

------
nbardy
A hundred million dollars would change my decisions, but it also changes the
nature of the problem I'll be working on. The contracts I get are on a much
smaller scale which means the problem I'm solving is also of a different
scale. The language I chose is based on the fact that my client wants their
project done on time and on budget along with the ability to change direction
when new requests come in.

If my client wants something as robust as a hundred million dollar budget;
they will need a 100 million dollar budget. More often then not they just want
something that works.

------
3pt14159
(Note: This article, while good, is from 2007 so perhaps its advice is
slightly out of date.)

Personally, if I were going in completely blind I'd choose Python (with C as a
backup). Numpy and Scipy are fast enough for most of the things that you need
to do. It's the "it does pretty much everything" language. Is it great at very
high performance games? No. But you can drop into C pretty easily (although I
find Ruby + C to be easier) and most of the code you'd be writing isn't going
to be core game logic.

Now as to why I'm not coding it now: Because I know what I'm building: Dynamic
web apps. Ember (with EmberUI) + a pure Rails JSON backend (with xdomain) will
get me there much faster than the needlessly verbose Python. It will also let
me hire people that won't have to figure out my stack. It is simpler and less
risky. I'd never write this in C.

~~~
robrenaud
What projects on the scale/complexity of Photoshop are implemented in Python?
Possibly large chunks of Youtube or Dropbox?

I love Python for small scripts, and even don't regret it too much for
programs with mild complexity (~5000 lines of Python), but the lack of
explicit typing makes figuring out what any given function/class does pretty
hard. What are the arguments? What do they do? What does the return type do?
Where is this thing initialized? Finding the answers to those kinds of
questions are greatly aided by an IDE/source code browsing that is anchored by
types.

~~~
wting
Yelp is a multi MLOC Python code base. AFAIK YouTube is slowly being ported
from Python to Go.

There are plenty of other large codebases in dynamic languages:

    
    
      - Wikipedia and Facebook written in PHP
      - Twitter was originally written in RoR
    

To be honest, it's my personal opinion that good static languages (i.e. type
inferenced) are better than dynamic languages. I believe it's situational
compared to traditional static languages (e.g. Java, C++).

~~~
jacquesm
Both twitter and facebook ran into some pretty hard limits on those platforms
which in the case of twitter led them to abandon RoR and in the case of
Facebook led them to create a compiler for the language (doable on that 100M
budget but still quite an undertaking).

------
apdinin
I have trouble accepting the general premise of the question. Not the $100m
part... even a $100k job or a $10k job... doesn't matter. It implies that the
program being developed has a definitive "done" moment. But software (at least
software in constant use) doesn't really get finished. So you build, and you
expand, and you rebuild, and new things happen that change your approach and
new tech gets developed and your software changes. I can (technically) start
coding in Language X and ultimately port the entire codebase to Language Y (or
take bits of Language A, B, and C). It's not so much about choosing the right
language as reacting to the ever-changing world in which your software is
being deployed.

------
chas
I feel like the corollary to this is how quickly one could solve the problems
in a pet programming language if $100M was on the line. If the compiler
doesn't scale well, hire one of the authors for a bit as a consultant. C with
Classes was once someone's pet programming language.

~~~
username223
... and if the baby isn't born in one month, hire nine women! A lot of the
development of mature software can't be parallelized. Paying Simon Peyton-
Jones twice as much, or even cloning him, won't make Haskell improve twice as
fast.

~~~
nhaehnle
Peyton-Jones is not the only person working on Haskell. In fact, if it were
_only_ Peyton-Jones working on Haskell, the language wouldn't be where it is
today.

Yes, the speed of development probably almost always scales sublinearly with
the number of people working on it, but beware that perception may also be
distorted by the old adage that "the last 20% of the work take 80% of the
time". That is, when people are added to a project that has finished the first
80%, it may _seem_ like progress scales terribly sublinearly with the people
added, but what is observed may simply be the fact that the same amount of
work leads to a smaller perceptible change once a project has sufficiently
advanced.

------
hacknat
A project this huge is likely to be a distributed system. I can't think of any
single binary that could provide $100m in value to one customer. Maybe some
really important financial database, but even then...distributed? Right? That
being said, we're going to have to use way more than one language, right? Like
most projects, right?

No. This question is a red-herring. I think a semi-decent point is being
attempted, but I think most Software Engineers ARE good at deciding when it's
appropriate to use the right tool.

Quick example: I'm a fan of NodeJS, but I would never use it to try to solve a
computation heavy problem, it's good at IO multiplexing, but very little
should happen in between connections. My experience has been that _most_ of
the Node community is aware of this.

~~~
imanaccount247
>I'm a fan of NodeJS, but I would never use it to try to solve a computation
heavy problem, it's good at IO multiplexing

No it isn't. It is very bad at it. It just uses the most primitive event loop
and foists all the complexity of that onto you as the developer using it.

~~~
Ixiaus
You're being downvoted but you're right.

~~~
BuckRogers
He is, and I'm seeing more of this on HN over time. Many people who use
<technology x> see a bothersome comment, and while snide- he was accurate.
People may use Node.js, drank the Kool-Aid, but in 5 years there's going to be
an industry movement off of it and the messes being created today. I've used
it, was not impressed by its technical merits, and wrote it off as yet more
technological churn.

I'm not a fan of churn, and keep a keen eye eye out for true innovation. Which
happens far less than people are convinced to believe. That's the biggest scam
the tech world convinced everyone, that innovation is rampant and fast moving,
when in reality everything moves at glacial pace.

I'm a late adopter of technology, proud of it because it's generally the smart
move for most of us. I test drive shiny things I can make time for, but
bringing it into my stack doesn't happen by reading a few blogs. It not only
requires significant technical merit, but someone has to maintain all this
shit.

------
edgyswingset
The need for reliable and extensible libraries is a huge one, and that is
largely why I am a huge fan of F#. Not that .NET is some silver bullet to
solve all your problems, but one could use F# to develop a $100,000,000 system
largely because it shares the same framework C# does.

~~~
jeremyjh
I don't know why people always say that. How many full-stack web frameworks
are there that use idiomatic F#? Haskell has at least three that were mature
three years ago and still under active development. In general the .NET open-
source ecosystem is weak; enterprise just waits for Microsoft to reveal the
One True Way or spends thousands per seat on clunky tool-kits. That same
enterprise will use hundreds of open-source Java libraries. Maybe its changed
in the last few years and certainly there is enough there to work with, but I
don't think it can rival Python or the JVM and I don't think F# has any edge
over Haskell at all when it comes to libraries and community. You get a large,
solid core to start with which is more coherent than the de facto Haskell
standards but as soon as you start looking for nice idiomatic test frameworks,
distributed application frameworks, embedded databases, FRP libraries or many
other examples Haskell really leaves it far behind.

~~~
pjmlp
Which Haskell libraries interoperate with classical enterprise stacks,
alongside existing code?

~~~
jeremyjh
There are lots of libraries for building and consuming web services or using
messaging products, which is the same as how any heterogeneous environment is
integrated. Big companies integrate .NET, Python and Java like this all the
time. There ARE also tools and libraries [1] for generating all the JNI code
necessary to call directly into JVM libraries but I don't think it has seen
much adoption. I'm not really arguing that Haskell is suitable for enterprise
applications - it is not. For greenfield SAAS product companies though I think
its as viable as F#.

[1] [http://hackage.haskell.org/package/java-
bridge](http://hackage.haskell.org/package/java-bridge)

------
elwell
Well healthcare.gov went with Java [0], but that was about 20x this budget
[1], so I think I'll have to go with Forth.

[0] - [http://www.randalolson.com/2014/05/22/programming-
language-b...](http://www.randalolson.com/2014/05/22/programming-language-
breakdown-for-the-healthcare-gov-website/)

[1] - [http://www.bloomberg.com/news/2014-09-24/obamacare-
website-c...](http://www.bloomberg.com/news/2014-09-24/obamacare-website-
costs-exceed-2-billion-study-finds.html)

------
percept
I was going to comment on a different article from the same author, but
decided this one was better:

[http://prog21.dadgum.com/57.html](http://prog21.dadgum.com/57.html)

------
im3w1l
>What would you do? And if a hundred million dollars changes your approach to
getting things done in a quick and reliable fashion, then why isn't it your
standard approach?

Ordinarily you try to strike a balance between "exploration and exploitation".
For a small project where success isn't that hugely important, the learning
that a less conservative choice offers can be more heavily weighted.

------
zzzcpan
Here's one idea: why don't we compile every new language into a pretty C code?
This way we will be able to use every available C library with a compiler of
our choice and any extra C code, necessary for our real world application.

~~~
lmm
Because if you use C libraries you will inherit their problems (and also their
interface is rarely idiomatic in the new language). Usually if you want to use
a different language it's because you think that language has advantages over
C. E.g. a large part of the point of using OCaml is to avoid the safety
problems of C, but that only applies if your libraries aren't written in C.
See [https://github.com/mirleft/ocaml-tls#why-a-new-tls-
implement...](https://github.com/mirleft/ocaml-tls#why-a-new-tls-
implementation)

(Almost all languages do offer _some_ "FFI" to call C. But in languages that
don't naturally fit, there will often be significant overhead to e.g. aligning
memory management so that the same memory doesn't get freed twice, as might
happen when a garbage-collected language called a C library)

~~~
zzzcpan
That's why you would compile it into C instead of having hand-written C when
safety is important and for most part of your program it probably is. But
sometimes you absolutely need to be able to write assembly and use existing
unsafe C libraries.

~~~
lmm
If you absolutely need a C library, almost all languages give you a way to do
that. But why hobble your compilation strategy for all programs just to
support this occasional use case better? You may well want to target a runtime
(JVM, .net, Javascript) for which no good C compiler exists. Even if you're
solely interested in building native executables, compiling to C would mean
throwing away a lot of information (e.g. knowing which values are immutable)
that an optimizer could use. Look at Haskell; performance-optimized Haskell
performs a lot better than machine-generated C ever would (indeed, it often
comes close to the performance of hand-tuned C).

(If you're worried about duplicating effort in optimizers, do what many
languages do and offer an LLVM frontend. LLVM optimizations are (mostly)
language-independent, and LLVM bytecode, while imperfect, often allows a
better representation of language semantics than C source would)

~~~
zzzcpan
I don't think it is so rare, that you could call it occasional. That's the
author's point and mine as well. Sometimes I need sqlite, other times libjpeg,
giflib, libpng and some simple image processing with at least some low-level
code. And it takes a lot of effort to make these things work with other
languages on different platforms. Golang tried to make it easier and it still
isn't.

Optimizers is also one thing I was worried about, LLVM is not the only
compiler out there. And also there is friction. Most of the systems already
have C compiler installed, if you target it, there is no friction in setting
up building environment and so on.

~~~
lmm
> Golang tried to make it easier and it still isn't.

How much of that is because Golang doesn't compile to C, and how much is
because the languages just don't fit? E.g. you'd still need a way to mark
whether Golang was supposed to garbage-collect a value that had been returned
from a C call, or not.

> Most of the systems already have C compiler installed, if you target it,
> there is no friction in setting up building environment and so on.

Yes there is, because you still need to build the thing that interprets to C.
If anything there's _more_ friction than having a compiler that directly
builds native binaries like e.g. ghc, because you need two tools - your
translate-to-C tool and your C compiler. "Most systems already have a C
compiler" is only really true on *nix; it will be easier for a Mac or Windows
user to run your system if you distribute e.g. java bytecode.

------
BuckRogers
If $100M were on the line you'd get a mess that met your requirements.

If I wanted good code, I'd take guys who worked on a project for free.

------
lwh
The language is irrelevant at that price. Given an architect of sub-average
skill they'll pick an acceptable one.

------
michaelochurch
Yes. There's risk with every technical decision. I'd probably be inclined to
use Haskell, for access to a top-notch community and a level of robustness
(esp. in the face of refactoring) that, while it can be achieved in a dynamic
language, is hard to hit without a static type system.

An upshot of Haskell is that it probably wouldn't take a $100 million budget
because you can do a lot with a small team. We're going to need far fewer
people than on a Java project, and you won't need the layers of management
that come with big teams, and you'll also have smaller code and more of an
ability to use the Unix philosophy (systems compromised to multiple, usually
small, programs). Those wins aren't additive, but multiplicative.

So, yes, I'd definitely trust Haskell (or any mature FP language) on a $100M
project. I'd trust Java on one too-- if I could find someone to do all the
work, and trusted that person-- but I'd rather use an FP language and really
kick ass.

------
lmm
I was nodding along until I realized that the author has a very different
notion of esoteric from mine.

The last month of Fridays I've been tinkering with Idris, trying to get it
to... well, do anything useful. I'd heard it was a dependently typed JVM
language, which it is... kinda. Turns out you need to install a full Haskell
toolchain to do anything with it. With the right esoteric option incantation,
you can make it build an executable that turns out to be a shell script
concatenated with a jar file. Nice. But you can't build a library with it,
even within the library. You can't invoke it to build some classes to use
later. You can't even call the build tool from anything remotely standard in
JVM-land, and when I asked about making the compiler selfhosting the response
was a kind of lukewarm "yeah, sometime". The current toolchain is in Haskell
but there's not even an FFI from Idris into Haskell, so porting it would be...
challenging. I tried to join the two together via their C FFI, but in between
undocumented linker options and the fact that the Idris runtime is already
linked to part of the Haskell runtime, I eventually gave up. If nothing else,
it gave me a real appreciation for how much work went into making Scala a
serious, commercially usable language, something I'd previously rather taken
for granted.

But C, holy shit, C? You'd write a program in C? And expect it to work? I've
seen programs go wrong in a lot of languages, but most of them can be
eventually fixed. In C you get irreproducible voodoo random crashing that,
sure, you can _usually_ track down with a static analyzer, valgrind,
debugging, intelligence and luck. But what if you couldn't? What if the
program was just broken, and you couldn't fix it? With $100,000,000 on the
line, there is no way on earth I would risk letting C (or any other unmanaged
language) anywhere near my codebase. It might mean more work, and less
features, if I were to use e.g. that pure-OCaml SSL library, or a JVM-native
multimedia library. But I'd do it in a heartbeat, all the same. "The final say
on overall data sizes" is such a tiny, trivial concern compared to using a
language where failure is understandable, reliably diagnosable.

(In fact I'd say the scenarios where a measly factor of 2x memory consumption
makes the difference between "working" and "not working" are just vanishingly
narrow. If you need to scale horizontally in OCaml, you're going to need to
scale horizontally in C a couple of weeks later. Particularly with
$100,000,000 on the line, sod it, buy a bigger server with more RAM if using
32GB rather than 64GB really makes the all-important difference).

Would I use a "pet language" I tinker with, like Idris? No, but I wouldn't use
that for any kind of serious commercial work. With $100,000,000 on the line,
I'd use the same language I use for almost all my work: Scala. And I would
stay the hell away from JNI, because I don't want C anywhere near this system,
lest it bring it all crashing down.

~~~
phkahler
>> But C, holy shit, C? You'd write a program in C?

Most of the worlds software rides on the back of C. Pick a language, the
runtime is probably written in C. Any high performance libraries are either in
C or C++ (QT anyone?) and for numerics you may still find <gasp> Fortran. If
performance is a requirement or running on a micro controller, you're going to
have some C or you fail to get the best performance. That said, stuff like
string processing sucks in C - some say that's why C++ was invented.

~~~
pjmlp
That is an historical accident of UNIX widespread into the industry.

I am old enough to remember C wasn't even an option unless the customer was
willing to shell out money for UNIX systems.

------
comment38296
is the quality of HN commenters going down, perhaps?

1.) It's about strategy. re-read Sun Tze, Art of Strategy (and avoiding war).
The title is amistranslation cleverly designed to fool non-native LANGUAGE
cunning linguists. - english joke - cunn ing lin gui st - end joek.

2.) those who armed welll with the RIGHT language, did not need to fight or
program well. that's why Haskell or Ocaml is at the 2nd stage.

3.) But it AINT COQ or provable...LANGUAGE.

4.) what's the second point U missed? Yes, the weak spot is "inter-team
dependency" \- ref: mathattack post. Yes, the ??? is true and necessary.

BUT THE CONDIITION is NOT SUFFICIENT for AI or artificial intelligence.

8.) so what is the closest to ai. Seems to me prolog, but darn... most of the
obscure conferences are in japanese....language barrier.

9.) We conclude the 'lesson of the day" by quote: nbardy: "client wants their
project done on time and on budget along with the ability to change direction"

So, THE CLIENT wants it FREE, Yesterday (get the time machine working) and
High quality and CHANGEABLE - agilie.

Yes, of course, as long as you pay up front, I THE VENDOR guarantees ALL FOUR
ASPECTS in the 'game.'

No wonder, the track record of software is so BAD compared to engineering,
building of the Pyramids in Egypt, farming using bio-dynamic techniques, other
human endeavor - spacecraft.

addendum: fun note: just simply download basic OPEN SOURCE that includes
libressl, openssl, gettext GNU, etc.

SOME OF IT DONT WORK, at least on basic systems.

