
The perils of Lisp in production - fogus
http://symbo1ics.com/blog/?p=1321
======
jwr
You can write spaghetti code in any language.

Whether enforced compile-time strong type checking is a benefit seems to
depend on the programmer. It apparently helps some people, it certainly does
not help me.

For what it's worth, I've been building "production" (is "production"
something you make money on? I find this word increasingly vague) systems with
both Common Lisp and Clojure for quite some time now. I prefer Clojure. But
both languages are lisps. My thoughts so far: you can build spaghetti code in
any language. You can either use a language that lets you run your spaghetti
quickly, or one that whacks you on the head repeatedly thus making your
spaghetti stiffer and straighter. But you end up with spaghetti anyway.

I agree that it is difficult to write good lisp code. In my code, I spend a
lot of time thinking about contracts and data structures. If I am not careful,
I end up with problems later on. But using a language like Java doesn't solve
that: you just get the illusion of "better code", because your spaghetti
design is now codified into neat factories and patterns.

The advantage of using a language from the lisp family is that reworking your
spaghetti into something better is much easier, if only because there is so
much less code.

~~~
dons
Have you ever worked on a 1M line code base? Or with dozens (or hundreds!) of
developers.

Once the team gets big, or the code gets so big you can't hold it all in your
head, _any_ machine-enforced QC can have a major impact.

(I'm working on a very large Haskell code base now -- the parts that tend to
cause trouble are the dynamically typed bits -- they can just go wrong in so
many unanticipated ways, and the complexity means its almost unavoidable that
devs miss things).

I'm increasingly convinced that to wrangle massive code bases, you need both
heavy abstraction abilities (higher order functions, DSLs, algebraic data
types, polymorphism), _and_ very strong typing, to keep the complexity under
control

~~~
rbanffy
A 1M line codebase can mean many things. It can mean code that belongs to many
systems mashed together as if it were a single thing. It can mean that the
code that should belong to different systems is tightly coupled into one
monolithic entity that should have been several little entities. It can mean
lots of boilerplate code too. It can indicate a lack of timely refactoring on
an aging codebase so interconnected nobody has the courage to separate in more
manageable pieces. None of those can be solved by clever language choices
alone.

You also mention things going wrong in unanticipated ways - this may signal
that the problematic code where errors bubble up is not problematic at all -
it is called by code written by people who don't really understand what the
functions do and who probably didn't write adequate tests for that - because
the tests should catch the unanticipated parts. The problematic code is the
one calling the parts where errors bubble up. The canary is not responsible
for the gases in the mine.

While you may be right that, in order to deal with multi-million-line
codebases you need static typing, I'd much rather split that codebase into
smaller units that could be more easily managed.

Wearing a straitjacket is often a consequence of an underlying condition that
can, sometimes, be corrected.

~~~
route66
I would expect that dons is talking about a 1M codebase which already consists
out of manageable pieces. It's only, that the pieces have to work together,
talk to each other, know about each other (but not too much).

Sometimes software solves problems which provoke incidental complexity because
of sheer size and my experience (albeit not above 500k) tells me that indeed,
all bits help, also compiler enforced type checks. I would never bet my life
on tests. As you write: "because the tests should catch the unanticipated
parts", that's the point: tests never catch unanticipated parts by their very
nature. Sometimes, by sheer luck, yes.

~~~
rbanffy
> tests never catch unanticipated parts

No, but calling a function with arguments of the wrong type is something tests
would catch.

~~~
maximilianburke
> No, but calling a function with arguments of the wrong type is something
> tests would catch.

Or the compiler.

~~~
rbanffy
A compiler can only go so far. It'll happily compile:

    
    
      #include <stdio.h>
    
      int main(int argc, char *argv[])
      {
        FILE *fp = fopen("outfile", "w");
        fclose(fp);
        do_something(fp);
        return 0;
      }
    
      int do_something(FILE *f)
      {
        fprintf(f, "This should work if you know what you're doing");
      }
    

And it won't work.

~~~
maximilianburke
In C++, proper scoping with RAII (similar to CL's with-blah idioms) would
alleviate that issue as do_something in the code below won't compile.

    
    
        struct WithOpenFile {
            FILE *fp;
            WithOpenFile(const char *fileName) : fp(fopen(fileName, "w")) {}
            ~WithOpenFile() { fclose(fp); }
        };
    
        int do_something(FILE *f)
        {
            fprintf(f, "This should work if you know what you're doing");
        }
    
        int main(int argc, char *argv[])
        {
            {
                WithOpenFile file("outfile");
            }
            do_something(file.fp);
            return 0;
        }
    

But Lisps will happily accept this:

    
    
        (defun do-something (foo)
            (1+ foo))
    
        (with-open-file (file "outfile")
            (do-something file))

~~~
rbanffy
Fine. But now you'll have to accept both files and network sockets. What do
you do?

~~~
anonymoushn
Pass the int representing the FD instead of the FILE*?

~~~
rbanffy
And then you are back to dynamic typing again, except that, when the integer
you send points to something fprintf doesn't like you'll get a segfault
instead of a doesNotUnderstand.

------
brlewis
If you presume that it's OK to rely on compiler errors rather than attempt to
exercise all code paths, then yes, Common Lisp, Scheme, Clojure, JavaScript,
PHP, Python, Ruby, Perl, and bash will be more perilous than C, C++, C#, Java,
ML and Haskell. (Which of the former group are Lisps?) I disagree with the
presumption.

~~~
rbanffy
I think the most dangerous piece in such system is, indeed, the programmer who
thinks the compiler will save him from himself.

~~~
cageface
Much less dangerous, in my experience, than the programmer that thinks he can
keep a huge system all in his head. I've moved back to static languages after
over a decade with dynamic languages and I couldn't be happier to let the
compiler take care of a lot of low-level consistency issues so I can focus on
the code.

~~~
ataggart
An alternative approach is to avoid building systems that require you to keep
the whole "huge system" in your head at once.

------
hbbio
Should I guess (from the domain name) that the link is running on Lisp? I
can't get past that!

~~~
prodigal_erik
Surprisingly no, blub strikes again. But this is just some guy's blog, I don't
see any definite connection to the Lisp Machine pioneers.

    
    
      Server: Apache/2.2
      X-Powered-By: PHP/5.1.6
    

HN itself is basically pg's testbed for prototyping <http://arclanguage.org/>
though.

------
enduser
Summary: use SBCL or CMUCL, declare your types, hire good programmers, and
don't commit to a deadline without getting a commitment to a specific set of
requirements.

------
lucian1900
Statically checked types are little more than very broad pre/post conditions
on your code. Both CL and Clojure offer such conditionals and both can do it
at compile-time if necessary.

I don't get it.

~~~
Kototama
Pre/post conditions at compile-time in Clojure? Nope. How could you know the
value returned by a function at compile time? Or the value of the inputs?

You could only know the types of the input if the compiler was smart enough
and define constraints on the types but as far as I know it's not possible in
Clojure.

~~~
lucian1900
You're right, Clojure doesn't support compile-time conditions. Must've been
confused.

------
peteretep
I thought the main peril was that the beards of the programmers would get
caught on things...

~~~
e40
This so childish. If this article was about Python or some other language,
this type of comment wouldn't be tolerated. Why is it when the topic is lisp?

~~~
peteretep
I like this Larry Wall quote:

"By policy, LISP has never really catered to mere mortals. And, of course,
mere mortals have never really forgiven LISP for not catering to them."

I say this as someone who's prototyped several things in Scheme before porting
this to other languages.

