
Ask HN: Modern approach to web application development with Common Lisp - kung-fu-master
Recently, we have started developing web application with Django. Everything is wonderful, but Python is not Lisp and I feel uncomfortable with Python. In the sense that with Python &#38; Django I am restricted. I can't build eDSL to describe application workflows, eDSL for DB access, etc.<p>I want to try rewrite it from scratch in Common Lisp. What web technologies stack to use?<p>SproutCore + Hunchentoot + MongoDB is it right choice?
======
vog
Personally, I'd leverage on PostgreSQL rather than MongoDB, because I think
relational databases are still the best fit for the majority of modern web
applications.

Unless, of course, you have a really simple schema so that you want to use the
DB as a mere persistence layer. Or you know from the beginning that you'll
have to scale really huge. But then, you'd need a highly customized framework
anyway, which would somewhat defeat the purpose of writing a general purpose
framework in the spirit of Django.

Also, PostgreSQL is allows to be _extended from inside_ , thus working much in
the spirit of Lisp. It can be extended with plain C code, but has also
language support for pgSQL, Perl, Python and Tcl:

[http://www.postgresql.org/docs/current/static/server-
program...](http://www.postgresql.org/docs/current/static/server-
programming.html)

Unfortunately, language support for Lisp languages is missing yet, but this
might be only a matter of time (and would be really great).

~~~
escanda
Following the above, if you're looking for an object store you'd find
interesting cl-prevalence which is entirely written in CL.

About web frameworks others have mentioned AllegroServe, which looks quite
good with the logic and view separation in templates and actions, similar to
Django. Other people prefer to code the HTML inline with cl-who i.e.

If you're developing an application in the web you may find worth a
continuation based framework like weblocks or UCW. Cliki contains a fair
amount of libraries categorized. Here you have the database libraries:
<http://www.cliki.net/database>

~~~
sedachv
bknr-datastore works on the same principle as cl-prevalence, but is actively
developed: <http://www.cliki.net/bknr-datastore>

------
Zak
With any luck, mahmud will be here to talk about real-world CL web
development. He's been using CL for web stuff in his startup for a couple
years, I think.

Another approach for people who are just trying to get something done rather
than write a framework to consider is to use CL to do the interesting bits and
something else for the web frontend. I've built a couple webapps in CL that
would simply have been easier in Rails.

~~~
silentbicycle
Mahmud's traveling and offline for a few weeks
(<http://twitter.com/#!/BigThingist/status/1947382564519936>), but he's
answered variants of this question before - try searching the HN archives.

------
gracchus
Hunchentoot is a bit odd, but works well. In particular, the fact that
handlers (think, like prefix-matching dispatch functions) are global, rather
than per-listener. There is, in general, an apparent tendency to use _var_
globals for expediency's sake, but they tend to infiltrate code, and make
things harder to debug.

I concur with other commentators that PostgreSQL is worth looking at. I wrote
the postgres bindings for what is now CL-SQL many years ago (hi stringbot!),
and postgres is fast and reliable (aside from SBCL having interrupt hander
sensitivities during ffi calls), though again, the CL-SQL interface is
littered with globals, the connection pooling is a bit .. unusual .. and the
reader macro syntax probably a pretty bad idea, all in all. If hunchentoot and
cl-sql were mildly rewritten with an eye towards looking more like, for
example, Java's JDBC and Servlet interfaces, you would have a simple base
environment where you could reap most of the big CL wins immediately (DSLs,
CLOS, coding closer to the speed of thought, etc.)

The 'IMHO' sources are kicking around here: they suffered a bit from
WebObjects envy (hey, it was the 90's..) and do somewhat suboptimal handling
of session state. It's still not clear to me how OO you really want to get
with the server-side representation of a page, especially in light of all the
nice JS client-side component kits that are around.

Mull.

------
macmac
How about going 3/4 of the way to Common Lisp and try Clojure instead. Great
lightweight web + mongo libs + all the java libs.

~~~
pavelludiq
Yes, taking a serious look at clojure is worth it, the language is awesome,
the libraries are maturing and growing in numbers, and the community is
awesome. But i wouldn't bet against common lisp either, im currently learning
it, and im amazed, clojure programmers can only gain by learning it.

------
stringbot
Back in the day (around the turn of the century) I worked for a startup
building a web application using Common Lisp. We used a framework called IMHO,
which was developed in-house by a couple of the lead developers. IMHO seems to
have been deprecated, per <http://www.cliki.net/IMHO> and the source code is
nowhere to be found. I am still in touch with the authors, so maybe I can
encourage them to make it more available.

Using Lisp for a web dev platform was pretty nice. I was a support engineer in
those days and I always found it useful to be able to pull up a console and
inspect the actual running sessions to debug problems in production.

------
gibsonf1
We've been using Portable Allegroserve + mysql (running on SBCL on EC2) with
very good results.

~~~
mark_l_watson
I second the Portable Allegroserve advice. It is an older project, but John
Foderaro really put a lot of good functionality in Portable Allegroserve. That
said, Edi's Hunchentoot project is very popular.

------
smanek
I'd hesitate about MongoDB - I'm not aware of a mature Mongo-backed ORM, for
starters. Personally, I like <http://marijnhaverbeke.nl/postmodern/>.

~~~
sqrt17
Since you directly pull JSON objects out of MongoDB (i.e. usable data
structures), MongoDB lends itself to a slightly different programming style
than you'd have with a relational database and an ORM. I'd hesitate to call
either one more idiomatic in a lisp-ish programming style.

EDIT: That said, MongoDB's reliability model (you have to have a synchronized
pair of DB servers to get the reliability you get from a single PostgreSQL
instance), PostgreSQL may be the safer choice.

------
vii
How about tpd2? If you're comfortable rewriting everything from scratch, at
least you'll be working from a high performance foundation --
<http://github.com/vii/teepeedee2/tree/master>

Weblocks is a very good concept if you're willing to get your head round it,
and work within its worldview (which is about displaying widgets rather than
manually generating HTML).

Everybody uses parenscript, which is a great concept.

~~~
sedachv
Both TPD2 and Weblocks rewrite your code using a continuation-passing style
transformer, which makes web apps written in them harder to debug than they
need to be. It also makes testing your apps or doing any kind of automated
HTTP client scripting much harder than it needs to be. There's no upside to
managing app state with continuations if you are able to use AJAX.

------
WalterGR
_SproutCore + Hunchentoot + MongoDB is it right choice?_

Even then you're not yet off to the races. In particular, Hunchentoot's
session management capabilities are anemic out of the box.

I haven't seen any drop-in replacements, so you're looking at reinventing at
the very least this extremely core part of any web development stack.

Hunchentoot also doesn't have any built-in capabilities for handling updates
to the website's source code.

~~~
sedachv
"In particular, Hunchentoot's session management capabilities are anemic out
of the box."

Do you mean user authentication? What would be really nice to have is a user
authentication library that offered a generic protocol that could be used with
several web servers and persistence mechanisms.

"Hunchentoot also doesn't have any built-in capabilities for handling updates
to the website's source code."

You just load the new code and it works.

~~~
WalterGR
_Do you mean user authentication?_

No, I mean session management. Example of anemia: it offers no way to back
sessions by persistent storage.

 _You just load the new code and it works._

Really? What about threads that are busy processing requests when you load the
new code, which may have different assumptions?

It seems like a better approach would be to queue up incoming requests, wait
for existing requests to be handled, reload the code, and then resume
processing requests.

~~~
sedachv
"No, I mean session management. Example of anemia: it offers no way to back
sessions by persistent storage."

The session protocol is a bit awkward in that you need to implement all of it
if you want to change only a small part, but it's there to be hooked into,
which is what the frameworks built on top of Hunchentoot do. The support is
there to have persistent session objects, you just need to find something that
does that (I agree it would be great to have libraries independent of
frameworks but built for specific DBs for this) or do it yourself (all that
Hunchentoot requires for this to work is to implement two methods).

A lot of the time what you'd really want to do is subclass the session class
to make it persistent through an ORM or prevalence, but the problem is that
you can't just drop in arbitrary metaclasses into arbitrary subclasses in
CLOS.

"Really? What about threads that are busy processing requests when you load
the new code, which may have different assumptions? It seems like a better
approach would be to queue up incoming requests, wait for existing requests to
be handled, reload the code, and then resume processing requests."

You're talking about atomic change management. Queueing up requests works on a
per-request basis, but what happens when your code changes affect existing
sessions (for example, people in the middle of a multi-step order confirmation
form)? The easiest way to do this is have multiple servers and switch them
over one by one. In the more general case you need to think in terms of
protocols and protocol versioning
([http://carcaddar.blogspot.com/2009/03/p-cos-blew-my-
mind.htm...](http://carcaddar.blogspot.com/2009/03/p-cos-blew-my-mind.html))

~~~
WalterGR
_Queueing up requests works on a per-request basis, but what happens when your
code changes affect existing sessions_

Good point.

 _The session protocol is a bit awkward in that you need to implement all of
it if you want to change only a small part, but it's there to be hooked into,
which is what the frameworks built on top of Hunchentoot do._

I notice you used the plural "framework _s_ " built on top of Hunchentoot. Are
there any aside from Weblocks?

~~~
sedachv
RESTAS (<http://restas.lisper.ru/en/>) and web4r (<http://web4r.org/en/>) are
both built on top of Hunchentoot.

------
sedachv
Take a look at RESTAS, I think it has some good ideas, and doesn't try to
force you to do too many things in certain ways, unlike many other CL web
frameworks: <http://restas.lisper.ru/en/>

------
pge
as someone who has written webapps in both lisp and python, I would recommend
sticking with python. you have to choose the right tool for the job and get
comfortable working with different languages. I am a huge fan of CL, but if
you go down that road for a webapp, you will spend a lot of time filling in
gaps in the framework instead of focusing on the functionality of your app.

Django may be the wrong tool for the job in your case. If so, I recommend
exploring some of the more flexible python frameworks like pylons before
making the switch to common lisp.

------
gruseom
If you're building web apps in CL you should take a close look at Parenscript.

