
Mongrel2 Says, "Goodbye Python" - spahl
http://sheddingbikes.com/posts/1285063820.html
======
nailer
I understand what Zed's saying here - particularly as I came to programming
from a sys admin background.

Just like Perl before it, Python (and to some extent, Ruby if you use Puppet)
are part of the OS (Linux OS or OS X that is).

 __ _Shit can and will break if you mess with your OS._ __

You never got this with PHP because people didn't use it in their OSs.

But there's a simple solution: not only does the Ministry of Packaging want
you to use virtualenv, your OS maker does too.

The faster pip and virtualenv become standard the better.

~~~
dagw
_Shit can and will break if you mess with your OS._

Updating a program I use should never constitute "messing" with my OS, it
should constitute "using". Why should I as an OS user have to constantly worry
about upsetting the OS maker? Why should I have to jump through obscure hoops
just to install some software without breaking my OS?

Wouldn't a better solution be for the OS makers to wall off their magic python
behind some wall where I cannot see, touch nor use it, and let me install
whatever python I want without fear.

~~~
gaius
In the ancient days, stuff the OS relied on was in /usr/sbin and stuff that
was "yours" was in /usr/local/bin (or /opt) and stuff that no-one was quite
sure whose it was was in /usr/bin. And lo, peace and harmony did reign upon
the face of Unix. And the sysadmin did lie down with the developer, and it was
good.

Who even knows how it works these days?

~~~
jacquesm
If there is one thing that ticks me off about modern unix it is that to
install <insert minor package here> I end up having to install / upgrade half
the rest of the system because of all the interdependencies between packages.

~~~
gaius
Tell me about it! I just wanted to install folding-mode for Emacs the other
day, dselect wanted to install a new sodding IM client as well! (Something in
emacs-goodies-el depended on it). About 50M it came to!

So I just did it by hand, more like 50k...

~~~
johnzabroski
That is a pretty wild example, but it is common in software today.

Microsoft does stuff like this all the time, deprecating good .NET libraries
for stupid ones just because of God knows what reasons (none of them
intelligible to non-political folk).

------
gfunk911
This thread is a classic example of something that happens over and over.

Someone says something is hard to use.

1/3 of the responses agree.

1/3 of the responses disagree, say it's simple.

1/3 of the responses say it's simple, all you have to do is use additional
software package Y to manage X and it works great.

This is usually a sign that there is some merit to the original assertion.

~~~
dschobel
We're talking about technology, not politics so consensus be damned. There is
a fact of the matter independent of perception and lack of consensus in and of
itself is not a valid attack on a position.

In this case maybe we can listen to those who claim the python situation isn't
that bad and have provided specific technical solutions for evaluation.

~~~
gfunk911
If the question is "Do a significant number of users run into problems with a
piece of software," then the consensus is absolutely relevant.

~~~
dschobel
The question isn't whether there is a problem but whether there is a solution.
The virtualenv/pip people seem to think so.

~~~
zedshaw
Yes, pip/virtualenv works for installing things like Django.

It totally fails for something like m2sh which has to live in the system PATH
so that you can run it from wherever you have your configs.

But you know, I wonder if the various distros could sort of invert this and
_they_ start using pip/virtualenv instead of everyone else working around
them?

~~~
jacobolus
The distros should be shipping two versions of python (if they need), walling
their required old version off somewhere but actively and explicitly including
a newer one higher up in the $PATH so that people don’t have to mess with the
“system”. The problem is the combination of “system”, “third-party tools”, and
“users” trying to use the same python with completely different upgrade
timelines. If all of these distros included by default a recent-ish version,
and their package managers made updating that one a snap, then third-party
tools could stop trying to use system python for anything at all, and users
could forget that the “system” python existed.

Unfortunately, since the problematic old python versions are on existing old
distro versions, it’s not clear that there’s any easy way to fix this problem.

------
mahmud
Instead of inventing your own syntax, why didn't you use a portable syntax
like XML, JSON, YAML, or better yet, libconfig[1]?

Please compare the libconfig test file to the mongrel configuration file:

<http://www.hyperrealm.com/libconfig/test.cfg.txt>

[http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-190003....](http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-190003.3)

The mongrel conf is a subset of libconfig; it doesn't use the more advanced
features. Plus libconfig has bindings and versions of it in nearly every
mainstream language.

[1]<http://www.hyperrealm.com/libconfig/libconfig.html>

~~~
zedshaw
I thought about something like that, but wanted to duplicate the existing
config file format so that people's configs would keep working (mostly). It
actually had some cool advantages, like a nice psuedo variable system letting
you setup variables for reuse in different parts of the config file.

I haven't really seen that in other config formats, and it is damn handy.

~~~
mahmud
libconfig has include directives, is far more useful to multi-instance setups.
Plus you get a full type system, all in a 33k binary.

------
antileet
Hi Zed,

Can you please elaborate with some details about how "broken" python is on
systems, and the situation with python 2.4? According to the article, "recent
Ubuntu releases" had broken/deprecated python installations. A quick search
reveals that python 2.5 has been shipping as the default "python" meta-
package(?)

<http://packages.ubuntu.com/search?keywords=python>

Can you also please elaborate any "features" that the install was missing -
are these specific modules ?(I think python's sqlite3 is missing by default on
Ubuntu. I'm not too sure.) I'm not questioning you or anything, I'm just
slightly concerned as we had plans to ship a Python application, and when
someone experienced enough has a valid complaint, it makes sense to understand
the basis behind that.

~~~
acqq
I'd very much like to know why it wasn't easier for him to simply write a
Python code that doesn't use anything not existing in 2.4.

Just because he downloaded to his computer whatever later version he likes he
can't expect from all distros and all users to have exactly the same version
across of all the currently running computers, it's more than unrealistic,
it's simply impossible and always will be.

~~~
ubernostrum
_I'd very much like to know why it wasn't easier for him to simply write a
Python code that doesn't use anything not existing in 2.4._

This isn't an "oh, a brand-new version just now came out and it's not in the
distros yet" problem. Python 2.4 is _six years out of date_ at this point.
Python 2.3, which Red Hat will support until 2012, is even older (and for the
longest time Red Hat didn't even use a 2.x Python at all -- they sat on 1.5
_forever_ ).

That's simply unacceptable, and creates endless headaches for people who want
to distribute software written in Python.

~~~
acqq
Why is it "simply unacceptable" to produce the code to the least common
denominator?

Anything that is successful must have more versions in the installed base. You
can target a single version of something only if it's not used at all. It
seems too much developers live in "Everybody _must_ have that what I have"
world.

I haven't tried myself, but I can't believe Python changed that much that you
can't write the plain Python code to the least common denominator of versions
>= 2.4. If you have the specific examples why you can't I'd like to know them.

Edit: I've rechecked, he actually mentions 2.2 as the lowest version he saw,
still I believe it doesn't change too much.

~~~
ubernostrum
You _can_ write code that runs on 2.4, yes. But you miss quite a few useful
features of more recent Pythons by doing so (context managers, for example,
are a big deal, as are several of the newer standard-library modules). The
same thing used to happen with Python 2.3; you _could_ write 2.3-compatible
code, but it meant no decorators, no generators, etc.

Python has these useful features and has had them for years; why is it
acceptable to say that it will be 5-10 years before we can reliably use
anything new?

~~~
acqq
In his particular case, he certainly was not missing the newer Python modules
as he was able to write a "small" C-only replacement. That what he wrote in C
would certainly be easier to write in Python 2.4 (or 2.2)

Did you understand that he wrote the big C program just to process a small
file with the syntax as in the following example (taken from his own file):

    
    
        main = Server(
            chroot="./",
            hosts = [mongrel2]
        )
        settings = {"zeromq.threads": 1}
        mimetypes = {".txt": "text/plain"}
        servers = [main]

~~~
zedshaw
"Big"? You keep using that word. I do not think it means what you think it
means:

    
    
      $ rm src/parser.c src/cli.c src/lexer.c src/linenoise.c 
      $ wc -l src/*.c src/*.rl  148 src/ast.c
      646 src/commands.c
      267 src/config_file.c
       93 src/constants.c
       36 src/m2sh.c
       13 src/token.c
      186 src/cli.rl
      156 src/lexer.rl
     1545 total
    

That's dinky tiny, even with the linenoise and generated files it's only 4061
lines long, which isn't much at all.

~~~
acqq
My count is 4636 lines, and I can imagine much. much smaller Python 2.3 code:

    
    
        ast.c 115
        ast.h 45
        cli.c 482
        cli.h 31
        cli.rl 143
        commands.c 498
        commands.h 5
        config_file.c 193
        config_file.h 27
        constants.c 82
        constants.h 14
        lexer.c 360
        lexer.rl 121
        linenoise.c 433
        linenoise.h 40
        m2sh.c 27
        mimetypes.csql 851
        parser.c 1074
        parser.h 15
        parser.y 69
        token.c 11

~~~
zedshaw
No, most of that is generated from the .rl files or just SQL that's common to
everything. Also, you'll want to throw in all of Storm and PyREPL if you want
a real comparison.

Then again, 4600 lines of _anything_ is tiny. You have a massively skewed view
of "Big" and haven't disproven anything by finding 600 lines of cruft in one
directory.

~~~
acqq
I fully agree with you that this is more than tiny for a C project, but it can
be big compared to a small script. I also develop/maintain the projects
measured in zipped megabytes (where I don't even have a wish to try to wait to
count the lines!) I only compared it to some script solution.

You're right, 600 lines more or less are irrelevant. I just thought you're
measuring something else as I saw the total of 1000 instead of 4000. Still it
is all tiny for a real C program.

I also think that the C solution _is_ more portable than the dependence on any
version of Python. I also cross-compiled the code for 32 MB RAM mipsel
platforms and I agree that only C dependencies are better than any script
language dependencies (not counting shell, when it's carefully written).

But I actively use both Perl and Python so I'd still really like to know what
was lacking in Python 2.2 or 2.3 or 2.4, of course only in case you already
knew that you were to have _any_ Python on the target computer. But then if
you couldn't expect to have any Python, then the fact that distros still use
older versions wasn't of much relevance.

------
ZitchDog
I definitely would have used Lua in this situation. The Lua runtime is tiny
and lightweight. You can write pretty nice config files in Lua which look
pretty similar to Zed's too.

[http://windrealm.com/tutorials/reading-a-lua-
configuration-f...](http://windrealm.com/tutorials/reading-a-lua-
configuration-file-from-c.php)

------
jedbrown
This is a major problem. PETSc uses Python for configuration and we have to be
careful not to use any features requiring >2.3. The last release (this spring)
required 2.2 because RHEL3 is still supported (in production phase 3, see
<http://www.redhat.com/security/updates/errata/>) until Oct 31, 2010. Python
is easy to install, but it's a lot to ask of users when they just want to get
your software up and running. Note that RHEL4 sees end of life Feb 29, 2012
and RHEL5 not until Mar 31, 2014. So Python-2.4 support will have to live for
a very long time.

I hate this, but don't see a decent alternative. Many of our users are
building in batch environments of varying ages and just want to get their
science done. Shell/bash is not very portable, it would be painful to write
all the configuration in C (and confusing because people would need to both
native compilers and cross-compilers).

~~~
igravious
This is very interesting and is what RVM in the Ruby world was designed to
solve in some way. So maybe we need a widespread and useful PVM. This feeds
into an observation of mine I made over five years ago but that seems to be
ever more relevant as time goes by. Linux and BSD seem to have "solved" the
packaging problem through mega-repositories with dependencies and access to
source code. However your favorite language and especially the scripting ones
and virtual machines one will have there own repositories that are _not_
linked into the systems way of doing things.

So at the system level we have RPM and Yum and Ports and Portage and Macports
and Fink and Homebrew and AptGet and Conary and yada yada yada but at the
language level we have (deep breath) Perl and CPAN, Ruby and GEMS, Python and
EGGS, and Emacs and ? and Java and Eclipse plugins and on and on.

I would switch IN AN INSTANT to the distro that integrated all this so that I
would only need to go to ONE place to upgrade system packages and language
packages. Honestly if nobody does it one of these years I might even be forced
to do it myself and make mega $ € ¥.

Hope this makes sense to somebody somewhere. :)

~~~
jedbrown

        have there own repositories that are _not_ linked into the systems way of doing things
    

One approach is to have a system that packages everything in the language-
specific repository for the distro. Haskell on Archlinux is an example of
this.

I'm not a fan of actually using the language-specific packages because I like
the rolling release model and end up having to do some manual work to
reinstall all my packages when the distro upgrades it's copy (or switch to the
next 6-month release with Ubuntu and similar).

~~~
regularfry
I've no direct experience of Arch, but the traditional problem with
integrating rubygems is that it expects to be able to install more than one
version side-by-side. That's alien enough from the packge management point of
view, but when it's de facto idiomatic to specify required versions and
manipulate the load path at runtime, it makes integrating with a distro
downright fiddly.

------
fauigerzigerk
He's absolutely right. I have come to the conclusion that we must do one
thing: Stop sharing. Shared libraries and language runtimes are a relic of the
past when memory and storage were orders of magnitude smaller. Bundle all
dependencies and throw out the package managers!

~~~
jedbrown
And reinstall your entire operating system when a core library needs a
security patch.

~~~
aidenn0
You mean upgrade applications that depend on that core library. Which you have
to do anyway since a patched library will not be bug-compatible with the old
one.

That's the biggest issue, if you test with library version X and run with
library version Y, then any sufficiently complicated program will have bugs
that would have been found by testing with the same version of the library you
run with.

~~~
jedbrown
Responsible library maintainers ensure that security fixes and subminor
releases are drop-in replacements. No distro rebuilds the world under these
circumstances.

------
zeemonkee
Completely agree. The Linux distros (CentOS is the one I work with most, not
out of choice) have antiquated Python versions.

You can get round it with virtualenv etc., and I do this as a matter of course
when installing Python web applications. For a generic "plug and play" package
like Mongrel2 however, where sysadmins just want something that runs straight
out of apt-get or whatever, it's a pain and is holding things back.

I mean WTF does yum still require ye olde Python 2.4 ?

~~~
joelmichael
CentOS is about as conservative an option as you can get, though. Unusably so,
in my opinion. None of the other popular distros have packages that old.

~~~
zeemonkee
Question to manager: WTF are we using CentOS ?

Reply: because it's Enterprise.

~~~
barfoomoo
Which distro you think is the best to run LAMP stack with an up to date
package manager?

~~~
zeemonkee
Some flavour of Debian would be my preference.

Generally with Python WSGI applications I tend to keep things sandboxed using
virtualenv and pip in any case.

~~~
barfoomoo
I am complete newbie when it comes to running a server but have a bit of
experience using Ubuntu in my local box. Is ubuntu a good choice? Since most
use a flavor of Red Hat like CentOS I am a bit hesitant to use Ubuntu in
production.

~~~
wwortiz
It depends on what you are using the server for and who is using the server.

Ubuntu and debian are used by plenty of people in production and are probably
the better choice for learning how to set up a server if you already use
ubuntu.

Apt is also a great package manager.

~~~
nailer
Apt and yum are not very different. In the early 2000s, before yum Debian was
a clear choice for packaging alone, but things have changed these days.

(I'm an Arch guy these days, but hey...)

~~~
wwortiz
Every time I set up a server with yum I have to look up how to install
development tools, groupinstall just doesn't really click with me.

Pacman and apt are both better imo, and apt-get build-dep is really great when
you need to do a source install but can still use normal dependencies.

(I do use aptitude now instead of apt though)

------
joelmichael
I'm not sure why he concerns himself with old distros. Are they the customers
for Mongrel2? The description on the website emphasizes _modern_ web
technologies.

Using Rails 3 is best with Ruby 1.9.2, and yet the only popular distros that
install that via their package management system are the rolling release ones,
Arch Linux and Gentoo (where it is masked). Most people willing to use this
brand new software are also willing to compile their own necessary bleeding
edge dependencies, and probably have a generally up-to-date distro preference
(like Ubuntu).

Even the latest Debian stable, a notoriously conservative distribution, which
is a year and a half old and due to be upgraded soon, uses Python 2.6. I'm not
sure it's reasonable to support much past that.

~~~
vidarh
Upgrading Ruby to 1.9.2 won't break your OS. Upgrading Python on some distros
can even break package management. That's the difference.

And if you have a well tested, rock stable environment you generally don't
want to mess with it if you don't have to, or may not easily be able to.

You want to not support older than Lenny. Well, we have machines that are
still on Sarge, though slowly being upgraded. Some of the machines haven't
been rebooted since around the time Etch came out and that's the main reason
they haven't been upgraded yet.

~~~
mfukar
That's only because package management "on some distros" (I won't name them,
we all know which they are and I'm sure we're talking about the same one) is
fubar. I'm not talking about the package manager alone, but also the over-the-
top package dependencies, incoherent dependency resolver, let alone the sheer
stubborness of some developers working on package management..

------
bcl
FYI - Fedora 13 shipped with Python 2.6, Fedora 14 is going to ship with
Python 2.7 and 3.1 in parallel. Not every distribution ships ancient versions.

I also recommend using virtualenv and pip, these tools make it much easier to
create an easily reproducable Python environment.

~~~
ubernostrum
_Not every distribution ships ancient versions._

RHEL, meanwhile, does not ship 3.1, 2.7 or 2.6. And to be perfectly honest, if
you're seriously distributing Python software you have to take RHEL into
account -- "Fedora ships something newer" doesn't help.

~~~
jrockway
I don't use Python at all. Despite that, I still somehow have Python 2.6.4
installed on my RHEL4 box at work. (Incidentally, the Linux version number is
only slightly higher -- 2.6.9!!!)

So it seems like it's possible to get 2.6 on RHEL. I did it, and I hate
Python.

~~~
gizmomagico
I can't help but note that _hating_ Python seems irrational.

------
jgalvez
I have found that the one system that doesn't give me any trouble running
Python is Ubuntu. I'm using Ubuntu Server 10.4, specifically, which is the
latest LTS (Long Term Support) version.

Ubuntu 10.4 has Python 2.6 but I wanted Python 2.7 (which is the LAST 2.x
version). So here's what I did: I installed all common dependencies (such as
OpenSSL with "apt-get build-dep python2.6", since they're all the same for
Python 2.7. Then I installed Python 2.7 with "make altinstall" without
disrupting the system's base 2.6 install. Then, I got pip and virtualenv
installed.

I've never been happier doing Python development. pip never failed to install
a single package I needed, and Python 2.7 is fast and gives me a safe path of
migration towards Python 3, whenever the whole byte/str issue in the network
libraries (or at least WSGI) is 'resolved'.

~~~
zedshaw
Oh yeah, that's no trouble at all. <shakes head>

~~~
jgalvez
Oh well, you know, it's definitely not trouble enough to drive me away from
Python.

Despite the whole version madness across Linux distributions, Python is still
my preferred language for web development, and even if that means going
through a bit of trouble getting it properly installed on a particular system,
hopefully that is something I'll only have to do and document once every few
years.

I surely hope all distros eventually catch up with Python 2.7 so we can all
stop worrying about this. Thanks for taking the time to make some noise about
it ;)

~~~
zedshaw
It's trouble enough to drive people away from Mongrel2. Also, I find it weird
that you take me saying "we ditched python in mongrel2 because of Linux" to
mean "Python SUCKS! Stop using it." Never said the latter, only the former.

~~~
jgalvez
Sorry. Indeed.

------
FraaJad
Zed Should have used Lua(the _entire_ lua source distro weighs couple of
hundred _kilo_ bytes). Lua is superb embedded language and even better
configuration language.

No one needs to know it is Lua, (which also happens to be a marketing problem
for Lua, heh).

His decision not to use Lua the second time around is perplexing because he is
not averse to reusing third-party libraries and he is already using the code
from Lua for URL pattern matching (IIRC).

~~~
JulianMorrison
Well, he said he made the new parser read the unchanged config file format.
Presumably, lua would have needed a change of syntax.

Also possibly he decided to step outside the language wars because it's less
hassle to write your own parser than wrangle language partisans.

~~~
zedshaw
Exactly. It was part technical and part social.

------
metamemetics
So what can we learn from this for designing future languages?

Make it standard idiom to have every file of code declare its version-dialect
at the top?

Default to throwing warning if interpreter and code version-dialect mismatch?

Have the main code interpreter called by all code files simply be a host for
routing code to the correct dialect-version interpreter? Perhaps include an
optional "slim" download where the host only includes the interpreter for one
dialect-version, but make this non-default\opt-in.

~~~
points
> "So what can we learn from this for designing future languages?"

We can learn that it's inevitable that your language will come into fashion,
be hyped up to be "the language", then will fall out of fashion and be
declared 'dead'.

The idea of a '100 year language' is solid as long as developers don't choose
languages like they choose what to wear. Unfortunately though, most developers
seem to choose based on current fashion rather than anything else.

~~~
philwelch
Programming language popularity works like fashion, but not because developers
are shallow and finicky. It's because it's actually important what other
developers are using. You could have the best programming language ever, but
libraries, resources, help, and advice won't exist until you have a lot of
people using it. There's enough adventurous programmers to bootstrap this
cycle with new languages, but the general principle holds.

~~~
points
I agree to an extent. That stuff matters when you're starting out and just
learning. But after you get to a certain level the 'language ecosystem' isn't
important at all IMHO.

~~~
lygaret
Massively disagree: once I get to a certain level, there's no way in hell I
want to waste my time implementing all of (java) commons-lang, or my own
sqlite bindings, etc.

When the languages ecosystem is flourishing, I am productive. When it's not,
I'm shaving yaks that prefer not to be shaved.

~~~
zaphar
_snarky aside_

No one wants to implement all of java commons-lang in their language of choice
because they'd be lynched by the languages users. I no of no other language
with a standard library as annoyingly verbose and complicated as java.

------
doki_pen
Including Gentoo is wrong. Gentoo uses a slotted ebuild for python. You can
have many concurrent versions running. Not only that, but the latest portage
works fine on the latest 2.6 release of python. If you don't want to learn how
to use your OS, pick a different one. I hear that Macs "Just Work".

------
dschobel
Zed's pronouncements are works of art. The man is a pro. A veritable Martin
Luther for our times.

~~~
auxbuss
Indeed. I got to the statement:

"The distros have killed Python."

Said out loud, "here we go", and settled down for some classic Zed.

Can't fault him on it, though.

~~~
ubernostrum
I'd go one further: distros in general haven't killed Python, Red Hat has. By
tying their flagship OS to a six-year-old Python release, they've made it
impossible for a lot of popular software to move on and take advantage of more
recent advances in the language (since "screw everybody who uses Red Hat"
simply isn't a viable option).

~~~
gaius
This is a complete non-issue. ActiveState for example installs Python, Tcl,
etc in /opt.

~~~
jnoller
Except no one installs ActiveState. For most, _most_ of the market, the RedHat
supplied version of python _is_ python.

~~~
joshfinnie
Does RedHat still have that much of a market share in linux to demand such? I
guess I am out of the loop, but I hear way more about ubuntu these days than
RedHat...

~~~
count
The US Govt can use Redhat, but not Ubuntu (Ubuntu hasn't been Common Criteria
/ EAL tested). That's a pretty huge market right there.

------
mfukar
Python, as per Guido Van Rossum's wishes, is pushing for suspension of all
language syntax and built-in changes and work towards preserving semantics,
precisely for giving time to alternative implementations time to catch up with
CPython, as well as the various distributions to catch up with Python's recent
progress.

In short: "...have killed Python" is a hyperbole for saying "I was tired to
hear people nagging about dependency on Python". Still, it's a very good thing
what Shaw did.

~~~
dschobel
I've never heard the distro argument used by any of the python maintainers and
I would be surprised if they had.

The solution to distros which are bundling three year old software is not to
halt your progress for three years, that should be obvious.

~~~
ubernostrum
_The solution to distros which are bundling three year old software is not to
halt your progress for three years, that should be obvious._

Speaking as Django's release manager: Red Hat's continued use and support of
ancient 2.x Python versions has been a factor in the schedule we're developing
for migration to Python 3.x. We can't simply pull the rug out from under
anyone who's using Django on RHEL.

~~~
dschobel
Seeing RHEL/CentOS keep coming up led me to some googling and the first result
for "RHEL python" is <http://www.python.org/download/linux/> (about as
official as it gets) which says

 _The IUS Community Project maintains recent versions of Python for Red Hat
Enterprise Linux (RHEL) and CentOS. Installations are parallel installed next
to stock versions of python therefore don't disrupt the functionality of
critical python utilities on the system._

Doesn't that solve the issue and keep you from being hamstrung by those
distros?

~~~
ubernostrum
Not quite, because not everyone will install that stuff, or will be able to
get approval from their higher-ups to install it (folks who run RHEL tend to
be pretty conservative...).

------
Tichy
Shouldn't python just be a library dependency like any other? I was hoping the
problem of libraries with different versions was solved somehow. Apparently
not :-(

------
zokier
Doesn't every non-stale language suffer from the same thing? So either your
choices are a) using language that hasn't evolved or b) using an old version
of language which has evolved. I'm not sure that alternative b is that much
worse than a.

------
aphexairlines
"older Debian"... but Python 2.5 has been packaged in Debian stable since at
least Etch.

------
rbanffy
It's funny how, for Zed, anything that doesn't work the way he thinks it's
supposed to work is "broken" and the result of some boneheaded decision and
that it "sucks".

I too am frustrated by ancient versions of server OSs that use equally ancient
versions of Python, but that's why virtualenv was invented. And source
distributions. I am happy using it when required.

Being "stable" in OS terms is not changing APIs very often. One of the good
things is that I know that code I wrote for RHEL 4 the day it was launched
will run flawlessly on RHEL 4 today. The primary goal of such a Linux distro
is that "it _has_ to work". I won't _touch_ the system parts.

~~~
Confusion
He doesn't blame it on 'some boneheaded decision'. As far as I can see, he
doesn't blame anyone, he just notes the current state of affairs, in which the
situation is indeed pretty much broken. Not just for him, because he is far
from the only one that thinks it should work a certain way.

    
    
      that's why virtualenv was invented
    

Yes and shouldn't it also be the OSes that use it to isolate their default
Python install?

~~~
rbanffy
It's a design decision. They have a default Python install _they_ support,
validate and keep working that you shouldn't mess with. I am pretty sure you
can't ruin a RHEL just by installing RPMs from RH's repos. If the Python the
OS vendor provides is somewhat inadequate to you, it's your job to provide
another one or to use another distro. However, if you choose to write your
code against the distro's Python (something that's not that hard, really - 2.4
is a modern language) you can rest assured your code will work for as long as
the distro is supported. It's not "broken", not a boneheaded decision. It's
just the way it is.

If you require 2.7 or 3.x goodness, you can always set-up different
environments separate from the OS's Python and just be happy with it.

And, BTW, I never put boneheaded in quotes ;-)

~~~
sophacles
It is completely possible, easy and reasonable to have a multiple versions of
python installed. Many OSes do this, and the convention is to have them named
like python2.4 python2.5 and so on.

Any system scripts that require 2.4 can then change their shebang to say
/usr/bin/python2.4. At the very same time, packages for newer pythons can be
made available for ease-of-use for the rest of the world. This breaks nothing.

------
apgwoz
_Hilariously enough, I took the time to make sure the new config file format
for m2sh was nearly perfectly backwards compatible with the old file. This is
pretty funny because I predict nobody will blink an eye at the config file,
even though it's basically Python._

This is an interesting point, especially given the amount of code (relatively
small) actually needed to parse the subset of Python the config format uses.
That said, the subset is fairly "generic scripting language" since there
aren't any declarations, only assignments, lists, etc (at least at a cursory
overview).

------
mcantelon
Python needs something like the "Go PHP 5" campaign (which pressured
projects/hosting companies to drop support of PHP 4: <http://gophp5.org/>).

~~~
briancurtin
What version should be promoted? As much as I like, use, and work on Python 3,
a campaign to get distros to use 3.x would leave a lot of projects in the
dust, broken and unusable, or resulting in a rushed conversion. On the other
side of the coin, a campaign to promote a newer 2.x version is
counterproductive to eventually getting everyone on 3.x.

~~~
mcantelon
The latest v2 would be the easiest step to take.

------
jrockway
All in all, the C code looks pretty good. I browsed around in a few files to
see whether he used strn* or strl*, but it turns out he uses a proper string
library instead.

Very rare, but very nice to see!

------
JabavuAdams
After several issues with Python, Lisp, Haskell, etc., I'm coming around to
the philosophy of just generating C or C++ code from Python or Lisp.

Productivity is a function of (familiarity, language, libraries,
configuration, runtime). Other languages may be great due to (language,
library), but configuration can be a show-stopper. Especially since most devs
seem to see library/wrapper writing as a chance to pull in all their favourite
cruft dependencies.

------
forgottenpaswrd
If "python for linux sucks" I don't know what to say about python for mac or
windows.

I have a lot of python programs that use OpenGL, matplotlib, and a lot other
libraries. No problem or very little problems on Linux.

On mac 10.6 you have a limited python version and is very difficult to add
matplotlib and other libs, with Windows is a similar pain in the ass, unless
you buy it from a commercial scientific distro.

------
doki_pen
It sounds like the correct solution to this problem is a package maintainer
for the Mongrel2 package on different OSs. The package maintainers job is to
make sure a package builds and runs on a given OS. The reason we have them at
all is that it is often a difficult job. Mongrol2 systems like a system
application, no?

------
tzury
I guess it was just about time for me to try mongrel on a fresh new project. I
feel I am blessed for that only. Seeing that 0MQ batteries are included is a
great bonus.

If all would be fine, perhaps my network appliance would be shipped with
mongrel2 which would be fantastic.

~~~
zedshaw
Cool give it a shot. We've got a bunch of stuff we're working on for the 2.0
push but what you have in 1.1 should be stable and usable. If not, let me know
by dropping a ticket.

------
kfool
I would be very curious to learn why you had not packaged Mongler2 with
cx_freeze, which allows you to provide your own copy of Python with any
modules you may want to add to it.

Doing so would have made your software immune to the broken Python
installations.

------
nwmcsween
Every unix based OS deals with package resolution in a stupid way. Didn't the
old lisp machines before Linux, FreeBSD, etc solve this with self contained
state in a single object? Everything old and solved is the new and hard
problem.

------
donaq
It's sad and ironic (for Python) that Zed actually had to switch from Python
to C to make Mongrel more portable.

Disclaimer: I love Python, but I hate that anything but the most trivial code
won't run across all versions.

~~~
rbanffy
> I hate that anything but the most trivial code won't run across all
> versions.

Please, define "all versions" and "most trivial code".

I have very little problems with Pythons ranging from the ancient 2.4 to
modern 2.7. Of course, I am careful with what I do. I know if I use a
dictionary comprehension I will be limited to 2.7+, so I try not to. I am
quite sure a lot of the code I write could run under any 2.x Python with
little modification. As for 1.x, I will agree that things get more
complicated.

~~~
zedshaw
generators

~~~
rbanffy
I know that if I really need generators, then I really need to install
something newer than the 2.4 that comes with RHEL/CentOS 4.

Generators is 2.6+, right?

BTW, an ugly approach I used (more or less as a joke) was to check for
generator availability and, in case I can't use them, use a list comprehension
instead. There is a performance/memory hit but, depending on what you are
doing, it's a usable alternative. And a nice place to hang humorous comments.

But I agree that, if your users don't want to install an alien (from the
package manager POV) Python in order to run Mongrel, getting rid of python
code is the right thing to do.

It's like having a weird dependency, kind of requiring a Fortran 77 compiler
in order to run Perl...

~~~
pdonis
Um...generators are Python 2.2+ (introduction of the yield statement):

<http://docs.python.org/release/2.2/ref/yield.html>

Generator _expressions_ are 2.4+:

<http://docs.python.org/release/2.4/ref/genexpr.html>

That said, I don't disagree with removing the Python dependency from the
"core" of Mongrel2. As was noted in an earlier comment, people can always
write a config file management tool in any language they want (or Mongrel2
bindings in general, for that matter).

~~~
rbanffy
> Generator expressions are 2.4+:

So, what is the big deal? I can swear the oldest Python I had to deal with for
the last couple years was 2.4.

------
aphexairlines
what about dumping a local python install in $HOME/local/opt/mongrel-python if
m2sh didn't find a version it likes?

(yes, it's overkill)

~~~
apgwoz
This is an _insane_ idea. I should _not_ have to have a whole other version of
python just to run a _language_ independent web server. The choice Zed made is
a great choice--all the functionality of the original python m2sh is present
in the new, and there's nothing external to support, or dump somewhere else.

~~~
lincolnq
It's not that crazy. Python is designed for embedding. The interpreter is
small. Commercial software that depends on Python (almost) always embeds it.

~~~
apgwoz
I know this, however, Zed was using this for a shell command, not as an
embedded scripting language.

------
c00p3r
Narcissism and self-promotion as usual.

People who have any trouble with a system perl or python or java are shipping
their own versions within the package.

Sap Bussiness Object, for example, comes with its own perl 5.004 and Sun JDK
1.4.

It is also possible to provide an up-to-date package for most used OSes. You
cannot replace system's default python interpreter, because to much things in
distros are depended on it, put it could be /opt/python

btw, it is only Debian-based systems and RHEL who are lagging behind. Fedora
is always up to date.

I'm considering this post as just a flame-generator. There is no problem at
all.

------
baq
zed can be a very good troll. not this time, though.

what he's saying is the very reason virtualenv exists.

~~~
zedshaw
virtualenv doesn't work for a tool like m2sh, and isn't usable for people
using other languages.

------
points
Who cares?

------
known
Is anybody using Google's
<http://en.wikipedia.org/wiki/Go_%28programming_language%29>

------
terra_t
Python brought this on itself by having no respect for compatibility between
versions.

There's really no such thing as the Python language, only "Python 2.2",
"Python 2.4", "Python 3". If you want to run Python scripts, you need to have
three or four different Python runtimes installed, and that's asking a lot of
both the distros and of users who need to keep everything straight, and
probably even hack the #! at the beginning of scripts so that they work
correctly.

~~~
jnoller
Bullshit. We (python-core) are obsessed with backwards compatibility. Adding
new features that aren't in old version is the source of this problem.

Python 3 is by far, the biggest intentional deviation from the "don't break
existing scripts" mantra that's been in place for years.

~~~
terra_t
Why didn't the official bittorrent client run on Red Hat Linux (w/o
downgrading your Python) for at least three years?

Why was NLTK stuck on Python 2.4 for years? I might use Python to use NLTK,
but why do I have to choose an old Python to get it to work... And what if I
want to use it w/ software that needs Python 2.6?

