

Runtime code modification. Erlang? No, Python - wulczer
http://blog.ducksboard.com/2011/08/runtime-code-modification-erlang-no-python/

======
jerf
It isn't that Erlang is anywhere near the only language that can do code
reloading, it is that Erlang is unusual in the scope of reloading it permits
and the care that it takes, to the point that reloading can be used in Erlang
not as a clever hack to bridge gaps, but used as a planned update mechanism
that causes zero downtime.

Since data in Erlang is immutable, you don't have to worry about what
references will be left dangling. Since Erlang has no user-defined data types,
you don't have to worry about redefining a class. (Big problem with Python
reloading, all existing objects still belong to the old class instance.) Since
Erlang is structured in terms of processes, and OTP can track which process is
which thing, OTP has a defined mechanism for taking running server processes,
examining their state, and allowing you to upgrade from old states to new
states cleanly, and since in Erlang a process is basically nothing but its
call stack and its current state, that upgrade is complete and relatively
safe. You can literally upgrade a server mid-TCP connection without the client
knowing.

Most languages, if poked properly, can change code at runtime. Even C can
unload a dynamic library and load a new version of it at runtime. It's just
that mutable values and poorly-defined boundaries between systems make it
range from effectively impossible through "usable only as a desparate hack"
(as seen here), Erlang makes it production-quality.

I say this only for educational purposes, not advocacy per se. I'd like to see
further understanding of how Erlang works so that other language communities
are more likely to pick up on the essential features of Erlang, instead of
getting stuck on the superficial ones which is the usual case. (OTP is what
you need to port, not "message passing"...)

~~~
perfunctory
> OTP is what you need to port, not "message passing"...

So true.

------
ojosilva
This reminded me of that story about Lisp at the JPL, when the author debugged
and hotswapped code midway through a mission thanks to a built-in REPL.

"Having a read-eval-print loop running on the spacecraft proved invaluable in
finding and fixing the problem."

<http://www.flownet.com/gat/jpl-lisp.html>

Since then, I always make sure I have a (safe) REPL backdoor into my long-
running Perl apps.

~~~
berntb
>>(safe) REPL backdoor into my long-running Perl apps.

Is it on CPAN?

~~~
ojosilva
Not that I know of. But it's so simple to hook input to an eval in Perl (or
any dynamic language for that matter), that I've always wrote it myself. It
can also be implementation specific (preforking server, event-loop, dispatcher
daemon...). I've used different approaches throughout the years:

\- a controller action in Catalyst that listens at <http://host/repl> which
evals the POST parameters. As a client, I use a simple admin-repl html page.

\- in the main loop of a daemon script, a quick check for a row in a DB table
or a special file in the server. When the server detects a new repl event, it
evals the contents and writes the response back (either to the DB or to a file
log).

\- a 3 line AnyEvent::JSONRPC::Lite server that listens on a given local port
for a string to eval. A matching JSONRPC client that calls that port with the
repl string. I've used it to run scripts, and recently with a Corona-based
Dancer app.

Maybe it could become a Plack::Middleware someday.

------
huyegn
We used this technique once in one of our twisted servers to quickly fix in-
memory data structures that had been loaded from old sources.

Recently, my team has been developing most of our newer network services using
gevent. It's probably the closest thing to erlang that python has when it
comes to programming asynchronous servers.

One of the things I always missed was this manhole feature, but while
rummaging around the other day, I found this:

<http://www.gevent.org/gevent.backdoor.html>

Haven't tried it yet, but it looks like it will achieve the same task :)

~~~
j2labs
Mix gevent and zeromq and you're even closer to Erlang.

------
perfunctory
> Soon enough I found the issue:

I am just wondering how long would it take if the author and the Bad Engineer
were different people.

Btw, it's still not the same as Erlang. So, I don't think the title is fare.

~~~
ericmoritz
I haven't tried this but I wonder if hot loading code could be done using the
manhole and the reload() BIF.

~~~
wulczer
Probably not, Python's reload has its fair share of caveats. There was an
effort in Twisted to provide robust reloading support for Python modules[1],
but I think it's a bit abandoned.

That being said, we _are_ using reload() with manhole to update code. You just
have to be extra careful and know what you're doing.

[1]
[http://twistedmatrix.com/documents/11.0.0/api/twisted.python...](http://twistedmatrix.com/documents/11.0.0/api/twisted.python.rebuild.html)

------
drdaeman
Sorry, but to my understanding that's not even close to Erlang's code hot-
swapping.

In Erlang one could easily update any module, and re-importing modules is
quite complicated in Python. Update will be seamless, too - old code will be
running its last reductions (so already connected clients won't experience
downtime) while new messages will be processed with a new code.

Surely, it is possible to achieve the same in Python, but that'd require a lot
more code than monkey-patching several values.

------
erikcw
I did some googling and it looks like this only works if you're running
twisted. Is anyone aware of any options that will work with Django or other
long running python processes?

------
michokest
That's one advantage of scripting languages :)

~~~
lloeki
You could do the same on compiled C by attaching gdb to the process and
changing the timeout value so this has not much to do with "scripting". Just
saying.

~~~
praptak
True but this was only about values. Changing values is not the most you can
do with dynamic languages at runtime. In Python it is possible to create a
completely new method implementation at runtime.

Not to say that dynamically changing code at runtime is totally impossible in
C - buffer overflow attacks do exactly that.

