

Beefing up the Python Shell to build apps faster and DRYer - bjpless
http://benplesser.com/2013/01/10/beefing-up-the-python-shell-to-build-apps-faster-and-dryer/

======
stcredzero
To make this even more robust, steal a clever mechanism from Smalltalk -- the
Change Log. Basically, all code changes and basically all of the manual
manipulation of state was kept in a transactional log, so you could be as
daring as you'd like with exploratory programming without the fear of losing
any code. (Mandatory for Smalltalk, because all coding was interactive runtime
exploratory programming.)

For iPython, you'd want to implement one change log per source file, so for a
source.py file, there'd be a source.py.log file in the same directory. Runtime
changes would first write a patch to the source.py.log file at parse/compile
time. Then if you ever crash iPython after doing an hour of exploratory
coding, iPython could reload your changes after the source.py file. (Actually,
you'd want a little ui to come up, giving you the option of leaving off the
last few changes, so you can omit something that causes a crash.)

~~~
voltagex_
Would automating Git be too slow?

~~~
stcredzero
There are problems with writing back out to source files and keeping pointers
in source files from debugging sessions indexed correctly. The change log
mechanism handily avoids this.

------
klibertp
There is also a wonderful extension to IPython, namely autoreload:
[http://ipython.org/ipython-
doc/dev/config/extensions/autorel...](http://ipython.org/ipython-
doc/dev/config/extensions/autoreload.html)

I use it with level 1 and %aimport magic all the time and it's really sweet,
especially with %ed. It probably does not work well with Django models
however. You could use deep_reload: [http://ipython.org/ipython-
doc/stable/api/generated/IPython....](http://ipython.org/ipython-
doc/stable/api/generated/IPython.lib.deepreload.html) but I'm not sure if it
would be enough.

~~~
aidos
Erm, wow. I didn't even know about IPython extensions. Or %ed for that matter.
Great tips, thanks!

EDIT: You seriously just changed my life with %autoreload

------
JulianWasTaken
If your interpreter takes 3 seconds to start up there's something seriously
wrong. Hopefully that's taking into account the human time to hit ^D and an up
arrow, plus whatever time it takes to do all your imports or whatever.
Assuming that was what was meant, then you probably really just want to learn
about `python -i`.

I agree that the real solution here is "write unit tests", but even if you
_like_ an interactive interpreter, seriously, just restart it, it should take
a half a second at most (and yes, reload() is broken by design, though
sometimes it happens not to matter).

~~~
bjpless
3 seconds includes closing AND starting the IPython shell. Typing "exit" or
ctrl + d + return adds a little time and is generally annoying.

The builtin Python shell is admittedly faster to start/stop but IPython's
features rock.

~~~
JulianWasTaken
Ah, OK, fair enough.

FWIW you might also like checking out <https://github.com/ludios/Pyquitter>. I
use it occasionally when I want something like this, you could rig it to start
iPython's main().

It also doesn't use inotify but that'd be fairly trivial to add.

~~~
bjpless
Ah I like it, thanks. From a quick glance, it seems like it just restarts the
entire child process on change.

The big (but admittedly) dangerous feature of my script is that you can
maintain state inside of the embedded shell on module reload.

I like pyquitter looks like a cool generalization, though, of the basic idea.

------
ianb
I think doctest and a UI around the doctest concept (there have been a couple)
is a better approach. There was one whose name I can't remember –
r...something. Unfortunately it used wxWindows, and was a pain to install...
native UIs suck.

But with a Doctest model you just develop a script, and if you change
something rerun the script, starting where you left off (assuming the script
reruns – if it doesn't then you probably want to start where it fails). You
can extend the script without reloading still, but changing the past requires
starting over... but it's just CPU cycles, assuming you aren't doing something
computationally expensive. But even if so, you could have a kind of cross-
session Pickling memoize function if you don't want to recalculate things.

These reloading tricks are fragile and break in weird ways, like it won't fix
badly initialized data, or classes that can't be upgraded because of state
changes. It leads ultimately to a distrust of the environment, until you throw
your hands up and go back to the old restart-frequently model like everyone
else. Recursive reloading certainly isn't new, but it's never satisfying.

------
tbatterii
seems a hard way to go to not write unit tests. FWIW, I use the shell too for
exploratory coding but the moment that I find myself making changes and
restarting the shell, chances are it's time to write it in a unit test.

~~~
bjpless
I write unit tests. This is still very useful to me...sometimes even in the
process of writing those tests.

~~~
AnIrishDuck
I usually just drop into pdb wherever I want to prototype my tests.

~~~
lost-theory
Yes, especially with nose's --pdb and --pdb-failures options.

~~~
tbatterii
or in ipython just type pdb

------
juiceandjuice
Or, you know, you could try not reinventing the wheel and use emacs.

~~~
pyre
How does emacs deal with:

    
    
      | Django models.py modules can’t be reloaded
      | normally due to the AppCache singleton

