
Reverse Debugging for Python - scribu
https://morepypy.blogspot.com/2016/07/reverse-debugging-for-python.html
======
wyldfire
First we get awesome CPy C-extension emulation and now this? Thanks, team!

Completely serious: when can PyPy obsolete CPython? Why don't more CPython
core devs work on PyPy? Besides perhaps some remaining gap for C-extensions
support, what else is missing?

~~~
d33
Platforms I guess. Plain C code is much more portable than JIT and PyPy is
reinventing JIT implementation as opposed to using something like LLVM, so
they're much more limited in terms of supported target platforms.

~~~
wyldfire
Oh, my mistake -- I had assumed that it did use LLVM. Good point about
portability, CPython has tons of supported targets and PyPy just a handful.
Also CPython is _really_ easy to build IMO.

~~~
loeg
LLVM isn't as widely portable as generic C, either. ARM and x86 are well
supported in LLVM, but CPython runs fine on MIPS, Sparc, PPC, etc (via GCC or
other C compilers).

------
gregthelaw
Great to see. I'm biassed of course, but reversible debugging is the future
(or should that be the past? Sorry.) If you'll excuse the shameless plug,
[http://undo.io/](http://undo.io/) for the same with C/C++. I'd like to think
we were at least a little influential in this! :)
[https://vimeo.com/160863576](https://vimeo.com/160863576)

Greg

~~~
tperry98
Also worth noting that Undo is hiring: [http://undo.io/about-
us/careers/](http://undo.io/about-us/careers/)

------
baq
amazing as usual :)

no python 3 as usual :( 99% of my production code is python 3.

~~~
masklinn
> no python 3 as usual :( 99% of my production code is python 3.

Contribute work or funding[0] to pypy3?

[0] [http://pypy.org/py3donate.html](http://pypy.org/py3donate.html)

[1] [http://doc.pypy.org/en/latest/release-
pypy3.3-v5.2-alpha1.ht...](http://doc.pypy.org/en/latest/release-
pypy3.3-v5.2-alpha1.html)

~~~
corysama
Is the tech world really holding back pypy3 for a lack of $40k? I would expect
there are multiple _individuals_ in this thread who could top that off without
feeling much sting. Let alone the _tech companies_ they work for who would
benefit greatly from it.

~~~
flubert
Just out of curiosity, at what level of wealth or yearly income would you
think someone would have to have in order to drop $40k on a project like that?

~~~
corysama
I meet a lot of different engineers in different situations around SF all the
time. Recently met one who is at a point that he owns several (>2) Teslas...
Someone with $400K in liquid assets could do it without being embarrassed. I
would be surprised if there wasn't as least one person reading these comments
in that situation. Would that person care deeply enough to say "Here! Just
finish it!" ?? Not likely. However, I'm very surprised that there isn't at
least one multi-million dollar company who would value this higher than few
person-months of internal engineering time.

------
d33
Cool, but why do you have to disable ALSR for recording to work?

~~~
DanWaterworth
Here's a python program that should be deterministic, but isn't without
disabling ASLR:

    
    
        print id(object())

~~~
masklinn
Why should this program be deterministic?

~~~
DanWaterworth
Ah, you've caught me out on my sloppy use of English.

What I should have said is that the address space layout is a source of non-
determinism that would otherwise need to be accounted for in the recording of
a program.

Further, I hold to the view that given a program, knowledge of the thread
execution schedule/scheduler and the results of external calls, a program's
execution should be deterministic.

Thinking about this now, I realize disabling ASLR shouldn't be enough to fix
this problem since recording and replaying are sufficiently different.

~~~
masklinn
> What I should have said is that the address space layout is a source of non-
> determinism that would otherwise need to be accounted for in the recording
> of a program.

Ah yes, it _needs to_ be deterministic in the context of a time-traveling
debugger, I was thinking about the general case, sorry.

> Further, I hold to the view that given a program, knowledge of the thread
> execution schedule/scheduler and the results of external calls, a program's
> execution should be deterministic.

Surely it is, if you precisely know all timings and IO and can replay them
there's nowhere for non-determinism to creep in.

That's not really an acceptable scale of required knowledge for a time-
traveling debugger though.

~~~
DanWaterworth
> Surely it is, if you precisely know all timings and IO and can replay them
> there's nowhere for non-determinism to creep in.

This was my original point, the address space layout information leaks into
python land via things like id.

> That's not really an acceptable scale of required knowledge for a time-
> traveling debugger though.

That's what recording captures (though, you don't need to know the times at
which things happened).

------
lucio
Shouldn't his be a "reversible debugger" ? or a time-travel debugger as the
Elm debugger is called?

~~~
lgas
No, the way things work is that people name things whatever they want.

------
denfromufa
How is this more powerful than what I can already do in PTVS or PyCharm? I can
move around the current executing line freely back-and-forth!

~~~
corysama
I believe in PTVS you can set the current executing line higher up in a
function and execute those lines _again_ , thus further mutating the state of
the program. But, with this, you can roll back to earlier points of execution.
Thus, un-mutating the state of the program. You can continue to roll backwards
in execution to before the function was called, further and further until you
observe some earlier code making a change that eventually led to the problem
you are debugging.

~~~
denfromufa
Then I guess [http://www.pythontutor.com](http://www.pythontutor.com) could do
this long before!

~~~
corysama
Can it do this for arbitrary programs running on your command line?

~~~
denfromufa
Not sure, seems only in browser

------
Dowwie
Does the author provide no link to the project source or have I missed it?

------
holri
Isn't logging enough to track down a lot of such problems?

~~~
heartsucker
If you logged the stack at every step along with every variable in memory,
yeah. The point of debugging like this is to not have to insert 40 log.trace()
statements with 30 variables just to track it all down. It's interactive and
does it all for you.

~~~
holri
Usually I do not need every variable in memory. Often the relevant spot can be
very well circumscribed.

