
Implementing a Debugger: The Fundamentals - ingve
http://backtrace.io/blog/blog/2016/08/11/debugger-internals/
======
pklausler
Anecdote: Three weeks ago, I needed to debug what might have been a problem in
a compiler. A small change to the packaging of some libraries had caused a
test program to fail, and there was no obvious relation between the small
change and the failure that was reported by the test.

Experimentation showed that I could make the problem appear or vanish
depending on the presence or absence of a small amount of additional read-only
data to the link step of the test program. But I couldn't add any code to the
test to try to isolate the root source of the failure by printing intermediate
values or "hey I got here!" messages -- it was a true Heisenbug that vanished
if I got too close to it.

Long story short (if still possible): I ended up having to write a debugging
tool that would drive the ptrace(2) API to fork and exec two binaries, let
them run for a certain number of system calls, and then a certain number of
breakpoint hits, and then single-step a certain number of instructions.
Bisection on these three parameters allowed me to eventually find the point in
execution where the PC registers of the two images first differed. Working
backward from a trace found the conditional branch that first diverged, and
then tracing register values helped nail down the address of a memory location
whose load first yielded distinct values.

(And yeah, it turned out to be a compiler bug that had caused the right value
to be stored into the wrong address much earlier in the execution.)

Anyway, thank goodness for ptrace(2). It's pretty easy to use and it's a very
powerful tool.

~~~
glandium
You should give a shot to rr's chaos mode.
[http://robert.ocallahan.org/2016/02/introducing-rr-chaos-
mod...](http://robert.ocallahan.org/2016/02/introducing-rr-chaos-mode.html)

------
clifanatic
> if we had to fundamentally understand something in order to use it, we’d use
> precious few things

But the world would be a better place.

~~~
AgentME
A world where most people don't use debuggers?

~~~
clifanatic
No, a world where people understand them.

