
Prototyping an ltrace clone using eBPF - ingve
https://jvns.ca/blog/2018/02/24/an-ltrace-clone-using-ebpf/
======
cjbprime
Nice work!

I wonder if it would be possible to use this infrastructure to write a trace
command that I've always wanted, which is one that records _internal_ function
calls in the binary.

When I'm feeling overwhelmed with a new codebase, one of the things I like to
do to get a handle on the code is to just run some representative command (or
make a connection, if it's a server) and see a log of every function that's
called with its arguments.

We have strace to do that for system calls, and ltrace to do it for libraries,
but nothing that just.. traces. It's shocking to me to think about how many
person-years have been lost to debugging by such a command not existing in the
C world!

I've seen some hacky and unstable scripts that use gdb to set tracepoints on
"*" to get this done, but having a native binary that could do it would be
amazing.

~~~
jvns
yeah I think this would be very possible! I think the hardest part would be
the "with its arguments" part. But there's a really useful Rust crate called
gimli for parsing DWARF (I found it a lot easier than the C DWARF libraries I
looked into).

~~~
elcritch
That’d be pretty nifty! Having a program that could show the arguments would
be very useful.

On a related note, do you know if it’d be possible to store non-exec data in a
dwarf binary and parse/load it with the rust libraries? Seems it’d be easier
than doing the same in C.

P.S. Great article!

------
cthalupa
>The main problem I have with ltrace is that even though there’s a -p option
(“Attach to the process with the process ID pid and begin tracing”), I don’t
think I’ve ever been able to get that option to work. When I run sudo ltrace
-p SOME_PID, nothing happens, even though I’m pretty sure the process I’m
tracing is calling library functions.

>I don’t fully understand why ltrace can’t attach to processes, but that’s not
what this post is about.

I use ltrace to attach to specific PIDs all the time, across a variety of
linux distributions and versions. Unsure what is going wrong here when Julia
attempts to do it.

~~~
striking
I suspect she's on a Mac with SIP on.

~~~
jvns
I'm on Linux. I did a tiny bit more investigation and it seems like ltrace
works on some processes (like top) but not on other processes (like Firefox).
I'm sure that the Firefox process in question is making library calls because
when I trace the same process with `ltrace-bcc`, it shows the library calls.

------
caf
The actual ltrace uses a set of text config files to understand the types of
arguments and return values ( [https://anonscm.debian.org/cgit/collab-
maint/ltrace.git/tree...](https://anonscm.debian.org/cgit/collab-
maint/ltrace.git/tree/etc) ) - you could re-use those same files in order to
format arguments nicely.

~~~
jvns
thank you so much! I didn't know about that and using those files looks
extremely doable. I'd assumed that ltrace used DWARF or something like that to
format arguments.

~~~
caf
I believe it does it this way so that it works even if your libc isn't
compiled with debugging information.

------
hawski
eBPF is a very cool technology. LWN did a great introduction about it [1]. The
tooling is getting mature with BCC.

I for one think about how eBPF bytecode and verifier could be used outside of
Linux kernel. It's a thin and easily JIT-able architecture, designed to run
untrusted code.

[1] [https://lwn.net/Articles/740157/](https://lwn.net/Articles/740157/)

------
js2
I feel like Julia starts many of her days by saying to herself “I wonder...”
and then figures out how to answer that question and for the benefit of the
rest of us, she blogs about it!

This topic she’s clearly extra excited about!!!

:-)

