Thanks for the additional details :) I found bpftime through a social share [0], where it is shown to run bpftrace without root permissions [1], and added it to my newsletter into my eBPF learning series [2].
I have been talking about someone needing to do this recently, uprobes are largely quite slow for tracing (e.g. with bpftrace) because it has to trap into the kernel every single time, unlike tracing anything in the kernel because you're already in the kernel - so no context switch is needed.
It was always possible to do something like this, to inject the bpf code to run in-process and only export data occasionally just like the kernel side version - and the pieces already existed but no one had put all the pieces together, now someone has. Awesome!
Just need someone to implement this seamlessly into bpftrace :) That would be amazing!
If you want to understand why it's slow without this, I recently did a talk explaining the difference in tracing speed between strace and bpftrace (when tracing things in the kernel) that may help! In this case (which I didn't really cover), most function uprobes have to trap into the kernel and back twice (once on function entry and once on return, I explain why) - not as bad as the full strace example I explain in detail in the video - but still comparatively expensive.
I re-ran this talk (mostly the same, a few minor updates) at the Riga Ubuntu Summit a couple of weeks ago but the videos aren't up yet. But the version above is mostly the same and was a pretty good recording :)
A full version of the preliminary descriptions and results can be found here http://arxiv.org/abs/2311.07923. Thanks for the advise for cross boundary communication brought on the LPC, we will integrate!
This looks really cool. Like all eBPF though, what sort of eBPF verification is it doing and is it possible to do the verification in CI?
With eBPF the common gotcha is you can compile totally invalid eBPF code, with no compile errors, only to find out at go time (and only to be able to find out at go time).
I happen to work on a system-wide profiler[1] that is eBPF-based, and what we do is load our bpf programs on various kernels[2] using qemu. This is important as the jit and verifier change significantly across kernel versions and we want to make sure we maintain compatibility.
Uprobes has—I believe—DTrace USDT-like functionality. This seems interesting and different: sending the tracing facility into the traced/victim process. Whereas DTrace and ePBF typically trap into the kernel to execute and event, this approach allows for potentially much less overhead.
How important is enabled probe effect? Of that I’m less sure.
Stream: https://youtube.com/watch?v=zDNZY0HQOMw&t=1765s
The draft paper: https://arxiv.org/abs/2311.07923