Hacker News new | past | comments | ask | show | jobs | submit login
“Give me 15 minutes and I'll change your view of GDB” [video] (undo.io)
425 points by DebugN on July 15, 2016 | hide | past | web | favorite | 169 comments

A summary of the 15 minute video:

1. introduction to 'tui' mode in gdb (https://sourceware.org/gdb/onlinedocs/gdb/TUI.html)

2. intro to python support in gdb (if it was compiled in)

  2.1 python print('hello world')

  2.1 set breakpoints with python gdb interface
3. Demo of reverse debugging in gdb, which helps isolate a stack-corrupting bug (https://www.gnu.org/software/gdb/news/reversible.html)

GDB + Python is really slick. I wrote a Python C module for the first time last month and GDB really came in handy for that. It just worked.

Yes. It can print the value of C variables and Python objects, using their _str() method I assume. Extremely useful. Even better when done within Emacs.

I'm more of a vim user, but have dabbled in emacs so I'm not sure what you mean by "Even better when done in Emacs". Is there some kind of cool gdb tools in Emacs you're referring to? I'm just asking because day by day I'm becoming more interested in making the switch.

You might find this article useful: http://blog.aaronbieber.com/2015/05/24/from-vim-to-emacs-in-...

Aaron gave a great talk about his switch here: https://www.youtube.com/watch?v=JWD1Fpdd4Pc , which caused several people at work to move off NeoVim and give emacs a go.

Thanks for the links it was interesting to watch as a former Emacs fan.

In the old days, Emacs alongside DDD was the only way for me to get some of the Borland IDE comfort in UNIX land.

"Meta-x gud-gdb" for the old fashioned, or "Meta-x gdb", starts a gdb session in Emacs. Give it the executable you want to debug, then at the prompt type "run arg1 arg2...".

"watch p_mydata[34]->blahblah" sets a hardware watchpoint;

Ctrl-x-a-b sets a breakpoint at the cursor in the Emacs;

Ctrl-x-a-u runs until the cursor;

Ctrl-x-a-j jumps at the cursor;

And many other niceties.

There's also a way to create a buffer to monitor some variables.

You can actually attach the emacs-gdb buffer to a process you're running through valgrind when a valgrind error occurs. Not to mention it runs undo-gdb, of course.

As someone who regularly switches back and forth between emacs and neovim, I do prefer emacs' gdb integration. It gives you an IDE-like view where you can set breakpoints visually and see registers/stack in their own windows.


Be civil. Low effort holy-war prods aren't welcome on HN.

I thought it was humor, and not a bad attempt at that.

That was not humor, and the editor holy wars haven't ever been funny or interesting. Comments like that belong in cesspits like Reddit, not here.

I believe it is a reference to increased control over available gdb buffers (ex: register's value, disassembly, source). Ability to use macro's while debugging. And probably more key bindings.

A broken ncurses text gui debugger with broken navigation and byzantine commands should somehow make us look more favorably upon gdb? We had better debuggers than this in the early 80s. Turbo pascal had a decent debugger and IDE that was right built in!

How come the open source world hasn't been able to create a debugger that's as good as the (still primitive) debugger in Visual Studio 6.0? It's astonishing to me how bad the state of the art is.

Without question, Windows has the best debugging facilities. windbg is the best debugger, and when you have source code, visual studio's debugger is super nice. But guess what, windbg has so many commands that it's kind of hard to use for newbies and there's some great commands that even experts don't know about unless you go digging. This is many ways is like the bad UI in gdb. Over time, you learn.

Unfortunately, we're stuck with gdb, and improving it or making a new debugger requires massive effort and likely backing by one or more organizations (e.g. lldb has apple).

The ugliness of gdb won't change, but this presentation isn't about gdb's shortcomings -- it's about a "version" of gdb that supports reverse-debugging, which is really awesome.

I use undodb-gdb all the time and it's amazing. Consequently, I use gdb all the time. The result is that over time I've learned to deal with gdb's ugliness and so I get used to it. It's similar to complicated or hard-to-discover user interfaces in GUI programs.

> improving it or making a new debugger requires massive effort

Seeing what a single person (Greg Clayton) has done with his gui experiment for lldb (just run lldb and type "gui" command), I'm not sure the effort is so massive. Rather it's the fact that OSS developers have a really weak spot when it comes to design a good UI.

Also, there's Voltron:


> windbg is the best debugger

How so? I saw this suggested in some places but I found windbg atrociously complicated, even compared to plain GDB. It can debug the windows kernel, but that's about the best I can say about it. x64dbg and IDA's debugger were far nicer to work with for all desktop reverse engineering purposes.

In terms of non-RE debuggers, yeah, VS's is pretty good, but I don't find it that much better than anything else to say it's a must-have. Most source-level debuggers offer about the same feature set in only slightly different ways. It does the job and once you know one you pretty much know them all.

The UI is pretty bad but the commands are extremely powerful. It has commands and plugins to help you find deadlock, list out all system locks in a process, who owns them, layouts of data structures and lots more.

> "Unfortunately, we're stuck with gdb, and improving it or making a new debugger requires massive effort and likely backing by one or more organizations (e.g. lldb has apple)."

What about rr?


It's built on top of gdb, but seems to have a straightforward workflow (at first glance at least, I've not used it yet).


You still use gdb for debugging with rr...

Being stuck with gdb doesn't mean you can't do those things in IDEs though. Visual Studio has a gdb front end so you can run gdb on the target and debug in Visual Studio.

Since VB is mentioned I guess I can mention another language that have good open source debuggers:


For all its warts Java has a pretty decent open source community and many of the best tools in Java land are open source.

For example all the three major IDEs are available in open source versions, two of them are completely open source (although you can get commercial plugins) and all have decent debuggers.

Yes. The one thing I miss from Java development the most is debugging code from IntelliJ IDEA. Its just a fantastic experience to be able to step through line by line and see what's really going on, where it branches on each iteration and what state is changed. IntelliJ shows the current values of each variable next to where they are used in the source view, which really helps to narrow down the variables which are currently important.

The debugger that is available in JetBrain's CLion IDE provides a good chunk of the IntelliJ IDEA debugger interface for C/C++. Likewise, AppCode does the same for Objective-C. They wrap around LLDB.

Hear hear! This is, however, because the VM abstracts the bulk of the low level nastiness and exposes a reasonable API, which reduces the effort involved by an order of magnitude.

>> How come the open source world hasn't been able to create a debugger that's as good as the (still primitive) debugger in Visual Studio 6.0? It's astonishing to me how bad the state of the art is.

I keep asking myself that same question every now and then, why can XCode have such a nice and pleasant LLVM-based version of gdb, while gdb on linux still doesn't have even the most bare-bones GUI that let's me inspect common data types without jumping through hoops?

On Xcode nowadays the debugger is lldb, not gdb, but it's a very similar debugger in the sense that it's fundamentally command-line-based. Xcode just happens to integrate with it to provide a GUI front end.

There are plenty of nice front ends for gdb on all platforms though, including Linux: https://sourceware.org/gdb/wiki/GDB%20Front%20Ends

When I was into Linux in the '90s, DDD was very popular (as was Emacs as a gdb front-end), but now there are also dedicated IDEs like Develop/QT Creator/Eclipse that provide similar front ends to gdb like Xcode does for lldb (and used to do for gdb).

It's also very valuable to learn the CLI (both for gdb and lldb), because it seems like there are inevitably useful features in both debuggers that aren't exposed by any GUI, not to mention being able to debug something over ssh. It's not a requirement though, there are tons of options out there.

> fundamentally command-line-based

I'm a bit skeptical. Like the rest of LLVM, it's probably "fundamentally" an API, but with a bundled command-line interface.

This is strictly true, of course. But it's a very useful CLI, which is exposed within Xcode as well. From a developer perspective, it's a very similar situation- the CLI gives you full access to the capabilities of the debugger, and the GUI gives you more convenient/intuitive access to a subset of that functionality. Even inside Xcode I spend as much or more time in the lldb console window than I do driving it with the GUI.

What you are looking for is GNU DDD.


I use ddd, but I don't particularly like it. It's telling that after all those years we still don't have anything better, it seems like ddd today is still the same crutch as when I first used it, over 10 years ago.

The market has spoken in terms of preferring free tools. I personally think that is one big reason why the quality of developer tools is as low as it is.

because free == low quality ??

For some reason the equality seems to hold for most applications with a complex user interface. Reasons may include that it is a cross discipline thing that requires skills/people such as interaction designers, artists, technical writers, linguists etc that are less common than developers in the OSS community. It could also be that good interface design benefits from a centralized decision process which can be a bad fit for OSS that is usually too incremental and democratic for radical design changes. Normally there is no money for things like paid UI testing/focus groups.

That said, there is great free software with good UI/UX too - only they are more or less always backed by a large corporation.

A market with only free offerings will cause a certain category of actors to be absent. I would think that a market with fewer categories of actors would correspond into a drop in quality among some dimensions.

No, but there's certainly a positive correlation

I find that the interface to that is somehow worse than GDB!

Now if ddd had python intégration... Surprise me rms. (Guile top)

At least GDB mostly works. LLDB constantly falls over for me doing even the most basic of tasks. Give me a functioning debugger over a pretty one any day.

A year ago I would've agreed with you, but lldb seems just as solid these days. I use lldb just about every day in all sorts of oddball, convoluted debugging scenarios, and I just haven't been hitting the issues I used to with it.

It's not perfect- there are still rare situations in which it's not using the right symbols, but at least that I can usually fix that with a 'target symbols add'. Otherwise it's because the thing I'm trying to debug is sandboxed or otherwise protected, which is almost always going to be a hassle, and requires workarounds outside of the debugger. Raw lldb is a lot better than it used to be though.

Are you on OS X? LLDB might work a bit better there than on other platforms.

GDB mostly works for x86. I used to swear at GDB many times daily when using it to debug an embedded ARM target with openocd a few years ago.

Very frustrating when it crashes just as you've got the target into the state where you think your bug will trigger. And then GDB repeats the performance reliably for you... On those days you end up resorting to a lot of creative thinking.

>> let's me inspect common data types without jumping through hoops?

could you elaborate on what's missing in GDB? data-types are written in the binary by the compiler (dwarf format), and GDB does nothing more / nothing less than parsing it and giving access to it through it user interface.

GDB has no knowledge about your code or it's library in itself (at least in C, Python extentions now include libc++ data-type-specific pretty-printers)

That is exactly what is missing in GDB. The most common things you do while debugging are treated as exotic special cases. Every other debugger I've ever used tries to show me the environment I've landed in, giving me context I can use to understand program state. That is, so far as I'm concerned, the first and most important job a debugger is supposed to do. GDB shows me nothing; I have to tug every scrap of knowledge out of it individually, and I have to tell it how this work should be done. Instead of feeling like I'm using a tool that helps me and does some of the work for me, enhancing my awareness, it feels like I'm fighting with a tool that wants to make my life difficult, that is grudgingly giving up scraps of jealously-guarded information only when I can prove my worth by guessing and reciting its arcane incantations.

I rarely bother; it's easier to fprintf() and read the log file, and this has the advantage of also working when you're trying to debug asynchronous processes.

QtCreator has a nice GUI over the GDB. I was pleasantly suprised to see it worked really well, but - on the other hand - I didn't use it for any advanced debugging.

> LLVM-based version of gdb

you mean lldb?

Have you tried Visual Studio Code with the C++ extension? It adds support for GDB, still some bugs left, but it's not bad.

Amen! And it makes you wonder what it's doing to our young programmers? I first used gdb in college and had I not already been using Turbo Pascal and Lightspeed C's debuggers, I would probably would have decided that debugging was too much trouble. I see this a lot on Stack Overflow, too. People writing fairly complex code who don't know how to check the value of a variable in the debugger. It's both shocking and depressing.

Because most people using 1% of the functionality of even a debugger like gdb, if they're aware it exists in the first place.

A debugger is something that helps sell an IDE by padding the feature list more than something most people will use much.

I'm not saying it ought to be like that, but in more than two decades of watching developers work, my conclusion is that most developers 1) are unfamiliar with debuggers, 2) if they do know about debuggers they rarely use them, 3) if they do use them, they rarely use more than the very basics .

I myself fall in category 3. I use gdb, and I'm fine with that because all I tend to use is going up and down the stack, displaying some expressions and occasionally single-stepping.

I've used Visual Studio, and I've used DDD, and frankly I fall back to the above because it's quick, and because it's very rare that spending lots of time in the debugger is as helpful as writing more test cases or spending more time reasoning about the code.

To displace more primitive debugging techniques, a debugger will need to be extremely quick to cut to the core of the problem and basically help automating what is often a very basic workflow to try to narrow down things like "that crash happened because of an incorrect value in x; when did x get that value and what contributed to that?" followed by "how did y get that value that contributed to the broken value in x".

You can provide lots of fancy ways of making those kind of steps more pleasant, but they're doing too little to beat having a very minimal UI to deal with.

Instead debuggers appears to largely have focused on making those hair-pulling frustrating and long but also exceedingly rare debugging sessions more pleasant.

At least that's my impression: 99.9% of the time, I need something that's a smidgeon above a printf; 0.1% of the time I get frustrated enough to consider looking at more advanced tools, but that's not often enough to usually be worth it.

A debugger is something that helps sell an IDE by padding the feature list more than something most people will use much.

Because you have the most experience in environments where debuggers can only reach a small fraction of their potential. (Barely better than printf) Let me assure you that there are very different environments out there, and have been for decades.

I don't see how that would change anything I said. I never claimed that there aren't people who will use debuggers more fully, because clearly there is. But most people won't. And I'd be very interested about hearing about these "very different" environments - I've worked on tiny embedded systems, and large distributed systems, and desktops, and a wide range of other varied systems, and personally I've never felt compelled to dig into more advanced debugger features. Not that I haven't wanted them - but what I'd like to see is not what debuggers tends to offer.

(What I'd like to see is more context aware automation of the environment; e.g. debugging a crash? I'd like to see attempts at tracing values backwards, and options for attempting to re-try runs with automatically added probes based on the state of the environment when it crashed; done right, that's save me time)

>How come the open source world hasn't been able to create a debugger that's as good as the (still primitive) debugger in Visual Studio 6.0? It's astonishing to me how bad the state of the art is.

Because nobody has hired developers to work on it. Most large scale projects (whether OSS or Proprietary) get done when someone bankrolls the development. And I suppose the reason nobody has hired developers is because its hard to make money on open source debuggers.

GDB development is driven mostly by Redhat and Mentors Graphics (ex Code Sourcery), and both of them often add new functionnalities, fix bugs and maintain GDB. But maybe their work is at too low level for most people to notice?

I still wonder what's missing in GDB. I got quite proficient with this tool, and never had access to VS. I personnally don't count GUIs built on top of GDB, as they don't provide new capabilities, just 'fancier' interfaces that GDB's CLI.

> I personnally don't count GUIs built on top of GDB, as they don't provide new capabilities, just 'fancier' interfaces that GDB's CLI.

A strong visualisation and comfortable interface is not a new capability per se, but still a very important (some would even say "killer") feature.

Back when I coded C a lot, I'd use DDD. It had an ugly, but functional UI, that supported editing and recompilation. DDD itself supposedly works with a variety of debugger back ends (eg pydb), but I only used it with gdb and C.

20 years later apparently it's still maintained, even if it has plataeued.

Personally, I'd like to see a more modern frontend made, perhaps for Atom.


They did: it's called Emacs.

I don't understand the downvote. Debugging C with gdb + emacs (albeit 20 years ago) was a decent experience. Even for a vi user...

It often feels like (as a non-emacs, non-vi user), that every conversation ever will end up at some point with a post that says "use emacs" or "use vi".

These tend to have zero actual content (emacs is a text editor. How would I use it with gdb?), and not provide any evidence, or guidance.

Emacs would split the terminal screen, showing the gdb interpreter/prompt in the bottom pseudo-window where you could interact with it, and the source code for the current function in the top "window", with a visual "pointer" at the current line.

I know you could put watch expressions into gdb, and I suspect there was a way to pull these into another emacs text window, as well.

Gdb also had commands to list the stack and select a frame further down to see the callers data. I don't remember if the source code view (in emacs) would track the selected frame.

You just run it like any other interaction. In emacs if I want to compile a file I type M-x compile, and it asks me how. That creates a new buffer in which errors are linked to the original buffer. M-x gdb works the same way.

I really feel like a cowboy when I am debugging C with `gdb`


Please re-read the commenting guidelines carefully:


Sorry, missed that second to the last one. No guidelines on downvoting, though???

You seem obsessed with other people's downvoting. Relax, seriously. If you think someone got a downvote they didn't deserve, upvote it yourself and move on. It's not your responsibility to police other people's voting behavior.

Edit: If it really bugs you that much, post a comment backing the person up, but don't bring up the votes. It never adds anything except drama and distraction.

I've had pretty good experiences with Eclipse's GDB interface - how does that stack up against VS6?

I recently switched from doing my C/C++ debugging in gdb to Eclipse here are some of my thoughts:

-> Eclipse can be easily be started from the command line - e.g. "cdtdebug.sh -e ./a.out". A minimal version of Eclipse opens up, and you can immediately step through code, no need to create projects.

-> While GDB TUI mode somewhat mitigates the issue of not having context when debugging it IMHO pales in comparison to seeing the full file in Eclipse.

-> In Eclipse I can see so much more information than gdb. I can see several expressions all being updated, memory views, I can highlight expressions in code. With gdb I could only ever see one thing at a time.

-> Although GDB has more functionality than Eclipse, I found since I couldn't remember certain GDB commands and so I just didn't use certain features that I find myself constantly using in Eclipse because it's just a click away.

Yeah, I've had decent luck with both Eclipse and Qt Creator's interfaces. There's also excellent debuggers for Python like WinPDB. GDB itself has reverse debugging support which could be integrated into IDEs... https://www.gnu.org/software/gdb/news/reversible.html

Buggier. Much buggier.

Priorities and Laziness. Open source produces some amazing stuff because of laziness. It also produces a lot of "good enough" stuff because of laziness.

If you count interpreted languages, then there are plenty of killer debuggers for JavaScript, at the very least. Chrome dev tools is just as good (probably better if you count all the tabs) as I remember Visual Studio's debugger being.

In his defence, he didn't say he'll make you think gdb is the greatest, just change your perspective of it. A broken ncurses text gui debugger is miles better than viewing your code via basically ed commands. If all you have is gdb, then knowing how to use it more productively is certainly welcome. That's the situation with e.g. embedded development or debugging the kernel via a serial interface.

It's still kind of crappy, but at least it's not ed.

I almost always use gdb from Eclipse but it's nice to be able to ssh into a small system without GUI and still get baseline functionality, kind of like how I use vi.

Because the Cult of Torvalds is convinced using a debugger makes you a bad programmer.

Because we already have the worlds greatest debugger. The Print (proper noun) statement.

That's fine for relatively simple code that you have written yourself, but is crap if you are trying to get an idea of how someone elses complex application works.

This is sarcasm!!!

IMHO, In this day and age, a proper debugger should by default show source code and execution point, should allow you to inspect variables, and change its values, you should be able to manually set the executing position, and you should be able to alter code and re-compile on the fly, continuing the debugging session.

I know this is really-really hard to accomplish, but some IDEs do it.

The only thing that's "optional" and a little over the top (but very useful) is time-travel debugging.

When you say "alter code and re-compile on the fly," do you mean and continue debugging without stopping the app and re-running? Because if so, that's a terrible way to debug. You now have state that may not be possible to achieve with the new binary you've made, and you may be debugging something that won't every exist in reality. And it may be very hard to tell that's the case. That doesn't sound very useful. It sounds very dangerous.

I find that's exactly where the debugger is most useful. I know I'm trying to reach or examine a state that my binary can't achieve. That's the problem. The debugger lets me poke around to figure out what exactly is wrong with my state, faster and more immersive than recompiling and rerunning every time. Then I fix the binary to match.

It occurs to me now that's essentially using the debugger as a REPL, but with access to a whole runtime's worth of external state. That's not a bad tool to have in your box.

And to be sure, this is exactly why Jupyter is so powerful for experimentation, especially when doing data science. When you can run lines independently, alter them and re-run them, but keeping all the state of the previous lines, it's so much better than needing to re-run from the beginning. Of course, you can shoot yourself in the foot if some of those lines are like (i = i + 1) and you end up incrementing i by (# times you've run the code) rather than (# times it appears in the code). But if you name your variables sanely and treat them as const, you can easily avoid this.

The question at the core here is: is a debugger meant for experimentation to find the (often subtle) root cause of a defect, or is it meant to be a read-only window onto state during execution?

Because if so, that's a terrible way to debug.

If you're in a codebase composed largely of side-effect free functions or well encapsulated Object Oriented code, it's a very good way to debug. I had great success with such debugging and even coding and new development in Smalltalk environments for over a decade.

On the other hand, if your codebase is full of side effects and doesn't have good encapsulation (perhaps there's a lot of fiddling with globals) then you're going to have a bad time. But to me this isn't because the debugging method is bad. To me, it's because your codebase is designed with lots of tight coupling and side effects. You have an architecture that makes it harder to reason, debug, and refactor your code. This isn't just spouting. I'm basing this on many years of experience. And yes, I saw both kinds of Smalltalk code, and the effect is exactly as I described. Guess which codebases were more productive?

I loved developing in eclipse with hot code reload. 0-second changes + debugging all in one.

Eclipse basically started out as a port of the VisualAge for Smalltalk environment to Java.

And they still keep a very tiny bit of it on the "Java Browser" perspective, which is basically the standard way Smalltalk presents the code navigation.

That port was horrible. It was only when they dumped the smalltalk codebase and started again that it became useful.

Pretty much par for the course for those days. Converting Smalltalk apps to Java ones usually yielded horrble Java apps, because Java was very immature, to be frank. While it was still VisualAge for Java, it was considered one of the best Java IDEs of the time.

Well, I have to use OpenGL for my development, and it's a giant state machine that works mainly via side-effects. It's notoriously frustrating to work with, but that's life. I realize not all code works that way, but I'd bet the majority of code does. So I stand by my statement that it's a terrible way to debug, maybe with the caveat "unless you can work with only pure code and no state."

Well, I have to use OpenGL for my development, and it's a giant state machine that works mainly via side-effects. It's notoriously frustrating to work with, but that's life. I realize not all code works that way, but I'd bet the majority of code does.

I suspect your sample is a bit skewed.

The Smalltalk class library wasn't stateless. It certainly wasn't pure. Really, code only has to be "pretty good" for such techniques to work well.

I believe Visual Studio does this by simply rewinding to the function entry point which addresses most of the concerns you've raised. A debugger is just an aid to reasoning about your code, not a substitute for it.

You might make the same argument against unit tests, or little hacks you write to separate out a problematic piece of code from an even more complicated context.

I believe Visual Studio does this by simply rewinding to the function entry point which addresses most of the concerns you've raised.

Live recompilation in Java (at least in Eclipse) behaves basically the same way.

If you've never written code in a live debugger session, moved the execution point back up, and immediately run over the code you just wrote, you have a crappy debugger. VB6 could do this, Smalltalk does this, it's not dangerous, it's damn productive.

VB6... got so much hate but folks just don't seem to see how useful it's debug mode was.

Oh sure... VB6 and good OO design and modern practices (unit tests, dependency injection, etc) don't mix. But... if you need to sit down and "hack out" a bunch of working code quickly it was pretty hard to beat.

Python with the VB6 ide/debugger experience would likely take the scientific computing field by storm.

I miss it... especially since I've never been able to get "edit & continue" to work at all in Visual Studio.

You can do 'proper' OO, unit tests, and dependency injection with VB6, it's just not as easy as it could be, and most developers didn't, partly because unit tests and dependency injection weren't as popular at the time, but (with 'proper' OO and dependency injection) many apps were simple enough that these things were perceived as of little importance.

When I last wrote some VB6, I was criticised by a colleague for 'over-engineering' because, to build a cancellable progress dialog, I used a (very simple) observer pattern. He didn't understand what it was or how it worked - and would have preferred to simply block the entire UI while a process ran (without the ability to cancel, even). To him, even basic OO ideas were pointless and dangerous over-'designing'.

If your application was crap, it could also remain incredibly simple in terms of design, though inevitably multi-thousand line methods and hacks upon hacks led to completely non-understandable, buggy, and brittle code, so I'm sticking to my 'enough design' principles!

One more aside: This developer complained that there was a bug in the VB6 IDE because it wouldn't let him add any more code to a file - he'd written such a huge code file that he'd actually hit the IDE's limit. I tried talking about modularisation, refactoring, etc... then just gave up.

I never got this to work properly.

VS does this and I've used .NET since its start. It's useful now and then, but a REPL is far more useful. Considering the cost of Edit-and-Continue I'm not sure it's worth it. I'd have much prefer MS to have spent the effort on improving language tech or working on REPLs.

REPL is OK, work-spaces are much better, ala Smalltalk. The best analogy is like a SQL editor, you can simply write anything and execute it in place. The notion one needs to enter one line at a time, aka the REPL, is quaint; not something to really be desired except from languages that have neither.

REPLs don't have to be one line at a time. F# interactive and C# interactive work well with Visual Studio. And at worst, it's a simple UI issue. Like various SQL UIs, nothing stops a REPL from being a big textbox then letting you select text and "run".

Do Smalltalk workspaces do something fundamentally different?

Well, in Smalltalk everything everywhere is an execution environment; anywhere you can type, you can highlight and execute code. But it seems we have different definitions of REPL, to me a REPL is a command line; a textbox isn't a REPL, it's a workspace. Sure they're both code executions environments, but working with them is vastly different.

When using common lisp on SLIME you can type your code in the buffer, select it and have have it eval'ed. The result will be printed in the repl buffer instead of next to the cursor, but that's a pro to me.

When I tried out pharo I was pressing backspace all the time because outputs are not valid code and thus would break the highlighting.

Anyway, a repl is just that, a read eval print loop. If you interface with it using a command line or other means is just an implementation detail.

How do you make sure there's no corrupted state and that the data structures in memory fit the new code?

You don't. Worst case is you had wrong assumptions and restart the program, same as if you didn't have the ability to change things.

Best case thought, it's so useful. It's also incredible for editing tunables on the fly(think videogame gameplay values, layout values for UI, etc).

Worst case is you don't know you broke consistency. Then you can waste hours hunting for ghost bugs that only occur because of the hot-code-reload.

As someone who worked on dynamic updating research projects for years I can attest to the really shitty bugs dynamic updating can cause.

Editing tunables is possible without recompiling new code. It's just setting the variables to new values.

Not if you want to change the formula the the tunable uses.

In Smalltalk, most of the class library, and hopefully your own codebase, is either side-effect free or at least well encapsulated. The classic trip-up in Smalltalk was with Streams. An experienced Smalltalker would make note of where the code was fiddling with Streams, then go down the stack until the debugger has effectively gone back in time before the Stream objects were instantiated. Try that for a few seconds, then restart the app. If your environment was set up intelligently, none of that would take that long.

I believe changing data structures gives a "you must restart to make this change" when resuming debugging. It's mainly for code flow type bugs rather than data structure type bugs.

For things that require restarting in Visual Studio, look in the docs for the topic of rude edits.

If the data structures in memory don't fit the new code, I suspect that's a compiler bug.

It's not dangerous. It's awesome.

There is all of this excitement around ideas like Light Table, allowing you to see the output of your code inline as you develop it. Being able to alter code and recompile on the fly in the debugger gives you a very similar experience.

there are some situations where execution of the rest of the program up to that point takes forever, and you don't want to have to spend however many hours/days/weeks reexecuting just for a typo. scientific computing comes to mind, with its huge simulations

Twenty-five years ago, there were commercial C environments that supported interactive coding close to what you got with Lisp or Smalltalk. There was Instant-C, Safe-C, RUN/C, CodeCenter, and a few others. The whole genre died out in the mid 1990s for various reasons. The most popular and powerful system was CodeCenter. People wanted to keep using it, but the developers couldn't handle the increased complexity of supporting C++, so they just gave up around 1996.

I only knew the one from Lucid Energize C++ after they moved out of Lisp Machines business.

IBM also had a version of C Set++ that used a Smalltalk like image database for C++ code, but they hardly managed to sell it as it was very resource hungry for hardware at the time.

It is very hard to find online references to it though, I think I read about it on Dr. Dobbs, back in the day.

A video from Energize C++ for those that never saw it


Yes if this video shows anything it's that you need a real GUI. The curse one is ok, but what's the point if nobody knows it exists and you need obscure commands just to scroll?

You scroll with the arrow keys. The "obscure commands" (ctrl-p and ctrl-n) are to get to the Previous and Next commands typed and are more or less universal in unix command line systems.

I've used Unix a lot and I've never used Ctrl-P or Ctrl-N for previous/next. Ask 99% of people what those do and they'll either have no idea or say Print, New. Ask 99.999999% of people how you scroll in gdb and none of them will guess Ctrl-P/Ctrl-N. It's shit.

I agree with this. While also holding the opinion that the best coders internalize what is happening anyway. Ideally, you should be able to use one of these tools in a codebase you are familiar with.

In C# I do this all the time. Not so clear if you can do this in C, especially if statically linked.

I prefer Emacs GUD over the built-in GDB TUI. I actually don't understand why anyone would use the built-in TUI; you have to learn new things anyway, why not learn the Emacs interface which not only looks way better but is more reliable and has more features? The keyboard commands for the GDB TUI are even (mostly) cloned from Emacs.

But as that person in the audience says, it looks like the presenter is just not aware that the GDB TUI is a substandard version of Emacs GUD, having mostly the same keybindings.

The second half is a good combined demo of some neat GDB features though.

The biggest problem there is that you're injecting the politics of emacs into your debugging then. GUD's hooks into LLDB are still a third party package because certain parties have deemed llvm-based utilities politically inconvenient.


How is that relevant at all? LLDB is a completely different debugger. The fact that I have to download an extra package to use LLDB doesn't affect my use of GDB at all.

If LLDB eventually becomes as popular as GDB, I'm sure it will be included into core Emacs then. But I have no interest in LLDB at the moment.

"I'm sure it will be included into core Emacs then"

No, actually, it won't :) RMS has said it won't, in fact.

RMS is not the Emacs maintainer...

And even if he was, I don't believe he ever said "If LLDB successfully replaces GDB as the most used libre debugger, even then we still shouldn't support it in Emacs". He's not insane, you know.

On the other hand, the historical record: https://www.phoronix.com/scan.php?page=news_item&px=RMS-Emac...

Personally I use the TUI, but I completely get where your coming from - the TUI is really pretty lame for all that it does compared to all that it could do.

That said, the debugging I do is all from the command-line anyway, so all I really want is a way to quickly view the assembly and registers when I break, which the TUI does quite well.

As a note - I think the person in the audience was trying to get at the fact that gdb (like tons of cli programs) uses readline() for input. readline() can be configured to use emacs or vi keybindings, with emacs the default - hence why ctrl-p and ctrl-n work. I personally use vi keybindings, so in gdb I can use ESC, and then jk to go up and down the command history.

Is this entire set of comments suffering from people conflating the debugger with an interface to the debugger? In much the same way that sometimes people conflate their compiler with their IDE.

GDB is a fine debugger. I've never had any trouble with it at all, and I don't recall ever needing a feature in it that it didn't support but other debuggers did, although I'm not a very demanding user of the debugger; I rarely need more than some conditional breakpoints and the ability to see the value of any given variable or memory location.

If WinDbg were so much better than gdb, someone at Google would have replicated it for Linux.

I'm fine with gdb and conditional breakpoints is usually the most advanced feature I need. Only once I used bigger guns. Detecting the error meant run into conditional breakpoint A, then enable and run into conditional breakpoint B (which would have triggered a lot before A), and then inspect a variable. The problem: The error only occured sometimes. Thus automate this via Python, so gdb could auto-reexecute everything until it found the failure case again.

What I still miss is multi-process debugging. Gdb can only attach to a single process, so you need multiple gdbs to watch multiple processes and this introduces race conditions.

You've just reminded me; sometimes I need to run something under Valgrind, and upon Valgrind not liking something, sparking a break using GDB. I expect that's possible using other debuggers, but it's so, SO easy using Valgrind and GDB. So by this reckoning, GDB is the best debugger I've ever used.

Bring it, WinDbg :)

  > people conflating the debugger with an interface to the 
  > debugger? 
It's very annoying. If there are any further cool gdb tricks in the thread, they're hard to pick out from the noise.

I get that he's being clever at the beginning by poking fun at Windows but many of the shortcuts he talks about are available in Visual Studio and it has a much saner set of defaults and initial information. Ask anyone who did XBox and PS3 development which environment they preferred :).

The reverse debugging stuff is pretty cool though.

>Ask anyone who did XBox and PS3 development which environment they preferred :)

I'm pretty sure if you asked anyone that did PS3/PS4 development they'll probably say ProDG.

I'll give you that PS3 had a better CPU profiler but ProDG was a mess compared to Visual Studio. Crashes, disconnects, problems with source maps. If I remember correctly it couldn't do data or conditional breakpoints.

Compared to VS and PIX it was night and day, although you probably already knew that from my comment.

ProDG has had support for conditional breakpoints for some time.

Really, they were both bad for PS3 debugging.

Oh man, using VS for PS3? Yeah, I bet. I was mostly talking about how it worked on X360 which is a bit better supported platform.

Windows can do reverse debugging (time traveling debugging) using Nirvana.

Unluckily time traveling debugging/Nirvana is not available to end users.

I use UndoDB a lot. It's been increadibly helpful for debugging dynamic binary translators, which in my case are x86-to-x86 JIT compilers.

I also use it for debugging unit tests for a static binary translator. When I'm unit testing, I will set a brakepoint at the reporting of a failure, look at how the native program and emulated program states diverged, then I'll use reverse execution and data/code breakpoints to go back and forth to try to figure out what went wrong. This is much better than having to restart each time.

Over the years I've found many bugs in UndoDB and they've been pretty quick to fix them. Their support is great.

I started using it as a student, on a non-commerical (i.e. free license), and it was an essential purchase for my current work.

There's an awesome free replay debugger GDB extension available for Linux that most people don't seem to know about: http://rr-project.org

The user interface to gdb is not very nice, otherwise a utility like RR looks pretty amazing.

Developers don't use debuggers because neither the benefits of using one nor the harm of not using one are immediately obvious. And when you first start learning how to program, the cognitive load of picking up a new language, its tooling and quirks, you simply don't have the energy to understand yet another piece of complex software. Worse, you may not even know there's such thing as a debugger! So we all do "printf debugging" because printing to the screen is the first thing you learn, and it's obvious how you can use it to inspect program state.

However, this is a novice trap, as more experienced developers would know that printf debugging may not work for subtle cases, and may even steer you in the completely wrong direction. But by the time said novices encounter such bugs, they've already picked up the "bad habit", and it's difficult to go back and unlearn it.

Some also feels that needing a specialized tool to find bugs is a weakness, as you ought to be able to reason about the code without aid. But that is folly, for if so, why use editors with syntax coloring (shouldn't you know int is a keyword w/o being pointed out) or auto-completion (shouldn't you memorize all exported functions)? In fact it's the opposite: a good debugger greatly aids in comprehension, since you get to catch corner cases happen in action, and with sufficient context for you to figure out how it'd happened.

There was a time when many developers also don't use a D(VCS) of some sort, for similar reasons. But of course git has a celebrity champion, and so the rest followed. Almost all the excuses from this thread people had raised about not wanting to learn how to use a debugger properly apply to git (and to all D(VCS) to varying extents), but you'd be ridiculed for not using it. IMO debuggers need a similar champion before our opinions change, but alas IIRC Linus himself is not too fond of debuggers...

That said, the state OSS debuggers is terrible, so they're not helping their own cause. Moreover, they are often "buggy" themselves. I put that in quotes because sometimes a "bug" is actually correct, but unintuitive behavior for inexperienced users (pop quiz: how do breakpoints work?), or just poor / nonexistent documentation. Nonetheless, the result is the same, and nothing loses your trust in a tool faster than finding bugs in it while trying to fix your own bugs.

I don't compare gdb to visual studio but to windbg.It would have been great if gdb has support for debugger extension s like windbg(or any windows debugging tools like cdb or ntsd etc). I can easily debug any .net programs on windbg and level of details sos,psscor or many other extensions show you on for .net debugging is amazing. There are windbg extensions where I can write SQL like queries on .net memory objects or write python scripts or JavaScript to control your debugger or write automated analysis very easily.I have seen the same for php and nodejs processes with windbg and it was not very difficult.On Linux the story is different and there is no easy replacement .When Microsoft ported .net to linux with dotnet core ,they chose lldb instead of gdb as gdb does not provide extensions support.I also heard that lldb is not as good or stable as gdb.

I think this a good prelude for a good talk. I'm wondering if the guy is going to release the full talk he gave later...

However, mentioning advanced-gdb usage nowadays and not mentioning RR (http://rr-project.org/) is a sin =)

TUI mode is cool, I didn't know about that!

Missing from this video, but the most powerful part of using gdb for me recently has been the ability to do procedural debugging. You can easily write debug scripts that are re-runnable instead of reproducing bugs manually. You can sequence together multiple conditional breakpoints, keep debug variables & state, and other stuff that is really tricky to do in more common GUI based debuggers. I haven't used Visual Studio in the last few years, so maybe they've added it. I had tried for years and never found it, resorted to gdb for the hard problems, when I could.

What I never liked about gdb was that I could not really figure out a way for it to show me what has changed when stepping over an instruction or a call, a feature that I often find very useful in other debuggers (especially in x64dbg and edb if you do not have the source).

The debuggers I have encountered seem a bit limited when regarding the debugger as a means of querying a subset of program executions (out of all possible executions), but they are usually helpful in picturing system dynamics. It has always been very helpful for me to allocate a portion of fixed screen real estate to memory, variables and registers that I am watching and see the changes as I step through, allowing me to observe the dynamics of the system.

Too bad I could not figure out a way for gdb to help with that, the closest I got was examining the memory I was interested in ('x' variants) before and after, but no highlighting. To be fair, it is good at other things though :)...

More recent talk from the same presenter http://undo.io/resources/presentations/accu-2016-become-gdb-...

My favorite gdb feature? write watches. Set a write watch on a variable (even by address) and get a break point when the variable is written to. Read watches work the same way but break when a thread reads the variable.

Reverse debugging is a nice feature but just skip to 9min in to watch the demo of reverse debugging. Surprised to see so much of the video talking about the UI of gdb which certainly didn't change my opinion of it. I've found emacs+gdb to be very useful over the years especially when debugging on remote servers but this demo just reinforced my opinion of gdb being a very powerful program that is pretty difficult to master. Also reinforced my opinion that you should open your talk with at least some of your best content rather than saving it for the last few minutes.

I'm glad LLDB exists now. It brought competition into the abysmal Open Source debugger space. It's just a pity that there are no good frontends for LLDB so far.

In which ways is LLDB better than GDB?

From the WWDC talks XCode looks quite close to VS experience.

Xcode does work, mostly, as you'd expect, since it's backed by lldb. And it's certainly more like VS than command line lldb is! But still, the UX isn't quite there. I found it rather limiting and fiddly to use - you don't get good control over where the various windows go, and too many things are forced to share a window. You can only see one of such things at once, and have to do something to switch to the other.

This criticism applies to the UI overall, which puts things in inconvenient places (the tall, narrow find results/compile results panel was particularly ridiculous), and then won't even let you move them, but I found it especially tiresome when debugging. This is the last thing you want in a debugger UI in my view! I want to see as much stuff at once as possible, and I don't want to have to switch between tabs unless I deliberately set things up that way myself.

There's no separate registers window (it shares its window with the locals), there's only one watch window, and the disassembly window never seemed to work very well (when stepping, execution still proceeds on a line by line basis). I recall the disassembly window also sharing a window with the source window too. I think the memory view might do that too. All in all, most unsatisfactory.

Visual Studio does have some odd limitations and annoyances, but you can at least put the windows where you like.

GDB tui is old. I remember using it 11 years ago, debugging a VM for Itanium. Also, if you know emacs, gdb emacs integration will blow your mind.

Looks like VS Code has a plugin that integrates with GDB.


Maybe polishing that plugin could go a long way in a better dev experience than the current GDB UI?

This is VS 2015, not VS Code. Though VS Code does have some C/C++ plugins that, I assume, integrate with GDB.

Well I stand corrected. I also have an open source project to start it seems :)

Who writes bugs that need a reversible db to find? or an embedded python interpreter in your debugger? I've done some hairy things in gdb to find bugs (actually valgrind is a big help with this aswell) but these other features, yikes..! If i ever need them it's gonna be a dark day for sure.

Well, "need" is a strong word. The features are definitely helpful though.

With large programs, there can be a really long distance between where something is created/modified and where it's used. If there's a bug you catch that occurs where the object is used, reversible debuggers help you trace back to where it's created, or who modified it last.

If you're at a place that's hard to put a useful breakpoint (e.g. a widely used function), and need to answer a what-if question, being able to manipulate state from the debugger is super useful. You might not be able to easily insert code to the program to create that state without messing up everything else (again, example is a widely used function, where inserting code could mess up other users of the function).

Just to let you know that the same guy is running a series of 15 minute webinars this week on how to become a GDB pro. First one is today (4pm London). http://undo.io/become-gdb-power-user/

I think GDB is fine architecturally for debugging.

One day maybe we'll see a good non-buggy GUI frontend for GDB that shows code, local variables, assembly, etc in separate windows, all clickable to change values, set breakpoints, change interpretations, etc.

The same guy is giving a series of 15 minute talks this week via webcast (first one is tomorrow at 4 pm London time)


Here is a free video course on GDB - covers most of things discussed in this video and more:


"Sorry to the Windows users out there"

Not anymore! My, how times have changed.

Interesting! I used to use in circuit emulators with trace buffers. You could use them to discover that an interrupt handler messed something up by looking back in time.

Wow, I didn't know gdb has built in reverse debugging or the Python interpreter. The reverse debugger looks awesome!

I'm curious what people use the Python interpreter for?

Reminded me of all the hours spent debugging with dbx. Turns out, even after all the yrs, they are still quite similar.

b _exit.c: 32

This guy is my hero.

Nice troll brain

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact