Hacker News new | past | comments | ask | show | jobs | submit login
GDB tricks (oracle.com)
177 points by luu on Nov 28, 2014 | hide | past | web | favorite | 34 comments

These are all useful, and I would even say fundamental, commands. But, as such, these are also very basic commands, not tricks. If you're looking for something more advanced, you'll be disappointed.

I strongly suggest to anyone using gdb to just read the manual [0]. Or at least skim it quickly and find a dozen of such useful "tricks" in as many minutes. The time gained in debugging later is well worth knowing and mastering your tools.

Here's a quick summary of the article, most of this stuff is covered in the first few chapters of the manual :

1. conditional breakpoints, yes, they exist

2. execute commands each time a breakpoint is hit, yes, that's possible

3. pass commands to the program directly from gdb's command line

4. define source directories

5. debug macros with the right compile options

6. you can use the results of commands, and even define variables !

7. you can access cpu registers

8. you can even examine memory

[0] http://sourceware.org/gdb/current/onlinedocs/gdb.pdf.gz : it looks big but a lot of it is annexes at the end, or how to extend GDB with your own scripts, which can be interesting but can easily be skipped in the beginning. The important chapters to begin with would be :

- Getting in and out of gdb

- gdb commands

- Running programs under gdb

- Stopping and continuing <-- breakpoints 101, learn about conditional breakpoints (you can also set the conditions after setting the breakpoint), hardware breakpoints on data (called watchpoints in gdb), how to skip quickly (to end of function|a whole file|...), etc.

- Examining the stack

- Examining source files

- Examining data <-- very important, discover the joy of artifical arrays, casting with @ or {type}, configuring the display etc.

- Altering execution

I found constructing small problems and solving them with GDB when not under pressure greatly expanded my working knowledge. Trying to learn GDB while attempting to solve a time critical production bug is not possible for me.

I can relate to that. When you are not under critical pressure is a great time to print the manual, sit back and try new stuff on some minor bug somewhere.

I actually developed most of my debugging skill when younger, by reverse engineering other programs and games. And after reversing foreign code from the binaries, debugging with the source code seems comfortable. It is also great to get more familiar with the inner working of the processor and to understand what instructions different languages and compilers produce.

It was not with gdb, but debugging is debugging. Switching debugger when you've known a few is like learning a new programming language when you know others, but much easier. You just need to find the new syntax, they're mostly similar.

One thing you learn quickly when REing is also not to jump head first in the debugger. Sometimes your problem can be solved easier using a different route, and that applies to debugging as well. A good read of a promising part of the source code or a well placed debug trace can sometimes be much faster.

For others who want to learn, there are plenty of reverse engineering tutorials you can find with a simple search. I have a particular place in my heart for fravia's (RIP) old website (should still be mirrored by woodmann [0]). It's a bit dated and there may be a better place to start now but it certainly covers some basics.

[0] http://www.woodmann.com/fravia/ You can check out http://www.woodmann.com/forum/content.php for more modern stuff while you're there

Have you got any of those problems written down or any pointers for creating them? That sounds like fantastic material for a tutorial.

on mobile ... play with a running program like wget, step through nginx request, debug a python script. wget is by far the richest playground. setup a site on local host for wget to spider, modify bandwidth, wait time and kinds of links to follow.

> By Ksplice Post Importer on Jan 24, 2011

Heh. When interning at Ksplice/Oracle in 2011, I was tasked to import all of the blog posts from the old Ksplice Wordpress blog into Oracle's system. By hand, because they didn't want to turn on the Apache Roller API, or something.

I couldn't set a custom author for each post, so instead I changed my name to "Importer, Ksplice Post".

You're famous! Or perhaps infamous, going from a blogging platform with the lackluster codebase to one that doesn't autofit on mobiles and was unreadable on my android chrome as it didn't reflow nor pinch-zoom and was just lots and lots of panning.

I would have left my internship if I was tasked with that.

Bad engineers are only effective when given tasks that are interesting to them. Good engineers are those who are equally effective when given tasks that aren't so exciting. If you're at the intern level and you're already picking out which tasks are worthy of you and which tasks are beneath you, you may want to think about a different line of work.

While not disagreeing entirely with the core of your argument, an internship should be about learning rather than handing off the work no one else wants to touch. Is is unreasonable to expect, in a position which is reasonably considered a part of your education, that mentorship, controlled exposure to production systems, and legitimate work will be provided?

An internship is not some magical excuse to hire someone likely overqualified to do extremely tangential work for low pay, its a partnership between the intern and the company to perform work that is useful for both the intern and the company.

Its not about being a good or bad engineer, its about the work being relevant. Tasking an apprentice welder to change the oil in the fleet of company cars would be more akin to the scenario described: somewhat related but utterly useless beyond a piece of paper saying they did an internship at X.

Knowing lwf, I suspect this took a day or two out of the whole internship...

Well, an internship in the software industry here is not really an internship. More like a summer job for CS students: intern just means you're temporary and lowest on the totem pole.

I certainly was given jobs that were considered semi-important but that the full-time staff had more important things to do.(They aren't going to pay an intern to do useless work) I still learned a lot, even when I wasn't coding because I was required to learn good practice.

To be clear, I did a lot more than this in my months there :) Most of the time was working on improvements to the customer-facing management webapp, Uptrack. https://www.dropbox.com/s/tqqup4ltz9tphga/uptrack-oracle-r.p...

However, somebody had to migrate the blog, might as well've been me.

I would add the skip command to register a function or file as something you don't want the step command to go into in the future. Really handy for all those trivial operator*s you get in C++ smart pointers and such.

My favourite one:

    attach to the process in question using gdb, and run:
        p dup2(open("/dev/null", 0), 1)
        p dup2(open("/dev/null", 0), 2)
From: https://stackoverflow.com/questions/593724/redirect-stderr-s...

"Oracle blogs is experiencing technical difficulty. We are aware of the issue and are working as quick as possible to correct the issue. Please try again in a few moments.

We apologize for any inconvenience this may have caused."

sounds like a perfect time to whip out that debugger Oracle, or upgrade to faster DB engine?

This lack the best option: ctrl-x a

Don't bother having a external GUI, just use the built-in Ncruses one.

The QtCreator debugger is actually pretty decent, and I prefer it to gdb -tui and cgdb. The debugger in the early-access version of CLion looks promising, but it lacks essential features like a disassembly view and a memory view, and sadly they aren't mentioned on the current 1.0 roadmap.

Emacs's gdb-mi mode has the potential to be the best out there. Unfortunately it has some critical weaknesses in its current form. The biggest is that it uses the awful speedbar extension for its watch window. And gdb-mode's local variables window is useless as it won't allow you to inspect any aggregate types like structs and arrays, so you're forced to use the aforementioned awful speedbar-based watch window just to inspect local variables. I'm hoping to fix this or rewrite it entirely when I have time.

Rocky Bernstein's GUD rewrite may be of interest: https://github.com/rocky/emacs-dbgr

I don't think it has a watch window yet, or a disassembly or memory view. It has a cleaner architecture than the original GUD though, which might make it more pleasant to do major development on. It supports more backend debuggers, multiple concurrent sessions and attaching to a debugger session in an existing Comint buffer (presumably that means it supports attaching to existing processes as well; don't know if the original GUD gdb mode does that).

It does have the nifty idea of a minor-mode that is activated in source buffers that makes the buffer read-only and maps debugger commands to single keystrokes, eg. n for next, s for step, etc. ('Insert' toggles it off in case you actually want to do some editing).

The gdb support seems currently a bit basic. For example if you highlight an expression and press 'e' it sends "eval <selection>" to gdb when of course it should be "print". (It sounds like the main dev effort has been on the Ruby debugger so far). I get the feeling it might just need a little TLC to make it great.

For now, I agree the speedbar, and the experience of a watch window in it, is awful. Having seen how powerful gdb's print command is (doing very well with aggregate types, and now with the latest gdb you can switch on some Python-based pretty printing for it as well), as a temporary personal init-file hack I brought the single-key mode to the original GUD making sure to include 'p' to 'print' and capital 'P' to 'print*' the currently highlighted text, or the current word if nothing is highlighted. If the gdb buffer isn't visible, it prints to the minibuffer, else if the gdb buffer is visible, it goes there, and then I can move the cursor around and use seperate Emacs folding commands to collapse and expand the braces. It's still a bit raw but I'm so pleased with it that I already don't miss Visual Studio (the supposed gold standard for user-friendly debugging, yet even in the latest version I find it distressingly weak at inspecting common STL types).

I might try and contribute to Rocky's mode myself when I have the time.

Thanks, I didn't know about the GUD rewrite.

After I wrote my post above, I started work on a GDB mode from scratch. So far, I have the process management and a PEG-based parser (peg.el is great) for MI output working. I'm not sure if you've looked at gdb-mi.el's code, but it was written by a GSoC student and does vile shit like munging MI output with regular expressions so it can be fed to json.el and parsed as valid JSON. In similar lazy fashion, it uses GUD just for process management. I don't feel there's anything worth salvaging or building on in there.

I find the built in one extremely flakey and hard to use. I usually have better luck with cgdb.

The one feature that has been present in almost all the other debuggers I've used, and is apparently missing from GDB, is a standard hexdump - addresses, byte values, and ASCII. It's possible to emulate this with a bunch of extra steps, but the omission of such a simple feature is surprising. All the times I've had to use GDB the feeling I get from it is "makes very difficult things easier, but makes very easy things more difficult."

It requires a little getting used to, but quite powerful:


Seems like here's the right time and place to ask. How do I comfortably inspect variables of types like std::vector<std::vector<boost::optional<string>>> with gdb ?

GDB 7.0 and up can be expanded with Python pretty printers to do exactly this. See here: https://sourceware.org/gdb/wiki/STLSupport

Isn't lldb better option now? Commands are more structured.

FWIW, gdb on OSX is a pain to setup, and attaching processes is broken.

I tried LLDB on Ubuntu 12.04, only to find out that it has multiples bugs and no one cares. So no.

My current standing is to use whichever compiler and debugger your platform comes default with: msvc/cdb for Windows, gcc/gdb for Linux and clang/lldb for OS X. Crossing them always reveals tons of bugs, at least for me.

On osx, perhaps. On Linux it's still a little rough. Or was last time I tried to use it. That said, I do look forward to using it more. But most importantly, I look forward to the tooling that can be built around it, since it's much easier to treat it as a library than gdb.

I heard they recently bit the bullet and implemented GDB/MI so lldb can be used by all mainstream IDEs. This doesn't stop lldb-specific targets in IDEs, but it does make them less likely is my guess.

Copying `/usr/bin/gdb` and `/usr/libexec/gdb/gdb-i386-apple-darwin` from an OSX Lion installation is one way to get a functional gdb on Mountain Lion or Yosemite.

Buried lede: Oracle uses GDB.

This blog post was written 6 months before Ksplice had anything to do with Oracle.

Its actually the ksplice blog. :/

Oracle got their filthy hands on it when they bought Sun.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact