Once, I used a chunk of ice to cool down a chip, and that made it work. The hardware guys were unimpressed. But hey, they've got cans of Chill and they use them a lot, and this software guy took a while to realize the reason the board worked in the morning and was dead by lunch, and worked for a little while again after lunch, was temperature related.
There were some devs who tracked down a nasty bug in a processor's TLB. I only heard about that one, wish I had been there. I only had to deal with the fallout in the hypervisor. Note: If you have to spend 20ms hunting down and killing lies with all interrupts turned off and everything basically stopped in its tracks, you are no longer a real-time operating system.
My ex-coworker has done the vanilla scope thing too and has a 400MHz scope at home. For some reason people like this are not too uncommon in Finnish oldskool[tm] IT scene. I remember how he isolated a latency and concurrency bug to an expensive interrupt handler. Rewriting isolated parts of core kernel code to make a really tricky problem go away was one of his more hardcore skills.
I'm not even near his level. My own experience is limited to slightly nibbling the edges of file system and block cache behaviour. It's a brave person who dares dive into that code. Not me.
But I do know one person who regularly works with decapped chips. He works for a company who do extremely low-level hardware investigations. Now that's hardcore.
When you find the problem it's usually a blinding flash of realization that illuminates a tiny, eensy bit of code that you tweak and make right in a couple of minutes. Invariably the mistake was pretty stupid. The glory moment is over quickly because you know all the test cases will pass and that you've just nailed another one.
You've got bragging rights during one lunch, but that's it. It's off to more mundane bugs in the mortal world, and you feel a little sad.
I need to do hardware again.
I would try it even before going for some harder software problems, because it's so easy.
I had a local-made spectrum clone, it didn't overheat, but I lost a multimeter on its power supply.