It happens quite often that bugs end up being obscure things like this.
The a-ha moment when you've figured one out is interesting. It's always a mixture of pleasure at having discovered a complex causal chain and horror at having written it.
Maybe one could ask the authors if they could donate their debugging articles to a cc licensed repository so the articles themselves could be hosted instead of linking? That would be very nice I think.
Don't want to write a real post but my most memorable simple story was with a C optimizing compiler. I was using a something like struct Item { char name[12]; int flag; } with code that would would use this in a btree and use memcmp(x,y,sizeof(Item)) for ordering. The name could be non-unique but name+flag would be and give specific ordering. To fill Item I used something like (much simplified):
So the bits after name are now significant in compares but expected to be zero because of memset. Well the optimizer would effectively remove the memset in release builds due to the strcpy so the code worked most of the time in release but would occasionally break because the bytes after the null terminated name could be anything and would only come into play when name was the same for 2 items.
I absolutely loved the "Things I won't work with" articles, very enjoyable to read. And I even learned some things about chemistry in the process. I can only recommend everybody who is even slightly interested in chemistry to read them.
seems he has since moved to sciencemag, but i don't see any new articles in that category over there (and all his old corante stuff has been copied over).
is really dumb. The moral is that you don't use global variables that are accessed by multiple threads (unless you're very careful). That's C programming 101.