
Yesterday, my program worked. Today, it does not. Why? (1999) [pdf] - maemre
https://www.st.cs.uni-saarland.de/publications/files/zeller-esec-1999.pdf
======
warpech
Reminds me of one of the nice features of Git: git bisect. Find the breaking
commit in the smallest number of steps.

Some random blog post about it:
[http://webchick.net/node/99](http://webchick.net/node/99)

~~~
azhenley
This is awesome. Didn't know about it, thanks!

My approach has been to step exponentially. I go back 1 commit, 2 commits, 4
commits... etc. I'll be using bisect from now on though.

~~~
maccard
Heh, I do a binary search. Find last known good build, go half way between,
etc

~~~
ChronosKey
That's what git bisect does

~~~
maccard
And not everyone uses git, unfortunately.

------
kossmoboleat
I took several courses with professor Zeller and with post-docs at his chair.
We applied the described concept of "delta debugging" to finding programming
errors in python programs.

Suprisingly it was possible to implement this in the course although one had
to fiddle so much with Python's internals.

Zeller's chair homepage has a lot of additional infos. Here's a page on how
the idea evolved after the initial paper in 1999: [https://www.st.cs.uni-
saarland.de/dd/](https://www.st.cs.uni-saarland.de/dd/)

~~~
sn9
He actually has a course on debugging on Udacity.

------
atonse
We once had a bug that would cause a failure in our test suite, but very
rarely. It was so rare that it would just "go away" and not show up for weeks.
So we thought it was odd (maybe bad test data, etc) but moved on.

When we decided to look into it when we had some downtime, we found out that
it was a date bug that only reared its head on the 31st day of the month,
hence only happening every two months.

I'm not sure bisect would've helped but still was really funny when we did
find the issue.

~~~
adrianratnapala
Thanks for the anecdote, because it gives me a new idea.

We should be monitoring the rate of test failures -- per test -- in a
timeseries and looking for periodicities. Just graphing them and seeing
regular spikes will help, but having real numerics looking for approximate
periodicities will catch this sort of thing (after a few months!).

~~~
bArray
Unless you are writing date specific code or have seen this bug arise, I
probably wouldn't bother. Time bugs will likely creep in regardless, for
example how would you find a bug that happens one month a year? How would you
find a bug once every four years? Sometimes human knowledge/experience can't
be tested out of a system. All test failures probably warrant investigation of
some type.

But, as a general rule, I would try to brute force or fuzz your "inputs",
a.k.a data out of your control. Random numbers, dates, user input, network
packets, etc. Users and environments often act in unexpected ways and you want
to make sure your bug finding isn't reduced to accidentally be working on the
right day! :)

------
dsw108
Here's an implementation of a modified version of his delta debugging
algorithm we did for minimizing file inputs that induce a failure in a
compiler or static analysis tool:
[http://delta.tigris.org/](http://delta.tigris.org/)

Also works for other kinds of files of course. Our version is easily seen as a
kind of simulated annealing, with the input providing sufficient randomness
that we did not need to add any internally.

Fun story: this was my first open source project. I had always wanted to be an
open source programmer, but the precipitating cause of me releasing delta was
that Microsoft Research asked me to. So back when Microsoft was saying that
open source is a "cancer" I became an open source programmer because Microsoft
asked me to.

------
ericfrederich
_The GDB people have done it again. The new release 4.17 of the GNU debugger
[6] brings several new features, languages, and platforms, but for some
reason, it no longer integrates properly with my graphical front-end DDD [10]:
the arguments specified within DDD are not passed to the debugged program.
Something has changed within GDB such that it no longer works for me.
Something? Between the 4.16 and 4.17 releases, no less than 178,000 lines have
changed. How can I isolate the change that caused the failure and make GDB
work again?_

I guess this was before Git. I love git bisect

~~~
khedoros1
I couldn't quickly find the 4.17 release, but gdb 4.17.0.4 was released at the
end of June 1998. The release notes mentioned Linux kernels in the 2.0-2.1
range.

------
T_D_K
Why does the article say there are 2^N different possible configurations?
Shouldn't it be N! ? Maybe I'm missing something, but I don't see how they can
rule out so many different orderings of the "configurations", since he started
out by saying that the ordering wasn't reliable.

Edited: I figured it out. It's because they essentially do a binary search of
the changes to find the issue, I should have read a bit further before
commenting. The tables / examples cleared it up pretty quickly.

------
known
Blue screen of death?
[https://en.wikipedia.org/wiki/Blue_Screen_of_Death](https://en.wikipedia.org/wiki/Blue_Screen_of_Death)

------
throwawaybbq1
If it was iOS, I'd say the provisioning profile expired :-p

------
candiodari
TLDR of the article: cross-application interfaces (like GDB's cli interface
has effectively become) break with version changes. Users don't care about the
reasons behind that breakage, and would like to see those treated as bugs.
Developers see that attitude as unworkable, and would rather just not care
about external interfaces at all.

I always wonder why we gave up on refactoring instead of switching it along so
that cross-program changes would become not just possible, but easy and
feasible.

What can be done using alt-shift-r in a distributed large scale java app with
a GWT frontend and 50 libraries used on the backend, each of which can run
distributed across machines ... is baffling.

It shouldn't be !

And now with microservices, it becomes more and more baffling that this worked
at one point. That's just horrible. Most of the complaints about it strike me
as being complaints that when a toolchain allows for complexity, people make
things complex. And of course, that's fair. If you give a 5 year old a laser
CNC machine and he finds how to turn it on, things are going to go south
quickly. But damn the things you can do with that !

~~~
infinity0
That is not what the paper is about. Stop pushing your own agenda and actually
read the thing.

