I just realised that, since I started doing free software development, debugging has become much easier than it used to be.

At first, I didn’t even knew gdb, and debugged with printf()s. Totally impractical, slow, prone to error (as it moves corruptions to other places or even completely hide them), requires recompilation and a new run each time one needs more info.

Then I learnt about gdb and how to use it. Once I could reproduce a bug, finding its root cause became much easier. Mid-2002, a crash handler was added to Claws Mail, which automatically attached gdb to a crashed process and got a backtrace. That increased the quality of our crasher bug reports, when Claws Mail was compiled with debug symbols. Distributions were more and more shipping with bug-buddy or equivalents, that did the same thing. It was still rather useless without debug symbols, but better than nothing…

Then Valgrind came around, and that gem made tracking down corruptions, double-frees, unitialized pointers, dangling pointers, etc, much easier. I remember tracking one of these in IMAP code for days, and Valgrind would have made it clear in a matter of minutes. Valgrind is the debugging tool that gives me the most « oh damn, how did I not notice! » moments.

As if it wasn’t enough, the glibc developers started integrating basic handling of memory allocations errors, basically making the program dump core right on such an error, instead of silently corrupting things and continuing to run for a few seconds/minutes. And now, the glibc itself is able to produce crude stacktraces when crashing on these conditions!

Finally, distributions developers start to get tired of crash dumps with no debug symbols, and some services appear that automatically re-trace crashes with debug symbols: Launchpad.net, for example, gets bugreports with stacktraces from Ubuntu’s apport. Then a bot automatically retraces with debug symbols, and we get a really nice report.

All of these really help, and kids these days have it easy!