Making a mountain out of a molehill (of bugs)

Rands In Repose has a wonderful new article called “Trickle Theory“. He talks about having a mountain of bugs right before a ship date (I think we work for the same start-up) and how to tackle the impossible task. I’ve covered that a lot of what he talks about in Getting to Deadline, but I wanted to mention a couple of points his article raises about bugs.

Having a mountain of bugs is a Bad Thing.

It happens and will continue to happen until a company is bit on the ass hard enough, but if you have 537 bugs in the database I can guarantee that 489 of them are not reproducible. I will lay cold hard cash down on the line to anyone who wants to make a bet that the combination of code to be tested, test environment and testcase will still reproduce the bug, or that the bug report has enough information to flick on a light bulb in someone’s head.

When it comes to triaging bugs “not being reproducible” is not that same as “is not a bug.”

I’ve been fortunate enough to work for a manager who took verification seriously and would not let the designers sit on a bug for more than two weeks. I don’t care how busy designers are Doing the Fun Stuff (implementing new features). They can take the time to look into the bug, check that it’s real and update the bug report with more details (if they can’t fix it in a short period of time). The hidden beauty is by taking a look at the bug they’ve started to let their subconcious knaw on it.

The biggest reason not to let a bug sit for long is because every bug can be just the tip of the iceberg. Who knows what is hiding because the verification engineer can’t go any deeper into the features?

Would you rather be Leonardo DiCaprio getting hot and sweaty in a car with a slightly-overweight but-I-can’t-say that-outloud oh-no-it’s-too-late Kate Winslet on the Titanic[1]? Or would you rather be the King of the World for real and see the iceberg before it sinks your ship[2]?

Bugs can also hilight design errors that may mean major restructuring and redesign. The sooner a bug is found and fixed, the less costly it is to the company.

One of the best time saving utilities I’ve written at work was a script for shoving all of the information to reproduce the bug into a bug report. It required that the verification engineer check in a branch copy of all code before submitting a bug report, but then they could run this script it would throw together all the information to reproduce it and attach it as a file to the bug report. When it came time to for a designer to tackle the bug they could run another script with the bug number and check out a sandbox of the entire environment of code, testcases and random seeds to reproduce it.

Triaging the bugs changes from eyeballing bug reports to re-running old bugs and seeing how they behave on the new code base. Because you can.

>> Rands In Repose: Trickle Theory[6]

>> All we are saying, is give bugs a chance.

[1] No one wants to be hot and sweaty in a car with Kate Winslet unless they are being paid $2.5 million[3] or they are Melanie Lynskey. Especially if they have to listen to Celine Dion.

[2] In this allegory[4] “your ship” could be your product, your start up, your ability to go out for drinks on friday afternoon or another name for Mr. Happy.

[3] Yeah, DiCaprio only got 2.5.

[4] Or is it a simile[5]? I hope my Mom isn’t reading this.

[5] Nesting and recursion[5]!

[6] I keep reading that as tickle theory.

Related Posts

• Getting to Deadline
• Bug 95849 and Bug 330884