[geeks] The value of regression tests!

[der Mouse]

I just got a rather sharp lesson - the hard way - in the value of
regression tests.

I have a hack to gcc, to implement a particular language feature I find
useful (the details aren't really relevant, but, for concreteness: it's
labeled control structure, supporting things like
	for <"tuesday"> (...;...;...)
	 { ...
	   for (...;...;...)
	    { ...
	      if (...) break <"tuesday">;
	      ...
	    }
	   ...
	 }
).

I implemented this years ago, for NetBSD 1.4T, meaning egcs-1.1.2.  It
was an interesting little project, and the extension has proven very
useful.  A while back, I rolled this forward to NetBSD 3.1; this was
not a big deal, since its gcc's internal structure is fairly similar to
1.4T's.  But when I started using NetBSD 4.x, I found that _its_ gcc
used a totally different parser (recursive descent instead of
yacc-style) and it would basically mean a de novo reimplementation.

Over the last few days, I finally bit the bullet and did that
reimplementation.  I was very unsure of it, though, so I wrote up some
small regression tests (white-box testing, focusing on the kind of
mistakes I thought were reasonably likely in view of the
implementation).  Sure enough, I found a bug or two and had to jump
through assorted hoops to fix them - if I didn't think it highly
unlikely the GNU people have even noticed my hacks to gcc, much less
cared either way about them, I'd think the new internal design (the one
with the recursive-descent parser) had been deliberately designed to
make it hard to implement labeled control structure!

But, once I _had_ the test suite, just for the hell of it, I threw it
at my 3.1 implementation.

Apparently I've got bugs in that version, bugs which must have existed
since day one (day one for the port to 3.1's compiler, that is).  If
I'd built a proper regression test suite at the beginning, I would have
caught and fixed these with the implementation still fresh in my mind.
Instead, I've got some rather unpleasant debugging to do.  (Mercifully,
the 1.4T implementation passes it handily.)

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse at rodents-montreal.org
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B