Frank Kastenholz (KASTEN@MITVMA.MIT.EDU)
Sat, 13 Feb 88 10:03:21 EST
Hal Peterson (hrp%windsor.CRAY.COM@uc.msc.umn.edu) at Cray wrote
>I should stress, though, that a verification suite is only part of a
>comprehensive testing strategy, and that bake-offs can be extremely
>valuable for finding bugs. A verification suite is software, and like
>all software it is imperfect. It is entirely possible that two
>implementations both pass the suite but don't interoperate. Bake-offs
>can catch that, and when they do they have found as many as FOUR bugs:
>1. a bug in one of the implementations.
>2. a bug in the other implementations; after all, they could both be
> at fault.
>3. a bug in the specification. It may be unclear or inconsistent or
> incomplete and so have misled the implementors.
>4. a bug in the verification suite. It should have caught the
> problem, and next time it will, since it's a simple matter to add
> a test to a well-designed suite. The suite gets better as time
> goes on.
His last two points bring two simple(?) questions to mind -
How do the Specification Bugs get resolved?
Who Verifies the Verification Suite???? (Qui Ipsos Custode Custodiet)
This archive was generated by hypermail 2.0b3 on Thu Mar 09 2000 - 14:40:42 GMT