Verification Suite

Frank Kastenholz (KASTEN@MITVMA.MIT.EDU)
Sat, 13 Feb 88 10:03:21 EST

Hal Peterson ( at Cray wrote

>I should stress, though, that a verification suite is only part of a
>comprehensive testing strategy, and that bake-offs can be extremely
>valuable for finding bugs. A verification suite is software, and like
>all software it is imperfect. It is entirely possible that two
>implementations both pass the suite but don't interoperate. Bake-offs
>can catch that, and when they do they have found as many as FOUR bugs:
>1. a bug in one of the implementations.
>2. a bug in the other implementations; after all, they could both be
> at fault.
>3. a bug in the specification. It may be unclear or inconsistent or
> incomplete and so have misled the implementors.
>4. a bug in the verification suite. It should have caught the
> problem, and next time it will, since it's a simple matter to add
> a test to a well-designed suite. The suite gets better as time
> goes on.

His last two points bring two simple(?) questions to mind -

How do the Specification Bugs get resolved?

Who Verifies the Verification Suite???? (Qui Ipsos Custode Custodiet)

Frank Kastenholz
Atex Inc.

This archive was generated by hypermail 2.0b3 on Thu Mar 09 2000 - 14:40:42 GMT