8 Feb 88 13:59:00 PST
With the recent discussion about TCP testing and verification, I thought
that I would pass on an observation. I don't know if this reflects on
just the specific instances or protocol testing in general.
I was involved with GM in testing of MAP protocols for Autofact '84 and '85.
Both times, all of the vendors eventually were able to successfully pass
the conformance test suite. When interoperability testing began, MOST of
the vendors found problems affecting interoperability. On other occasions,
I have seen systems working on real networks fail conformance tests and
systems which passed the conformance tests fail on the real network.
I feel that this probably stems from a tendency by the test writer(s)
to concentrate more on the protocol details (such as which diagnostic
error code should be returned), than on the basic functionality of the
protocol. The protocol writer may then end up spending effort just trying
to make the test suite happy at the expense of a robust design.
This archive was generated by hypermail 2.0b3 on Thu Mar 09 2000 - 14:40:42 GMT