Re: Benchmarks for high-level protocols

David Haynes (mnetor!yetti!geac!
20 Sep 87 12:26:42 GMT

In article <> (Erik Nordmark) writes:
>Reply-To: (Erik Nordmark)
>Distribution: world
>Organization: Swedish Institute of Computer Science, Kista
>Keywords: benchmark performance
>To my knowledge the standard way of comparing protocol performance
>is to transfer a large file between two machines using the different
>My question to the net is if anybody uses more sophisticated benchmarks
>that take into account e.g. real-time communication, transaction-style
>communication, the overhead for multiple connections, the protocol's
>ability to handle packet loss due to the sender(s) overrunning the
>receiver et.c. et.c.
>(Pointers to any litterature on the topic are appreciated too!)
>If the response to this message is significant I'll summarize to the net!
>Erik Nordmark

I have been using a home-brew version of the "ping" program to
test the relative performance of STREAM and DATAGRAM messages
in the UNIX and INTERNET domains of BSD unix.

Primarily, the code asks for a packet size and the number of
messages to be sent and then sets up a bounce-back type
connection between the server and the client.

Time is taken to complete all the messages and the throughput
in messages per second and bytes per second are taken.

Modifications I want to make (if I ever get the time :-)) are
to allow for different sized packets (random and double-bell
curve distribution) and to have the packet carry some of the
timing information.

As a second test, I have almost completed the internet version
of Stonebreakers' concurrency control model for (relational)
databases. The UNIX domain version works just fine. The idea
is to have a client process which simulates a "user" or a
number of users entering transactions of various size and length.
The actual properties of these users can be configured by
the tester. The system then moves the transactions through the
concurrency controller model and performs such things as
re-tries, transaction commits, transaction aborts, etc.
until all transactions are completed. (Actually, the model
runs forever as it sits now, we just watch it run for the
first 1,000 transactions or so and see how it doing.)

The entire system is controlled via a master clock which
simulates the system clock in a machine.

The whole system requires the co-operation of 19 distict processes.

This, I think, makes a nice benchmark set for any message-based
communications system.


David Haynes			      [	mnetor,	yetti, utgpu] !	geac ! david
Geac Computers International Inc.	   Wise	words in mouths	of fools
350 Steelcase Road,Markham, Ontario,	   do oft themselves belie.
CANADA,	L3R 1B3	   +1 416 475 0525 x 3420

This archive was generated by hypermail 2.0b3 on Thu Mar 09 2000 - 14:39:16 GMT