Response to Long Distance NFS Query

Frank Kastenholz (KASTEN@MITVMA.MIT.EDU)
Sun, 20 Dec 87 15:45:21 EST

A few months ago I made a general request to the list about
running NFS internetwork gateways. The specific configuration
that I have to deal with is two Ethernets separated by some
physical distance (possibly intercontinental). There would be
some kind of gateway/bridge/router/thing at each Ethernet and the
things are connected by a medium to high speed serial link (anything
from about 38K to 1.544M bits/sec).

The responses that I received (minus the "gee, what a great idea's"
or "Well, it had to come someday" etc etc responses) are:

(I have left the originating party's name on each on - the rest of the
SMTP/RFC822 junk has been removed).

From: "James B. VanBokkelen" <JBVB@AI.AI.MIT.EDU>

By default, none of Sun's implementations of NFS use UDP checksums. If
you enable them, the last release I heard anything about still had the
4.2 UDP checksum mis-calculation. They assume they're running one hop,
on a CRC-protected medium like Ethernet.

Accordingly, you're not too likely to catch any situation where the NFS
packet is corrupted on the way through a gateway, or over an error-prone
link. Instant filesystem corruption. It can certainly be fixed if you
have source, but I don't own any Suns (2nd hand info, this), so I can't
say exactly what must be done.

From: jas@MONK.PROTEON.COM (John A. Shriver)

The nature of the Sun NFS fragmented UDP-grams causes many routers and
bridges to have fits. You get 6 back-to-back IP fragments. If ANY of
those fragments is lost, the entire UDP-gram must be retranmitted.

You can, however, reduce the size of the UDP-gram. In /etc/fstab, you
need to add the undocumented rsize & wsize switches. For example:

across_gw_machine:/usr /usr nfs rw,noquota,soft,rsize=2048,wsize=2048

This will reduce the size of the UDP-gram to 2048 btyes of data, as
opposed to the default 8192. This will cuase only two fragments
instead of eight. (Do keep this parameter a multiple of 1024, as all
the network code likes page-aligned buffers.)

For reference, see bug 135 in Sun Software Technical Bulletin,
February 1987, part number 812-8701-01. (The page numbering is
botched in this one.)
From: (Stuart Levy)

One problem we've had in NFSing between disparate machines is with
naming them. The mount request passes the originating machine -name-
rather than having the server use gethostbyaddr(). It's important
to check that "hostname" on the client yields a name known to the
server and vice versa. That's probably not the whole problem but
can cause things to break.

A guy from Proteon, Mick Scully ( recently visited here
and mentioned that he had mounted NFS filesystems at Berkeley across ARPAnet.
From: hedrick@ATHOS.RUTGERS.EDU (Charles Hedrick)

We run NFS over cisco routers, either directly connecting two
Ethernets or connecting Ethernets via a T1 line. The only problem is
that the Ethernet cards used by cisco (and others) can't handle large
numbers of consecutive packets. So you need to specify
rsize=nnn,wsize=nnn in the mount. Typically we use 2048, though I
think someting a bit closer to 3000 might give better performance. I
haven't tried it over anything slower, though we understand that
somebody a Univ. of Maryland mounted one of our disks over NSFnet.
From: John Romkey <ROMKEY@XX.LCS.MIT.EDU>

One problem you'll run into is that NFS does not checksum its packets.
NFS packets are UDP-based instead of TCP-based, and the UDP checksum is
optional. On a single ethernet, the ethernet's CRC is possibly reliable enough
to detect bad packets, but through an IP router there is too high a probability
of losing (.1% would mean one out of 1000 packets was damaged; you really
desparately want 0% errors).

The reason is that there are chances for corruption of data in the ethernet
interface of the IP router, in the IP router's memory, and in the
other interface it routes the NFS packet too. The corruption can be due
to hardware errors, electrical noise, memory errors and software problems.
In fact, you've got the same problem with just an NFS server and client on
the same LAN, but since fewer components are involved, the chances of error
are much smaller.

I've spoken with people who've used NFS over a router, and they've actually
seen files corrupted due to the lack of checksums. I'd recommend against it.

BTW, the reason they turn off checksums is to up performance.
                                        - john romkey

Any further discussion should go to the list, to the original author
or directly to me (unfortinately I have recently moved from MIT-Multics
to MITVMA but the list has yet to catch up to me (whoever is running
the distribution list must be on a verryyyyy loooonnnnggg vacation:-))

Seasons greetings to all

Frank Kastenholz

This archive was generated by hypermail 2.0b3 on Thu Mar 09 2000 - 14:40:16 GMT