I recently wired my house for Cat6 and gigabit ethernet and learned that this is a very tricky business. A lot of things can go wrong, such as cables can have too sharp bends, not crimped properly, the list goes on, and on, and on. Also, NFS had performance problems when run over gigabit ethernet, not giving me gigabit throughput. I just did a test. I can transfer files with HTTP at the speed of 50 megaBYTES per second, but I can transfer files via NFS at only 5 megabytes per second. i

On Fri, 26 Oct 2007 08:09:32 -0500, Ignoramus7291 <ignoramus7291@NOSPAM.7291.invalid> wrote: >I recently wired my house for Cat6 and gigabit ethernet and learned >that this is a very tricky business. A lot of things can go wrong, >such as cables can have too sharp bends, not crimped properly, the >list goes on, and on, and on. >Also, NFS had performance problems when run over gigabit ethernet, not >giving me gigabit throughput. I just did a test. I can transfer files >with HTTP at the speed of 50 megaBYTES per second, but I can transfer >files via NFS at only 5 megabytes per second. What buffer sizes do you have on your NFS? I'm not running jumbo frames since I have 100mbps devices on the same network, and I get 70MB/s on my gigabit connection from my linux client to my solaris server. I use wsize/rsize=32K for NFS. I also noticed differences between the two gigabit interfaces on my asus motherboard used for linux. The Marvell 88E8001 gigabit I/F does much better than the Nvidia CK804.

On 2007-10-26, AZ Nomad <aznomad.2@PremoveOBthisOX.COM> wrote: > On Fri, 26 Oct 2007 08:09:32 -0500, Ignoramus7291 <ignoramus7291@NOSPAM.7291.invalid> wrote: > > >>I recently wired my house for Cat6 and gigabit ethernet and learned >>that this is a very tricky business. A lot of things can go wrong, >>such as cables can have too sharp bends, not crimped properly, the >>list goes on, and on, and on. > >>Also, NFS had performance problems when run over gigabit ethernet, not >>giving me gigabit throughput. I just did a test. I can transfer files >>with HTTP at the speed of 50 megaBYTES per second, but I can transfer >>files via NFS at only 5 megabytes per second. > > What buffer sizes do you have on your NFS? > I'm not running jumbo frames since I have 100mbps devices on the same network, > and I get 70MB/s on my gigabit connection from my linux client to my solaris > server. > I use wsize/rsize=32K for NFS. I also noticed differences between the two > gigabit interfaces on my asus motherboard used for linux. The Marvell 88E8001 > gigabit I/F does much better than the Nvidia CK804. That's what I am doing also. 32k. I have much better luck with http speed. i

I've probably missed some context, but a couple things: NFS is a request/response protocol. And 32KB isn't enough data at one time to get link-rate on GbE. So, on reads your NFS client has to be doing read-ahead, and on writes write-behind to get maybe four or more requests outstanding onto the wire at one time. If using UDP, keep in mind that all 22 or so fragments of that 32KB read/write have to make it through for any of that 32KB read/write to be good. That means if you have a packet loss probability of p, (1-p) is the prob of a given fragment getting there, which means that (1-p)^22 is the probability of all the fragments getting there. For TCP if one of the segments is lost, it only need retransmit that segment, not the entire 32KB read/write. rick jones -- Process shall set you free from the need for rational thought. these opinions are mine, all mine; HP might not want them anyway... :) feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...