Gigabit Ethernet

Backing up a 250GB hard drive over 100Mbit Ethernet takes a l..o..n..g time. At 10 megabytes per second (theoretically), that's 600 megabytes per minute, or about eight hours. Compression can cut this number down, but the backup rate is severely constrained by the speed of the link.

In contrast, backing up to an external hard drive over Firewire or USB2 yields about 30MB/second, which brings it down to a manageable 2 1/2 hours.

I decided to take the plunge to Gigabit Ethernet (GigE), something I've been wanting to do for quite a while but was waiting for prices to drop. I brought in a contractor and upgraded my eight year old Cat5 wiring to Cat6. Strictly speaking, GigE is supposed to run over Cat5 for short distances, but I was having difficulties and it wasn't worth taking the time to individually check each wire.

I've learned several things. First, the "jumbo frames" that everyone gets so excited about are useless on a general purpose network. If you turn on jumbo frames, then every device on the network must support them and be set to use the same jumbo frame size. Since devices like printers and Tivos don't support GigE, much less jumbo frames, it means that jumbo frames are a non-starter. The place that jumbo frames *can* be useful is for a dedicated connection between servers or a backbone link.

The next thing I learned is that today's network application protocols don't scale well to GigE. When you upgrade from 10Mb to 100Mb Ethernet, your performance typically jumps almost 10x. However, when you go from 100Mb to 1000Gb, your performance typically only jumps 2.5x, even if you've optimized your system configuration. Ugh. That's not what I was hoping for.

The primary optimization required for Microsoft Windows 2000 and XP is to update the receive window size in the registry. This change isn't required for Windows Vista because Vista automatically tunes its networking parameters for optimal performance.

You can see what the practical maximum speed of your link is using pcattcp.exe. You run it on a server and on a client and it reports the speed at which it was able to send data. Most importantly, you can set the write buffer size, which allows you to determine the effect of larger buffers. In my case I determined that a 64K buffer provides throughput at 90% of the link capacity.

I still have more research to do, but that's the beginning.

[Update 6/20/2007] After spending far too many hours testing various GigE configurations, I've determined that the original Cat5 wiring was probably working fine. The problems I was having were a combination of the malfunctioning GS108 and a configuration error in my router/switch.

There's a great article at Hardware Secrets that describes how Gigabit Ethernet works and why Cat5 cables are sufficient.