GigE File Sharing Performance - 96MB/sec!

I wrote numerous blog entries last year about my difficulties getting GigE (Gigabit Ethernet) to work properly. Yesterday I upgraded both my Vista client and my Windows 2003 Server to 4GB RAM (see the end of this article for hardware configurations.) Suddenly my resource-constrained systems had lots of room to play. I reran my performance tests over my Gigabit Ethernet and came up with some very unexpected results.

First I used the DOS "copy" command on the Vista client to copy a 256MB file from the server to the client. The file was not cached on the server, and it transferred about 15MB/sec. This was the same performance I was getting before the RAM upgrade.

Next I repeated the same copy. The file was completely cached on the server and the file transferred at about 25MB/sec.

Finally, I again used the DOS "copy" command on the Vista client to copy the file from the client back to the server. The transfer peaked at 96MB/sec!! (A 256MB file copies VERY quickly at that speed.)

On the one hand, this is a contrived test - in the real world we almost never have the luxury of copying a file that's already cached in RAM. However, the tests lead me to several useful conclusions:

1. The tests show the peak performance of Windows file sharing when you take the disks out of the equation. At 96MB/sec, that's 85% of the practical maximum of 112MB/sec. Considering that the filesharing protocol itself has overhead, we are actually running at somewhere between 90% and 95% of the theoretical max. That's fantastic.
   
   
2. The file sharing protocol's latency is negligible. If it were significant, I wouldn't have reached the above performance numbers. Instead, the transfer rate of the hard drives is the primary performance constraint. Both test systems have SATA 7200RPM drives. I expect that if I had 10,000 RPM RAID 5 drives, my maximum performance when copying non-cached files would improve dramatically.
   
   
3. Something very strange is happening when copying from the server back down to the client. Why this operation is peaking at 25MB/sec is not clear.
   
   
4. Jumbo frames are completely unnecessary for peak performance. They might cut down on the CPU load, but even that's debatable when interrupt coalescing is enabled on the Ethernet card.

One set of datapoints that still need to be measured is to rerun from the command prompt on the server. In prior tests, it mattered which system initiated the file copy.

Finally, I accidentally performed the above tests with Virtual PC 2007 running on the Vista client. Virtual PC cut the transfer rates by 30 to 60%. The final copy peaked at 40MB/sec instead of 96MB/sec. Oddly enough, VMware Server was running on the Windows 2003 Server for all of the tests, and it had two virtual machines active. So while Virtual PC had a significant impact on network performance, it appears that VMware Server had no impact.


Client Configuration
Vista SP1
P5B Deluxe Wifi
Core 2 Duo 2.4 GHz
On-board Marvell Yukon Ethernet Card
  - Interrupt coalescing enabled
  - Jumbo frames disabled
4GB RAM
GigE registry update applied

Windows 2003 x64 Server Configuration
P5B Deluxe Wifi
Core 2 Duo 2.13 GHz
On-board Marvell Yukon Ethernet Card
  - Interrupt coalescing enabled
  - Jumbo frames disabled
4GB RAM
Ethernet adapters on the VMware virtual machines were running in Bridged mode.