Backing up a 250GB hard drive over 100Mbit Ethernet takes a l..o..n..g time. At 10 megabytes per second (theoretically), that's 600 megabytes per minute, or about eight hours. Compression can cut this number down, but the backup rate is severely constrained by the speed of the link.
In contrast, backing up to an external hard drive over Firewire or USB2 yields about 30MB/second, which brings it down to a manageable 2 1/2 hours.
I decided to take the plunge to Gigabit Ethernet (GigE), something I've been wanting to do for quite a while but was waiting for prices to drop. I brought in a contractor and upgraded my eight year old Cat5 wiring to Cat6. Strictly speaking, GigE is supposed to run over Cat5 for short distances, but I was having difficulties and it wasn't worth taking the time to individually check each wire.
I've learned several things. First, the "jumbo frames" that everyone gets so excited about are useless on a general purpose network. If you turn on jumbo frames, then every device on the network must support them and be set to use the same jumbo frame size. Since devices like printers and Tivos don't support GigE, much less jumbo frames, it means that jumbo frames are a non-starter. The place that jumbo frames *can* be useful is for a dedicated connection between servers or a backbone link.
The next thing I learned is that today's network application protocols don't scale well to GigE. When you upgrade from 10Mb to 100Mb Ethernet, your performance typically jumps almost 10x. However, when you go from 100Mb to 1000Gb, your performance typically only jumps 2.5x, even if you've optimized your system configuration. Ugh. That's not what I was hoping for.
The primary optimization required for Microsoft Windows 2000 and XP is to update the receive window size in the registry. This change isn't required for Windows Vista because Vista automatically tunes its networking parameters for optimal performance.
You can see what the practical maximum speed of your link is using pcattcp.exe. You run it on a server and on a client and it reports the speed at which it was able to send data. Most importantly, you can set the write buffer size, which allows you to determine the effect of larger buffers. In my case I determined that a 64K buffer provides throughput at 90% of the link capacity.
I still have more research to do, but that's the beginning.
[Update 6/20/2007] After spending far too many hours testing various GigE configurations, I've determined that the original Cat5 wiring was probably working fine. The problems I was having were a combination of the malfunctioning GS108 and a configuration error in my router/switch.
There's a great article at Hardware Secrets that describes how Gigabit Ethernet works and why Cat5 cables are sufficient.
Wednesday, April 25, 2007
Sunday, April 22, 2007
ReadyBoost Part IV
I finally ordered a 4GB Apacer HT203 and installed it. I also installed the Vista USB patch. The hang problem is now fixed (see my earlier blog entries about ReadyBoost.) I can browse the thumb drive after resuming from sleep without difficulty.
However, ReadyBoost still doesn't work reliably after resuming from Sleep. After the computer wakes up, ReadyBoost works for a while, then mysteriously turns off (as seen by opening the Properties for the drive and looking at the ReadyBoost tab.) If I reenable ReadyBoost, it just turns off again almost immediately.
At this point, I'm only slightly better off than I was before I applied the Vista USB fix. The computer no longer hangs, but ReadyBoost still becomes disabled after resuming from sleep. Doing a "Safe Remove" and then reinserting the thumb drive, even into another port, doesn't help.
However, ReadyBoost still doesn't work reliably after resuming from Sleep. After the computer wakes up, ReadyBoost works for a while, then mysteriously turns off (as seen by opening the Properties for the drive and looking at the ReadyBoost tab.) If I reenable ReadyBoost, it just turns off again almost immediately.
At this point, I'm only slightly better off than I was before I applied the Vista USB fix. The computer no longer hangs, but ReadyBoost still becomes disabled after resuming from sleep. Doing a "Safe Remove" and then reinserting the thumb drive, even into another port, doesn't help.
Saturday, April 21, 2007
Symbols for CRT and MFC Source Code
[12/19/2014: Updated to include Visual Studio 2013.]
I'm not sure why I keep getting stuck on the littlest things. Ever since I installed Visual Studio 2010, I haven't been able to debug into the C Runtime (CRT) or the MFC source code. This is an operation that never gave me any difficulties in years of using VC6, so it's been quite frustrating. I've made several attempts to fix the problem, but no success until today.
Reproducing the problem is straightforward. Set a breakpoint in one of your MFC message handler functions, go to the Call Stack window, and try to click on any of the functions in mfc80d.dll. Similarly, any attempt to set a breakpoint on an MFC function (such as CWinApp::CWinApp) simply fails to work.
'MyApp.exe': Loaded 'C:\WINDOWS\WinSxS\x86_microsoft.vc80.debugmfc_1fc8b3b9a1e18e3b_8.0.50727.762_none_29a8a38855141f6e\mfc80d.dll', Symbols loaded (source information stripped).
The key part is "source information stripped", which is strange, because the PDB files for MFC ship with Visual Studio (and the service pack) and those PDB files include source information.
I searched my system for mfc80d.pdb and I found two copies. The first copy is in the symbol cache at c:\cache\mfc80d.i386.pdb\A12C75C3E6A244E3B3BFBE577AB27642e. This file was about 2MB. The second copy of mfc80d.pdb was in c:\windows\symbols\dll. This file was 13.5MB, significantly larger and probably the version of the PDB file that I needed. The problem was making VC2005 use it. That's the hard part.
First go to Tools/Options, open Debugging, and select Symbols. Add the directory "C:\Windows\Symbols\dll" to the beginning of the list. This is different than how VC6 was configured, which required that you not include the "dll" part of the path.
Click OK. Now close Visual Studio. Go to Task Manager and make sure its gone. Now kill the process MSPDBSRV.EXE, if it's running.
Finally, the critical operation to making all of the other steps work is to remove the smaller copy of mfc80d.pdb from the cache. To do this, delete the directory c:\cache\mfc80d.i386.pdb. The path may be slightly different if you are on a 64-bit system.
At this point you can restart VC2005, load your project, and run your application. In the Modules window, you should see that the symbol file for mfc80d.dll is being loaded from C:\Windows\Symbols\dll.
I'm not sure why I keep getting stuck on the littlest things. Ever since I installed Visual Studio 2010, I haven't been able to debug into the C Runtime (CRT) or the MFC source code. This is an operation that never gave me any difficulties in years of using VC6, so it's been quite frustrating. I've made several attempts to fix the problem, but no success until today.
Reproducing the problem is straightforward. Set a breakpoint in one of your MFC message handler functions, go to the Call Stack window, and try to click on any of the functions in mfc80d.dll. Similarly, any attempt to set a breakpoint on an MFC function (such as CWinApp::CWinApp) simply fails to work.
Part 1
If you look at the Output window, you'll see a message such as this one:'MyApp.exe': Loaded 'C:\WINDOWS\WinSxS\x86_microsoft.vc80.debugmfc_1fc8b3b9a1e18e3b_8.0.50727.762_none_29a8a38855141f6e\mfc80d.dll', Symbols loaded (source information stripped).
The key part is "source information stripped", which is strange, because the PDB files for MFC ship with Visual Studio (and the service pack) and those PDB files include source information.
I searched my system for mfc80d.pdb and I found two copies. The first copy is in the symbol cache at c:\cache\mfc80d.i386.pdb\A12C75C3E6A244E3B3BFBE577AB27642e. This file was about 2MB. The second copy of mfc80d.pdb was in c:\windows\symbols\dll. This file was 13.5MB, significantly larger and probably the version of the PDB file that I needed. The problem was making VC2005 use it. That's the hard part.
First go to Tools/Options, open Debugging, and select Symbols. Add the directory "C:\Windows\Symbols\dll" to the beginning of the list. This is different than how VC6 was configured, which required that you not include the "dll" part of the path.
Click OK. Now close Visual Studio. Go to Task Manager and make sure its gone. Now kill the process MSPDBSRV.EXE, if it's running.
Finally, the critical operation to making all of the other steps work is to remove the smaller copy of mfc80d.pdb from the cache. To do this, delete the directory c:\cache\mfc80d.i386.pdb. The path may be slightly different if you are on a 64-bit system.
At this point you can restart VC2005, load your project, and run your application. In the Modules window, you should see that the symbol file for mfc80d.dll is being loaded from C:\Windows\Symbols\dll.
Part 2
[This applies to Visual Studio 2010, 2012, and 2013.]
You may need to tell Visual Studio where the source code is installed. Right-click on a C++ solution and select Properties. Under Common Properties, select Debug Source Files. For Visual Studio 2013, it looks like this on my machine:
Problem solved.
Tuesday, March 13, 2007
Solution to Vista USB ReadyBoost Hang
Microsoft has released a patch for Vista that cures several problems with USB devices, including devices that hang when resuming from Sleep. Here's the solution:
Reliability update for the USB stack in Windows Vista
http://support.microsoft.com/kb/925528/en-us
Finding this patch isn't particularly easy if Google is your only search engine. The knowledgebase article doesn't mention ReadyBoost at all. It comes up as the first result if you search support.microsoft.com for vista and usb, but it's on page three if you search Google for vista usb hang.
My Apacer USB thumb drive is dead, so I can't test out the patch until I order a new one.
[Update 4/25/2007]
The Microsoft Knowledgebase article has been retitled Stop errors occur on a Windows-based computer that has 2GB or more of RAM and is using an NVIDIA nForce USB controller. My computer isn't based on the nVidia chipset, but the update still fixed my hang problem, so I'm not sure why Microsoft retitled the article. I do have 2GB of RAM.
Reliability update for the USB stack in Windows Vista
http://support.microsoft.com/kb/925528/en-us
Finding this patch isn't particularly easy if Google is your only search engine. The knowledgebase article doesn't mention ReadyBoost at all. It comes up as the first result if you search support.microsoft.com for vista and usb, but it's on page three if you search Google for vista usb hang.
My Apacer USB thumb drive is dead, so I can't test out the patch until I order a new one.
[Update 4/25/2007]
The Microsoft Knowledgebase article has been retitled Stop errors occur on a Windows-based computer that has 2GB or more of RAM and is using an NVIDIA nForce USB controller. My computer isn't based on the nVidia chipset, but the update still fixed my hang problem, so I'm not sure why Microsoft retitled the article. I do have 2GB of RAM.
Making VC2005 Remote Debugging Work
I use Virtual PC extensively for testing older operating systems and non-standard configurations. When I have to debug configuration-related problems, I use Remote Debugging to connect to the Virtual PC instance and debug in-situ.
Before I upgraded to VC2005, remote debugging with VC++ 6 was straightforward. I updated the PATH on the remote computer to include MSVCMON.EXE, the related debugger DLLs, and the debug versions of the C runtime library and MFC. Worked great.
Remote debugging with VC2005 is more challenging. Between security problems and Windows Side-by-Side assembles, it can take quite a few steps to make everything work. Here is what I had to do.
Your debugger and Windows firewall double-team you to make configuring remote debugging as painful as possible. My first problem was to fix the firewall, since I couldn't connect to the Windows XP SP2 computer at all. I fixed this by disabling the Windows Firewall (if you are following along, don't do this unless you are absolutely certain it's okay for your environment!) Probably overkill, but simple.
My next problem was debugger authentication. This proved to be a challenge because my Virtual PC test machine is on a domain and my development machine is not. After several hours of effort, I determined that they simply were not going to agree on a trust relationship, so I disabled authentication. (Handy tip - you can do this from the command line with /noauth. Also add /nosecuritywarn if you don't want to go crazy with security prompts.)
I thought I was golden at this point. I clicked F5 to debug my executable and was told, "Could not start App.exe". So I tried to start the application from Explorer. Same result. I ran Dependency Walker to check my DLL dependencies and was reminded that I needed to update my path to include the VC2005 debug libraries for the CRT and MFC. I did so, clicked F5 again, and... it still didn't work.
Several hours and much frustration later, I had received a solid education about Windows Side-by-Side Assemblies, otherwise known as WinSxS. In order to cure "DLL Hell," Windows XP and Windows Vista provide support for an executable's manifest to hardcode the desired version of dependent DLLs. For executables created with VC2005 that are dynamically linked to MFC and the CRT, a manifest is automatically inserted into the generated executable that includes this version information.
Once this is done, Windows XP and Windows Vista become absolutely hell bent on loading the correct version of the DLLs. Trying to work around this protection is futile. None of the following "obvious" solutions work:
\Program Files\Microsoft Visual Studio 8\VC\redist\Debug_NonRedist\x86
Inside this directory, you'll find two more directories named Microsoft.VC80.DebugCRT and Microsoft.VC80.DebugMFC. These directories must be copied to the same directory as your EXE and magically everything works right.
At this point you'd think the story is over, but it's not. Windows 2000, Windows NT and Win9x don't support WinSxS, so the manifest is ignored. To debug your application any of THOSE versions of Windows, you need to include the appropriate DLLs on your PATH (or copy the appopriate DLLs into the same directory as your EXE.) I set the path in Virtual PC to point directly to the WinSxS directories on my development machine, which were: ("duo" is the host name of the development computer, which drive C shared under the name "C")
-
Finally, I used the following command in my batch file to start the debug monitor:
start "" "\\duo\c\Program Files\Microsoft Visual Studio 8\Common7\IDE\Remote Debugger\x86\msvsmon.exe" /noauth /anyuser /nosecuritywarn /timeout 7200
Finally, that made everything work properly.
Before I upgraded to VC2005, remote debugging with VC++ 6 was straightforward. I updated the PATH on the remote computer to include MSVCMON.EXE, the related debugger DLLs, and the debug versions of the C runtime library and MFC. Worked great.
Remote debugging with VC2005 is more challenging. Between security problems and Windows Side-by-Side assembles, it can take quite a few steps to make everything work. Here is what I had to do.
Your debugger and Windows firewall double-team you to make configuring remote debugging as painful as possible. My first problem was to fix the firewall, since I couldn't connect to the Windows XP SP2 computer at all. I fixed this by disabling the Windows Firewall (if you are following along, don't do this unless you are absolutely certain it's okay for your environment!) Probably overkill, but simple.
My next problem was debugger authentication. This proved to be a challenge because my Virtual PC test machine is on a domain and my development machine is not. After several hours of effort, I determined that they simply were not going to agree on a trust relationship, so I disabled authentication. (Handy tip - you can do this from the command line with /noauth. Also add /nosecuritywarn if you don't want to go crazy with security prompts.)
I thought I was golden at this point. I clicked F5 to debug my executable and was told, "Could not start App.exe". So I tried to start the application from Explorer. Same result. I ran Dependency Walker to check my DLL dependencies and was reminded that I needed to update my path to include the VC2005 debug libraries for the CRT and MFC. I did so, clicked F5 again, and... it still didn't work.
Several hours and much frustration later, I had received a solid education about Windows Side-by-Side Assemblies, otherwise known as WinSxS. In order to cure "DLL Hell," Windows XP and Windows Vista provide support for an executable's manifest to hardcode the desired version of dependent DLLs. For executables created with VC2005 that are dynamically linked to MFC and the CRT, a manifest is automatically inserted into the generated executable that includes this version information.
Once this is done, Windows XP and Windows Vista become absolutely hell bent on loading the correct version of the DLLs. Trying to work around this protection is futile. None of the following "obvious" solutions work:
- Putting mfc80d.dll in the same directory as the EXE, even if it's the correct version.
- Including mfc80d.dll on the user's path.
- Manually copying the appropriate directories from the \Windows\WinSxS on the development machine to \Windows\WinSxS on the remote debuggee.
\Program Files\Microsoft Visual Studio 8\VC\redist\Debug_NonRedist\x86
Inside this directory, you'll find two more directories named Microsoft.VC80.DebugCRT and Microsoft.VC80.DebugMFC. These directories must be copied to the same directory as your EXE and magically everything works right.
At this point you'd think the story is over, but it's not. Windows 2000, Windows NT and Win9x don't support WinSxS, so the manifest is ignored. To debug your application any of THOSE versions of Windows, you need to include the appropriate DLLs on your PATH (or copy the appopriate DLLs into the same directory as your EXE.) I set the path in Virtual PC to point directly to the WinSxS directories on my development machine, which were: ("duo" is the host name of the development computer, which drive C shared under the name "C")
\\duo\c\Windows\winsxs\x86_microsoft.vc80.debugmfc_1fc8b3b9a1e18e3b_8.0.50727.762_none_29a8a38855141f6e
\\duo\C\windows\winsxs\x86_microsoft.vc80.debugcrt_1fc8b3b9a1e18e3b_8.0.50727.762_none_24c8a196583ff03b
-
Finally, I used the following command in my batch file to start the debug monitor:
start "" "\\duo\c\Program Files\Microsoft Visual Studio 8\Common7\IDE\Remote Debugger\x86\msvsmon.exe" /noauth /anyuser /nosecuritywarn /timeout 7200
Finally, that made everything work properly.
Tuesday, March 6, 2007
Visual Studio 2005 SP1 Update for Windows Vista Released
A little bit later than I really needed it, but Visual Studio 2005 Service Pack 1 Update for Windows Vista has been released. Get it while it's hot!
http://www.microsoft.com/downloads/details.aspx?FamilyID=90e2942d-3ad1-4873-a2ee-4acc0aace5b6&DisplayLang=en
http://www.microsoft.com/downloads/details.aspx?FamilyID=90e2942d-3ad1-4873-a2ee-4acc0aace5b6&DisplayLang=en
Friday, February 23, 2007
Virtual Server 2007 versus VMware Server
With the advent of the Core 2 Duo processors, it's finally possible to build a server for virtual machines without breaking the bank. I have three computers in my office that run various versions of Exchange Server for testing. Most of the time, these boxes are little more than space heaters, so I was looking forward to integrating everything into a single box.
I started by installing Windows Server 2003 x64, which is required for Exchange 2007. My plan was to run Exchange 2007 on the host x64 operating system and the older versions of Exchange in virtual machines.
The Microsoft solution for creating virtual servers is Microsoft Virtual Server 2005 R2. I had tried to install the software before the R2 version and I found it to be difficult to install and even more difficult to use. The documentation, while extensive, didn't have any Getting Started section that made any sense. My experience installing the R2 version was only slightly better and I spent several hours figuring out how to do things that should have been obvious.
The next step was to convert my physical servers into virtual servers (P2V). I spent almost a day researching how to do this and finally gave up. Microsoft's tools are immature and very difficult to use. The P2V conversion process relies on having a Domain Controller, Microsoft Operations Manager (the 32-bit version), and other requirements, none of which I had the time or inclination to set up. I uninstalled Virtual Server 2005 R2.
The alternative was VMware's product, VMware Server, which just recently became free. I found this software to be very easy to use. I installed it, opened the desktop icon labeled "VMware Server Console," and was able to immediately get started. This was a stark contrast to my experience with Microsoft's product. (Note that I've used Virtual PC since before Microsoft purchased it, so I have no shortage of experience with the technology that Microsoft Virtual Server is based on.)
Converting my physical servers to virtual servers was equally easy. I downloaded VMware Converter and followed the simple wizard. Once the conversion process was finished, I went back to VMware Console, added the new virtual machine, booted it, and it worked. The only caveat was that you have to power down the old physical computer before booting the virtual machine, or you end up with duplicate names. Also, if you've set your DHCP server to hand out a particular IP address to the MAC on your old server, you need to update your DHCP server.
It's also possible to run the VMware console on other computers by downloading the Client installer from the VMware downloads page. Although this works well when you are managing multiple servers, the sluggish screen update over the LAN can be annoying. I use Remote Desktop when I need to work with one of the virtual servers for any period of time.
I'm really impressed with VMware Server. It works well and has some impressive upgrades if I need better manageability than I have now.
I started by installing Windows Server 2003 x64, which is required for Exchange 2007. My plan was to run Exchange 2007 on the host x64 operating system and the older versions of Exchange in virtual machines.
The Microsoft solution for creating virtual servers is Microsoft Virtual Server 2005 R2. I had tried to install the software before the R2 version and I found it to be difficult to install and even more difficult to use. The documentation, while extensive, didn't have any Getting Started section that made any sense. My experience installing the R2 version was only slightly better and I spent several hours figuring out how to do things that should have been obvious.
The next step was to convert my physical servers into virtual servers (P2V). I spent almost a day researching how to do this and finally gave up. Microsoft's tools are immature and very difficult to use. The P2V conversion process relies on having a Domain Controller, Microsoft Operations Manager (the 32-bit version), and other requirements, none of which I had the time or inclination to set up. I uninstalled Virtual Server 2005 R2.
The alternative was VMware's product, VMware Server, which just recently became free. I found this software to be very easy to use. I installed it, opened the desktop icon labeled "VMware Server Console," and was able to immediately get started. This was a stark contrast to my experience with Microsoft's product. (Note that I've used Virtual PC since before Microsoft purchased it, so I have no shortage of experience with the technology that Microsoft Virtual Server is based on.)
Converting my physical servers to virtual servers was equally easy. I downloaded VMware Converter and followed the simple wizard. Once the conversion process was finished, I went back to VMware Console, added the new virtual machine, booted it, and it worked. The only caveat was that you have to power down the old physical computer before booting the virtual machine, or you end up with duplicate names. Also, if you've set your DHCP server to hand out a particular IP address to the MAC on your old server, you need to update your DHCP server.
It's also possible to run the VMware console on other computers by downloading the Client installer from the VMware downloads page. Although this works well when you are managing multiple servers, the sluggish screen update over the LAN can be annoying. I use Remote Desktop when I need to work with one of the virtual servers for any period of time.
I'm really impressed with VMware Server. It works well and has some impressive upgrades if I need better manageability than I have now.
Subscribe to:
Posts (Atom)