NUnit Unit Testing with C++

I switch back and forth between C++ and C#. When doing C# development, NUnit rules the day for unit testing. Whether I'm doing automated tests from the command line or using the GUI to run selective tests (show below in a screenshot from SourceForge) NUnit is a pleasure to use.



If you've ever tried to run unit tests for C++, the landscape is much less appealing. C++ does not have a reflection API, nor does it have attributes that are embedded in the executable code, so it's much more tedious in C++ to do all of the housekeeping to initialize the framework and it's much more difficult to integrate external GUI tools. In short, unit test in C++ is a sub-par experience compared to more modern languages.

But I have good news for you - it's possible to use NUnit with C++. There are two minor caveats:

1. You must be using Microsoft Visual Studio 2005 or later (sorry MinGW and cygwin users.)
   
2. The code to be tested must either be a DLL or a static library. If you need to test code in an .EXE, you should factor that code out into its own static library.
   

The secret is to place your tests in a separate DLL compiled for C++/CLI, which you enable on the Visual Studio C++ properties page in the General properties, immediately under Configuration Properties. Under Project Defaults, set Common Language Runtime support to "/clr". Don't use any of the other variants - they won't work for this task.

If you haven't built this kind of project before, a C++/CLI project is a curious hybrid that contains all of the power of C++ as well as much of the power of .Net. (Access to certain features, like LINQ, is not available in C++/CLI.)

With a C++/CLI project, you can use NUnit attributes for classes and member functions, just like in C#. When you run NUnit, your target is your test DLL, which implicitly loads either your static library code or your DLL code.

This strategy can also be used in the test environment build into Visual Studio if you have the Professional edition or better.

AppCrash in StackHash_1703

I've wasted an hour solving this one. Here's the symptom. Your application crashes on Windows Vista almost immediately after it starts. If you look at the "Details", you see something like:

Problem signature
Problem Event Name: APPCRASH
Application Name: xxxxx
...
Fault Module Name: StackHash_1703
Fault Module Version: 0.0.0.0

The problem is that there's no such module as "StackHash_1703", so this appears to be some special case in Windows. I tried turning off antivirus and enabling compatibility mode, but the application still would not work.

The problem was that I had DEP (Data Execution Protection) enabled. For whatever reason, the error message gave me the AppCrash error above instead of the standard message about "DEP has closed the program."

To solve the problem, I added the application to the DEP exclusion list and everything worked again.

If the application that's crashing is video related (such as MovieMaker or Media Player), then your problem is probably caused by an old versions of Nero or an old video Codec.

Downloading symbols for .dmp files

We have an automated system for processing .dmp files received from the field, either our own collection or Microsoft's crash report system. We use cdb (the command line version of WinDbg) to automatically create a text file that contains the stack dump.

The batch file looks like this:

@set _NT_SYMBOL_PATH=SRV*c:\cache*http://msdl.microsoft.com/download/symbols
Cdb -lines -c "!analyze -v;q" -z %1

(Thanks to John Robbins for showing me how to use cdb in this manner.)

In theory, setting _NT_SYMBOL_PATH should provide cdb with enough information to automatically download symbols as needed. However, I wasn't seeing that happening. Without symbols, the debugger can't properly processes callstacks using FPO (Frame Pointer Omission), which means that the callstacks were often missing a lot of information.

Today I found a workaround. The symchk utility will examine a dmp file and verify that all of the required pdb and dbg files have been downloaded. Here is an example:

symchk /id Demo_000000.dmp /s SRV*c:\cache*http://msdl.microsoft.com/download/symbols


Remote Debugging Managed Code

Last year I blogged about how to get Remote Debugging working. That article was for Visual Studio 2005, but it works fine with some minor modifications for directory and filenames for Visual Studio 2008.

Now I've had to take the next step, which is remote debugging managed code instead of native code. Turns out that debugging managed code is even more complicated than native code.

One of the key problems I originally had was that I was never able to get authenticated debugging working because one computer was on a workgroup and one was on a domain. After reading several articles, I was able to solve the problem by creating an account on the remote computer that had the same username and password as the account of the development computer. If your development system isn't on a domain, make sure that the remote account is created on the local computer and not the domain. You may need to use the "NET USER" command from the command line to create such an account.

If you are running Windows Firewall, the remote debug module msvcmon.exe will automatically unblock the right ports for you. At least that was easy.

The next trick is to convince .Net to run your application from a network drive. This technique is described in the .Net Security Blog.

By the way, the Remote Debugger Configuration Wizard was useless for me.

Visual Studio 2008 Feature Pack: Manifest Problems

If you install the Visual Studio 2008 Feature Pack, be aware that the manifest handling in the release is buggy, to put it politely. It's very easy to end up with the manifest requiring both the RTM DLLs (9.00.21022.8) and the Feature Pack DLLs (9.00.30411.0). This can cause weird, untraceable errors at runtime. I blogged about this problem before, but in that case it was caused by changing from a Beta build to the Release build.

There's a helpful blog post about the problem in the Visual C++ Team Blog.

The important point is that there are two #defines that force the manifest to point to the newer DLLs:

#define _BIND_TO_CURRENT_CRT_VERSION 1
#define _BIND_TO_CURRENT_MFC_VERSION 1

There's a single #define that does the same thing, but it's broken in the Feature Pack. [9/7/2008 - It's fixed in Service Pack 1. Thanks "anonymous"!]

#define _BIND_TO_CURRENT_VCLIBS_VERSION 1

The blog points out that defining the explicit BIND macros is not always the right thing to do.

This problem bit me because I was following the directions in this blog post, which call for making copies of certain directories from Debug_NonRedist. In a system where the CRT and MFC DLLs have been installed, there are pointers in the registry from the old versions of the DLLs to the new versions, so everything "just works." However, when using the Debug_nonRedist directories, these redirect pointers don't exist. Since the manifest was calling for the old versions of the DLLs, the application wouldn't open.

If you get error c101008a, either Rebuild All or delete all of your "xxx.exe.embed.manifest" files and build the solution.

Debugging .Net Framework Code

When I started trying to write C# code, I spent most of my time with the Reflector trying to figure out what the .Net Framework was doing under the hood. My start routine was: start the debugger, see unexpected results, read source code in Reflector, repeat. I spent a lot of time searching for functions in the Reflector.

I spent years doing MFC programming. MFC shipped with complete source code from the very early days. Without that source code, it would have been almost impossible to solve many of the problems I ran into. (See my earlier blog about fixing problems with MFC symbols in Visual Studio 2005.)

I was tickled to discover today that Microsoft has made the source code of the Framework available for debugging purposes - even specialized code like Asp.Net. You must be running Visual Studio 2008. Take a look at Shawn Burke's Blog showing how to set it up.

Make sure you install the Visual Studio hotfix (QFE)!

WinSxS Breaks Old Libraries

After my previous experiences with handling Windows Side-by-Side Assemblies (WinSxS) in remote debugging and Isolated COM, I thought I was actually starting to get a handle on how it worked. Today I got stuck on another WinSxS problem, this time while porting our application to Visual Studio 2008.

The problem was that the application would fail to fail to start, with an error about the manifest being wrong. I used sxstrace, a handy tool under Vista, to try and determine what was happening. Sxstrace generated 180 lines of information about Vista's attempt to find and load the correct assemblies. It ended up being too much information. The only obvious problem I saw was that one of the DLLs being loaded was from Visual Studio 2005, not 2008.

I used Depends to look at the file and received the same errors. I looked in the Event Viewer and saw this error:

Activation context generation failed for "s:\csi\xsales\debug\XSALES.EXE". Dependent Assembly Microsoft.VC90.DebugCRT, processorArchitecture="x86", publicKeyToken="1fc8b3b9a1e18e3b", type="win32",version="9.0.20706.1" could not be found. Please use sxstrace.exe for detailed diagnosis.

This is even stranger, because 20706 was the version ID of Visual Studio 2008 Beta 2. I'm running the final release.

I used Visual Studio to open XSales.exe in Resource mode so I could look at the manifest itself. Here I found references to four versions of DebugCRT. The question was, where were they coming from? My code makes no explicit reference to assembly versions.

What I discovered was that the intermediate manifests generated by the compiler and linker include assembly information from all objects used by the linker, including objects from libraries, of which I had two. One of my .LIB files was generated by VS2005 and another was generated by VS2008 Beta 2, which is what caused the references to old assembly versions. Once I rebuilt those .LIB files with VS2008, the problem went away.

The lesson learned from all of this is that .LIB files are no longer easily portable between versions if they rely on any of the CRT or MFC DLLs. The painful part of this is that the problem doesn't show up until you try and run the software because none of the development tools warn about the inconsistency.

[Update 11/19/2008] You can find another solution here.

Legacy Code from ... CP/M?!?

Remember CP/M? Unless you are over 40, my guess is probably not. This week I ran into a bug in our software that traces its roots all the way back to CP/M in the 1970s. Over thirty years later, the code still exists in the Visual Studio 2005 C Runtime Library, waiting for the next innocent victim to run afoul of it.

Here's what happened. We had reverse engineered a data file format and had shipped the first pass to customers. Our only sample of the data file was quite small, about 10 records, but it was enough to determine the file format and make it work. The ten records in the file converted cleanly.

After the product shipped, several customers reported that only 26 records were being loaded. This was obviously incorrect as most of their files had hundreds of records. Our end-of-file handling code was common with numerous other modules and worked fine. The 26 records that did convert did so correctly.

Even when we instrumented our software to give more insight into what was happening at customer sites, we found nothing. Our software converted 26 records, detected end-of-file, and exited. WTF?

When we finally found a customer willing to share his data file, we ran our software in the debugger and got exactly the same results. 26 records were converted and the software exited cleanly, with no errors.

The lightbulb didn't go off over my head until I was looking at the data file with the cygwin "cat -v" command, which shows ASCII codes 0 through 31 as control characters. This particular data file had two bytes for each record ID and record numbering started at 0. The 27th record contained ID 26 (0x1a) which showed up as ^Z. Does that ring any bells? If you ever developed for CP/M (or for MS-DOS 1.0) it should.

Thirty years ago, CP/M only tracked the number of blocks in each file, not the number of bytes. By convention, Ctrl-Z was used to denote "end of file" for text files. MS-DOS 1.0, which bore a striking resemblance to CP/M internally, followed this same convention. At the time, the C Runtime Library understood this convention and automatically generated an "end of file" condition when ^Z was encountered. Today, the VS2005 C Runtime Library still contains that code and generates the end-of-file condition even if the exact length of the file extends beyond that point.

The bug in our software was that the file had been opened in text mode instead of binary mode. Normally this is easy to detect because the records become out of sync as they are read, but by some coincidence of the data layout, the data in this file was read in perfectly up until that ^Z.

So now I have one more reason to dislike CP/M, although (admittedly) in this day and age it seems somewhat pointless to carry a grudge against a dead operating system that was designed to run off of 8" floppy disks. Old habits are hard to break.

Static Code Analysis with Visual Studio 2005

Not too long ago I received a jarring reminder that "char" in Visual C++ is signed by default. The problem was related to the ctype functions, such as isalnum() and isspace(), all of which take an integer. When you sign-extend an eight-bit multibyte character (MBCS), you end up with a negative number. The ctype functions then do a table lookup on a negative number and your program crashes. This wouldn't be so interesting, except that this bug existed in our production code for six years before the software crashed for a customer in Europe. If the customer hadn't been willing to give me sample data to reproduce the problem, I'd still be scratching my head.

Apparently I'm not the only person who has run into this issue. Visual C++ 2005 includes a command line option named "/analyze" for finding this problem and many others. I had tried /analyze during the Whidbey Beta two years ago but didn't get useful results. Now the results were much more helpful.

The "/analyze" options enables the static code analysis feature in the C++ compiler. This feature is based on prefast, a technology that came out of Microsoft Research in 2001. Prefast knows about hundreds of common programming problems. I ran it on our codebase, which after eight years is quite mature and already compiled under warning level 4 with minimal problems.

What I learned was quite interesting. Prefast knows about the ctype problem I mentioned earlier and pointed out several lines I missed when I tried to fix the problem. All of those lines were crashes waiting to happen.

Prefast found several places where I expected one return type but was getting another. For example, at one point I checked for an HRESULT but was actually getting a bool, which meant that the sense of my error check was inverted.

Another error that prefast found was where I was calling sizeof on a variable that was defined as pointer instead of as an array, which meant that the length being handed to strncmp was wildly wrong. The code still worked, but that was a happy accident.

Prefast also had numerous warnings about the Boost C++ Library. It should be possible to suppress those warnings, but I haven't done so yet.

Prefast supports annotating your code with the Standard Annotation Language (SAL) to better describe the static behavior of the code. If you've looked at recent versions of the Windows SDK, there are numerous annotations such as __out_ecount(). These annotations provide additional information to prefast that allows for better analysis. You can learn more about these annotations at blogs.msdn.com.

Prefast also pointed out two problems with my code that I would never have found, even with a close code inspection. I'm sure you've seen stricmp, the case-insensitive version of strcmp. I was using stricmp to check for keywords. Turns out that this is a bad idea. Prefast gives warning C6400, which explains that some languages interpret combinations of letters as a single letter, which changes the behavior of stricmp. The correct solution is to use the Windows API call CompareString(), which should be set to LOCALE_INVARIANT or LANG_ENGLISH, depending on the version of Windows.

Prefast also warned me that using _alloca in a loop could cause a stack overflow. Normally I'd consider this obvious, but in this case the warning was being given about the W2A Unicode to MBCS conversion macro, which uses _alloca. I looked at the source code to W2A (in atlconv.h) and the problem seems to be handled properly, but it was a worthwhile exercise.

I'm adding prefast to my bag of recommended tricks. For more insight into the use of this tool, I invite you to read Scalable Defect Detection from the Center for Software Excellence at Microsoft.

Visual Studio 2008 Beta 2 Test Results

Today I tried building our product with Visual Studio 2008 Beta 2. The last two times I tested compiler upgrades (VC6 to VS.net 2003, then VS.net 2003 to VS2005), the upgrades were extremely painful and took days to complete.

I was pleasantly surprised that rebuilding our product in VS2008 Beta 2 went quite smoothly. The issues were minor:

• The switch /OPT:NOWIN98 option is no longer supported.
  Impact: None.
  
• Project could not be linked against a library originally built in VC6.
  Impact: Rebuild library.
  
• Visual Studio 2005 manifest that set requireAdministrator caused this error: manifest authoring error c1010001: Values of attribute "level" not equal in different manifest snippets.
  Impact: Removed the explicit manifest file and set the UAC level in the Linker node on the project properties.
  
• MT.EXE gives an error on a valid manifest.
  Impact: Move $(WindowsSdkDir)\bin to the top of the list for Executables files in VC++ Directories under Tools/Options.

In terms of the size of the generated executables, here are the stats:

Product	VS2005 size	VS2008 Size	Net Change
#1 (C++/MFC GUI)	1,198,592	1,180,672	-1.5%
#2 (C++/MFC GUI)	1,589,760	1,549,312	-2.5%
#3 (C++/MFC Console)	2,043,904	1,998,848	-2.2%

While not earth shattering, these numbers do show that they were able to contain bloat in MFC and the C runtime library. All of these applications are statically linked against MFC and the CRT.

The biggest surprise was that VC2008 worked fine with version 1.34.1 of the Boost library, which has not yet been tested on VC2008. (although, admittedly, our software only uses one or two of the Boost modules.)

That's it! So far, backwards compatibility is excellent.

Initial testing has shown no problems with the generated code. Common dialogs under Vista were automatically updated to Vista styling, as promised.

My only disappointment is that very little was done for the IDE for C++ developers in this version of Visual Studio. The tools for editing dialogs and other resources are still awful compared to VC6. For a look at the future of the IDE, take a look at Somasegar's blog. Attention is being renewed on the IDE for native C++ developers, but improvements won't ship until 2010.

Symbols for CRT and MFC Source Code

[12/19/2014: Updated to include Visual Studio 2013.]

I'm not sure why I keep getting stuck on the littlest things. Ever since I installed Visual Studio 2010, I haven't been able to debug into the C Runtime (CRT) or the MFC source code. This is an operation that never gave me any difficulties in years of using VC6, so it's been quite frustrating. I've made several attempts to fix the problem, but no success until today.

Reproducing the problem is straightforward. Set a breakpoint in one of your MFC message handler functions, go to the Call Stack window, and try to click on any of the functions in mfc80d.dll. Similarly, any attempt to set a breakpoint on an MFC function (such as CWinApp::CWinApp) simply fails to work.

Part 1

If you look at the Output window, you'll see a message such as this one:

'MyApp.exe': Loaded 'C:\WINDOWS\WinSxS\x86_microsoft.vc80.debugmfc_1fc8b3b9a1e18e3b_8.0.50727.762_none_29a8a38855141f6e\mfc80d.dll', Symbols loaded (source information stripped).

The key part is "source information stripped", which is strange, because the PDB files for MFC ship with Visual Studio (and the service pack) and those PDB files include source information.

I searched my system for mfc80d.pdb and I found two copies. The first copy is in the symbol cache at c:\cache\mfc80d.i386.pdb\A12C75C3E6A244E3B3BFBE577AB27642e. This file was about 2MB. The second copy of mfc80d.pdb was in c:\windows\symbols\dll. This file was 13.5MB, significantly larger and probably the version of the PDB file that I needed. The problem was making VC2005 use it. That's the hard part.

First go to Tools/Options, open Debugging, and select Symbols. Add the directory "C:\Windows\Symbols\dll" to the beginning of the list. This is different than how VC6 was configured, which required that you not include the "dll" part of the path.

Click OK. Now close Visual Studio. Go to Task Manager and make sure its gone. Now kill the process MSPDBSRV.EXE, if it's running.

Finally, the critical operation to making all of the other steps work is to remove the smaller copy of mfc80d.pdb from the cache. To do this, delete the directory c:\cache\mfc80d.i386.pdb. The path may be slightly different if you are on a 64-bit system.

At this point you can restart VC2005, load your project, and run your application. In the Modules window, you should see that the symbol file for mfc80d.dll is being loaded from C:\Windows\Symbols\dll.

Part 2

[This applies to Visual Studio 2010, 2012, and 2013.]

You may need to tell Visual Studio where the source code is installed. Right-click on a C++ solution and select Properties. Under Common Properties, select Debug Source Files. For Visual Studio 2013, it looks like this on my machine:

Problem solved.

Visual Studio Debugger Hangs

Ever since I first tried to run our software under Visual Studio 2003, I've had trouble with the debugger and the debuggee freezing solid. This problem cropped up again today in Visual Studio 2005. I knew the problem was something to do with the symbol server, but that was all I knew. Fortunately, the developer of VirtualDub has pinpointed the causes. You can find his discussion at:

http://www.virtualdub.org/blog/pivot/entry.php?id=118

The easiest solution to the second problem in the article is to "seed the cache." Visual Studio shows in its status bar (at the bottom) which DLL is being processed. In your application startup code you need to call LoadLibrary for each DLL that causes a hang, as shown in that status bar. This will force Visual Studio to download the symbols over the Internet while the process is in a "safe" state.

In my case, on a Windows Vista RTM system, I placed the following code in InitInstance() before any network calls were made:

LoadLibrary("SENSAPI.DLL");
LoadLibrary("SCHANNEL.DLL");
LoadLibrary("CREDSSP.DLL");
LoadLibrary("RASAPI32.DLL");

Note that these aren't all of the DLLs, just the DLLs at the top of the dependency hierarchy. The dependent DLLs will be loaded automatically. It's not a bad idea to just leave this code in your debug build because the problem will recur as soon as any of those DLLs are updated in a service pack. Make sure you add the related "FreeLibrary" calls if you do this.