Monday, May 17, 2010

Using MsgWaitForMultipleObjects in MFC

One of the problems that I didn't solve in my Multithreading book was how to use MsgWaitForMultipleObjectsEx with MFC. I have always felt guilty about this because it was an important problem, but I simply ran out of time to do the implemenation. Recently I finally had a need to solve the problem. MFC has come a long way since 1996 and it's clear that this problem was planned for. With MFC in Visual Studio 2008 and 2010, I was able to solve the problem in minutes instead of days.

In short, the solution is to replace MFC's call to GetMessage with your own call to MsgWaitForMultipleObjectsEx. This will allow you to dispatch messages, handle signaled objects, and dispatch Completion Routines. Here's the code, which goes in your MFC App object:

// virtual
BOOL CMyApp::PumpMessage()
{    HANDLE hEvent = ...;
    HANDLE handles[] = { hEvent };
    DWORD const res =
      ::MsgWaitForMultipleObjectsEx(
        _countof(handles),
        handles,
        INFINITE,
        QS_ALLINPUT,
        MWMO_INPUTAVAILABLE | MWMO_ALERTABLE);
    switch (res)
    {
        case WAIT_OBJECT_0 + 0:
            // the event object was signaled...
            return true;
        case WAIT_OBJECT_0 + _countof(handles):
            return __super::PumpMessage();
        case WAIT_IO_COMPLETION:
            break;
    }

    return TRUE;
}


There are several obscure points in this code worth mentioning. Getting any of these wrong will cause it to break:
  • The MWMO_INPUTAVAILABLE is required to solve race conditions with the message queue.
  • The MWMO_ALERTABLE is required in order for Completion Routines to be called.
  • WAIT_IO_COMPLETION does not require any action on your part. It just indicates that a completion routine was called.
  • The handle array is empty. You do NOT wait on the directory handle. Doing so will prevent your completion routine from being called.
  • MsgWaitForMultipleObjectsEx indicates that a message is available (as opposed to a signaled object) by returning WAIT_OBJECT_0 plus the handle count.
  • The call to __super::PumpMessage() is for MFC when this is running in your application object. Outside of MFC, you should replace it with your own message loop.
Note that this strategy breaks whenever a dialog box is displayed because dialog boxes use their own message loop.

Wednesday, May 12, 2010

Visual Studio 2010 Tab Key Not Working

Several days ago, my tab key stopped working when editing C++ code in Visual Studio 2010. Everything else worked fine, even Ctrl-Tab. But pressing Tab on the keyboard would not insert a tab into the file.  I tried disabling add-ins. No change. I reset the keyboard in Tools / Options / Environment / Keyboard. Still no change. I tried running devenv /safemode. That didn't work either.

Finally I looked at the other command line switches for devenv.exe, where I saw devenv /ResetSettings.  Finally, that fixed the problem.

Wednesday, April 28, 2010

Static Library Dependencies in Visual Studio 2010

I've been porting my Visual Studio 2008 C++ application to Visual Studio 2010. One rough spot I've hit is the change in how static library dependencies works. In VS2005 and VS2008, we completely gave up on automated static library dependencies because they only worked in the IDE, not from automated builds with VCBUILD. VS2010 was supposed to fix this problem. (Note that MSBUILD replaces VCBUILD for command line builds in VS2010.)

The problem manifested itself during Batch Build with errors indicating that Debug libraries were being linked into Release builds. For example, these errors happen if you are using STL:

Core.lib(HttpHelpers.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '2' doesn't match value '0' in Activation.obj

MSVCRTD.lib(MSVCR100D.dll) : error LNK2005: _strcspn already defined in libcmt.lib(strcspn.obj)


I've also seen these errors for vanilla MFC projects:

msvcrtd.lib(ti_inst.obj) : error LNK2005: "private: __thiscall type_info::type_info(class type_info const &)" (??0type_info@@AAE@ABV0@@Z) already defined in libcmt.lib(typinfo.obj)

msvcrtd.lib(ti_inst.obj) : error LNK2005: "private: class type_info & __thiscall type_info::operator=(class type_info const &)" (??4type_info@@AAEAAV0@ABV0@@Z) already defined in libcmt.lib(typinfo.obj)

Visual Studio 2010 has multiple ways of specifying dependencies.  The actual effect of each setting and settings' interdependence does not appear to be officially documented, although there is some information at http://msdnrss.thecoderblogs.com/2010/02/16/project-settings-changes-with-vs2010/ and at http://blogs.msdn.com/vcblog/archive/2010/03/02/visual-studio-2010-c-project-upgrade-guide.aspx.

The first way of specifying dependencies continues from previous versions of Visual Studio in Project | Project Dependencies. However, unlike earlier versions of Visual Studio, this only affects project build order, not dependency library linking. Also, this only affects project build order in the IDE. It does not affect project build order in MSBUILD. Therefore, you need to configure that page, but it's useless for command line builds. Changes on this page are reflected in the .sln file.

The real action is now on the Framework and References tab in the project properties. This tab used to be exclusively for .Net dependencies, but now it's been expanded for C++ dependencies. When I looked at this page for my project, it looked like this:



There are a couple of things to notice. First, the Configuration in the top left is grayed out. The Debug/Release settings do not apply to the Framework and References page. Second, you'll notice that the Full Path in the properties on the right points to the Debug build. In other words, the Debug library is always linked, even in the Release build. I've blogged about this problem in the past with .Net libraries, but this is pretty fundamental for C++ libraries.

I first thought that the solution was to set Reference Assembly Output to False. However, after reading the blog of one of the Microsoft Program Managers and following up with him, he said that the Reference Assembly Output option is only for managed libraries and does not do anything for native libraries. He said that the behavior I'm seeing is anamolous and that it shouldn't be happening.

[Update 7/18/2010] I have confirmed that this problem only happens with Batch Build. The problem doesn't happen if you explicitly choose a configuration, nor does the problem happen when running MSBUILD from the command line.

I have opened a bug report on Microsoft Connect. Please vote for it at:
https://connect.microsoft.com/VisualStudio/feedback/details/576146/link-library-dependencies-for-c-does-not-work-with-batch-build

[Update 7/18/2010] Microsoft has reproduced the problem, but has decided not to fix it. If you are reading this, PLEASE let them you that fixing this problem is important to you!

Thursday, April 22, 2010

Visual C++ 2010 Apps Don't Support Windows 2000

One of the rather important things missing in the list of Breaking Changes for Visual C++ in Visual Studio 2010 is that applications generated by VS2010 no longer support Windows 2000.  Usage of W2K has fallen under 0.5%, but that's still a very large number of users. This was pointed out by Martin Richter in the Community Content. However, the actual story is a little more complex.

I built a C++ application to see what functions are actually being called. When examined in DEPENDS under Windows 2000, these are the functions that are undefined:
  • DecodePointer
  • EncodePointer
  • ReleaseActCtx
  • CreateActCtxW
  • ActivateActCtx
  • DeactivateActCtx
The interesting thing is the DecodePointer and EncodePointer are documented as only being available in Windows XP Service Pack 2 or later. This means that the minimum system requirements for an application generated by Visual C++ 2010 is WinXP SP2, or Windows Server 2003 SP1 for server versions of Windows.

In addition, DUMPBIN shows that the required version of Windows has been bumped from 5.00 to 5.01, which is Windows XP.

What's really annoying is that none of these calls are required under Windows 2000. These calls appear to be an artificial limitation purely for the purposes of preventing end-of-life versions of Windows from working properly.

Tuesday, April 13, 2010

Targeting C++/CLR v2.0 with Visual Studio 2010

I just spent far too many hours recovering from trying to change the target .Net Framework version for a C++/CLR project in Visual Studio 20010 (final release.) I hope to save someone else the same frustation. If you don't need the features in .Net 4.0, there's a big advantage to targeting the old version since it is much more likely that your customers will have the old version installed.

If you go into the Framework and References page for a C++/CLR project in Visual Studio 2010, you will see that the Platform droplist is grayed out. There's nothing you can do to ungray it, so don't bother trying.

Underneath it says, "Targeted framework: .NETFramework, Version=4.0" There's no button you can click to change that. If you RTMF, the documentation says that, "The IDE does not support modifying the targeted framework, but you can change it manually." So that's what I tried to do. I went into the vcxproj file, changed the header to say ToolsVersion="2.0" and reloaded the project. Big mistake. The updated project file wouldn't load. I reverted back to the original file from source control and that wouldn't load either.

The first error I had to fix was:

The imported project "C:\Microsoft.Cpp.Default.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

This turned out to be relatively easy to solve. I went into the vcxproj file and did a "Replace All" changing UserRootDir to VCTargetsPath.

The second error was something about ClCompile and not finding AssemblyInfo.cpp. I'm still not entirely clear what caused this because now I can't reproduce the problem. In the end, I removed the project from the solution, closed the solution, closed Visual Studio, deleted the .sdf file, deleted all of the build directories (Debug, Release, etc.), deleted the ipch directory, restarted Visual Studio, added the project back into the solution, and it worked.

None of which got me any closer to solving the original problem.

I created a new C++/CLR project from scratch, which pointed me at the answer. The project file I was using had been created by the Upgrade Wizard in Visual Studio 2010 and it turns out that an important line was left out. Underneath these two lines:

<PropertyGroup Label="Globals">
   <ProjectGuid>{96FD1FEF-8A07-44E6-9CC1-
D4284218B186}</ProjectGuid>


there should be another line that says:

   <TargetFrameworkVersion>v4.0</TargetFrameworkVersion>

Change it from v4.0 to v2.0 and everything will magically switch to v2.0. The value v3.5 is also supported. There's some discussion of this at http://blogs.msdn.com/vcblog/archive/2010/03/02/visual-studio-2010-c-project-upgrade-guide.aspx halfway down under Known issues for conversion in VS2010.

Note that using v2.0 forces Visual Studio to use the Visual Studio 2008 compiler. Visual Studio 2010 can only compile v4.0 code.

Update 6/18/2010: To debug your code, you may have to explicitly tell Visual Studio 2010 that the .Net 2.0 CLR should be used. I ran into this when my EXE was native and my DLL was C++/CLR. Create a file named .exe.config and put the following information in the file:

<configuration>
    <startup>
        <supportedRuntime version="v2.0.50727"/>
    </startup>
</configuration>
 
More information can be found in this article on the Visual Studio Debugger Team Blog. Note that their sample did not work for me under Windows Vista until I removed the line that said:
 
<?xml version ="1.0"?>

Thursday, March 18, 2010

Managing multiple configurations using Virtual PC

We use Virtual PC extensively for testing in our lab. We have about 12 different configurations for each version of Windows we test. We've spent a lot of time learning how to create these configurations so that they work reliably and to create a reasonable balance between updates of individual configurations versus a global rebuild as the security updates and service packs pile up.

This blog entry describes how we create the VHDs and configure them in Virtual PC.

Layer Overview

Our VHD tree is built based on a set of defined layers. Layer 1 is comprised of single root VHD for each version of Windows, such as WinXP SP3. Layer 2 is built from differencing VHDs that use the root VHD as the parent. This means that the root VHD needs to be configured as far as possible, without requiring lower layer VHDs have to "undo" any configuration. Here is the VHD structure we use:

1. Root VHD - Contains base Windows install and common configurations
2. Configuration specific VHD - Includes apps, service packs, etc. that are different from the root. This is where configuration-dependent changes are done.
3. Working VHD - Includes temporary changes that are used over the course of several tests.
4. Undo file.

Layers are defined by how often they are changed, because when a layer changes, everything underneath it is invalidated. Layer 1 changes every six to eighteen months, usually when the number of security updates becomes overwhelming. Layer 2 changes every few months. Layer 3 is only used when a customized configuration needs to be tested for several days (or when you hit the wrong button and say "commit change.) Layer 4 changes on a minute to minute basis. It is discarded whenever a user selects "Discard changes" when a VPC is closed.

Roadblocks

Creating the VHD hierarchy is tricky. There are several problems we've seen that must be solved by making the changes in the correct order in the correct order. Some of these problems include:
  • Duplicate Ethernet MAC addresses.
  • Duplicate computer names.
  • Domain disconnects.
  • Accidental overwriting of a parent vhd file.
  • Windows Update automatically overwriting desired configurations.
The MAC address will be updated automatically by Virtual PC as long as you create a new .vmc file from the menu. In other words, use the New Virtual Machine Wizard, don't make a copy of your existing .vmc file. If you insist on copying your .vmc file, you can manually remove the MAC address from the XML file and it will automatically be recreated.
Many people are also concerned about "SID duplication" and want to use something like NewSID or SysPrep. Recent reports indicate that this is no longer a concern. Please see NewSID Retirement and the Machine SID Duplication Myth.

Create the Layer 1 Root VPC

The first task in creating the hierarchy is to create the Layer 1 VHD, which contains the OS install and will be common to all dependent virtual machines.
  1. Create the .vhd file using the Disk Wizard. Set to dynamic size, 20 to 32GB is usually a good size.
  2. Create the Virtual PC .vmc using the New Virtual Machine Wizard under the File menu. Make sure you give it enough RAM, the defaults are almost always too small.
  3. Install Windows. Install a version of Windows that includes the desired service pack. Set the system name to be something like RootWinXp so you know when you've forgotten to change it. Do not join a domain during installation.
  4. Log in as an Administrator and activate Windows, if necessary. Don't do this in a child virtual machine or you'll need an additional activation key for every child.
  5. Shut down Windows and make a backup of the VHD. Name it "Level 1 - Clean" This is a "clean machine" that's been activated. Use this VHD to create completely new configuration hierarchies.

Configure the Layer 1 Root VPC

  1. Boot the virtual machine.
  2. Install the Virtual PC Additions.
  3. Set the vpc to be part of workgroup, if it's not already. Do NOT join the root vhd to a domain! That must be done in a lower layer.
  4. Install desired Windows service pack, if any.
  5. For Windows XP, go to Windows Update in Internet Explorer and enable Microsoft Update.
  6. Install latest security updates through Windows Update. Pay attention to the following:
    • For Windows XP, go to Windows Update in Internet Explorer and check High Priority and Optional updates. The task bar "Update" icon does not show these updates.
    • Under Optional, I recommend you install Update for Root Certificates.
    • Also under Optional, install Microsoft .NET Framework, if needed.
    • Windows Update may automatically upgrade Internet Explorer if you don't do a "Custom" update and uncheck Internet Explorer. OTOH, IE8 for Windows XP is under Optional.
    • After each reboot, go back to Windows Update and check again. You'll probably need to run through Windows Update several times to get all updates.
  7. In Control Panel, set Automatic Updates to "Notify me", not to automatically download or to automatically install. Otherwise you end up fighting Windows Update every time you boot a configuration.
  8. Disable the requirement for Ctrl-Alt-Del. This can save a lot of mousing around when running the virtual PC in a window. For Windows XP, this can be found under Control Panel / User Accounts / Advanced (or use Group Policy.)
  9. Create any desired local users (as opposed to domain users.) Make sure you set them as Admin or Limited, appropriately. At a minimum, I recommend creating a "Limited" user for testing.
  10. Disable the Internet Connection Wizard if you have Windows XP virtual machines. In Group Policy, it's under User Configuration\Administrative Templates\Windows Components\Internet Explorer\Internet Settings\Advanced Settings\Internet Connection Wizard\
  11. Set firewall exceptions in Control Panel | Windows Firewall | Exceptions:
    • File and Printer Sharing must be enabled to allow access the system by name on the network.
    • Remote Desktop must be enabled to access the system remotely.
  12. If desired, enable Remote Desktop under My Computer | Properties | Remote. Note that Remote Desktop is not supported in any of the "Home" versions of Windows. Local Administrators will automatically have rights to access the system via Remote Desktop.
  13. If you use Remote Debugging in Visual Studio, start up MSVCMON.EXE and tell it unblock the firewall for your subnet. You will discover that this adds numerous exceptions to the firewall. MSVCMON.EXE can be found in:

    C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\Remote Debugger\x86\msvsmon.exe
  14. Modify options in My Computer | Tools | Folder Options | View, such as "Display the contents of system folders", "Show hidden files and folders", "Hide extensions", and Hide protected operating system files". This can also be done in the domain logon script with:

    reg add hkcu\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced /v Hidden /t REG_DWORD /d 1 /f

    reg add hkcu\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced /v HideFileExt /t REG_DWORD /d 0 /f

    reg add hkcu\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced /v WebViewBarricade /t REG_DWORD /d 1 /f
  15. Share any desired folders.
  16. Shut down the root vpc using the "Shutdown" from the Start menu.
  17. Commit all changes if you are using Undo disks.
  18. Set the VHD to be read-only.
  19. Create a backup of the file. Name it "Level 1 - Common".
  20. Optionally, go to the Virtual PC Console and remove this virtual machine so you don't accidently try to use it. If you don't delete it, make sure you enable Undo disks so you don't accidentally change it (Virtual PC will helpfully remove the read-only attribute.)

Create a Layer 2 VPC

  1. Use disk wizard to create a differencing hard disk to the root.
  2. Make a backup of this vhd file. Name it "Level 2 - Empty".
  3. Create the Virtual PC .vmc using the menus. Again, make sure you give it enough RAM. Turn off Undo for now.
  4. Boot the new virtual PC.
  5. Log in as an adminstrator.
  6. Change the system name.
  7. Join a domain, if desired. This can be done from the command line with the NetDom command, which is available in the Windows XP Support Tools. (This should all be on one line.)

    netdom join myvpc /domain:test.com
    /userd:Administrator
    /passwordd:<domain admin password>
  8. Give any desired domain users or groups permission to login via Remote Desktop.
  9. Log in as any domain users you plan to use frequently. This does the one-time configuration of the VPC for that user. Change Folder Options again for these users.
  10. Install any configuration-dependent software.
  11. Install updates and service packs for the software you just installed.
  12. Map network shares with a domain logon script. If a computer on a domain must map a drive on a non-domain computer, make sure the non-domain computer has an account with the same name and password as the domain computer. Alternatively, you use this command to save the username/password on the domain computer, then say "net use x: \\acme\share" in the domain logon script.

    net use x: \\acme\share
    /savecred /persistent:yes
  13. Check Windows Update one more time. On Vista and Windows 7, make sure you explicitly tell it to check again.
  14. Once everything is configured to your satisfaction, shut down Windows from the Start menu so it shuts down cleanly.
  15. Set the VHD to be read-only.
  16. Create a backup of the file. Name it "Level 2 - Configured".

Create a Layer 3 VPC

  1. Create a third vhd, differencing, whose parent is the vhd you just set to be read-only. I normally name it the same as the parent with the word "Child" appended. This is what I called the "Working VHD" earlier in this document. You will use this when you need to reuse a configuration a few times, then discard the configuration. Normally this vhd is empty.
  2. Change the settings in Virtual PC for your virtual machine to point to this working file.
  3. Enable Undo for this vhd.
  4. Backup this third VHD. Label it "Level 3 - Empty". You'll need the backup whenever you discard your current working configuration.
At this point you should have five backup files. The files labeled "Empty" in the below should be small, between 44KB and 106KB:
  • Level 1 - Clean root VHD, with a clean Windows install.
  • Level 1 - Configured root VHD, with a mostly configured Windows install.
  • Level 2 - Empty configuration VHD, which is an empty differencing disk whose parent is the Level 1 vhd.
  • Level 2 - Configured VHD, which is your fully configured test environment.
  • Level 3 - Empty VHD.

Create Remaining Configurations

  1. Copy your Level 2 - Empty vhd to a new file that will be the new configuration. Under Windows 7, you can create all of them at once using the DISKPART utility. For example: (DiskPart requires elevated privileges and so must be run from a command prompt started with Run As Administrator.)

    DISKPART < Parts.txt

    And Parts.txt is one or more lines similar to this:
    create vdisk file="c:\vpc\ParentDisks\WinXP Office 2007.vhd" parent="c:\vpc\ParentDisks\WinXP SP3 Root.vhd"
  2. Go to Step 3 under Configure a Layer 2 VPC and continue through Configure a Layer 3 VPCs.

Licensing

Licensing is also an issue. Strictly speaking, it appears that you need one Windows license per virtual PC. In practice, we have a dozen different variants of each root virtual PC, with only minor changes between each, such as whether it's a member of a workgroup or a member of a domain, or whether it's Service Pack 2 of WinXP or Service Pack 3. Since we rebuild these fairly routinely from scratch, it is hard to believe that we are supposed to own thirty different licenses and rebuy all of them every few months. Our solutions for Windows has been to use the volume licensing versions of Windows, which are available to Microsoft Certified Partners on MSDN Downloads.

Tuesday, March 9, 2010

Create separate references for Debug and Release DLLs in C++/CLR

I'm one of the crazy people who has worked with C++/CLR to make native C++ code coexist with .Net code. It runs really well, with a minimum of frustration.

Except for one thing. We rely on some components supplied by a third party. These components have both Debug and Release versions. You'll notice that the References are listed under "Common Properties" in the Properties window, so you can't create separate references for Debug and Release by simply switching to the appropriate property set.

This has been annoying me now for at least two years, and I finally found a solution, thanks Marco Beninca in this post:

http://social.msdn.microsoft.com/Forums/en-US/clr/thread/9087fb4f-149f-4d66-b33e-f2f280c65fa6

The solution is to use #using in your source code instead of defining your assembly references on the "Framework and References" page. Then you can go to the General page for C/C++ and set different directories for Debug and for Release.

The other advantage to this solution is that you don't have to reset all of your references when you get a new version of the components.

One disadvantage to this strategy is that the compiler no longer takes care of copying the appropriate DLLs to the appropriate directories. This can easily cause you to build with a different set of DLLs than you are running against, which will certainly cause a crash. My solution was to create a Pre-Link step that copies the DLLs to $(OutDir).