Sunday, August 26, 2018

Quicken Won't Download Transactions

I've been having a recurring problem for the past year with Quicken refusing to download recent instructions. I've spent *DAYS* trying to solve this problem. Tech support from Quicken and the bank were not helpful. In desperation, I built a completely new data file. This worked for a couple of months, and then the problem came back. I've seen the problem on both Quicken 2014 and Quicken 2017, so I don't think the problem is tied to the version of Quicken.

Cause

If you try to Update while you have transactions that you haven't accepted, then Quicken doesn't download anything, but DOES change the start date for "interesting" transactions. This prevents new transactions from downloading.

Solution

  1. Accept the transactions from ALL of the accounts for this financial institution.
    For example, if you have Checking, Savings, and Overdraft for a particular financial institution, then the accounts will transactions to reconcile will show a red flag in the list of Accounts on the left side of the screen. Go to each of those, click on "Downloaded Transactions" underneath the register, and click Accept All.
  2. Look at the date of the last downloaded transaction in your register and remember that date.
  3. On the Quicken menu, select Tools | Online Center.
  4. While holding down Ctrl and Alt on your keyboard, click Contact Info with your mouse in the Online Center window. You should see this window:

  5. In the Account droplist, select the account you need to update.
  6. Note the date next to the Reset DTSTART button. It most likely has today's date. In my case, I took this screenshot on 8/26/2018.
  7. Click Reset DTSTART and you'll see this window:

  8. Change the date next to Change to be the date you took note of in Step #2.
  9. (Note that you can't check the box to the left of Reset DTSTART, so it's not important.)
  10. Click OK.
  11. Click Refresh.
  12. Click Update/Send... in the top right.
  13. Make sure the relevant financial institution is selected.
  14. Click Update Now.
Good luck!



Thursday, January 4, 2018

Intel NUC Won't Install Windows 10 Fall Creators Update

I've spent many hours trying to install the Windows 10 Fall Creators Update (16299) on my Intel NUC D54250WYK. The symptom was that the update would appear to proceed correctly, reboot, get most of the way through processing the update, reboot again, and return to 15063. When I looked at the updates history, it would say Restart Required, but the restart would never succeed.

The other symptom I saw was that Windows would routinely hang trying to do Shutdown or Restart.

The final solution was to uninstall the following drivers:

  • Bluetooth
  • HD Audio
  • Trusted Platform

Once I did that, Shutdown and Restart started working reliably and the Fall Creators Update installed without incident.

Saturday, August 13, 2016

Installing a Code Signing Certificate, 2016

This is an update to my earlier article about the same subject. This process is much easier than it used to be.

This post was edited on August 1, 2019 to reflect minor changes in the process.

Install a certificate from Sectigo (was Comodo)

  1. Use IE11 for your browser for everything related to the purchase. Don't use Edge or Chrome.
  2. Use Sectigo (formerly Comodo) to buy your Code Signing Cert. They're still the cheapest provider, especially if you go through Tucows and buy a three year certificate.
  3. When filling out the information during the purchase process:
    • Make SURE you click the checkbox to allow the private cert to be exported, or you will be very unhappy.
    • You want a SHA2 certificate, which will do SHA1 and SHA256 signing.
  4. Your private key is generated when you initiate the order, but it is NOT stored in your Cert Store. Therefore, you must start and complete your order on the same machine with the same browser.
  5. After your identity is confirmed, you will receive an email from Sectigo with a subject like ORDER #12345678 - Your Code Signing Certificate is ready!
  6. Install the new certificate by clicking the link in the email. Again, use IE11. When it's done, you'll see something like this:


  7. At this point, your new certificate has been automatically added to your Cert Store.
  8. Verify the installation. Go to Control Panel / Internet Options / Content, click Certificates, select your new certificate, and click View in the bottom right. The certificate will probably be on the Personal page. Make sure you are looking at your new certificate and not the old. You can tell the difference by the expiration date.
  9. Make sure you have the private key. Again on the Certificate page, at the end of the information, right under the "Valid from" dates, you should see something that says "You have a private key that corresponds to this certificate." If this isn't there, you may not have checked the box during the signup process as described in Step 3 above. You will probably need to get the certificate reissued (this is free with Sectigo).
  10. Remove your old certificate. If you are renewing an existing certificate, then keeping the old certificates installed isn't usually useful, and having multiple certificates will break SIGNTOOL if signtool is searching the certificate store. Go to Control Panel / Internet Options / Content, click Certificates, select your old certificate, and click Remove. The old certificate will probably be on the Personal page.

Export the PFX file

A PFX file can be used by many third party utilities. One advantage is that PFX files can be created without a password, which is handy in automated builds if you are using SIGNTOOL. You can see the complete process with pretty pictures at Acmetek. Here's the abridged version:
  1. Go back to the Certificates Page on Internet Options.
  2. Select your new certificate
  3. Click Export...
  4. Click Next.
  5. Select "Yes, export the private key."
  6. Click Next.
  7. Select "Personal Information Exchange - PKCS #12 (.PFX)." (If this option is grayed out, then your private key was not imported).
  8. Check the box labeled "Include all certificates in the certification path if possible." THIS IS VERY IMPORTANT.
  9. You may want to read about the new Windows 10 option, Enable certificate privacy.
  10. Click Next.
  11. Check the Password box.
  12. Provide a password and the confirmation.
  13. Click Next.
  14. Enter a filename.
  15. Click Next.
  16. Click Finish.
  17. If you want to remove the password from the PFX file, use openssl as described in this post from the Wayback Machine. Quick summary:
    openssl pkcs12 -in mycert.pfx -out tmpmycert.pem -nodes
    openssl pkcs12 -export -out mycert2.pfx -in tmpmycert.pem

Sign your code!

If you need some hints on this, see my earlier post.

As of January 1, 2016, all Windows executables destined for Windows 7 or later must be signed with SHA256. If you still support Windows XP and Vista, then you must also dual sign with SHA1. For details, see this excellent article.

Sunday, May 22, 2016

Remote Debugging Visual C++ 2015

This describes how to configure your system to do remote debugging when you are using Visual Studio 2015. There is no installer for the debug DLLs, so you need to work around this problem.

Among other things, these instructions solve this error message:

The program can't start because ucrtbased.dll is missing from your computer.

These instructions take extra steps to handle the case where you are doing DLL builds instead of static linking MFC and CRT.

Note: These instructions should not be used for release versions of MFC and CRT. Those files can be installed with the files in C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\redist\1033.

1. Install the Remote Debug Tools


The version you install depends on the bitness of Windows. Most people will therefore want to install the 64-bit version. It will debug both 32-bit and 64-bit applications.
  • Click on Tools for Visual Studio 2015 on the left.
  • Click Remote Tools for Visual Studio 2015.
  • Choose the desired version. The section on top is for Visual Studio Update 2. The second section is for other versions of Visual Studio 2015.
  • Select the desired bitness
  • Click Download
  • Run the installer

2. Create a shortcut to the Remote Debug tool

Next create a link to the remote debugger on your desktop. Below is the command I used. All security is disabled because I'm the only user on my local network. Note the useful "/anyuser" parameter that is very helpful when your test VMs are configured with generic user accounts.

"C:\Program Files\Microsoft Visual Studio 14.0\Common7\IDE\Remote Debugger\x64\msvsmon.exe" /noauth /anyuser  /nosecuritywarn /timeout 10000

3. Copy the MFC and CRT DLL's to your application's Debug directory.

There is no installer for the debug DLLs, so you need to work around this problem.

Let's say your application is in:

C:\Projects\MyApp\Debug

Copy the CRT and MFC files as follows. Change MYARCH to be x64 if you are debugging a 64-bit application. This depends on the bitness of your application, not the bitness of Windows.

set MYDEBUG=C:\Projects\MyApp\Debug
set MYARCH=x86

copy "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\redist\debug_nonredist\%MYARCH%\Microsoft.VC140.DebugMFC" %MYDEBUG%

copy "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\redist\debug_nonredist\%MYARCH%\Microsoft.VC140.DebugCRT" %MYDEBUG%

4. Copy ucrtbased.dll into your application's Debug directory

The Universal CRT DLL is from the Windows SDK. It is not in the Visual Studio directory. This is a new file required as of Visual Studio 2015. Make sure you set MYDEBUG and MYARCH as shown in the last step.

copy "C:\Program Files (x86)\Windows Kits\10\bin\%MYARCH%\ucrt\ucrtbased.dll" %MYDEBUG%

Note that the discussion on MSDN about ucrtbased.dll being part of Windows 10 is wrong.


5. Define a network share

Create a network share to your Debug directory. It's a lot easier than copying files around. Let's assume you shared your entire C drive. You can share read-only.

On the VM, mount it as a standard drive. I like to use K:.

Finally, in Visual Studio, in the debugger properties, set the application location to use the network share as the path.

6. Run with remote debugging

Start the remote debugger msvcmon.exe on the VM using the shortcut you created in Step 2.

7. Run the debugger in Visual Studio



Sunday, May 17, 2015

Fix for "ICWrapper.dll failed to register" when installing QuickBooks

QuickBooks, how many ways do I hate you? Let me count the ways.

Every version of QuickBooks I've ever installed has had horrible install bugs that made installation difficult or impossible. For five years I couldn't print anything because QuickBooks didn't support printing in 64-bit Windows. The newest version, QuickBooks Pro 2015, has yet another problem.

The error is:

ICWrapper.dll failed to register


If you search the web, you will find many people who are unable to install QuickBooks because of this error. The pathetic part is that the solution is easy. Install the VC90 redistributable (the 32-bit version). It can be found at:

https://www.microsoft.com/en-us/download/details.aspx?id=29

Intuit, are you listening?

Here's a really really easy solution for Intuit that will save your customers endless frustration. Try installing QuickBooks on a Windows clean machine. That's a version of Windows that's just been installed with nothing else. No Microsoft Office. No Visual Studio. Especially no Chrome and no Firefox. Just the latest security updates.

Once you've installed QuickBooks (assuming that works), try testing things like printing.

This would trivially have found the problem on Windows 8.1.

Here's how to debug this particular problem. Perform the following steps:
  1. RegSvr32 ICWrapper.dll. You'll get an error about an error in the side-by-side configuration.
  2. Open the DLL in Visual Studio. The resource section should appear.
  3. Open RT_MANIFEST.
  4. Notice that it wants vc90, version 21022.
  5. Now look in %WINDIR%\WinSxS
  6. Notice that Windows 8.1 includes 30729.1, not 21022.8. Therefore it won't run, especially since your manifest doesn't allow later versions.



Monday, March 17, 2014

Intel NUC - Can't re-enter BIOS through F2

This blog entry is about how I solved the problem where I was no longer able to enter the BIOS using F2 on my Intel NUC D54250WYK.

The problem started when I upgraded my BIOS from 22 to 25 and enabled SecureBoot. Once I did that, the NUC would boot directly into Windows without showing the NUC boot logo nor the F2/F7 prompt.

This is not an uncommon problem. In my case, it took a combination of solutions to solve the problem.

First, the fact that the NUC logo wasn't showing was a clue that the NUC BIOS wasn't detecting the monitor properly. Apparently there's some sort of detection for the monitor to support 1024x768. This detection seems flaky. I tried the DisplayPort connector on two separate monitors without success.

The solution for this problem was to use an Apple mini-DisplayPort to DVI adapter to go from mini-DisplayPort on the NUC to DVI on the monitor. Once I connected the monitor in this manner, the boot screen showed up immediately. I got the idea from this article, which recommended mini-DisplayPort to HDMI.

The 1024x768 detection is particularly problematic if you are connecting the NUC to a TV or through a receiver. In these cases, you may need to connect to a "real" computer monitor.

The next problem was the SecureBoot prevented the BIOS from showing. To solve this it was necessary to enter Maintenance Mode, as described at https://communities.intel.com/message/190524.


Intel NUC - Three flashes and won't boot

This blog entry is about how I solved the problem where my Intel NUC D54250WYK wouldn't boot after I upgraded my BIOS from 22 to 25. (I should have paid to attention to the note that said, "Don't install this if you don't need it." However, I was installing Windows 8.1 and I wanted to make sure I had the latest fixes.)

The symptom was that the NUC would turn on, would flash three times, wait a few seconds, flash three times, and then turn off.

I was running the NUC with 16GB of DDR3 1866 (PC3 14900) Laptop Memory. This memory is faster than most users run, but recent versions of the BIOS are documented as supporting this speed of RAM.

I tried taking out one of the DIMMs, which had worked for me in the past. However, this didn't work this time. (If you try this, make sure you take out the correct DIMM. The NUC will operate on one DIMM, but this configuration is only supported for one of the two slots.)

Eventually I just left the NUC alone for half an hour in its reboot cycle. It eventually booted successfully and the problem did not come back.

Followup, Oct 2014: My system would crash every couple of weeks after I wrote this article. I finally ran Windows memory tests, which failed. I changed the NUC BIOS to slow down the memory and it worked fine after that.

Sunday, January 13, 2013

Outlook 2010: Fixing "An object could not be found"

I just wasted most of a day troubleshooting Outlook 2010 giving me the error message, "The operation failed. An object could not be found."  I'd like to thank Malik Shadid, who wrote a long and very useful blog post that pointed me to the tools below that helped find the solution.

Important: There are MANY things that can cause this error, most of which I'm not remotely qualified to diagnose. Therefore, I cannot answer questions about these procedures. This post describes my experiences, yours may be quite different.

The symptoms were as follows:
  • Outlook would give the "object could not be found" error when trying to download the Offline Address Book (OAB) from the Send/Receive | Send/Receive Groups | Download Address Book.
  • Attempts to create a new profile in Outlook would hang for at least several minutes.
  • Outlook 2007 and 2010 talking to Exchange 2007, all on a LAN.
  • The LAN is a small business LAN that had a new provider and a new firewall/router. 
  • Exchange was running under Hyper-V.
  • Configured for direct RPCs, not RPC-over-http.
  • The OAB did not have the latest changes from the GAL.
  • The OAB had worked before the new router.
  • The problem appeared on multiple computers.
  • Outlook would connect and download mail/contacts/calendar successfully.
My first thought was that my firewall settings were wrong. Indeed, when I looked at the machine running Hyper-V, it had reset itself to be on a "Public Network" and was blocking many ports that it shouldn't have been. So this was the first thing I fixed. Unfortunately, this only made a minimal improvement.

 

HOSTS file

My next step was to check for DNS resolution. My domain was "abc.local" with the server name being "jupiter.abc.local". For the purpose of testing, I added abc.local, jupiter.abc.local, and jupiter to the HOSTS file in C:\Windows\System32\Drivers\Etc. After doing this, I pinged the host by the various names to make sure it worked. (You do not need to reboot after updating the HOSTS file.) Unfortunately, this didn't fix it either.

 

PowerShell

At this point it's important to understand that the OAB is a rather strange beast that sometimes uses a different set of protocols (and ports) than MAPI. The OAB can get the connection information from the Autodiscover.xml file, so it's critical that the Autodiscover.xml file be accessible, even in a small test environment. The actual process used to track down the AutoDiscover file is quite complex, as Malik describes in his TechNet blog.

To help find the Autodiscover file, there's an applet in the Exchange PowerShell called test-outlookwebservices that will give you ther URL, as well as telling you whether the server responds to basic "are you alive" tests. It's documented on this  TechNet Page. Here's an example:


[PS] C:\Documents and Settings\JimB>test-outlookwebservices | format-table -wrap -autosize
 

  Id        Type Message
  --        ---- -------
1003 Information About to test AutoDiscover with the e-mail address Administrat
                 or@abc.local.
1007 Information Testing server jupiter.abc.local with the published name http
                 s://jupiter.abc.local/EWS/Exchange.asmx & .
1019 Information Found a valid AutoDiscover service connection point. The AutoD
                 iscover URL on this object is https://jupiter.abc.local/Autod
                 iscover/Autodiscover.xml.
1006 Information The Autodiscover service was contacted at https://jupiter.abc
                 .local/Autodiscover/Autodiscover.xml.
1016     Success [EXCH]-Successfully contacted the AS service at https://jupiter
                 .abc.local/EWS/Exchange.asmx. The elapsed time was 31 millis
                 econds.
1015     Success [EXCH]-Successfully contacted the OAB service at https://hamme
                 r.abc.local/EWS/Exchange.asmx. The elapsed time was 0 millis
                 econds.
1014     Success [EXCH]-Successfully contacted the UM service at https://jupiter
                 .abc.local/UnifiedMessaging/Service.asmx. The elapsed time w
                 as 31 milliseconds.
1006     Success The Autodiscover service was tested successfully.



600 Error

To eliminate any firewall or routing issues, I first tried opening the Autodiscover URL in the browser on the server itself.

https://jupiter.abc.local/Autodiscover/Autodiscover.xml

This yielded the an error 600 with the following content:

<?xml version="1.0" encoding="utf-8" ?>
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
<Response>
<Error Time="20:34:08.3587762" Id="147247660">
<ErrorCode>600</ErrorCode>
<Message>Invalid Request</Message>
<DebugData />
</Error>
</Response>
</Autodiscover>



A half hour of research indicated that the "600" error actually meant that it was working, but it wasn't authenticated. So the above result was actually good.

I tried the same URL from two client computers. It gave the same result in both cases. So all of this looked okay.

 

Connectivity Analyzer

Malik's blog also pointed me to a tool called the Microsoft Exchange Connectivity Analyzer (beta). (Requires .Net 4.5.) Get the tool from:

https://www.testexchangeconnectivity.com/

After starting the Analyzer, click the Client tab, download the file and install it.

The results of this finally showed me the problem, which was that autodiscover.abc.local wasn't resolved by DNS. I put that domain in the HOSTS file, tried the Connectivity Analyzer again and... it failed again. But it got further than last time.

I chased down the errors by clicking the little arrows under each error message in Analyzer. By drilling down, I discovered that it didn't like that my server certificate was for jupiter.abc.local instead of *.abc.local. Fortunately Outlook doesn't care about this, so Outlook works even if the Connectivity Analyzer does not.

The other domain names I mentioned at the beginning of this article also had to be in the HOSTS file, so that effort wasn't wasted.

Test E-mail AutoConfiguration

The next step was to go to a client and use an undocumented feature in Outlook 2007/2010 to test the Autodiscover resolution. Ctrl-right-click the Outlook notification icon on the right in Windows. Select Test E-mail AutoConfiguration. Enter your Exchange email address and your password. Make sure Use AutoDiscover is checked. Click Test.

If it's successful, you can switch to the XML tab and see the Autodiscover.xml file.

 

Wrapping Up

Finally I backtracked to find out why changing the router broke everything. It turns out that the old router was configured to forward all DNS requests to the Windows Server, which knew how to resolve *.abc.local to the proper server. Anything else was forwarded to Google's DNS. (Google's public DNS servers are blisteringly fast and are definitely worth using. Their IP addresses are 8.8.8.8 and 8.8.4.4)

Sunday, July 15, 2012

5 Tips for Curing Feature Creep

[I wrote this article in 2000 and it was published in the Visual C++ Developers Journal. This article has always been one of my favorites and it's no longer available on the web. So I'm reposting it here for posterity.]

Longtime software engineers know that feature requests are as inevitable as the daily tides. These requests are usually made by impassioned people who preface their requests with statements such as, “I’m sure this will be easy,” or “I know you can squeeze this in for me.”
    Most textbooks say you can combat feature creep by performing comprehensive up-front planning, creating detailed time estimates, and establishing a track record of on-time delivery to back up your claims when you try to control the feature set. This method works great, assuming you have a specification to design against, you have the time to do a detailed design, and you have developers good at estimating. In many companies I’ve worked with, particularly at the startup stage, these conditions don’t hold true. In this article I’ll talk about why people make feature requests, and I’ll give you five tips for successfully managing feature creep that can be used in the typical imperfect organization.

1. Avoid Saying No

This tip is easy to put into practice, and you’ll be surprised how far it gets you. Instead of saying “No” to people and putting them on the defensive, give them a positive answer instead. People like to get their way, and when something prevents that, it upsets them. If you’re a manager, consider how you’d feel if you asked a subordinate to do something reasonable and that person said “no” to you. You’d probably be irritated and have to spend the next few minutes curbing your temper to try and figure out why the employee is so difficult.
    Now, consider how others in the organization feel about development. They make what they consider a reasonable request (it may not be from development’s point of view, but they don’t know that), and development reacts by saying, “No. There isn’t time.” If you’re representing development and you make such a statement, you’ve put the other person on the defensive, left no room for negotiation, and set the stage for a hostile conflict.
    Instead, try this answer, “I think development can do that by either dropping features X and Y or by scaling down the feature you described.” With this type of answer, you’ve told them how you can put the feature in as well as given them the choice of what tradeoff to make, thereby giving them a psychological feeling of control. Both alternatives may be poor, but at least you’ve left room open for discussion. And you never said “no.”
    Don’t confuse providing alternatives with saying “yes.”  Say what you can do, even if it bears no resemblance to what you are being asked for. You may not be able to generate these alternatives off the top of your head, so don’t hesitate to ask for a couple days to study how the request will affect you . Let people know you need the additional time, then get back to them when you say you will.
    If you’re comfortable playing politics, you can sometimes conveniently “forget” about features you don’t think make sense. Tell the person you’ll get back to them about the feature, then don’t get back to them. If the request really was important, the person will remind you about it, and if it wasn’t, he or she will forget about it. However, you can only get away with this if you have a consistent track record of following up when you say you will.
    I can think of many meetings that would have been a lot more pleasant for me if I’d learned a long time ago to avoid saying “no. Saying “yes” makes you appear a more reasonable person and you guide the discussion instead of controlling it.

2. Make Decisions Based On Business Goals

Remember, engineers don’t think like people in other departments. This leads to serious communication gaps between departments. Often, the people requesting a feature speak “business,” the development representative speaks “techie,” and the two sides can’t make the other understand their positions. The CEO ends up arbitrating, but because the CEO also speaks business, development loses. As this scenario repeats, the development manager loses more and more credibility and eventually fails, by not delivering the massive number of requested features on time, even though the manager never agreed to the features or the schedule in the first place.
    The textbook way to solve this problem is to hire program and product managers whose job it is to understand both the business side and the technical side. Outside of textbooks and large organizations, these jobs often fall to the senior people in development who have little or no training in business.
    Resolving this problem requires you, the developer, to learn to speak business. If you don’t explain your position in a way the rest of the organization understands, it will be very difficult to break out of the vicious cycle described previously. The rest of the organization considers software development a black art, where worker bees can churn out any amount of code in no time at all, if they’re pushed hard enough. Engineers know it doesn’t work this way, but trying to explain this rarely works.
As a first example of this technique, let’s look at a situation where it might be applied inside development. It’s Monday morning and another developer wanders into your office. Plunking down onto the nearest flat surface, he says, “I’ve been thinking about our rendering code all weekend, and I think I can make it 50 percent faster with about three days of work.” So which of the following do you tell him?
·         That’s a great idea, faster is always better.
·         Absolutely not, there’s no time budgeted.
·         “If it’s really important to you, you can do it after hours.”
·         “That section of code is already working and we don’t want to destabilize it.”

All of these are typical engineering answers (or excuses, depending on your point of view). But the goal is to focus developers on corporate business priorities, so you need to try another solution. Appeal to the developer’s profit side. Developers may not always understand business, but they usually have a keen interest in their own financial stake.
    The first point is that you shouldn’t say no to the developer.  However, instead of offering alternatives, teach the developer the process of making the decision. How about an answer such as, “You’ve obviously put a lot of thought into your proposal, but I know you’ve shown a lot of enthusiasm about our company’s upcoming IPO. How would making that change get us closer to that objective?” You force the developer to analyze the reasons for the change, provide the metric for making similar decisions, and demonstrate how that metric personally impacts the developer.
    This technique won’t work with all developers, but it is important that it should work for architects and other developers who handle important parts of the project. If the technique doesn’t work, you might want to question whether that developer can make decisions in the company’s best interest.
    I’ll make two final comments  about deciding based on business goals. First, in some applications, such as air traffic control software, protecting human life is more important than the bottom line. Second, good business decisions depend on having a central strategy and sticking to it. For example, if your application is for XML text markup, then adding SOAP support probably doesn’t make sense. Even though SOAP is linked tightly with XML, SOAP is for interoperability, not for text markup.

3. Use a Prioritized Master Feature List

Detailed lists are a cornerstone of engineering. A master list of all feature requests can be an important tool in controlling feature creep. If you’re following a formal development methodology, you should have the beginning of this in a Marketing Requirements document. However, if you are a typical small company, your process may be somewhat more ad hoc.
    The secret to the list is to keep it prioritized.  It doesn’t need to be perfectly ordered, just categorizing each item as 1 to 5 is probably enough, with 1 the most important and 5 the least important. You can prioritize development resources by having developers work first on the priority 1 and 2 features.  Once these are finished, move on to  the quick and easy priority 3 and 4 features. Ultimately, development will probably ignore most priority 4 and 5 features.
    You’ll discover that everyone will try to put everything into the highest priority, but here is where speaking the business language pays off. As the design phase moves into the implementation phase, and feature requests start rolling in, you need to have a conversation with senior management that goes something like this:
    “Do you agree our goal is increased revenue?” Unless you’re a nonprofit company, senior management will almost certainly agree. (Note to techies: a business can have valid goals other than revenue, such as increased market share or increased profits, but a company typically focuses on just one goal, because multiple goals can conflict. For this discussion, I’ll use revenue as the goal.)
    You next question is, “Is it therefore reasonable that any changes to features be based on the feature’s impact on revenue?” Again, it will be difficult for management to disagree. Note the pattern of agreement we’re creating here.
    “Is it therefore reasonable that, given that development can’t do everything, we prioritize our features based on the feature’s ability to increase revenue?” This statement is the clincher. It is very difficult for management to disagree with the chain of logic you’ve created. Once you’ve had this discussion, you can use management’s own words to control it. The only other discussion point is how much revenue a feature should be able to bring in before designating it a Priority 1. This is a business decision, so you can stay out of it.
    To see the prioritized list in action, consider this situation: In the lunch-time status meeting, one of the sales people has come back from a show where she has gathered dozens of hot leads. One lead is for a large deal with a Fortune 50 company, and closing the deal would be an important milestone for the product. The only problem is that deal requires four new features, one a major change. Predictably, the CEO is excited and wants to know how engineering can “Help out” to bring in this customer. Which of the following should you respond with?
·         The developers are already working weekends.
·         Feature freeze was last week, and it’s too late for a change.
·         It was already agreed that those features would be in version two.
·         This has happened over and over again, and you won’t add yet another feature for a customer who probably won’t buy anything

    If you made any of the preceding statements, you’ve already succeeded in uniting the rest of the room against you. It will be almost impossible to avoid the feature request. Instead, speak their language by asking, “What’s the revenue potential?” The sales person will name some figure, for example, two million dollars. Now ask, “How did you arrive at that figure?”  If the sales person can’t offer a convincing argument, everyone else in the room will do the arguing for you.
    When the CEO agrees it’s worth moving forward on the feature, you say something like, “It sounds like the possibility of closing is only 25 percent. Therefore, I’ll prioritize this feature at $2M * 25 percent=$500,000.” Sales organizations commonly use this technique to project future revenue, so there’s no reason development can’t use it, too. A rule of thumb is that the possibility-of-closing percentage never goes above 75 percent unless a contract is actually signed. It is very difficult for others in the room to argue with this logic, and the final dollar value defines the feature’s priority.
    Notice you never mentioned development’s schedule or manpower. This is important. The moment you start talking about development’s schedule, you also have to start talking about estimates, and estimates are controversial and difficult to defend. If possible, avoid discussing estimates. It’s often better to accept the new feature and then quietly drop something (or several things) to the bottom of the priority list. If someone asks why the feature isn’t in a release, you can reply that the revenue potential for the feature wasn’t high enough, and again avoid discussing engineering schedule and estimates.
    Unfortunately, this technique may indicate you need to add a feature at the last minute, and the CEO doesn’t want to slip the release. The easiest way to handle this is to capitalize on everyone’s belief that engineering can’t adhere to a schedule. Instead of saying you don’t have enough time, say the estimates for the task are too uncertain, and you aren’t confident you can finish it on time. From a risk-management point of view, you believe it would be better to finish what you have, then do a small point release with that one feature. Business people understand risk, and you’re again speaking their language.

4. Understand the Root Requirement

There is a big difference between what someone wants and what someone needs when they make a feature request. This happens because of a lack of understanding about how to effectively link a real-world solution to a technical solution. Therefore, when a developer meets with a nontechnical person to talk about a feature, the developer must first understand the nontechnical requirement driving the feature request before determining the technical solution.
    Here’s an illustration. At the end of the day, the product manager stops you on your way to the car to talk about a magazine review of a competitor’s product and the new “must-have” items reviewers expect in that kind of product. The product manager believes the new product you’re working on needs these features for “parity,” and he wants to have a meeting with you tomorrow to talk about adding these features. Before the meeting the next morning, you could prepare notes about:
·         Why magazine reviews are useless.
·         How far the release date needs to slip to add those features.
·         How many people you’ll need to hire to implement those features.
·         What features the product manager recommends should be dropped to make room for new features.
    At least two of these responses represent ideas from this article, but none is the right answer. The solution is to go through each feature, determine the end-user requirement driving that feature, then see if you already have the feature, but no one knew about it, or if you have an alternative solution to get the same result.
    For example, say that one feature request from the magazine article is to “support FTP.” Many reasons might drive such a request. Because the FTP protocol is built into many libraries, adding FTP support might not be hard, but no work is still quicker than a little work. By going back and reading the magazine article yourself, you might discover the competitor’s product needs FTP support do file transfer through firewalls. However, maybe the application you’re building already supports HTTP file transfer. Because http goes through firewalls even more easily than ftp, it would not be necessary for you to add support for FTP. You can now explain to the product manager that you are already able to meet the spirit if not the letter of the requirement. Scratch one feature request.
    Another example might be a feature request to “support XML.” Such a broad, ambiguous description could require years of development work. You might assume that only a developer could properly interpret what “support XML” means. In this example, further discussion with the product manager reveals the requirement is simply to be able to say on the box that the product supports XML. You ask the product manager what she means by “support.” And the answer is, “I don’t know. I just need to be able to put those words on the box.”
    Now, you probably think this is stupid. However, XML is a big buzzword right now, and lots of people have heard of it. It’s likely someone comparing products on the shelves at CompUSA might buy a box with a big “Now supports XML” sticker, even if that person has no idea what XML is and has no plan to use it! So, your job is to technically decide what “supports XML” means, and to decide the minimum amount of work you can get away with. XML is evolving so fast that anything you do will be out of date before it ships, so one strategy might be to create something simple and see how customers react. This is dramatically different than a requirement to “successfully interoperate with three popular B2B, XML-based application servers” that a statement like “supports XML” might mean.

5. Plan Your Development Strategy

All too often, despite your best efforts, so many features get stuck into the product development can’t finish it in time. You can combat this by changing your development strategy. In most organizations, the strategy is to determine the feature set, pick a ship date, then, when the date is missed, keep slipping the date until the feature set is finally finished. I’ll call this the “feature-driven” strategy. Unless your organization is actually skilled at project estimating (and, judging by the number of slipped dates in the industry, few are) this strategy rarely works.
    An alternative to the feature-driven strategy is the date-driven strategy. In a date-driven strategy, whatever you have on a certain date, you ship. This strategy has logic behind it. In most cases, a date is chosen for a good reason. Perhaps it’s the end of the quarter, perhaps investors are reviewing the company, or perhaps a customer has a “drop dead” date for certain new features. In all these cases, developers need to recognize that hitting the date is often more important then completing all the features. Sales prefers to have something to sell, even if it isn’t perfect, rather than nothing at all. So, the solution is to change the development strategy so the company can ship something on any given date.
    The key change to the development process that this strategy requires is ensuring that features as they’re built. In too many organizations, development does months of feature development, then tests for bugs during a huge “bug bash” at the very end, which invariably takes much longer than anyone expected. A date-driven strategy calls for lots of little “bug bashes” along the way to keep the product as stable as possible. With this strategy, when the deadline rolls around, development can offer the organization two realistic alternatives—ship on time with a limited feature set, or slip the ship date and add more features. Remember, with two alternatives, you avoid saying no!
    Development often presents these alternatives, but frequently unsuccessfully, because the development process doesn’t allow for shipping on time with a limited feature set. If you leave bug fixing to the end, you can’t estimate how long they’ll take to fix, so you can’t ship on time, even if you leave many features are left out. By finding and fixing bugs as you go, you can actually offer these two alternatives and be able to deliver on either of them.
To follow this strategy, it is an absolute requirement to maintain the prioritized feature list to make sure the features you do deliver are the ones that will have the greatest impact on revenue or market share.
Today’s reality is that the computer industry is changing very rapidly and successful companies must be able to change direction on a dime. To stay competitive, a company must have the flexibility to add features as a product is developed. Ultimately, you don’t “win” if you successfully fend off all feature requests that appear after the design phase is over. In a product-based company, you win by doing whatever grows the business, enhances market share, and/or drive revenue. Computer science classes rarely teach this fundamental point, but it is important to remember when managing feature creep.
Being successful at doing this is much more a matter of working with people and understanding business needs than it is of maintaining detailed schedules and estimates to prove your points. At the end of those never-ending meetings, the most important result is that other people feel good about themselves and the decisions that were made. They don’t care about why you’re right—they care about why they’re right. If you can consistently make people feel good about themselves, you’ll always be a hero.

Monday, January 16, 2012

Windows 7 Forgets My Multiple Monitor Layout

For the last several months I've had trouble where every time I rebooted, my second monitor would become the default and my primary monitor would stop working. I think this started when I upgraded drivers, but I'm not positive. I've searched and searched for a solution to this, including reversing cables, adjusting the multiple monitor settings in Windows 7, and various obscure registry entries - but nothing worked.

The really strange thing was that Windows showed the login screen on my main monitor and the problem didn't happen until after I logged in, when the secondary monitor would suddenly become my primary. This symptom provided the key to the problem. After you log in, the nVidia Control Panel overrides your Windows display settings, so you must make sure that everything is set the way you want in the nVidia Control Pane.

To solve the problem, I did this:
  1. Opened nVidia Control Panel > Display > Set up multiple displays.
  2. Dragged the displays to the proper location.
  3. Checked all of the displays.
  4. Right-clicked on the desired primary monitor.
  5. Selected "Make this the Windows primary display."
  6. Clicked Apply in the bottom right.

Tuesday, November 29, 2011

Base64 for Unicode UTF16

Recently I needed to store a JPEG file in a Unicode UTF16 string. In UTF8 or ASCII, this is a trivial problem - use base64 encoding. But base64 encoding, which is 75% efficient for a single or multibyte character set, drops to 37.5% efficiency with double bytes - which means that the storage required for my JPEG data would almost triple. Since I had to store potentially thousands of images in memory simultaneously, this was a problem.

I ran across a great blog entry by Markus Scherer about a technique called Base16k. This seemed to be exactly what I needed, so I translated the JavaScript sample code into C++.

Note that this technique is designed particularly for UTF16. You will lose the efficiency if you write it to a file as UTF8. If you use UTF8, then don't use Base16k. Use base64 instead.

You can download the source code and sample project from:

Friday, October 28, 2011

Convert a Locale Name to LCID in C++

I've spent most of the afternoon today figuring out how to convert a locale name, such as "en-US", into a locale identifier (LCID). Locale names are also called RFC 1766 language codes, or ISO 639 language names. As it turns out, this is easy to with the function LocaleNameToLCID(), but this function does not exist in Windows XP.

Microsoft provides a redistributable library that adds this functionality for XP, but it's 2MB and yet another module to install. Since my entire installation is 4MB, this isn't an option for me. You can read about that library in the article Mapping Locale Data on Downlevel Systems. In that article, they discuss the function DownlevelLocaleNameToLCID().


For handy reference, Microsoft provides a list of Language Identifier Constants and Strings on MSDN.

If you then need to convert the LCID to a CODEPAGEID, you can use this call to retrieve the code page for a particular LCID:

GetLocaleInfo(lcid, LOCALE_IDEFAULTCODEPAGE, ...)

Additional information on this topic can be found on The Old New Thing blog, including a discussion of using the MLang library.

// This is a version of LocaleNameToLCID() that works on Windows XP. 
// On Windows Vista or later, it automatically delegates to the 
// built-in version of LocaleNameToLCID(). 
// This function is not case-sensitive. 

// Tested with Unicode builds only, not tested under ANSI builds, but should be pretty close. 
// Example value for szLocale is "en-US" 
LCID CompatibleLocaleNameToLCID(LPCSTR szLocale)
{
CString strValue(szLocale);
CStringW wstrValue(strValue);

static  bool  bDone;
LCID (WINAPI * pfnLocaleNameToLCID)(LPCWSTR, DWORD);
if  (!bDone)
{
bDone = true ;
*(FARPROC*)&pfnLocaleNameToLCID = ::GetProcAddress(GetModuleHandleA("Kernel32" ), "LocaleNameToLCID" );
}

if  (pfnLocaleNameToLCID)
return  LocaleNameToLCID(wstrValue, 0);
 

CComPtr<IMultiLanguage> iLang;
HRESULT hr = iLang.CoCreateInstance(CLSID_CMultiLanguage);
LCID lcid;
if  (SUCCEEDED(hr) && SUCCEEDED(iLang->GetLcidFromRfc1766(&lcid, (BSTR)wstrValue.GetString()))) {
return  lcid;
}
 

return LOCALE_USER_DEFAULT;
}
 


Tuesday, October 25, 2011

Reading XML from C++

Reading XML from a URL is easy with by MSXML::IXMLHttpRequest.

Sample code is available at:
http://msdn.microsoft.com/en-us/library/ms992615(v=exchg.65).aspx

However, there are some disadvantages to this interface, which does not give you access to the low level handles if you need to tweak anything. For example, some servers require you to authenticate with a certificate and IXMLHttpRequest provides no way to do this. This interface will also display errors to the user and prompt for required information, so it's not appropriate for an application that will run silently.

Therefore, Microsoft provides a second interface called IServerXMLHTTPRequesthttp://msdn.microsoft.com/en-us/library/windows/desktop/ms762278(v=VS.85).aspx. This interface inherits from IXMLHttpRequest, but it does not use WinInet and so is appropriate for use in servers. It also allows you to explicitly provide a certificate using SXH_OPTION_SELECT_CLIENT_SSL_CERT.

Tuesday, September 20, 2011

Google C++ Style Guide

I've authored at least a couple of C++ style guides for companies over the years. Therefore, it was very interesting today for me to see Google's C++ style guide:

http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml


One point I saw that was interesting is Exceptions, where it says, "We do not use C++ exceptions." I find this interesting considering that some popular libraries (notably STL) do use exceptions - whether you like it or not. If you don't have exception handling, nasty things happen when things go wrong. I discuss one such debugging nightmare in an earlier blog post:

http://qualapps.blogspot.com/2009/11/handling-exceptions-from-stl.html

Wednesday, January 19, 2011

Display Unicode in a Windows console window with printf/wprintf

Did you know that you can print Unicode to a console window under Windows? I didn't. Here are some blog entries that discuss how to do it. You need Visual Studio 2005 or later to display Unicode using the C run-time library. Thanks to Microsoft C++ MVPs Jochen Kalmbach and Igor Tandetnik for introducing me to these solutions:

http://blog.kalmbachnet.de/?postid=98

https://archives.miloush.net/michkap/archive/2010/10/07/10072032.html

https://archives.miloush.net/michkap/archive/2008/03/18/8306597.html

http://illegalargumentexception.blogspot.com/2009/04/i18n-unicode-at-windows-command-prompt.html

With the techniques in those articles, you can display both UTF-16 and UTF-8 Unicode characters in a console window.

There are some caveats:
  • The console window must be set to a TrueType font. By default it is set to a raster font, which does not support Unicode characters.
  • Not all languages are supported. Lucida Console, the most common console TrueType font, will display Russian characters, but not Arabic or Hebrew.
This article discusses the issues with Hebrew characters and provides some workarounds:
http://groups.google.com/group/microsoft.public.win32.programmer.international/browse_thread/thread/788eaf7082088b35

Friday, January 7, 2011

How to: Debug with Code Center Premium Source (In Detail)

This article is an extensive update to the MSDN article of the same name. I saw that article two months ago and it was only with the help of Microsoft C++ MVP John Czopowik that I was able to make Windows Source Debugging work under Visual Studio 2010. I hope this article saves other people from the same frustration.

These instructions are for developers who have a Windows source code license and access to Code Center Premium.

To prepare for debugging with Code Center Premium

  1. Close all copies of Visual Studio that are running (this is required or VS will overwrite your registry changes.)
  2. Connect your SmartCard reader and insert the card you obtained from the Shared Source Initiative.
  3. Windows Vista and Windows 7 will need to contact Windows Update to install a Gemalto driver. Make sure that you allow this to happen. (Windows XP: See the instructions that came with the card reader.)
  4. Make sure your smart card certificate is working by opening Internet Explorer and going to https://codepremium.msdn.microsoft.com/. You should see prompts as shown in Steps 3 to 6 under Sample Walkthrough, below. This must work properly before you proceed with these instructions. Do not use Firefox because Firefox does not support smart card authentication!
  5. Open the Registry Editor and go to: HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0\Debugger
  6. Set both SecureSourceLocalDirectory and SourceServerExtractToDirectory to the same directory. The directory does not have to already exist.
  7. Launch Visual Studio.
  8. On the Tools menu, click Options.
  9. In the Options dialog box, open the Debugging node and click General.
  10. Clear the Enable just my code (Managed only) check box.
  11. Select Enable Enable source server support.
  12. Select Print source server diagnostic messages to the Output window.
  13. Clear Require source files to exactly match the original version.
  14. Under the Debugging node, click Symbols.
  15. In the Symbol file (.pdb) locations box, clear the Microsoft Server Symbols check box and add the following location:

    https://codepremium.msdn.microsoft.com/symbols

    Move this location to be second in the list to ensure that these symbols are loaded first. (You can't move it to the top. Microsoft Symbol Servers is always first, but you should have unchecked it.) Visual Studio 2005 and 2008 users: In the screen shot below, note the third entry in this list, C:\Windows\symbols\dll. This directory is where the MFC and C run-time symbols are installed by Visual Studio. These symbols are not part of Code Center, so you need to include them if you want to debug into MFC or the C run-time library. Visual Studio 2010 users: You don't need the reference to C:\Windows\symbols\dll. Visual Studio always looks there, even if it's not in this list.
  16. In the Cache symbols in this directory box, enter a location such as C:\symbols where Code Center Premium can cache the symbols. This location should be DIFFERENT from the location used to store public symbols when Microsoft Symbol Servers is checked because symbols already downloaded from the Microsoft Symbol Servers will prevent symbols being downloaded from Code Center Premium.
  17. Click OK.
  18. Right-click the Solution under Solution Explorer and choose Properties:

  19. Under Common Properties, select Debug Source Files.
  20. Add a link for the desired operating system (complete list at the end of this article.) For example:

    https://codepremium.msdn.microsoft.com/source/Windows%207/RTM

    Note that spaces must be replaced with %20, and you cannot browse to this location in your browser.

  21. In the same window, make sure that the list under Do not look for these source files is empty.
  22. Click OK.
  23. Close and restart Visual Studio to ensure that settings are persisted.

Sample Code

Here is sample code that I used for testing. It's a native code console application that can be tested under x86 or x64:

#include "stdafx.h"

#include <windows.h>
#include <shlwapi.h>

#pragma comment(lib, "shlwapi.lib")

int _tmain(int argc, _TCHAR* argv[])
{
  // Can't step into this function. You get a "not indexed" error.
  //char buf[256]={'\0'};
  //::PathRemoveExtensionA(buf);

  // Set a breakpoint on this function, then try to Step Into it.
  GetConsoleMode(0,0);

  // Stepping into this function should work too, but not on XP.
  //UnregisterApplicationRestart();

  return 0;
}



Sample Walkthrough

  1. Set a breakpoint at GetConsoleMode in the sample code.
  2. Start debugging (F5).
  3. You should see the Confirm Certificate window. If not, your smart card isn't working.
  4. Click OK.
  5. You'll be asked to enter your PIN. You received this in your email confirmation that welcomed you to Code Center Premium.
  6. Enter your PIN and click OK.
  7. It may take a while to get started because the private symbols must be downloaded. Since they contain full debug information, they are much larger than the usual symbols.
  8. When Visual Studio stops at the breakpoint, look at your Modules window. Here's how it appears on my Windows 7 64-bit machine: (click it to see it full size.)

  9. Note that kernel32.dll shows "Symbols loaded", whereas KernelBase.dll shows "Symbols loaded (source information stripped). If kernel32.dll also shows "(source information stripped)", then you are using the wrong symbols. The most likely cause is if you didn't set a different cache directory for private symbols.
  10. Also note that the timestamp for both kernel32.dll and KernelBase.dll is 7/13/2009. This means that they are RTM DLLs for Windows 7. The timestamp for ntdll.dll is 3/23/2010, which means that it's been updated and you can't step into it. So it is possible to debug on a patched system, but it's hit or miss because many DLLs have been patched and so can't be stepped into.
  11. Now you will see a window asking to execute an untrusted command named SD.EXE:

  12. This command will be executed by the Source Server at Code Center Premium. It will NOT be executed on your computer. You do not need to have SD.EXE on your computer.
  13. Click Run.
  14. You should now be looking at Windows source code.
  15. If you look at the Output window, you should see messages such as the following: (some NDA information removed.)

    SRCSRV: sd.exe -p windowsdepot.sys-ntgroup.ntdev.microsoft.com:2016 print -o "C:\Users\JimB\AppData\Local\SOURCE~2\WIN_WINDOWS\win7_rtm\windows\yyy\yyy\yyy\yyy.c\1\yyy.c" -q //depot/win7_rtm/windows/Core/xxx/xxx/xxxxx.c#1

    SRCSRV: Source server cannot retrieve the source code for file 'd:\w7rtm\windows\zzz\zzz\zzz\zzz.c' in module 'C:\Windows\System32\kernel32.dll'. The system cannot find the file specified.
  16. Note that Source Server is actually showing an error. In spite of this, debugging still works properly.

Things That Don't Work

  • You can't debug into 32-bit code running under a 64-bit operating system. In other words, you can't debug into any DLL that lives in \Windows\SysWOW64. (Remote debugging into a 32-bit virtual machine running under a 64-bit operating system is okay.)
  • You can't debug any hot fixes, security fixes or operating system updates. You can ONLY debug the RTM release or a service pack. In practice, this usually means that you are remote debugging a virtual machine that was installed from the DVD and Windows Update has been turned off.
  • You can't debug when running Visual Studio under Remote Desktop because the smart card doesn't work when you are logged in under Remote Desktop. (LogMeIn and GotoMyPC are not tested.)
  • The debugger won't step into some Windows functions. This usually means that you are trying to step into a DLL that's been updated since the original release.
  • Some functions are in files that Code Center Premium can't find. One example is PathRemoveExtensionA in shlwapi.dll. For files that can't be found, you'll see a "not indexed" error in your Output window:

    SRCSRV: o:\w7rtm.obj.amd64fre\shell\shlwapi\srca\objfre\amd64\patha.cpp not indexed

Troubleshooting

Everything has to be exactly right for source code debugging to work properly. If things aren't working, here are some ideas:
  • Is your smartcard working?  (Repeat Step 4 under Prepare for debugging, above.)
  • Did Visual Studio overwrite your registry entries? This usually leads to the error "SRCSRV: ... The system cannot find the file specified." (Repeat Step 5.)
  • Did the file get added to the list of Do not look for these source files? (See Steps 18 through 22.)
  • Are you debugging a 32-bit app on 64-bit Windows? You can't debug Windows source in this case. Run the 32-bit code on a 32-bit version of Windows in a virtual machine and use Remote Debugging.
  • Are you trying to step into the Windows kernel? Kernel debugging is different from user mode debugging. You must use WinDbg for kernel mode debugging.
  • Is the source code missing on Code Center Premium? This would be the case if you are getting the error SRCSRV: ... not indexed. Please see the section below titled ANSI versus Unicode Functions.
  • Are you seeing a warning about an "Untrusted Command"? This is normal. Just click Run.
  • Can't step into a Windows API function? Make sure you have symbols for the DLL. Look in the Modules list and make sure that it does NOT say "Source information stripped." If it does, clear your symbol cache and run your app again.

Running Visual Studio in a Virtual Machine

You can install Visual Studio 2010 on your Windows 7 clean machine and run Visual Studio 2010 in the virtual machine so you don't have to use remote debugging. However, you must use a virtual machine that supports USB devices, such as VMware or Virtual PC in Windows 7. Virtual PC 2007 will not work (not even with SP1.)

In Virtual PC on Windows 7, go to the USB menu and select USB SmartCard Reader. You will be prompted to install the drivers in the virtual machine. The smart card will be DISCONNECTED from the host system, so you will not be able to use it in the host machine until the virtual machine releases it.

You do not need to do this if you are using Remote Debugging to run your application in the virtual machine.

ANSI versus Unicode Functions

Debugging into the "A" and "W" versions of functions frequently doesn't work. Here is the explanation from C++ MVP Jochen Kalmbach:

The problem with " PathRemoveExtensionA" seems to be a "build feature". They implemented the file only once for A and W and then you uses this file without "A, W"... In this case, the source file is called "path.c" and is referenced as "patha.c" (A) and "pathw.c" (W).

If you want to debug into this files, you need just to copy the source from the CCP-Website for the "TCHAR" version and save it wherever you want with the name "patha.c" and/or "pathw.c". Then if VS askes you for the file, you can just use this file. It will exactly match.

It seems that this file is generated during the build-process, because it seems that it is also compiled from the output directory...

Source Server Paths

(Thanks again to C++ MVP Jochen Kalmbach for providing this list.)

Here is the list of all currently available source server paths. Links are not clickable because these links are only for use in Visual Studio, not in your browser.

Win7 RTM: https://codepremium.msdn.microsoft.com/source/Windows%207/RTM


W2k8 RTM: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202008/RTM

W2k8 Hyper-V-RTM: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202008/Hyper-V%20RTM

W2k8 SP2: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202008/SP2

W2k8 R2: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202008/R2


W2k3 RTM: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202003/RTM

W2k3 R2: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202003/Windows%20Server%202003%20R2

W2k3 SP1: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202003/Windows%20Server%202003%20SP1%20RTM

W2k3 SP2 RC: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202003/Windows%20Server%202003%20SP2%20RC

W2k3 SP2: https://codepremium.msdn.microsoft.com/source/Windows%20Server%202003/Windows%20Server%202003%20SP2%20RTM


Vista Beta1: https://codepremium.msdn.microsoft.com/source/Windows%20Vista/Beta%201

Vista Beta2: https://codepremium.msdn.microsoft.com/source/Windows%20Vista/Beta%202

Vista RC1: https://codepremium.msdn.microsoft.com/source/Windows%20Vista/RC1

Vista RTM: https://codepremium.msdn.microsoft.com/source/Windows%20Vista/RTM

Vista SP1: https://codepremium.msdn.microsoft.com/source/Windows%20Vista/SP1

Vista SP2: https://codepremium.msdn.microsoft.com/source/Windows%20Vista/SP2


XP RTM: https://codepremium.msdn.microsoft.com/source/windows%20XP/RTM

XP SP1: https://codepremium.msdn.microsoft.com/source/windows%20XP/SP1

XP SP2: https://codepremium.msdn.microsoft.com/source/windows%20XP/SP2

XP SP3: https://codepremium.msdn.microsoft.com/source/windows%20XP/SP3

XP 64-bit:  Not available.


W2k Datacenter RTM: https://codepremium.msdn.microsoft.com/source/Windows%202000/Datacenter%20RTM

W2k SP3: https://codepremium.msdn.microsoft.com/source/Windows%202000/SP3

W2k SP4: https://codepremium.msdn.microsoft.com/source/Windows%202000/SP4

Friday, November 12, 2010

Extracting Icons with the Windows SDK

This is an article about the technical aspects of dealings with Windows Icons, including icon resources and HICON handles. But first, let me share with you how I feel about icons. I hate icons. I hate creating them. I hate working with them. I hate rendering them. I hate converting them. I hate dealing with transparency. About the only thing I like about icons is the artistry, and that's sadly lacking lately with the one and two letter icons now popularized with Adobe, Microsoft, Pandora, and Facebook. And let's not even talk about the new dark blue Visual Studio icon, that's pretty much invisible on the task bar.
Life got a little better about four years ago when I bought Axialis IconWorkshop. That's one handy piece of software, with free upgrades for life. If you are in the business of writing software, IconWorkshop will save you a lot of headaches.

But I digress.

Let me start out by pointing out the "official" documentation on icons. See the article on MSDN titled simply, Icons. Note the date of the article: 1995. Things have changed a lot in the last 15 years.

In the beginning, there were 16-color (4-bit) icons, with one color reserved for transparency, so there were really 15 colors available for use. As computers because more powerful, Microsoft introduced 256 color icons, then 24-bit truecolor icons, then 32-bit icons with 24-bit color and 8-bit transparency. Then, as a final coup-de-grace, Vista introduced 32-bit icons sized at 256x256 pixels. These took up so much space that they were stored in a completed different format (PNG). Ouch!
 

A good case study is the application icon for Outlook Express/Windows Mail in the file MSOERES.DLL, which has existed for most of the life of Microsoft Windows and has evolved as operating system support (and video card support) for icons has improved. 
  • Vista and Windows 7: eight different sizes (256 pixel, 64, 48, 40, 32, 24, 22, 16) at three bit depths (32-bit, 8-bit and 4-bit.)
  • Outlook XP SP3: three sizes (48 pixel, 32, 16) at at three bit depths (32-bit, 8-bit and 4-bit.)
  • Windows 2000 SP4: three sizes (48 pixel, 32, 16) at at three bit depths (32-bit, 8-bit and 4-bit.)
  • Windows 95: two sizes (32 pixel and 16 pixel) at one bit depth (4-bit.)

The only surprise for me in this list is that the Windows 2000 DLL included 32-bit icons, which would have transparency. However, that DLL was part of Service Pack 4 in 2002 and so was probably shared with Windows XP. I'm not aware that Windows 2000 was able to handle transparent icons. Also, you might wonder why I list "dead" operating systems like Windows 95. The reason is that there are some APIs that are stuck in the days of Win9x and never improved, notably the toolbar code in MFC. The common controls also behave differently depending on whether the Common Controls 6 is enabled, either explicitly or with a manifest.
Here's an important point about dealing with icons that took me quite a while to understand. First, there's an icon resource with a DLL or EXE that contains the icon in multiple sizes and multiple bit depths. On the other hand, an HICON contains only a single one of those formats. If you want a different size or a different bit depth, you need to load a new HICON.

The most basic icon-handling function is LoadIcon. It loads an icon from a particular HINSTANCE whose "size conforms to the SM_CXICON and SM_CYICON system metric values." So let's say you call this function and get an icon back. What bit depth? I have no idea. Presumably the same as your desktop, but even that's a hazy concept, because your bit depth can change on the fly, especially when you start a Remote Desktop session. And there's no such thing as a 32-bit desktop - there's no transparency on your desktop. So a 32-bit icon kind of/sort of matches a 24-bit and a 16-bit desktop. But if the icon matches the bit depth of your desktop, how would the icon keep its transparency bits? Again, I don't know.

The documentation for LoadIcon says that "This function has been superseded by the LoadImage function."  The LoadImage function takes the desired x and y size as parameters. That's helpful, now I can load those 256x256 bitmaps. But there's still no documentation on the bit depth.

What lead me down the path to this point is that I'm trying to extract some application icons for a web page that is being created for the local user. Those icons needs to be in PNG format to preserve transparency, so the goal was to load them from the original EXE as 32-bit icons, then save them as compressed PNG files.

Let me summarize my success for this project: Total Fail.

The first task was to get the icon for the particular file extension, such as ".doc".  This problem is easy to solve and well documented - use SHGetFileInfo, which returns the HICON in either small or large (however those happen to be defined.) I spent several hours trying to convert that icon to a PNG file. The popular strategy seemed to be using OleCreatePictureIndirect, as described by DanRollins in the comments in this article and by neilsolent in this article The problem I saw was that my PNG file was always 16 colors. In later comments of that article, neilsolent said that he was seeing a similar problem, which no one was able to provide an answer to. I wasn't able to solve that problem.

Note that the algorithm offered in the initial question in that article is utterly wrong. An HICON is not an HGLOBAL and you can't access it using GlobalLock.


My fallback strategy was to just write a .ico file instead of automatically converting to a .png file. With a .ico file, converting to .png is trivial using IconWorks.

This was nowhere near as simple as I'd hoped. Remember that an icon resource contains multiple sizes and multiple bitdepths? A .ico file is really a container for manager all of those formats, and there is no Windows API for writing that file. Some documentation is given in that 1995 article I mentioned earlier, but it's a non-trivial problem to solve. There is a sample from Microsoft called IconPro (run the executable, open the .chm help file, look for IconPro, and it will give you the option to extract file sample code.) IconPro hasn't been updated in fifteen years. It knows how to write .ico files, so I was hoping that I could make some minor modifications and feed it the HICON from SHGetFileInfo. No dice. IconPro only reads the original EXE or DLL file.

No problem. I'd just retrieve to pointer to the original DLL or EXE and pass that information to the appropriate routine in IconPro.

Unfortunately, it's not that easy. In fact, it's really ugly. There's no Windows API that will automatically tell you the filename and index to retrieve an icon. SHGetFileInfo only returns an index to the system image list (aka the Icon Cache.) There's no hint of information about where the icon came from. I was surprised because I thought that this problem was simple - just look up the DefaultIcon key in the registry. Turns out that this is just one way of specifying icons. To even begin to understand the other methods, you have to delve into the shell namespace and the IExtractIcon interface. I didn't care that much.

I spent several more hours searching for a method that could take an HICON and write it to a .ico file. I found numerous other people asking this same question, but I found no answers that worked and would create a .ico file with 32-bit color depth.


At this point I gave up on an automated solution. I'll use IconWorks to manually extract the resources that I need from the relevant EXE and DLL files.