Troubleshooting Specific ASP.net 2.0 (64-bit) Exceptions with ADPlus.vbs

The following troubleshooting steps are recommended asp.net web apps running under the 2.0 framework, 64-bit framework—not the 2.0, 32 bit framework. For similar instructions to use for use with other framework versions, or if you’re unsure which version of the .net framework your web application is using, please click here.

SYMPTOM:

When browser to a an asp.net web application the browser relays a complaint from .net about System.<SomeSpecificException>. Some examples include:

· System.InvalidCastException

· System.CannotUnloadAppDomain

· System.Threading.SynchronizationLock

· System.InvalidOperationException

TROUBLESHOOTING:

1. Install Debug Tools

Please install the Debugging tools for Windows onto the server that you’re seeing the exception on.

Install Debugging Tools for Windows 64-bit Versions
Download page for the latest 64-bit packages.

By default the tools are installed to: C:\Program Files\Debugging Tools for Windows 64-bit. This page assumes the default location. However, you can install the tools to the directory of your choice—it doesn’t have to be the system partition. If for instance you have limited freespace (less than 2 gigs) on the C:\ drive or if you’re just not comfortable installing anything to the system partition, you can choose an advanced install and have them installed, for example, to the D:\ drive. In fact, if you’re uncomfortable with or restricted from installing tools onto the server at all, you can install the tools onto a workstation and then copy the “Debugging tools for Windows” folder to the server.

If you happen to be running another dump tool, you may want to uninstall or disable it. For instance, if you are running DebugDiag, please deactivate or delete any crash rules that might be active.

2. Create and Modify a Config File

Copy the following text into notepad and save it as “Exception.cfg.” Note the comments in yellow and consider whether any edits need to be made. At least one change is mandatory; others are optional. Regarding the mandatory change, you’re going to want to change the “System.YourException” to “System.[WhateverYourExceptionYourTroubleshootingForIs]. Regarding the optional changes, you may, for instance, prefer the output directory (where .dmp files are going to be created) to be whichever volume on your server has the most free diskspace available. Or you may want to specify the exact pid number of a w3wp.exe which you know to be suffering from the exception rather than create dump files of all w3wp.exe’s when the exception is seen in any of the w3wp.exe’s. Those can be changed in the config file.

<RunMode> CRASH </RunMode>

<OutputDir> c:\dumps </OutputDir> <!—Note, you may want to change this to a different directory if you have limited freespace on the C drive-->

<Option> Quiet </Option>

<ProcessName> w3wp.exe </ProcessName> <!—Or you can change this to <PID> and specify a specific PID number. You can use iisapp.vbs to find the pid number-->

</Settings>

<cmd>!threads</cmd>

</PreCommands>

<Code>AllExceptions</Code>

</Config>

<CustomActions1>!soe System.Exception 1;!soe System.YourException 2;.if(@$t1==1 | @$t2==1) {.echo 'System.Exception or System.YourException occurred';.dump /ma c:\dumps\myExceptionDump.dmp;gn} .else {.echo 'some other exception occurred';!clrstack;gn}</CustomActions1>

</Config> <!—YOU WILL WANT TO CHANGE System.YourException to System.[whatever your exception is]-->

</Exceptions>

</ADPlus>

3. Move the Config File

Copy the Exception.cfg file to the “C:\Program Files\Debugging Tools for Windows 64-bit” folder of the server suffering from the exceptions.

4. Copy SOS.dll

Find and copy sos.dll from C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727 to “C:\Program Files\Debugging Tools for Windows 64-bit” folder.

Note: Be sure to not confuse the C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727 directory with the C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727 directory.

5. Attach Debuggers to W3WP.exe’s

The idea here is we want to use the adplus.vbs script to attach cdb.exe debuggers to the w3wp.exe’s which are seeing exceptions. To do so, open a command prompt and cd to C:\Program Files\Debugging Tools for Windows 64-bit. Enter the following command:

cscript.exe adplus.vbs –c exception.cfg

You should see one or more minimized adplus process windows appear in the task bar—you’ll get one for each w3wp.exe. You also might see in Task Manager (show all processes) that you have one new cdb.exe for each w3wp.exe. This is your sign that debuggers have been attached to all the w3wp.exe processes and they’re waiting for your specific managed exception to occur. When that exception is encountered, all that is in the memory spaces of those w3wp.exe processes will be written into .dmp files.

6. Wait for the Exception, Wait for Dumps

The writing may require anywhere from 20 seconds to 200 seconds depending on the size of the w3wp.exe’s. Be sure to not run IISRESET until all the dmp files have finished being written to or else the dmp files become worthless.

Do NOT log off of the server. You may CTRL+ALT+ DELETE > Lock Computer, but don’t log off. If the server is rebooted, iis is restarted, or you are logged off the server, you’ll need to re-run the command above to re-attach the debuggers to the IIS processes.

When the exception is seen again in the browser or the event logs, check to see if the minimized adplus windows have vanished. And see if new dumps were written to the output directory specified in the configuration file.

7. Zip and Upload Dumps

Please zip these files and upload them to your ASP.net support engineer for analysis. He or she should create a secure https workspace to upload to. If you need an asp.net support engineer, visit http://support.microsoft.com/ to begin the process.

If dumps were not triggered, examine the txt files found in the “C:\Program Files\Debugging Tools for Windows 64-bit\logs” folder to try to get an idea of what went wrong. Also provide them to your asp.net support engineer. Examine the config file to make sure the exception was not misspelled. Make sure the output directory had at least 1-2 gigs of freespace.