Troubleshooting IIS6.0 Data Access Problems

with Debugdiag, Netmon, and Perfmon

Overview:

These steps focus on the IIS servers and include:

1. Two hang dumps made with Debugdiag

2. One Perfmon capture (with IIS and ASP.net related objects)

3. One Netmon trace

Ideally we’ll get two set of debugdiag dumps in combination with a concurrent perfmon capture and netmon trace during a five minute period when the performance problem is occurring. If there is not time to get all three of these things, please focus on the debugdiag hang dumps.

INSTALL DEBUGDIAG v 1.1

Please install Debug Diagnostics v 1.1 onto at least one of the web servers.

Download link: https://www.microsoft.com/downloads/details.aspx?FamilyID=28bd5941-c458-46f1-b24d-f60151d875a3&DisplayLang=en

Accept all the defaults during installation unless you want to avoid having .dmp files created on the system partition.

If you have a front end web server (asp.net web application) that talks to a middle-tier application server (asp.net web services) you’ll want to install Debugdiag on at least one front end web server and its corresponding application server.

PERFMON

Set up Perfmon before the hang or slow performance begins.

Start Button > Run > Open: Perfmon [Enter]

Expand "Performance Logs and Alerts"

Right Click on "Counter Logs"

Choose "New Log Settings..."

Enter a descriptive name (such as “IIS-Perf”)

Click the "Add" button

Click the "All Counters" and "All Instances" radio buttons

Select the following from the "Performance Object" dropdown, being sure to "Add" each one as you select it:

Add every Object that begins with “.NET” (such as, .NET CLR Data, .NET CLR Exceptions, .NET CLR Interop, etc.)

Add every Object that begins with ASP.NET (such as ASP.NET, ASP.NET Applications, etc.)

Memory

Process

Processor

Thread

Web Service

Click "Close"

Click "OK"

Feel free to stop the capture until you’re ready to start performance testing.

Ensure that the perfmon capture is started when the performance testing begins.

Set Time Taken (IIS logging)

Please ensure that the “time taken” option is check-marked in the properties sheet of the website(s) in focus.

Install Netmon 3.2 from here on the IIS server(s)

If Netmon will not install please feel free to install Wireshark or Ethereal.

Start the load test and wait for the performance problem to become noticeable.

When the performance problem is noticeable,

1. Ensure that the perfmon capture has been started

2. Start the Netmon trace

a. Launch Network Monitor from All Programs menu

b. Allow it to load parsers (if this is the first time it is being run)

c. Click “Create New Capture Tab”

d. Click the > Start button

3. Manually Trigger two Hang Dumps with Debugdiag

a) Launch DebugDiag from the All Programs menu

b) Click Cancel if given the choice of making a Crash Rule or Hang Rule

c) Expand the Tools menu

d) Select “Create IIS/Com+ Hang Dump” and wait for dump creation to occur (may take 30 seconds or more depending on size of the w3wp.exe)

e) Wait for two minutes after the first set of hang dumps is finished

f) Trigger a second set of hang dumps with the “Create IIS/Com+ Hang Dump” again

After the second set of dumps has been made, please stop the netmon trace (by clicking the Stop button) and save the netmon capture.

Also please stop the perfmon capture.

Advanced data collection

After the load test is over, expand the Tools menu in debugdiag, choose advanced data collection, create full cabinet file.   

Creating the full cabinet file will compress the event logs, the most recent IIS log, the metabase, the httperr log, all .net config files, and all dmp files together into one convenient .cab file.

At some point in the wizard you will have the option to select one or more websites to include their IIS logfiles or not. Please highlight all websites that are available and ensure that 1 day’s worth of IIS logs is included.

After the compression process is finished, the easiest way to find the cabinet file is to click the icon of the manila folder in debugdiag.

Please collect and zip and send/upload/share out the netmon’s .cap file, the perfmon’s .blg file, and the debugdiag .cab file.