IntelliTrace Tips and Tricks: IntelliTrace Everywhere

Series Links:

• Part 1: The Basics (this post) – also guest posted on MSDevDiv SA Blog
• Part 2: IntelliTrace Everywhere – also guest posted on MSDevDiv SA Blog
• Part 3: Enable Custom IntelliTrace Events with a Right-Click

In my previous post I showed you how to enable IntelliTrace for debugging – F5 IntelliTrace. That’s all well and good, but what about getting IntelliTrace logs from your test environments? Or from production?

Here’s where you can use IntelliTrace:

• .NET 2.0 and above managed code (note: the collector requires .NET 3.5, so you’ll need that on the target server even if you app is in .NET 2.0)
• Enable the IntelliTrace diagnostic adapter in Test Manager to collect logs during test runs
• IIS: Use PowerShell to attach to an application pool and collect logs
• Desktop Apps: Use IntelliTraceSC.exe to launch an app and collect logs
• Windows Services: This one is tough, but possible. Involves some registry tweaking. Read more here.

Here’s where you can’t collect IntelliTrace:

• Silverlight applications
• Windows Phone applications
• .NET 1 applications
• Native code applications

The IntelliTrace Standalone Collector

You can get this from the Visual Studio installation folder, or you can download it here. I recommend downloading it, since this will be the latest and greatest collector available. The page has a link to instructions about how to extract the cab file. Once you’ve expanded the cab, you’ll have the PowerShell module as well as the IntelliTraceSC.exe for collecting application data.

Symbols

Before we look at how to collect logs, let’s talk about symbols. In order to open up code from the iTrace logs, you’re going to need to supply symbols. Ever seen a pdb file when you compile your apps? The pdbs map source code to compiled code. But of course no self-respecting developer ever deploys pdbs, right? So if you’re not deploying your pdbs (and you shouldn’t be) then where do you put them? You get the build to publish them to a shared folder, or Symbol Server. (If you don’t use Team Build, you can simply keep your pdbs somewhere – when you open an iTrace file, you can provide the location of your pdbs).

Here’s an image showing where in the DefaultTemplate you can set the symbols location:

One more thing – you’ll need to set this location in the Debugging options of VS in order for it to look there for the symbols. In VS, go to the Tools->Options dialog. Then expand the Debugging->Symbols section. There are some buttons in the top right of the dialog – click the “New Location” icon (between the yellow warning icon and the ‘x’ button) and type in the same directory that you used in the Build Process settings (above).

You’ll also need to open the “General” tab under Debugging and set the following checkboxes:

Now you’re ready to open the logs – let’s see how you can collect them.

Collect IntelliTrace from Web Applications

If you’re developing and deploying web applications, you get a lot of love from IntelliTrace. Here are the steps you’ll need to follow to start logging:

1. Download the IntelliTrace collector and expand it.
2. Find the name (and identity) of the application pool that your web app is running under.
3. Create a log folder. Make sure the app pool identity has write access to this folder.
4. Open a PowerShell prompt. Go to the IntelliTrace folder. Run “Import-Module Microsoft.VisualStudio.IntellITrace.PowerShell.dll”
5. To list the commands, type “Get-Command *IntelliTrace*”. This will list the 5 IntelliTrace cmdlets.
6. To start logging, type

Start-IntelliTraceCollection –ApplicationPool AppPoolName –CollectionPlan plan.xml –OutputPath logPath

where

• AppPoolName is the name of the app pool your application is running under
• plan.xml is the collection plan (more on this in the next paragraph)
• logPath is the path you want the log files dropped into

The plan.xml is the settings file for what IntelliTrace events you want to collect (and if you’re running in Events Only or Events and Call Information mode). You’ll see 2 xml files in the IntelliTrace folder that ship with IntelliTrace – the lightweight collection_plan.ASP.NET.default.xml and the diagnostic collection_plan.ASP.NET.trace.xml. I recommend starting with the default plan (Events Only) and if you get stuck then dial it up to the trace plan (Events and Call Information). In the next post, I’ll show you how to customize the collection to get fine-grained control over the events.

Be aware that when you run Start-IntelliTraceCollection, IntelliTrace will attach itself to the app pool, but part of that will require a recycle.

Don’t leave this on too long – especially if you’re using the trace plan. One the collector is running, you can use the following commands when you want to grab the log and open it:

• Stop-IntelliTraceCollection – which stops the collector entirely
• Checkpoint-IntelliTraceCollection – which unlocks the log file and starts logging to a new log file

Use checkpoint when you want to open the log and leave the collector running (when the collector is running the file is locked by the collection process, so you won’t be able to open it).

Collect IntelliTrace from Desktop Applications

Collecting IntelliTrace from Desktop apps gets some love – not as much as the IIS scenario. Instead of launching your application directly, go to the IntelliTrace collector folder and run the following command:

IntelliTraceSC.exe help launch

to see the help on how to launch. Here’s the basic launch command:

IntelliTraceSC.exe launch /cp:plan.xml /f:pathToLogFile application

where

• plan.xml is the collection plan – use the same ones as the web collection command – don’t worry that it’s called ASP.NET – it’ll work for most default scenarios
• pathToLogFile is the full path and filename of the log file
• application is the path to the application you want to collect the log from

As soon as you exit your application, the logging completes and you’ll have your log file.

Happy logging!

IntelliTrace Tips and Tricks: The Basics

Series Links:

• Part 1: The Basics (this post) – also guest posted on MSDevDiv SA Blog
• Part 2: IntelliTrace Everywhere – also guest posted on MSDevDiv SA Blog
• Part 3: Enable Custom IntelliTrace Events with a Right-Click

This brief series of blog posts will show you how to get the most out of IntelliTrace – a historical debugger that allows you to record (and replay) program execution.

Using IntelliTrace

You can use IntelliTrace in the following scenarios:

1. During debugging in VS using F5
2. In Test Manager (enabled as a Data Diagnostic Adapter)
3. Anywhere (using the standalone collector)

Understanding IntelliTrace “Modes”

Out of the box, IntelliTrace has 2 broad modes – “Events Only” and “Events and Call Information”. Events Only is much more lightweight, and allows you to record “events” that occur when your program is running. These “events” are “interesting occurrences” – like ASP.NET calls or ADO.NET calls – or exceptions. The product team tried to record events which would help you understand how your code works so that you can understand long “cause and effect” chains.

Events and Call Information turns the dial up a lot – it collects not only all of the events from the Events Only mode, but also method calls (including in and out arguments). There are some limits placed on what data is collected – otherwise the logs would become even more enormous than they are using the default settings.

F5 IntelliTrace

Use this when you’re coding. It means you can debug and move back and forwards in your debug session without having to stop your application and add breakpoints. To turn it on, go to Tools->Options and go to IntelliTrace settings. Here you’ll see the 2 modes:

Once you’ve turned it on (in this case I’ve got the Events and Call Information mode on) you can start debugging. Click around your app, and then click the “Break All” link in VS in the IntelliTrace window:

(Here I am debugging my Calculator application – click “Break All” to go to the IntelliTrace log).

When you look at the log, you’ll initially see all the events that occurred. In this case, button clicks are “Gesture” events, so that’s what I see:

Let’s say that I remember something weird happened when I clicked “=” when multiplying 2 numbers – well, I don’t have to restart the application, I can simply click on the “=” gesture after the “*” button click (in the above log, I’ve clicked 3 x 6 =). So I can click on that event and “rewind” to that point.

What I can now do is click “Calls View” to start walking through the log from that point.

Double-click on the call to btnEqual_Click to “zoom” to that point of the execution. You’ll see the IntelliTrace glyphs in the gutter of the source window:

If you mouse-over the icons, you’ll get a tooltip. The functions (in the order they appear) are:

• return to calling method
• step back
• step into
• step over
• return to Live Debugging

(Just a handy tip: If you press the bottom button “return to Live Debugging” remember that this now puts you at the current debugger point – you’ll need to press F5 again if you want to start running the application again.)

If you advance using F11, the current pointer will advance. Pressing twice from the previous image gets me to the point where the program enters the switch statement:

Notice that I don’t need to guess which case statement was selected – the log records exactly what the program did. I’ve also pinned the mouse-overs for val1 and val2 – you can see their values. Not all values are collected, but since these are primitives and are being passed into (or out of) a method, IntelliTrace dutifully collects their values.

I’ll press F11 again to step into the Multiply() method, and then F11 again to advance one more event:

That puts me onto the closing curly brace of the Multiply method. If I look in the Autos window, I’ll see that the method in parameters (val1 and val2) were 3 and 6 respectively, and that the return value was 9.

Voila – with 3 or 4 clicks we found the bug in our program. And we didn’t even have to set a breakpoint!

Filtering Events

As you use IntelliTrace, you’ll start seeing large amount of events – to make the logs easier to navigate, you can use the category, thread and search box at the top of the events window to filter the events.

For example, expanding the dropdown with “All Categories” in it I can filter just exceptions:

Search For This Line…

Another useful tip is the “Search For this line” or “Search For This Method”. You suspect that a method was hit sometime during your debug session – but the log is quite long. No problem – right click a line (or method) and select “Search For This Line / Method in IntelliTrace”.

Once you do, you’ll see the search results in a bar at the top of the current code window – press the arrows to go to the previous or next instance of that line or method in the log file:

Here I’ll click on the arrow icon directly after the work “Multiply” to go to the first call to this method in the log, and I can start “debugging” from there.

In the next post, I’ll show you how to run IntelliTrace anywhere and everywhere.

Happy debugging!

Automated Builds–Why They’re Absolutely Essential (Part 2)

In my previous post I wrote about why you should be doing automated builds in general terms. In this post I’ll show you how TFS’s automated build engine gives you consistency and quality in your build processes. There are other build engines, but if you’re using TFS for source control (and/or test management and/or lab management and/or work item tracking) then Team Build makes the most sense as a build engine since it ties so many other parts of the ALM spectrum together.

TFS Team Build uses Workflow Foundation as the engine underneath the build. When you create a Team Project you get a few workflow XAML files out-the-box. For this post I’ll primarily discuss features of the DefaultTemplate11.1.xaml (the default build template).

Environment

When you configure a Build Agent, you install it on a build server. Ideally this is some Virtual Machine that is “clean” – the only things installed on the machine are the things that you need to compile (and test) your code. No rogue libraries or experimental settings – just a clean, controlled environment.

Installing the build agent is a snap – mount the TFS install media and install TFS. Then run the Build Configuration wizard and connect to a build controller (which can reside on the build machine if you want).

Labelling Sources During a Build

The build agent checks out the latest version of source control when it starts the build. As it does so, by default it labels the code that it checks out with the build name. This means that you can get the exact point-in-time code that the build used to compile, test and package. To see the labels, open the Source Control Explorer, find the folder that your build workspace is configured to download (in the Sources section of the build workflow), right-click and select “View History”. Then click on the Labels tab.

In the above picture you can see the build reports on the left, and the labels for the root folder of the build workspace on the right.

If you don’t want the build to label the sources on each build, then go to the Process tab of your build definition, expand the Advanced parameters and set “Label Sources” to false.

Perform Code Analysis During a Build

Most of us developers know we should be doing code analysis, but few teams that I work with actually do it. Most of the time it comes down to the fact that it’s hard to monitor. However, if you include code analysis as part of your build, you’ll easily be able to track Code Analysis over time.

If you have particular projects that you care about and don’t want to run Code Analysis on all projects, then you can configure that in the Build. The default build template sets “Perform Code Analysis” to “AsConfigured” which means if a project is configured to do code analysis on build, then it does so.

Of course you can set the Code Analysis to “Never” or “Always” too.

And as easy as that you now have Code Analysis as part of your build process:

Layer Validation

If you’ve got Visual Studio Ultimate, you’ll be able to draw Layer Diagrams. These diagrams allow you to visualize (and validate) layering within your architecture. Team Build can validate layering when building – all you have to do is right click your modelling project (that contains your layer diagrams) in the Solution Explorer, select Properties and set the “Validate Architecture” property to true.

As long as this project is part of the solution(s) being built, you’ll get layer validation as part of your build.

Oh dear – someone broke our layering!

Symbols

You should never deploy pdb files (symbol files) to production environments. But there are times when you’ll need the symbols files – for example remote debugging, for IntelliTrace or for Manual Test Coverage (see below). Team Build effortlessly publishes your symbols to a network share and indexes them for you, so you never have to think about them or hunt for them again.

Configuring Symbols and indexing on a build – the build creates a folder structure, so just supply the root folder and the build takes care of the rest.

Unit Testing and Code Coverage

Having a automated build without unit tests is like brushing your teeth without toothpaste. Once you’ve got a build in place, add unit tests and code coverage. This will increase the quality and consistency of your releases exponentially.

So let’s assume you have unit tests. You can easily configure Team Build to run the tests and perform code coverage. Set the automated test settings (you can have multiple) appropriately. By default the discovery filter is **\*test*.dll (which is any dll with the word “test” in it in any subdirectory). Click on “Edit” to enable Code Coverage and you’re done. I’ve even configured a build engine to run QUnit js files to test JavaScript in my web projects! Of course you can add category filters too if you want to filter which tests the build should be running.

Besides being a metric for each individual build, the pass/fail rates and coverage percentages go into the TFS Warehouse so that you can report off them and trend them.

If you’re not using the MSTest framework and you’re using nUnit or XUnit or some other framework and you have a corresponding Test Adapter in VS for running your unit tests within VS, then make sure you install the same Test Adapter on your build machine to enable it to run those tests during the build.

This build output shows a failed test.

That’s better – a 100% pass rate. Looks like the coverage is a little on the low side though…

Code Coverage for Manual Tests

At present this only works for Web Applications running in IIS. Get your testers to run test cases out of Microsoft Test Manager (MTM) against your test servers, and then enable the “Code Coverage” diagnostic adapter. You’ll have to tell it where to find the symbols files (which you hopefully configured on your build anyway) and you’re good to go.

Setting the Code Coverage diagnostic adapter (and the path to the symbols) in the Test Settings section of a Lab Environment.

The great thing about Team Build is that the manual code coverage is fed back onto the build report as testers execute their manual tests. Each time a manual test run is completed, the build report is updated.

This build report has been updated to show an additional test run (the manual test run) and the coverage has been merged into the total coverage (so it’s showing total unit test plus manual test coverage).

Build Reports – or what the heck is in this build?

Get your developers into the habit of associating checkins with work items. By default, the build lists all associated checkins between “good builds”. (The last good build is the last build that was successful – no compiler errors or test failures). If those checkins are associated with work items, the work items get associated with the builds too. That means that you can look at the build report and quickly answer the question, “What work is included in this build?”. This works for “direct” associations, such as when a developer checks in code against a Bug, but also “indirect” – when a developer checks in against a Task, the Tasks parent Product Backlog Item (or User Story) is also associated with the build.

Here we can see that Bug 82 was fixed in this build. We also see that Task 84 of PBI 83 is in this build.

Unfortunately this won’t work out-the-box for merges. If you queue a build that has only merges as changesets, the only changesets you’ll see will be the merges themselves. Never fear though – I created a custom build task that pulls in the merged changesets and work items into the build report. You can get it here.

Found In and Integrated In – Tracking Bugs Effectively

If you’ve got a build, and your testers specify that build number as the build they are testing, then any bug logged during testing has it’s “Found In” field automatically set. When you fix the bug and check it in, the Integrated In field is set so you know which build the bug was fixed in.

The System Tab of the default Bug work item: the “Found In” field gets populated automatically when logging a bug from MTM (where the build under test is specified) and the “Integrated In” field gets populated automatically when you resolve the bug during checkin.

Test Impact Analysis

Let’s say you have 100 manual tests. You run them all successfully. The developer then changes some code. Which tests should you run again? Ideally, all of them – but you may not have time to run all of them. So Team Build narrows down the list by doing test impact analysis. When you enable this diagnostic adapter in MTM, TFS builds a map of test vs code – it tracks which code is hit for each test the tester is executing. Then on the next build, for each passed test case, TFS does a lookup to see if any tests hit the code that changed since the last build. Each test that is “potentially impacted” is flagged during the build so that you can test is again to make sure the changes didn’t break the code. Of course TFS assumed you’ll re-run failed tests, so this only works against passed test cases.

Two tests were impacted by changes to the code – clicking on the “code changes” link opens details about what methods changed.

Build-Deploy-Test Workflow

I won’t go into any details on this workflow, but you get a LabDefault.xaml template out-the-box when you create a Team Project. This build doesn’t compile code – it takes the output of another build (TFS or even another engine), allows you to specify scripts that automate deployment to a Lab Environment (that you’ve set up using MTM’s Lab Manager) and even run automated tests, which could include manual test cases that you’ve converted to Coded UI Tests.

Metrics

I’ve mentioned that the build data go into the TFS warehouse – so you can see test results, coverage and code churn over time. Then you can slice-and-dice and create dashboards and reports.

The out-of-the-box Build Summary report.

The out-of-the-box Build Success Over Time report.

Some of the build-specific measures available when you pivot against the TFS Cube from Excel.

Summary

There are other build engines that you can use (such as Hudson or TeamCity or Jenkins). Where they cannot compete with TFS is in the rich integration you get into work items, source control, lab management, testing and reporting. And you get most of it for free – out-the-box. In short, if you want to take a quantum leap in consistency and quality, you need to get building! The small investment up-front will be well worth it in the long run. And you’ll be able to sleep at night…

Happy building!