NDepend Blog

Improve your .NET code quality with NDepend

Include IL Offset into Production Exception Stack Traces

February 2, 2021 3 minutes read

In a previous post The proper usages of Exceptions in C# I explained that it is important to get as much information as possible from production crash logs. One such key information is IL Offset for each method call in a stacktrace.


The Problem of Production Crash Naked Stack Trace

One recurrent problem with exception stack trace in .NET is that one needs to either release PDB files along with assemblies installed in production or setup a symbol server. Without PDB files, you get a naked stack trace: you lose the exact source file line from where the exception popups. One problem is that usually the PDBs files weight around twice the weight of the target assemblies. Hence, releasing PDBs is usually not an option because this consumes more resources (download size / install footprint / process memory at run-time). There is also the point about Intellectual Property protection:

  • not only PDB files contain extra info about your source that shouldn’t be public.
  • but also if your assemblies are obfuscated their original PDB become useless.

The exact source file line from where the exception popups is not just a convenient info, it is an essential info. The typical awkward situation is when a method throws a NullReferenceException, but contains several references that can potentially be null: in such case without the exact location that caused the exception you are blind. By knowing the exact source file line from where the exception popups, it would become easy to identify the faulty null reference. Notice also that the longer your methods (in terms of lines of code), the most likely you will fall into the awkward situation described.

Rich Production Stack Trace with IL Offset

Several years ago we developed some code to gather IL offsets in stacktrace. Now our production stacktraces look like:

Thanks to such stack trace containing IL offsets, it is straightforward to infer a source code line from an IL instruction. You just have to keep in mind that if the Nth IL instruction throws the exception, you’ll get the offset of the N-1th IL instruction logged. The L_00e3 offset in stack trace translates to L_00e4 in IL code.

On the screenshot below you can see that we obfuscate our code. Hence there is an extra step to resolve obfuscated identifiers to actual source file identifiers.

Below you will find our source code and corresponding tests. It can be compiled with .NET Standard. This code implements another useful facility: we remove all localization info from the stacktrace. This way we are able to build a hash code from the string of a stacktrace. Such hash code is useful to group similar crash logs, independently from the underlying OS localization settings. We then infer the number of similar crash logs. By logging other environment info we can distinguish between situations like:

  • Many users faces the same crash
  • A single user faces a crash multiple times

Thus we can prioritize problems to fix from production data. We are running this code for many years. It helped us countless times to fix non-trivial and rare bugs. With such relentless bug-fix approach the number of crash reported over time significantly decrease and is getting close to zero. Having a stable product has two essential benefits:

  • This avoids unnecessary friction with users.
  • We don’t lose time on bugs already fixed and have more time to implement new features.

Source Code with Tests