Since the NDepend inception more than 15 years ago, we stuffed our code with calls to Debug.Assert(). This results today in more than 23.000 assertions calls.
Few developers realize that assertions in code are as important as assertions in tests. In both cases they prevent corrupted states at both smoke test-time and automatic test-time. Smoke testing means when you manually test your application. It is like smoke: your effort into testing manually is lost once done.
The scientific name for assertion in code is Design by Contract. Ok, contracts integrated in a language are more powerful than raw assertions, especially concerning inheritance, but the ideas are quite similar.
Our code was stuffed with 23.000 calls to Debug.Assert() and we wanted to improve that:
- To use more fluent and modern assertions like
Assert.IsNotNull(obj)instead ofDebug.Assert(obj != null) - To be able to get some
AssertionExceptionat test-time instead of a blocking assertions dialog - To eventually have a Release version that throws
AssertionException. As its name suggests calls toDebug.Assert()are removed in the Release version.
This task might appears daunting due to the large number of calls. However thanks to the feature Resharper > Find > Replace with Patterns feature, it took just half a day to complete. In this post I would like to highlight this powerful feature and our case-study. Hopefully you’ll adapt it to your own refactoring scenarios.
Defining our own assertion library
There are many open-sources assertions libraries. Here we’d like to stick with a basic NUnit-like approach. Hence it is not complicated to define a small library fully adapted to our needs like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 |
using System.Collections.Generic; using System.Linq; namespace System.Diagnostics { public static class MyAssert { internal enum Context {Debug, Test, Release} internal static Context s_Context = #if DEBUG Context.Debug; #elif RELEASE Context.Release; #endif private static void DoAssert(bool b, string str) { switch (s_Context) { case Context.Debug: Debug.Assert(b, str); return; default: if(!b) { throw new AssertionException(str); } return; } } sealed class AssertionException : Exception { internal AssertionException(string msg) : base(msg) {} } private static string GetTypeName<T>() { return typeof(T).FullName; } [Conditional("DEBUG")] public static void IsTrue(bool b) { DoAssert(b, "Expression must be true but it is not."); } [Conditional("DEBUG")] public static void IsFalse(bool b) { DoAssert(b, "Expression must be false but it is true."); } // // IsNotNull // [Conditional("DEBUG")] public static void IsNotNull<T>(T obj) where T : class { DoAssert(obj != null, $"{{{GetTypeName<T>()}}} reference with type must be not null but it is null."); } [Conditional("DEBUG")] public static void IsNotNullValue<T>(T? valNullable) where T : struct { DoAssert(valNullable != null, $"{{{GetTypeName<T>()}}} nullable value must be not null but it is null."); } // // String Equality // public static void StringAreBothNullOrEqual(string str1, string str2) { if (str1 == null && str2 == null) { return; } if (str1 != null && str2 == null) { StringsShouldBeEqualButOneReferenceIsNull(str1); return; } if (str2 != null && str1 == null) { StringsShouldBeEqualButOneReferenceIsNull(str2); return; } //Apply string value equality, and not reference equality DoAssert(str1 == str2, $@"Strings should be equal but they are not. str1=""{str1}"" str2=""{str2}"""); } private static void StringsShouldBeEqualButOneReferenceIsNull(string str) { DoAssert(false, $@"Both strings should be null or equal but one string reference is null and the other string is equal to ""{str}""."); } // // Reference Equality // [Conditional("DEBUG")] public static void ReferenceEquals<T>(T obj1, T obj2) where T : class { if (obj1 == null && obj2 == null) { return; } if ((obj1 == null) != (obj2 == null)) { DoAssert(false, $"{GetTypeName<T>()} references should be both null or equal, but one is null and the other one is not null."); return; } DoAssert(object.ReferenceEquals(obj1, obj2), $"{GetTypeName<T>()} references are both not null but they are not equal."); } // // String and collections IsNotNullOrEmpty // [Conditional("DEBUG")] public static void IsNotNullOrEmpty(string str) { DoAssert(str == null, $"String reference must be not null nor empty but it is null."); DoAssert(str.Length > 0, $"String reference must be not null nor empty but it references an empty string."); } [Conditional("DEBUG")] public static void IsNotNullOrEmpty<T>(IEnumerable<T> seq) { DoAssert(seq != null, $"IEnumerable<{GetTypeName<T>()}> reference must be not null nor empty but it is null."); DoAssert(seq.Any(), $"IEnumerable<{GetTypeName<T>()}> reference must be not null nor empty but it is empty."); } ... } } |
A few remarks:
- We name our assertion class MyAssert and it is declared in the
System.Diagnosticsnamespace. Since the Debug class is also declared in theSystem.Diagnosticsnamespace changing calls to Debug to MyAssert don’t provoke compilation errors. Also our tests already used a class named Assert so having our own MyAssert class avoid ambiguous calls. Both MyAssert andSystem.Diagnosticsidentifiers can be temporary. All the 23K calls to MyAssert can be changed through a single refactoring action after the major refactoring session. - We can choose between a Debug.Assert() style or a throw new AssertionException() style depending on the running context.
- [Conditional(“DEBUG”)] attributes strip off all assertions calls from the release version. Executing assertions consume performance so we prefer a super fast Release version and have our own strategies to decipher production crashes logged (stategies explained in Toward Bug Free Software: Lines of Defense).
- If needed, we can still comment all [Conditional(“DEBUG”)] and send release version with assertions to a user that experiences a bug.
- We use generic methods to be able to differentiate between assertions on references or assertion on values. As explained just below, this will let the compiler detect some refactoring issues for us.
Doing the Refactoring
First click Resharper >Search >Find with Pattern:
Below, for example, we choose the search pattern Debug.Assert($var1$ != null); and the replace pattern MyAssert.IsNotNull($var1$);. Note that you need to select the Replace button in the top-right corner of the dialog. Else you only get the search feature.
Then we can preview the 16.475 calls that will be refactored. We can eventually discard some calls thanks to the checkboxes.
A few remarks:
- Calls to Debug.Assert(val != null); where val is a nullable value (like int?) will be refactored to MyAssert.IsNotNull(val) instead of MyAssert.IsNotNullValue(val). The compiler will then emit some errors since the IsNotNull<T>() method forces T to be a reference type. Hopefully this concerns only a few dozens of call that can be handled manually once spotted by the compiler.
- We’re not using yet the awesome C#8 non-nullable reference feature. When we’ll refactor our code to support this feature Resharper > Replace with Patterns won’t be powerful enough. However Resharper can help also with migration to nullable reference types. But having all those 16K assertions will be precious to determine which reference can be null or not. Maybe writing our own refactoring program based on Roslyn guided by our numerous calls to MyAssert.IsNotNull() will be the best approach. I’ll blog on this when we’ll be there.
- With the pattern Debug.Assert($var1$ != null); Resharper is smart enough to also handle reversed cases like Debug.Assert(null != x);.
- Replacing thousands of expressions in thousands of files is an expensive operations. Resharper freezes Visual Studio during several minutes. I wish Resharper could propose a progress dialog with eventually a Cancel button. So far we didn’t experience any crash even when replacing 16K expressions.
Non-trivial Refactoring
Refactoring expressions like Debug.Assert(obj1 == obj2) is not trivial. Indeed if the type of both objects is a reference type that overloads the equality/inequality operators refactoring such expression to MyAssert.ReferenceEquals(obj1,obj2) won’t work. You might think that it is a rare case but actually System.String compare strings by value! Hopefully we can match only expressions typed as string:
Other reference types overriding the equality and inequality operators like…
|
1 2 |
public static bool operator ==(Vector v1, Vector v2) { public static bool operator !=(Vector v1, Vector v2) { |
…can be found with this NDepend code query:
|
1 2 3 4 5 |
from m in Methods where m.ParentType.IsClass && (m.NameLike(@"op_Equality\i") || m.NameLike(@"op_Inequality\i")) select m |
We don’t use equality operator overloading because it is wise to reserver this feature to types like BigInteger, Vector, Matrix… . Hopefully this query tells us to take care of BCL types like System.Version and System.Delegate. Of course now with C#9 records many classes will be compared by value. However we don’t record yet in our code.
As before, replacing to MyAssert.ReferenceEquals($var1$,$var2$) will emit some compiler errors when the underlying type is a value type. We don’t provide a MyAssert.ValueEquals(val1,val2) method. Here we refactor manually to MyAssert.IsTrue(val1 == val2) because value type comparison can be tricky. To illustrate this complexity, we allow comparing Debt value with TimeSpan value. This is because a technical-debt estimation is expressed in cost-to-fix which is measured by human-time.
From there we don’t need to detail other assertion refactoring because hopefully you now get a sense on how to use this feature on both trivial and non-trivial situations.
Conclusion
Thanks to the Resharper Structural Search and Replace feature, we’ve been able to refactor 23.000 calls within half a day. As explained there have been some edge cases and sometime we had to rely on compilation errors. But without this feature we would have had to write our own program (or analyzer) based on Roslyn.
In a 2020 post I explained the top 10 Visual Studio 2019 refactoring actions. I got many thankful feedback from developers that didn’t know that Visual Studio was catching up Resharper on the refactoring field. However this Structural Search & Replace feature could justify alone a Resharper license. This is the kind of feature for which one can regularly imagine use-cases. For example it also works on .aspx code and we used it a lot to refactor and improve substantially the technical SEO of our website.
The other part of the blog is about assertions in code. Once again I underline that this is a common misbelief that assertions should be reserved to test code only. Assertions in code is a major weapon to both express your intention and detect bugs early. Also in many situations it can simplify testing. For example when testing UI, typically it is a pain to find relevant states to assert in test code. But if your UI code is stuffed with assertions, the tests just need to pilot the UI and assertions in code are then checked at test-time. For example here is a video of our test piloting the NDepend Graph UI extracted from the post Case Study : Complex UI Testing. Many time this test broke some assertions in code and helped us detect bugs early.