mandag den 4. april 2011

Optimization by Release build making nunit test fail?

A strange failure on the build server today: 


Given the test
        [Test]
        public void TestAcceptsDatetime()
        {
          Assert.That(validatorUnderTest.IsValid(DateTime.Now), Is.True);
        }
Which tests the method 

        public bool IsValid(object o)
        {
          return o is DateTime;
        }
I got a passing test when compiling in debug mode. But when submitted to the build server building and executing in release mode, the tests fails. 

The bastard defied manual debugging, since transforming the method being tested to produce a debug print also made the test pass. That is after making the following change
        public bool IsValid(object o)
        {
          Console.WriteLine("o is a " + o.GetType());
          return o is DateTime;
        }

made the test pass and indeed also produced the expected output "o is a System.DateTime".

To me the only reasonable explanation is that the compiler or runtime somehow optimizes away the type information on the object in the method without debug printing, since it sees that the object is not really used. The compiled version seemed to be behaving correctly. 

The test was made to pass by assigning the Datetime.Now to a value in the test. This seem to make the assumptions about the is operator correct, but it is a strange resolution. 

        [Test]
        public void TestAcceptsDatetime()
        {
          var testvalue = DateTime.Now;
 
          Assert.That(validatorUnderTest.IsValid(testvalue), Is.True);
        }