AssertAll doesn't work in TestCleanup

8 posts, 0 answers
  1. Dave
    Dave avatar
    31 posts
    Member since:
    Apr 2014

    Posted 17 Feb 2015 Link to this post

    My general pattern for testing has been to use TestInitialize to set up mocks that are used by all tests, and then in TestCleanup to call AssertAll on each mocked object. However I have just discovered that this doesn't (always?) work. In one of my tests I put an Arrange on one of the mocked objects that I knew would never happen, but the call to AssertAll did not fail the test. However, if I move the AssertAll into the body of the test itself it does indeed fail as it should.
    Is this a known bug, or am I doing something wrong? I have used this pattern for a lot of tests which it now seems may not be testing as rigorously as they should.
  2. Stefan
    Admin
    Stefan avatar
    198 posts

    Posted 19 Feb 2015 Link to this post

    Hello Dave,

    It is not a bug, it is by design. In TestCleanup you can only assert expectations arranged in TestInitialize. Expectations arranged in a test method can only be asserted in that test method.

    Even though it's by design, right now I can't say for sure that it actually makes sense. I will give it some thought, but in the mean time you should move your asserts into the test methods.

    Regards,
    Stefan
    Telerik
     

    Check out the Telerik Platform - the only platform that combines a rich set of UI tools with powerful cloud services to develop web, hybrid and native mobile apps.

     
  3. DevCraft R3 2016 release webinar banner
  4. Dave
    Dave avatar
    31 posts
    Member since:
    Apr 2014

    Posted 19 Feb 2015 in reply to Stefan Link to this post

    It's also very dangerous because it is completely non-intuitive. It would be completely natural for the test writer (as I did) to factor all the asserts into the TestCleanup function, and unless he/she actually noticed (as I did) that an expectation had been arranged that never actually happened, he/she would never know that the tests were incorrect. I now need to go back through the unit tests for several products, as well as the one I am currently working on, to refactor these cleanup methods. I think it is something that at the very least needs flagging IN BIG LETTERS in the documentation.
  5. Dave
    Dave avatar
    31 posts
    Member since:
    Apr 2014

    Posted 19 Feb 2015 in reply to Stefan Link to this post

    I am now hopelessly confused. I have written a small test project - you can infer the contents of Class1 and Class 2, the tests look like this:
    [TestClass]
    public class Class1Tests
    {
        Class2 _mockClass2;
        [TestInitialize]
        public void TestInitialise()
        {
            _mockClass2 = Mock.Create<Class2>(Behavior.Strict);
        }
        [TestCleanup]
        public void TestCleanup()
        {
            _mockClass2.AssertAll();
        }
        [TestMethod]
        public void ShouldFailButDoesnt()
        {
            // Arrange
            _mockClass2.Arrange(x => x.DoesSomething()).Returns(false);
     
            // No act
     
            // Assert by TestCleanup, arrangement has not happened.
        }
    }

    Intuitively the test should pass, however according to what you have just explained I would expect it to fail - HOWEVER it does actually pass!! Can you explain what is going on?

    Dave
  6. Dave
    Dave avatar
    31 posts
    Member since:
    Apr 2014

    Posted 19 Feb 2015 in reply to Dave Link to this post

    Oh dear, you see how confused I'm getting. The last sentence should read:
    "Intuitively the test should fail, however according to what you have just explained I would expect it to pass- HOWEVER it does actually fail!! Can you explain what is going on?"
  7. Stefan
    Admin
    Stefan avatar
    198 posts

    Posted 24 Feb 2015 Link to this post

    Hello Dave,

    Ah, you've found a bug in JustMock. Apparently Mock.AssertAll(o) is not the same as o.AssertAll(), which is a bug. If you replace _mockClass2.Arrange with Mock.Arrange(() => _mockClass2... and _mockClass2.AssertAll(); with Mock.AssertAll(_mockClass2), the behavior that I explained in the previous post will manifest.

    I have logged the bug in our bug tracker. I have also logged a bug that the TestCleanup method doesn't retain the mocking context of the test method preceding it. After these bugs are fixed and the fix is made available in one of our internal builds, you will be able to write asserts in the TestCleanup method that work correctly for expectations arranged in the test methods.

    As a token of gratitude for your feedback I have given you some Telerik points.

    Regards,
    Stefan
    Telerik
     

    Check out the Telerik Platform - the only platform that combines a rich set of UI tools with powerful cloud services to develop web, hybrid and native mobile apps.

     
  8. Dave
    Dave avatar
    31 posts
    Member since:
    Apr 2014

    Posted 24 Feb 2015 in reply to Stefan Link to this post

    Thanks Stefan (I'm not quite sure what I do with Telerik points, but I'm sure they'll come in handy :-)),
    Will I be, or can I arrange to be, notified when the relevant fixes are released?
    Dave
  9. Kaloyan
    Admin
    Kaloyan avatar
    872 posts

    Posted 26 Feb 2015 Link to this post

    Hi Dave,

    We will do our best to let you know when the fixes of the above mentioned bugs are present.

    Further, you can check this page inside your Telerik account for more information about the Telerik points. I hope it helps.

    Regards,
    Kaloyan
    Telerik
     

    Check out the Telerik Platform - the only platform that combines a rich set of UI tools with powerful cloud services to develop web, hybrid and native mobile apps.

     
Back to Top
DevCraft R3 2016 release webinar banner