Test approach: restore database each time (????)

4 posts, 0 answers
  1. Ian
    Ian avatar
    3 posts
    Member since:
    Nov 2010

    Posted 10 Oct 2011 Link to this post

    Please help to guide us as we foray into the world of automation (we've been doing unit testing for years but not behavior testing).

    We're going to have the following rule: All tests must be independent tests (ie be capable of being run regardless of what tests have run before). This will allow us to use "Dynamic Test Lists" (which will allow us to create a list of all tests that relate to XYZ business requirements).

    The problem that I foresee is that each "test setup" will be quite complex and time consuming.

    Let me explain... In order for a complex business process to be tested, there are perhaps another 4 or 5 complex business processes that have to happen first (in order that the correct data gets in the database). This means that before the test can even start, there will be 4 or 5 long time consuming tests that have to execute EVERY time a test need to run:

    Setup:
    1. Create location hierarchy structure
    2. Import lots of users
    3. Import other related data
    4. Schedule events to happen
    5. Run a calculation process

    The problem is that this setup could take maybe 2 or 3 minutes to execute to completion. And only then can we start the actual test that we want to conduct.

    This cycle of "Setup" then "Test" is going to be repeated lots so I can see that the overall testing is going to be very slow (yes but faster than manual).

    The way we do this manually is that we break the "each test must be independent". So the tester manually runs the setup and then runs many tests using the data that is setup. They have to use their intelligence to ensure that the data left over from the previous tests does not affect the current test (or is taken into account when checking to see if the results are correct (and it is possible).

    I don't think we can break that rule (all must be independent) for automatic tests because we do need to be able to run tests in any order (using Dynamic Test Lists).

    Suggested Solution

    Instead of running several recorded tests to build up the required setup data, we get the setup script to login to a "Super Admin" area of the application. It can then restore the database to a known state and then run the test script.

    So the cycle will be:

    Test #1: Restore database, run test #1
    Test #2: Restore database, run test #2

    Question

    Does anyone see any maintenance issues with this approach?
    Are other pitfalls to watch out for?


    Thanks in advance,

    Ian.


  2. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 11 Oct 2011 Link to this post

    Hello Ian,

    Your proposed solution can work. Depending on the complexity of your application, an IIS Reset (or whever the equivalent is for your web server software) may be required as well. This insures that web application does not maintain any application or session states.

    Another common solution is to add a cleanup section to the test that restores the database to a known good state. In Test Studio define a Cleanup method and add your cleanup steps in there:

    public override void CleanUp()
            {
                // Perform any cleanup required, such as removing data from the database


                // Call the base cleanup method when you are done with your cleanup
                base.CleanUp();
            }


    Best wishes,
    Cody
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
  3. Ian
    Ian avatar
    3 posts
    Member since:
    Nov 2010

    Posted 11 Oct 2011 Link to this post

    RE: OnCleanUp() function...

    Is there an equivalent for initialise setup?

    I think we will have several database states that we'll start from (so it's better to restore the data before the test runs rather than after it runs)
  4. Cody
    Admin
    Cody avatar
    3360 posts

    Posted 11 Oct 2011 Link to this post

    Hello Ian,

    Yes there is:

    public override void OnBeforeTestStarted()
            {
                base.OnBeforeTestStarted();
            }


    Kind regards,
    Cody
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
Back to Top