Null pointer exception occurring when running multiple tests at once

[Robin]

Hi

I'm running two tests that on their own succeed without any issues, however when I run them both together I get a null pointer exception in the Telerik framework code.

The tests both use a custom build framework that uses the following structure:

MyWindow.MyPanel.MySubPanel.MyControl

and achieve this by exposing the sub elements of the application structure using properties with the following code:

public class MyPanel : CustomPanelWrapper
{
    private CustomPanelWrapper subPanel;

    public CustomPanelWrapper MySubPanel
    {
        get
        {
            return subPanel ?? (subPanel = this.Find.ByAutomationId<CustomPanelWrapper>("MySubPanel");
        }
    }
}

public class CustomPanelWrapper : FrameworkElement
{
    ....
}

The application is being started up in a separate process for each test and then closed afterwards using the following code:

public class MyTestRunner
{
   private TelerikWpfTestRunner telerikTestRunner;

   public void StartTest()
   {
        var appProcess = new Process();
        appProcess.StartInfo.FileName = <filePath>;
        appProcess.StartInfo.WorkingDirectory= <App Directory>;

        appProcess.Start();

        telerikTestRunner = new TelerikWpfTestRunner(appProcess, <logPath>);
    }

    public void StopTest()
    {
        telerikTestRunner.CleanUp();
        telerikTestRunner.Dispose();
    }
}

public class TelerikWpfTestRunner :  BaseWpfTest
{
   public TelerikWpfTestRunner (Process appProcess, string logPath)
   {
       Initialize(logPath);

       Application = Manager.ConnectToApplication(appProcess);
       Manager.ActiveApplication.MainWindow.RefreshVisualTrees();
       Manager.ActiveApplication.QuitOnDetach = true;
       Application.WaitForWindow(<MainWindow>);
   }
}

(I've omitted some code for clarity that just deals with catching null values in our code)

The exception occurs when I try to do anything with the elements themselves (i.e. User.Click(), Refresh()) and I get the following error when trying to set a value in a comboBox:

[Error]

Object reference not set to an instance of an object.

[StackTrace]

at ArtOfTest.WebAii.Messaging.Process.PipeCommunication.WriteCommandToPipe(PipeCommand command, PipeStream pipe, Boolean waitForDrain)
   at ArtOfTest.WebAii.Messaging.Process.BrowserRemoted.ProcessBrowserRequest(BrowserCommand command, String requestId)
   at ArtOfTest.WebAii.Wpf.WpfProxy.ExecuteSLCommand(SilverlightCommand cmd)
   at ArtOfTest.WebAii.Wpf.WpfProxy.SetProperty(AutomationProperty property, IAutomationPeer peer, Object value)
   at ArtOfTest.WebAii.Silverlight.AutomationObject`1.SetProperty(AutomationProperty property, Object value)
   at ArtOfTest.WebAii.Controls.Xaml.Wpf.ComboBox.set_IsDropDownOpen(Boolean value)
   at ArtOfTest.WebAii.Controls.Xaml.XamlControlHelper.OpenComboBoxDropDown(IComboBox combo, Boolean simulateRealUser)
   at ArtOfTest.WebAii.Controls.Xaml.Wpf.ComboBox.OpenDropDown(Boolean simulateRealUser)
   ....(rest of our code)


I get the same error popup when also dealing with other controls on the second test only and when I run that test in isolation again the test passes with no exceptions.

Any ideas why this is happening?

I've tried adding some wait code in between tests to make sure the last test exits before the next one starts but this doesn't seem to do the trick.

Also both tests run one after the other and we are using MSTest as our test runner with Resharper 7.1 and Visual Studio 2012.

Thanks

Robin

[Plamen]

Hi Robin,

I am sorry to hear you are running into this problem. Would it be possible for you to provide us with a copy of your WPF app, or ask your developers to create a small sample app that demonstrates the issue. Place it into a zip file along with a sample test that we can run and reproduce the issue. This will help us to determine what is causing the problem and find a working solution for you. If you feel any of this information is sensitive, you can submit a support ticket which is confidential, unlike this forum.

In the meantime, one thing you can try is to temporarily disable or uninstall the ReSharper plugin. The ReSharper test runner is untested by Telerik and can cause different problems, like this one for example. We recommend using the JustCode tool instead, which also includes a unit test runner feature.

Kind regards,
Plamen
the Telerik team

[Robin]

Hi Plamen,

We've done some further investigation and believe it could be to do with some aspect of either our application or Telerik test framework hanging around after a test has been completed and hence locking the next test from running correctly.

I'll report back once I know more and have some code to give to you to reproduce the error (as I am unable to give you our project code).

Thanks

Robin

[Plamen]

Hello Robin,

Ok, we'll stand by for an update and a sample test demonstrating the issue if you continue to have difficulty getting this to work. 

All the best,
Plamen
the Telerik team

[Robin]

Hi Plamen,

We found out in the end that a singleton instance was the cause of the issue rather than the Telerik framework. Seems that even when clearing the references within the singleton, something was still being kept alive so we have instead removed the singleton from our code.

Thanks anyway.

Robin

[Robin]

Hi Plamen,

Although we have solved this issue for now (as described in my previous post), why would creating a BaseWpfTest in a singleton (and renewing that instance every test) cause such an issue?

The only thing we can come up with is that the communication piping is somehow staying around even when the BaseWpfTest.Dispose() is called (and hence not being collected by the GC due to its reference to the singleton instance).

Thanks

Robin

[Plamen]

Hello Robin,

I really have no idea why this is happening. In order to be able to provide you with a straightforward answer to this question I need to see the complete code you are using to implement this. Please provide a copy of your project(in a zip file), so I can try to replicate the exact same case against one of our WPF sample apps. Once we have a repro, we'll debug it in our environment and we should be able to give you the answer you're looking for.  

Also there is no need to call both methods, CleanUp() and Dispose(). You can only use Dispose() which internally calls the CleanUp() method.

Regards,
Plamen
the Telerik team

[Robin]

Hi Plamen,

We did notice when looking at the code in the end that the CleanUp() was being called during the dispose phase so this has already been changed but thanks anyway.

I'll try to create some code to reproduce the issue next week once I get some free time to do so.

Thanks

Robin

[Robin]

Hi Plamen,

Since my last post we have changed our code to not run our application in a separate process and then attach to it. This now seems to work without any issues so I can only assume that our null pointer issue was either to do with the framework attaching at the wrong time in the application start-up cycle or the framework searching the UI/Automation trees too early.

I'm aware this isn't a definitive solution to our problem (as we haven't been able to create a controlled test that reproduces the issue) but it may help others in trying to hunt down why this issue may occur. A solution therefore would be to either increase the delay of the attachment of the framework or increase the wait time that the framework has between the application starting up and the UI/Automation trees being used in searches of the UI.

An example of our new approach is below for anyone who wishes to know how to do the above:

var settings = new Settings
           {
               ClientReadyTimeout = TimePeriod.OneMinute,
               ExecuteCommandTimeout = TimePeriod.ThirtySeconds,
               ExecutionDelay = TimePeriod.HalfSecond,
               UnexpectedDialogAction = UnexpectedDialogAction.HandleAndFailTest,
               LogLocation = "<LogPath>",
               Wpf = new Settings.WpfSettings
               {
                   DefaultApplicationPath = "<ApplicationExePath>"
               }
           };

           Manager = new Manager(settings);
           Manager.Start();
           Application = Manager.LaunchNewApplication(settings.Wpf.DefaultApplicationPath);
           var window = Application.WaitForWindow("<MainWindow>", TimePeriod.TwoMinutes);

Thanks

Robin            

[Plamen]

Hello Robin,

Thank you for providing your solution to this problem. This might be helpful to other customers facing the same issue.

I have updated your Telerik points accordingly for sharing your settings.

Regards,
Plamen
the Telerik team