Sometimes dialog windows are not identified by testing framework

36 posts, 0 answers
  1. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 15 Jan 2015 Link to this post

    Problem
    Sometimes our tests fail due to problems finding a component in a dialog window, and reports that the dialog itself cannot be found. According to the screenshot the dialog is visible though. This is not a very frequent issue, but it generates lots of "faulty" test results, which is very annoying. Sometimes the issue isn't seen in any testrun on any agent for several days. That means successful opening of thousand of dialogs. Then suddently the problem could occur at the 23:rd iteration of a memory leak test run that navigates back and forth between views / dialogs...  


    Implementation
    Worth mentioning is that we have created a wrapper around the testing framework and have implemented our own Proxies and Waiting classes in order to have more control of the "identification procedure". To summarize and simplify how our identification procedure works:

    1) Identify that the Manager.Current.ActiveApplication and it's process is responding
    2) Get the Window in which to look for the UI component by the expected window caption - Manager.Current.ActiveApplication.GetWindow(<<<WindowTitle>>>)
    3) Refresh the visual tree
    4) Use the VisualFind as root to find the expected element

    The Waiter classes loops over this over and over again until the element is found OR the loop times out OR a critical issue occurs during the identification procedure. There is no "waiting" going on in any of the steps to wait for a window or a process to respond or similar. 

    When the tests fail because of that it can't seem to find the dialog which is apparently visible on screen according to the screenshot, it has been looping over the mentioned steps with a timeout of 100000 ms... With a polltime set to a minimum of 25ms it means lot's of refreshing has been done etc, and all of the failures are reported to occur in the 2nd step, about finding the window with a certain caption.


    Investigation
    According to the tracelog it looks like the Manager._windowMonitor_WindowOpened() never get's called in this case as it usually does when the system identifies a new window. As far as I have understood by reverse engineering the framework source, that would mean that this window never gets added to the list of windows in WpfApplication.Windows either, which is the list in which the framework looks when calling Manager.Current.ActiveApplication.GetWindow(<<<WindowTitle>>>). Unfortunately I cannot provide any full trace log when this issue occur at the moment.


    Has anyone else experienced this problem? Is it a known issue..? The issue still exists running the 2014.4 version of the testing framework.

    Any comments or help would be very much appreciated.
     













  2. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 20 Jan 2015 Link to this post

    Hello Andreas,

    Thank you for the detailed explanation.
    We've been trying to reproduce this behavior but to no avail. Unfortunately this seems to be quite sporadic, since we never got it reported.

    We would be happy to investigate further in case we cat get a reliable repro.

    Thank you for your understanding.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  3. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 28 Jan 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Thanks for your reply. I have created a smaller project that can reproduce this issue, excluding all the additional stuff that we added around the framework, together with log files, tracelogs etc and I would be happy to send this to you in an email, or if we can set up a meeting to look into it together. 

    I have managed to find out that the issue seems to be that a deadlock in the framework causes a event listening thread to hang in when the previous dialog is closed:

    Tobii.Studio.Executable.exe(10576:1),Framework] BrowserRemoteClient.Disconnect() : Sending ClientDisconnectRequest(ClientId="Client_3fd21b76-1fa0-4133-be2d-261d76f14307") to pipe server...
    JetBrains.ReSharper.TaskRunner.CLR4.exe(6560:379),Framework] BrowserRemoted.AsyncListenerThreadEntry() : A ClientDisconnectRequest(ClientId="Client_3fd21b76-1fa0-4133-be2d-261d76f14307") has been received from the pipe client.
    Tobii.Studio.Executable.exe(10576:1),Error] BrowserRemoteClient.Disconnect() : Could not join the listener thread even after two 1-second wait attempts and an interrupt.
    Tobii.Studio.Executable.exe(10576:1),Framework] BrowserRemoteClient.Disconnect() : Disconnecting command pipe (Handle = 3752).

    When the next dialog is spawned after this Manager._windowMonitor_WindowOpened()  is never triggered.

    Please let me know how to contact you so that you can get the reproduction project and instructions.

    BR
    Andreas

  4. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 02 Feb 2015 Link to this post

    Hello Andreas,

    Thank you for creating a smaller project. Could you please archive it and attach it to this ticket so I can take a look?

    Looking forward to hearing from you.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  5. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 02 Feb 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Please install the following application:
    https://s3.amazonaws.com/eagle.tobii/TobiiGlassesController_1.0_1.0.537_x86.exe

    Then build the attached solution and run the following testcase. It will launch the installed application and spawn lots of dialogs:
    TestFixture.OpenDialogs()

    What will happen is that one of the dialogs will suddenly not become recognized, and the system will time out waiting for that dialog to occur. The expected outcome can be seen in the attached file: expected_output.txt

    It’s definetly not a 100% reproduction even with 100 iterations so you might have to extend the number of iterations in the test or run it multiple times.

    When tracelog is enabled the log file will end up here:
    Reproduction\bin\Debug

    Please note: For some reason the tracelog seem to have a negative effect on the chance to reproduce the issue, but I have managed to get this reproduced with tracelogging enabled as well. The issue seems to be a deadlock in the testing framework that occurs when the listener thread on the previous dialog is about to be stopped, and can’t be stopped. The test execution then goes on and my guess is that a new listener thread cannot be created so that we don’t get the upcoming event from when the next dialog occurs. I have attached the tracelogs from where this occurs as well, and also another one where it works as expected. You can compare the two attached files:
    NotIdentifiedShort.txt
    and 
    IdentifiedShort.txt in BeyondCompare or similar.

    When the dialog is not identified the line that prints out what originallty goes wrong seems to be:
    Tobii.Studio.Executable.exe(10576:1),Error] BrowserRemoteClient.Disconnect() : Could not join the listener thread even after two 1-second wait attempts and an interrupt.

    Also worth looking into is also that our application sometimes crashes because of a crash in the callback to the testing framework. The stacktrace seems to be referring to the very same method as mentioned above. Probably you won’t run into that issue when running the above testcase but if you do the application will throw up a crash dialog containing that stacktrace. But that is a separate issue that is not as common for us as the issue identifying the dialogs.

    To clear the contents of the application, such as participants and studies please delete the folder:
    %localappdata%\Tobii Glasses Controller

    This has been issue for a really long time and really ruins the quaility and reliability of our UI Tests. We would appreciate any help in what we can do to avoid this issue.

    BR
    Andreas
  6. Rajesh
    Rajesh avatar
    1 posts
    Member since:
    Jan 2015

    Posted 04 Feb 2015 Link to this post

    The similar kind of problem I am also facing. My scenario is when we open the Dialogbox to save the file then it works fine but when we try to open the second file save dialogbox then it gives the following exception:

    02-04 18:01:36,Telerik.TestStudio.exe(32072:31)] First trace message from foreground unnamed thread (managed ID = 31, native ID = 28888).
    [02-04 18:01:36,Telerik.TestStudio.exe(32072:31),TestStudio] StandaloneHost.LoadResults() : ------- Function Start -------
    [02-04 18:01:36,Telerik.TestStudio.exe(32072:31),Uncategorized] MainWindow.LogMessage() : Partially Loading '9' results..
    [02-04 18:01:36,Telerik.TestStudio.exe(32072:31),Uncategorized] MainWindow.LogMessage() : Results are loaded!
    [02-04 18:01:36,Telerik.TestStudio.exe(32072:31),TestStudio] StandaloneHost.LoadResults() : ------- Function End -------
    [02-04 18:01:46,IEXPLORE.EXE(33508:1),Framework] BrowserRemoteClient.Disconnect() : Sending ClientDisconnectRequest(ClientId="Client_ef6b4c77-a538-4e4d-b269-3def9dc1dcf2") to pipe server...
    [02-04 18:01:46,IEXPLORE.EXE(33508:1),Warning] BrowserRemoteClient.Disconnect() : Command pipe broke.
    [02-04 18:01:46,IEXPLORE.EXE(33508:1),Framework] BrowserRemoteClient.Disconnect() : Command pipe is already disconnected.

    I am writing the following code to perform the operation:

    {
          String b= System.DateTime.Now.ToString().Replace(":","-");
         c = /*@"C:\Users\kos\Desktop\result\"+ */ "VisitorReport " + b +".xls";   
            Log.WriteLine(c);
            filename= "VisitorReport " + b +".xls";
    string saveLocation = System.IO.Path.Combine(c, "");
     Desktop Desktop= new Desktop();       
    DownloadDialogsHandler handler = new DownloadDialogsHandler(ActiveBrowser, DialogButton.SAVE, saveLocation, Desktop);
            handler.WaitUntilHandled(2000);
            Find.ByTagIndex<HtmlImage>("img", 108).Click();
            //Wait until dialog is handled
          
        }
            catch (Exception e)
        {
            string strE = e.ToString();
            Log.WriteLine(strE);
            //Console.WriteLine("An error occurred: '{0}'", e);
        }

    Where I am making a mistake?
  7. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 04 Feb 2015 Link to this post

    Hello Andreas,

    We are still working on this, as soon as we have any additional info we will update this post.

    Thanks for your understanding and patience.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  8. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 09 Feb 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Thanks for investigating but did the reproduction project actually reproduce the issue for you?

    BR
    Andreas
  9. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 11 Feb 2015 Link to this post

    Hello Andreas,


    Unfortunately I am not able to reproduce the behavior. And in order to investigate this we will need a reliable repro.

    Thank you for your understanding.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  10. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 11 Feb 2015 in reply to Ivaylo Link to this post

    Hi,

    Have you made sure that you have disabled TraceLogging while running the test? It's a LOT more likely to reproduce the issue without TraceLogging. How many iterations have you run? It's set to iterate 100 times, but if you increase it to a 1000 I would be surprised if you don't run into the problem.

    Probably you are aware of this but unfortunately when enabling tracelogging from scripts a flag was set in the registry that was never re-set after a testrun. Which then means that it could cause inconsistency between testruns where you didn't set / unset this flag explicitly. I don't know if it's still working this way but at least in previous versions of the framework it was.

    BR
    Andreas

  11. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 16 Feb 2015 Link to this post

    Hello Andreas,

    By not being able to reproduce the behavior I mean that I am having troubles executing the test in the solution provided. Since I do not have the physical time to investigate why this is not running it would be nice to provide reliable repro that is working out of the box.

    Furthermore this seems to be way too specific for the application you are using since this behavior has not been reported, and it is not guaranteed that it can be easily reproduced.

    Thank you.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  12. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 18 Feb 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    I'm sorry, but unfortunately I have made a mistake. I actually linked to an old version of the application that you need to install as a prerequisite for running the test.

    The correct version should be:
    https://s3.amazonaws.com/eagle.tobii/TobiiGlassesController_1.4_1.4.951_x86.exe

    If you have any problems running the test after installing that version of the application please let me know.

    BR
    Andreas

  13. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 19 Feb 2015 Link to this post

    Hello Andreas,

    I will give it a try and will let you know.

    Thank you.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  14. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 25 Feb 2015 Link to this post

    Hi Ivaylo,

    Any progress in running the tests..?

    BR
    Andreas
  15. Boyan Boev
    Admin
    Boyan Boev avatar
    1040 posts

    Posted 27 Feb 2015 Link to this post

    Hi Andreas,

    Ivaylo will respond to you with a further information shortly.

    Regards,
    Boyan Boev
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  16. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 04 Mar 2015 Link to this post

    Hi again,

    Any updates? Tests keep failing here because of this issue and I'm very interested in any workarounds or if you have managed to reproduce the issue.

    BR
    Andreas
  17. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 04 Mar 2015 Link to this post

    Hello Andreas,

    Please excuse me for the delayed response, but I was out of the office for the last 10 days. I am now downloading the correct app version and will give it a try.

    Will update this ticket shortly.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  18. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 04 Mar 2015 Link to this post

    Hello Andreas,

    I was able to run the test with the new application installed, but as soon as I run it the application is launched and I got the failure attached.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  19. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 12 Mar 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    From what I can tell from the image that you attached, it looks like an output from Test Studio..? We are not using it and run these testcases straight from within Visual Studio, using the resharper test runner or from console using the nunit-console test runner on build servers. Since we are not using Test Studio it's hard to tell what kind of problem the attached image is supposed to illustrate..?

    Do you get any prints in the output when running this? I guess you should at least have got the first one: "Launching application..." when the application was launched. 

    BR
    Andreas


  20. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 16 Mar 2015 Link to this post

    Hello Andreas,

    It looks like I've attached the wrong file, please accept my apology. I have recorded a short video demonstration (no audio) which you can see here.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  21. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 27 Mar 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Thanks for the video. According to your output seen in the video, it seems a bit weird that the exception thrown is occuring at: TestFixture.cs line 21. Looking at line 21 of your source it refers to a place where I can't see how this exception can occur...? Do you see what I mean? 

    The next thing of what is weird is that you don't get any windows at all printed under "Current Windows". The window is clearly visible and it should definitely be listed under the available windows (even if it would have had another title due to mismatching versions of test code / application under test). 

    Are you running the 2014.4 version of the Testing Framework?

    I have tried downloading the reproduction project to another computer and installed the same version of the application that I referred to and it works perfectly..!

    BR
    Andreas
  22. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 27 Mar 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Thanks for the video. It seems like the exception that gets thrown when the testcase fails actually occurs at a line where I can't see how an exception can be thrown at all (TestFixture.cs line 21). Does the source in the video match the binaries that you actually ran in this case..?

    It seems weird that you don't get any windows recognized at all. The waiting for the window should time out in 25000 ms and the application definitely launches faster than that. I don't see why your framework waits 60 000 ms and then fails the test..? 

    I have downloaded the attachment with the reproduction project and installed it on another computer, together with the expected version of the application to be tested and it works great.

    Please let me know if I can help you out somehow. Launching an application and finding the main window should be pretty straight forward I guess, and that is the part that is failing now as far as I understand.

    BR
    Andreas
  23. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 27 Mar 2015 Link to this post

    I had to repost that post but apparently the first one got submitted after all. Unfortunately I never stored what I wrote somewhere else so now you have two similar posts to enjoy :-) 
  24. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 01 Apr 2015 Link to this post

    Hello Andreas,

    Excuse me for the delayed reply.

    No worries about the double posting and I am glad that on the new computer you've tested it it is working perfectly.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  25. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 01 Apr 2015 Link to this post

    Hi Ivaylo,

    Do you think you could have another look and maybe investigate a bit why it can't find the main window on your end..? It launches the correct application, and since the main window becomes visible shortly after the application is launched I can't understand why no windows can't be recognized at all.

    BR
    Andreas
  26. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 03 Apr 2015 Link to this post

    Hello Andreas,

    Sure, I've started the test again and  it runs successfully now.

    Have a good week-end!

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  27. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 07 Apr 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Good to hear! Please let me know if you can reproduce the problem by rerunning this a couple of times. What is to expect is that the test will then basically fail when not being able to identify one of the dialogs. You could increase the number of iterations easily in the testfixture. 

     

    BR

    Andreas

     

  28. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 10 Apr 2015 Link to this post

    Hello Andreas,

    Not a problem. Should I have any additional information I will update this ticket.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
  29. Andreas
    Andreas avatar
    15 posts
    Member since:
    May 2014

    Posted 15 Apr 2015 in reply to Ivaylo Link to this post

    Hi Ivaylo,

    Now when the reproduction project is running fine on your end, does it actually reproduce the issue it is supposed to reproduce? 

     

    BR

    Andreas

  30. Ivaylo
    Admin
    Ivaylo avatar
    750 posts

    Posted 20 Apr 2015 Link to this post

    Hello Andreas,

    When I executed for the last time the issue was not reproduced on my end.

    Regards,
    Ivaylo
    Telerik
     
    Quickly become an expert in Test Studio, check out our new training sessions!
    Test Studio Trainings
     
Back to Top