Problem
Sometimes our tests fail due to problems finding a component in a dialog window, and reports that the dialog itself cannot be found. According to the screenshot the dialog is visible though. This is not a very frequent issue, but it generates lots of "faulty" test results, which is very annoying. Sometimes the issue isn't seen in any testrun on any agent for several days. That means successful opening of thousand of dialogs. Then suddently the problem could occur at the 23:rd iteration of a memory leak test run that navigates back and forth between views / dialogs...
Implementation
Worth mentioning is that we have created a wrapper around the testing framework and have implemented our own Proxies and Waiting classes in order to have more control of the "identification procedure". To summarize and simplify how our identification procedure works:
1) Identify that the Manager.Current.ActiveApplication and it's process is responding
2) Get the Window in which to look for the UI component by the expected window caption - Manager.Current.ActiveApplication.GetWindow(<<<WindowTitle>>>)
3) Refresh the visual tree
4) Use the VisualFind as root to find the expected element
The Waiter classes loops over this over and over again until the element is found OR the loop times out OR a critical issue occurs during the identification procedure. There is no "waiting" going on in any of the steps to wait for a window or a process to respond or similar.
When the tests fail because of that it can't seem to find the dialog which is apparently visible on screen according to the screenshot, it has been looping over the mentioned steps with a timeout of 100000 ms... With a polltime set to a minimum of 25ms it means lot's of refreshing has been done etc, and all of the failures are reported to occur in the 2nd step, about finding the window with a certain caption.
Investigation
According to the tracelog it looks like the Manager._windowMonitor_WindowOpened() never get's called in this case as it usually does when the system identifies a new window. As far as I have understood by reverse engineering the framework source, that would mean that this window never gets added to the list of windows in WpfApplication.Windows either, which is the list in which the framework looks when calling Manager.Current.ActiveApplication.GetWindow(<<<WindowTitle>>>). Unfortunately I cannot provide any full trace log when this issue occur at the moment.
Has anyone else experienced this problem? Is it a known issue..? The issue still exists running the 2014.4 version of the testing framework.
Any comments or help would be very much appreciated.
Sometimes our tests fail due to problems finding a component in a dialog window, and reports that the dialog itself cannot be found. According to the screenshot the dialog is visible though. This is not a very frequent issue, but it generates lots of "faulty" test results, which is very annoying. Sometimes the issue isn't seen in any testrun on any agent for several days. That means successful opening of thousand of dialogs. Then suddently the problem could occur at the 23:rd iteration of a memory leak test run that navigates back and forth between views / dialogs...
Implementation
Worth mentioning is that we have created a wrapper around the testing framework and have implemented our own Proxies and Waiting classes in order to have more control of the "identification procedure". To summarize and simplify how our identification procedure works:
1) Identify that the Manager.Current.ActiveApplication and it's process is responding
2) Get the Window in which to look for the UI component by the expected window caption - Manager.Current.ActiveApplication.GetWindow(<<<WindowTitle>>>)
3) Refresh the visual tree
4) Use the VisualFind as root to find the expected element
The Waiter classes loops over this over and over again until the element is found OR the loop times out OR a critical issue occurs during the identification procedure. There is no "waiting" going on in any of the steps to wait for a window or a process to respond or similar.
When the tests fail because of that it can't seem to find the dialog which is apparently visible on screen according to the screenshot, it has been looping over the mentioned steps with a timeout of 100000 ms... With a polltime set to a minimum of 25ms it means lot's of refreshing has been done etc, and all of the failures are reported to occur in the 2nd step, about finding the window with a certain caption.
Investigation
According to the tracelog it looks like the Manager._windowMonitor_WindowOpened() never get's called in this case as it usually does when the system identifies a new window. As far as I have understood by reverse engineering the framework source, that would mean that this window never gets added to the list of windows in WpfApplication.Windows either, which is the list in which the framework looks when calling Manager.Current.ActiveApplication.GetWindow(<<<WindowTitle>>>). Unfortunately I cannot provide any full trace log when this issue occur at the moment.
Has anyone else experienced this problem? Is it a known issue..? The issue still exists running the 2014.4 version of the testing framework.
Any comments or help would be very much appreciated.
55 Answers, 1 is accepted
0
Hello Andreas,
Thank you for the detailed explanation.
We've been trying to reproduce this behavior but to no avail. Unfortunately this seems to be quite sporadic, since we never got it reported.
We would be happy to investigate further in case we cat get a reliable repro.
Thank you for your understanding.
Regards,
Ivaylo
Telerik
Thank you for the detailed explanation.
We've been trying to reproduce this behavior but to no avail. Unfortunately this seems to be quite sporadic, since we never got it reported.
We would be happy to investigate further in case we cat get a reliable repro.
Thank you for your understanding.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 28 Jan 2015, 02:10 PM
Hi Ivaylo,
Thanks for your reply. I have created a smaller project that can reproduce this issue, excluding all the additional stuff that we added around the framework, together with log files, tracelogs etc and I would be happy to send this to you in an email, or if we can set up a meeting to look into it together.
I have managed to find out that the issue seems to be that a deadlock in the framework causes a event listening thread to hang in when the previous dialog is closed:
Tobii.Studio.Executable.exe(10576:1),Framework] BrowserRemoteClient.Disconnect() : Sending ClientDisconnectRequest(ClientId="Client_3fd21b76-1fa0-4133-be2d-261d76f14307") to pipe server...
JetBrains.ReSharper.TaskRunner.CLR4.exe(6560:379),Framework] BrowserRemoted.AsyncListenerThreadEntry() : A ClientDisconnectRequest(ClientId="Client_3fd21b76-1fa0-4133-be2d-261d76f14307") has been received from the pipe client.
Tobii.Studio.Executable.exe(10576:1),Error] BrowserRemoteClient.Disconnect() : Could not join the listener thread even after two 1-second wait attempts and an interrupt.
Tobii.Studio.Executable.exe(10576:1),Framework] BrowserRemoteClient.Disconnect() : Disconnecting command pipe (Handle = 3752).
When the next dialog is spawned after this Manager._windowMonitor_WindowOpened() is never triggered.
Please let me know how to contact you so that you can get the reproduction project and instructions.
BR
Andreas
Thanks for your reply. I have created a smaller project that can reproduce this issue, excluding all the additional stuff that we added around the framework, together with log files, tracelogs etc and I would be happy to send this to you in an email, or if we can set up a meeting to look into it together.
I have managed to find out that the issue seems to be that a deadlock in the framework causes a event listening thread to hang in when the previous dialog is closed:
Tobii.Studio.Executable.exe(10576:1),Framework] BrowserRemoteClient.Disconnect() : Sending ClientDisconnectRequest(ClientId="Client_3fd21b76-1fa0-4133-be2d-261d76f14307") to pipe server...
JetBrains.ReSharper.TaskRunner.CLR4.exe(6560:379),Framework] BrowserRemoted.AsyncListenerThreadEntry() : A ClientDisconnectRequest(ClientId="Client_3fd21b76-1fa0-4133-be2d-261d76f14307") has been received from the pipe client.
Tobii.Studio.Executable.exe(10576:1),Error] BrowserRemoteClient.Disconnect() : Could not join the listener thread even after two 1-second wait attempts and an interrupt.
Tobii.Studio.Executable.exe(10576:1),Framework] BrowserRemoteClient.Disconnect() : Disconnecting command pipe (Handle = 3752).
When the next dialog is spawned after this Manager._windowMonitor_WindowOpened() is never triggered.
Please let me know how to contact you so that you can get the reproduction project and instructions.
BR
Andreas
0
Hello Andreas,
Thank you for creating a smaller project. Could you please archive it and attach it to this ticket so I can take a look?
Looking forward to hearing from you.
Regards,
Ivaylo
Telerik
Thank you for creating a smaller project. Could you please archive it and attach it to this ticket so I can take a look?
Looking forward to hearing from you.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 02 Feb 2015, 08:51 AM
Hi Ivaylo,
Please install the following application:
https://s3.amazonaws.com/eagle.tobii/TobiiGlassesController_1.0_1.0.537_x86.exe
Then build the attached solution and run the following testcase. It will launch the installed application and spawn lots of dialogs:
TestFixture.OpenDialogs()
What will happen is that one of the dialogs will suddenly not become recognized, and the system will time out waiting for that dialog to occur. The expected outcome can be seen in the attached file: expected_output.txt
It’s definetly not a 100% reproduction even with 100 iterations so you might have to extend the number of iterations in the test or run it multiple times.
When tracelog is enabled the log file will end up here:
Reproduction\bin\Debug
Please note: For some reason the tracelog seem to have a negative effect on the chance to reproduce the issue, but I have managed to get this reproduced with tracelogging enabled as well. The issue seems to be a deadlock in the testing framework that occurs when the listener thread on the previous dialog is about to be stopped, and can’t be stopped. The test execution then goes on and my guess is that a new listener thread cannot be created so that we don’t get the upcoming event from when the next dialog occurs. I have attached the tracelogs from where this occurs as well, and also another one where it works as expected. You can compare the two attached files:
NotIdentifiedShort.txt
and
IdentifiedShort.txt in BeyondCompare or similar.
When the dialog is not identified the line that prints out what originallty goes wrong seems to be:
Tobii.Studio.Executable.exe(10576:1),Error] BrowserRemoteClient.Disconnect() : Could not join the listener thread even after two 1-second wait attempts and an interrupt.
Also worth looking into is also that our application sometimes crashes because of a crash in the callback to the testing framework. The stacktrace seems to be referring to the very same method as mentioned above. Probably you won’t run into that issue when running the above testcase but if you do the application will throw up a crash dialog containing that stacktrace. But that is a separate issue that is not as common for us as the issue identifying the dialogs.
To clear the contents of the application, such as participants and studies please delete the folder:
%localappdata%\Tobii Glasses Controller
This has been issue for a really long time and really ruins the quaility and reliability of our UI Tests. We would appreciate any help in what we can do to avoid this issue.
BR
Andreas
Please install the following application:
https://s3.amazonaws.com/eagle.tobii/TobiiGlassesController_1.0_1.0.537_x86.exe
Then build the attached solution and run the following testcase. It will launch the installed application and spawn lots of dialogs:
TestFixture.OpenDialogs()
What will happen is that one of the dialogs will suddenly not become recognized, and the system will time out waiting for that dialog to occur. The expected outcome can be seen in the attached file: expected_output.txt
It’s definetly not a 100% reproduction even with 100 iterations so you might have to extend the number of iterations in the test or run it multiple times.
When tracelog is enabled the log file will end up here:
Reproduction\bin\Debug
Please note: For some reason the tracelog seem to have a negative effect on the chance to reproduce the issue, but I have managed to get this reproduced with tracelogging enabled as well. The issue seems to be a deadlock in the testing framework that occurs when the listener thread on the previous dialog is about to be stopped, and can’t be stopped. The test execution then goes on and my guess is that a new listener thread cannot be created so that we don’t get the upcoming event from when the next dialog occurs. I have attached the tracelogs from where this occurs as well, and also another one where it works as expected. You can compare the two attached files:
NotIdentifiedShort.txt
and
IdentifiedShort.txt in BeyondCompare or similar.
When the dialog is not identified the line that prints out what originallty goes wrong seems to be:
Tobii.Studio.Executable.exe(10576:1),Error] BrowserRemoteClient.Disconnect() : Could not join the listener thread even after two 1-second wait attempts and an interrupt.
Also worth looking into is also that our application sometimes crashes because of a crash in the callback to the testing framework. The stacktrace seems to be referring to the very same method as mentioned above. Probably you won’t run into that issue when running the above testcase but if you do the application will throw up a crash dialog containing that stacktrace. But that is a separate issue that is not as common for us as the issue identifying the dialogs.
To clear the contents of the application, such as participants and studies please delete the folder:
%localappdata%\Tobii Glasses Controller
This has been issue for a really long time and really ruins the quaility and reliability of our UI Tests. We would appreciate any help in what we can do to avoid this issue.
BR
Andreas
0
Rajesh
Top achievements
Rank 1
Rank 1
answered on 04 Feb 2015, 12:40 PM
The similar kind of problem I am also facing. My scenario is when we open the Dialogbox to save the file then it works fine but when we try to open the second file save dialogbox then it gives the following exception:
02-04 18:01:36,Telerik.TestStudio.exe(32072:31)] First trace message from foreground unnamed thread (managed ID = 31, native ID = 28888).
[02-04 18:01:36,Telerik.TestStudio.exe(32072:31),TestStudio] StandaloneHost.LoadResults() : ------- Function Start -------
[02-04 18:01:36,Telerik.TestStudio.exe(32072:31),Uncategorized] MainWindow.LogMessage() : Partially Loading '9' results..
[02-04 18:01:36,Telerik.TestStudio.exe(32072:31),Uncategorized] MainWindow.LogMessage() : Results are loaded!
[02-04 18:01:36,Telerik.TestStudio.exe(32072:31),TestStudio] StandaloneHost.LoadResults() : ------- Function End -------
[02-04 18:01:46,IEXPLORE.EXE(33508:1),Framework] BrowserRemoteClient.Disconnect() : Sending ClientDisconnectRequest(ClientId="Client_ef6b4c77-a538-4e4d-b269-3def9dc1dcf2") to pipe server...
[02-04 18:01:46,IEXPLORE.EXE(33508:1),Warning] BrowserRemoteClient.Disconnect() : Command pipe broke.
[02-04 18:01:46,IEXPLORE.EXE(33508:1),Framework] BrowserRemoteClient.Disconnect() : Command pipe is already disconnected.
I am writing the following code to perform the operation:
{
String b= System.DateTime.Now.ToString().Replace(":","-");
c = /*@"C:\Users\kos\Desktop\result\"+ */ "VisitorReport " + b +".xls";
Log.WriteLine(c);
filename= "VisitorReport " + b +".xls";
string saveLocation = System.IO.Path.Combine(c, "");
Desktop Desktop= new Desktop();
DownloadDialogsHandler handler = new DownloadDialogsHandler(ActiveBrowser, DialogButton.SAVE, saveLocation, Desktop);
handler.WaitUntilHandled(2000);
Find.ByTagIndex<HtmlImage>("img", 108).Click();
//Wait until dialog is handled
}
catch (Exception e)
{
string strE = e.ToString();
Log.WriteLine(strE);
//Console.WriteLine("An error occurred: '{0}'", e);
}
Where I am making a mistake?
02-04 18:01:36,Telerik.TestStudio.exe(32072:31)] First trace message from foreground unnamed thread (managed ID = 31, native ID = 28888).
[02-04 18:01:36,Telerik.TestStudio.exe(32072:31),TestStudio] StandaloneHost.LoadResults() : ------- Function Start -------
[02-04 18:01:36,Telerik.TestStudio.exe(32072:31),Uncategorized] MainWindow.LogMessage() : Partially Loading '9' results..
[02-04 18:01:36,Telerik.TestStudio.exe(32072:31),Uncategorized] MainWindow.LogMessage() : Results are loaded!
[02-04 18:01:36,Telerik.TestStudio.exe(32072:31),TestStudio] StandaloneHost.LoadResults() : ------- Function End -------
[02-04 18:01:46,IEXPLORE.EXE(33508:1),Framework] BrowserRemoteClient.Disconnect() : Sending ClientDisconnectRequest(ClientId="Client_ef6b4c77-a538-4e4d-b269-3def9dc1dcf2") to pipe server...
[02-04 18:01:46,IEXPLORE.EXE(33508:1),Warning] BrowserRemoteClient.Disconnect() : Command pipe broke.
[02-04 18:01:46,IEXPLORE.EXE(33508:1),Framework] BrowserRemoteClient.Disconnect() : Command pipe is already disconnected.
I am writing the following code to perform the operation:
{
String b= System.DateTime.Now.ToString().Replace(":","-");
c = /*@"C:\Users\kos\Desktop\result\"+ */ "VisitorReport " + b +".xls";
Log.WriteLine(c);
filename= "VisitorReport " + b +".xls";
string saveLocation = System.IO.Path.Combine(c, "");
Desktop Desktop= new Desktop();
DownloadDialogsHandler handler = new DownloadDialogsHandler(ActiveBrowser, DialogButton.SAVE, saveLocation, Desktop);
handler.WaitUntilHandled(2000);
Find.ByTagIndex<HtmlImage>("img", 108).Click();
//Wait until dialog is handled
}
catch (Exception e)
{
string strE = e.ToString();
Log.WriteLine(strE);
//Console.WriteLine("An error occurred: '{0}'", e);
}
Where I am making a mistake?
0
Hello Andreas,
We are still working on this, as soon as we have any additional info we will update this post.
Thanks for your understanding and patience.
Regards,
Ivaylo
Telerik
We are still working on this, as soon as we have any additional info we will update this post.
Thanks for your understanding and patience.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 09 Feb 2015, 03:31 PM
Hi Ivaylo,
Thanks for investigating but did the reproduction project actually reproduce the issue for you?
BR
Andreas
Thanks for investigating but did the reproduction project actually reproduce the issue for you?
BR
Andreas
0
Hello Andreas,
Unfortunately I am not able to reproduce the behavior. And in order to investigate this we will need a reliable repro.
Thank you for your understanding.
Regards,
Ivaylo
Telerik
Unfortunately I am not able to reproduce the behavior. And in order to investigate this we will need a reliable repro.
Thank you for your understanding.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 11 Feb 2015, 12:56 PM
Hi,
Have you made sure that you have disabled TraceLogging while running the test? It's a LOT more likely to reproduce the issue without TraceLogging. How many iterations have you run? It's set to iterate 100 times, but if you increase it to a 1000 I would be surprised if you don't run into the problem.
Probably you are aware of this but unfortunately when enabling tracelogging from scripts a flag was set in the registry that was never re-set after a testrun. Which then means that it could cause inconsistency between testruns where you didn't set / unset this flag explicitly. I don't know if it's still working this way but at least in previous versions of the framework it was.
BR
Andreas
Have you made sure that you have disabled TraceLogging while running the test? It's a LOT more likely to reproduce the issue without TraceLogging. How many iterations have you run? It's set to iterate 100 times, but if you increase it to a 1000 I would be surprised if you don't run into the problem.
Probably you are aware of this but unfortunately when enabling tracelogging from scripts a flag was set in the registry that was never re-set after a testrun. Which then means that it could cause inconsistency between testruns where you didn't set / unset this flag explicitly. I don't know if it's still working this way but at least in previous versions of the framework it was.
BR
Andreas
0
Hello Andreas,
By not being able to reproduce the behavior I mean that I am having troubles executing the test in the solution provided. Since I do not have the physical time to investigate why this is not running it would be nice to provide reliable repro that is working out of the box.
Furthermore this seems to be way too specific for the application you are using since this behavior has not been reported, and it is not guaranteed that it can be easily reproduced.
Thank you.
Regards,
Ivaylo
Telerik
By not being able to reproduce the behavior I mean that I am having troubles executing the test in the solution provided. Since I do not have the physical time to investigate why this is not running it would be nice to provide reliable repro that is working out of the box.
Furthermore this seems to be way too specific for the application you are using since this behavior has not been reported, and it is not guaranteed that it can be easily reproduced.
Thank you.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 18 Feb 2015, 07:05 AM
Hi Ivaylo,
I'm sorry, but unfortunately I have made a mistake. I actually linked to an old version of the application that you need to install as a prerequisite for running the test.
The correct version should be:
https://s3.amazonaws.com/eagle.tobii/TobiiGlassesController_1.4_1.4.951_x86.exe
If you have any problems running the test after installing that version of the application please let me know.
BR
Andreas
I'm sorry, but unfortunately I have made a mistake. I actually linked to an old version of the application that you need to install as a prerequisite for running the test.
The correct version should be:
https://s3.amazonaws.com/eagle.tobii/TobiiGlassesController_1.4_1.4.951_x86.exe
If you have any problems running the test after installing that version of the application please let me know.
BR
Andreas
0
Hello Andreas,
I will give it a try and will let you know.
Thank you.
Regards,
Ivaylo
Telerik
I will give it a try and will let you know.
Thank you.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 25 Feb 2015, 03:18 PM
Hi Ivaylo,
Any progress in running the tests..?
BR
Andreas
Any progress in running the tests..?
BR
Andreas
0
Hi Andreas,
Ivaylo will respond to you with a further information shortly.
Regards,
Boyan Boev
Telerik
Ivaylo will respond to you with a further information shortly.
Regards,
Boyan Boev
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 04 Mar 2015, 07:05 AM
Hi again,
Any updates? Tests keep failing here because of this issue and I'm very interested in any workarounds or if you have managed to reproduce the issue.
BR
Andreas
Any updates? Tests keep failing here because of this issue and I'm very interested in any workarounds or if you have managed to reproduce the issue.
BR
Andreas
0
Hello Andreas,
Please excuse me for the delayed response, but I was out of the office for the last 10 days. I am now downloading the correct app version and will give it a try.
Will update this ticket shortly.
Regards,
Ivaylo
Telerik
Please excuse me for the delayed response, but I was out of the office for the last 10 days. I am now downloading the correct app version and will give it a try.
Will update this ticket shortly.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Hello Andreas,
I was able to run the test with the new application installed, but as soon as I run it the application is launched and I got the failure attached.
Regards,
Ivaylo
Telerik
I was able to run the test with the new application installed, but as soon as I run it the application is launched and I got the failure attached.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 12 Mar 2015, 03:19 PM
Hi Ivaylo,
From what I can tell from the image that you attached, it looks like an output from Test Studio..? We are not using it and run these testcases straight from within Visual Studio, using the resharper test runner or from console using the nunit-console test runner on build servers. Since we are not using Test Studio it's hard to tell what kind of problem the attached image is supposed to illustrate..?
Do you get any prints in the output when running this? I guess you should at least have got the first one: "Launching application..." when the application was launched.
BR
Andreas
From what I can tell from the image that you attached, it looks like an output from Test Studio..? We are not using it and run these testcases straight from within Visual Studio, using the resharper test runner or from console using the nunit-console test runner on build servers. Since we are not using Test Studio it's hard to tell what kind of problem the attached image is supposed to illustrate..?
Do you get any prints in the output when running this? I guess you should at least have got the first one: "Launching application..." when the application was launched.
BR
Andreas
0
Hello Andreas,
It looks like I've attached the wrong file, please accept my apology. I have recorded a short video demonstration (no audio) which you can see here.
Regards,
Ivaylo
Telerik
It looks like I've attached the wrong file, please accept my apology. I have recorded a short video demonstration (no audio) which you can see here.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 27 Mar 2015, 02:44 PM
Hi Ivaylo,
Thanks for the video. According to your output seen in the video, it seems a bit weird that the exception thrown is occuring at: TestFixture.cs line 21. Looking at line 21 of your source it refers to a place where I can't see how this exception can occur...? Do you see what I mean?
The next thing of what is weird is that you don't get any windows at all printed under "Current Windows". The window is clearly visible and it should definitely be listed under the available windows (even if it would have had another title due to mismatching versions of test code / application under test).
Are you running the 2014.4 version of the Testing Framework?
I have tried downloading the reproduction project to another computer and installed the same version of the application that I referred to and it works perfectly..!
BR
Andreas
Thanks for the video. According to your output seen in the video, it seems a bit weird that the exception thrown is occuring at: TestFixture.cs line 21. Looking at line 21 of your source it refers to a place where I can't see how this exception can occur...? Do you see what I mean?
The next thing of what is weird is that you don't get any windows at all printed under "Current Windows". The window is clearly visible and it should definitely be listed under the available windows (even if it would have had another title due to mismatching versions of test code / application under test).
Are you running the 2014.4 version of the Testing Framework?
I have tried downloading the reproduction project to another computer and installed the same version of the application that I referred to and it works perfectly..!
BR
Andreas
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 27 Mar 2015, 03:03 PM
Hi Ivaylo,
Thanks for the video. It seems like the exception that gets thrown when the testcase fails actually occurs at a line where I can't see how an exception can be thrown at all (TestFixture.cs line 21). Does the source in the video match the binaries that you actually ran in this case..?
It seems weird that you don't get any windows recognized at all. The waiting for the window should time out in 25000 ms and the application definitely launches faster than that. I don't see why your framework waits 60 000 ms and then fails the test..?
I have downloaded the attachment with the reproduction project and installed it on another computer, together with the expected version of the application to be tested and it works great.
Please let me know if I can help you out somehow. Launching an application and finding the main window should be pretty straight forward I guess, and that is the part that is failing now as far as I understand.
BR
Andreas
Thanks for the video. It seems like the exception that gets thrown when the testcase fails actually occurs at a line where I can't see how an exception can be thrown at all (TestFixture.cs line 21). Does the source in the video match the binaries that you actually ran in this case..?
It seems weird that you don't get any windows recognized at all. The waiting for the window should time out in 25000 ms and the application definitely launches faster than that. I don't see why your framework waits 60 000 ms and then fails the test..?
I have downloaded the attachment with the reproduction project and installed it on another computer, together with the expected version of the application to be tested and it works great.
Please let me know if I can help you out somehow. Launching an application and finding the main window should be pretty straight forward I guess, and that is the part that is failing now as far as I understand.
BR
Andreas
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 27 Mar 2015, 03:08 PM
I had to repost that post but apparently the first one got submitted after all. Unfortunately I never stored what I wrote somewhere else so now you have two similar posts to enjoy :-)
0
Hello Andreas,
Excuse me for the delayed reply.
No worries about the double posting and I am glad that on the new computer you've tested it it is working perfectly.
Regards,
Ivaylo
Telerik
Excuse me for the delayed reply.
No worries about the double posting and I am glad that on the new computer you've tested it it is working perfectly.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 01 Apr 2015, 09:08 AM
Hi Ivaylo,
Do you think you could have another look and maybe investigate a bit why it can't find the main window on your end..? It launches the correct application, and since the main window becomes visible shortly after the application is launched I can't understand why no windows can't be recognized at all.
BR
Andreas
Do you think you could have another look and maybe investigate a bit why it can't find the main window on your end..? It launches the correct application, and since the main window becomes visible shortly after the application is launched I can't understand why no windows can't be recognized at all.
BR
Andreas
0
Hello Andreas,
Sure, I've started the test again and it runs successfully now.
Have a good week-end!
Regards,
Ivaylo
Telerik
Sure, I've started the test again and it runs successfully now.
Have a good week-end!
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 07 Apr 2015, 03:28 PM
Hi Ivaylo,
Good to hear! Please let me know if you can reproduce the problem by rerunning this a couple of times. What is to expect is that the test will then basically fail when not being able to identify one of the dialogs. You could increase the number of iterations easily in the testfixture.
BR
Andreas
0
Hello Andreas,
Not a problem. Should I have any additional information I will update this ticket.
Regards,
Ivaylo
Telerik
Not a problem. Should I have any additional information I will update this ticket.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Andreas
Top achievements
Rank 1
Rank 1
answered on 15 Apr 2015, 02:07 PM
Hi Ivaylo,
Now when the reproduction project is running fine on your end, does it actually reproduce the issue it is supposed to reproduce?
BR
Andreas
0
Hello Andreas,
When I executed for the last time the issue was not reproduced on my end.
Regards,
Ivaylo
Telerik
When I executed for the last time the issue was not reproduced on my end.
Regards,
Ivaylo
Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 14 Nov 2016, 11:58 AM
Is this problem has been resolved? I'm using 2015.3.1314.0 version and have the same problem with my application. This error isn't deterministic, but when it apperas the rest of test cases can't been executed - it's very annoying.
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 15 Nov 2016, 10:53 AM
I found that this bug was under investigating.
https://feedback.telerik.com/Project/161/Feedback/Details/162947-intermittent-error-on-closing-wpf-window-causes-failure-to-find-window-in-open-cl
Is there any solution to avoid this problem?
Sorry for the double post, but this bug is blocking huge part of my tests and the link above seemed to me important.
0
Hi Tomasz,
Thank you for contacting us.
The issue you have encountered and found in our public feedback portal is still opened. Its investigation is being postponed since reproducing it is quite difficult and therefore working on finding a solution is not an easy job. Additionally its appearance is inconsistent which makes it even harder for investigation in depth.
Though I would like to ask for your assistance. If you could provide us a stable repro we could continue the investigation. A sample WPF application and a test which execution leads to that result would be of great help for us.
Thanks in advance for your time and cooperation! I hope to hear from you soon!
Regards,
Elena Tsvetkova
Telerik by Progress
Thank you for contacting us.
The issue you have encountered and found in our public feedback portal is still opened. Its investigation is being postponed since reproducing it is quite difficult and therefore working on finding a solution is not an easy job. Additionally its appearance is inconsistent which makes it even harder for investigation in depth.
Though I would like to ask for your assistance. If you could provide us a stable repro we could continue the investigation. A sample WPF application and a test which execution leads to that result would be of great help for us.
Thanks in advance for your time and cooperation! I hope to hear from you soon!
Regards,
Elena Tsvetkova
Telerik by Progress
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 18 Nov 2016, 02:15 PM
Hi Elena,
I will try give you my assistance with pleasure.
In the attachment you will find a sample Wpf Applicatin and the log with result of my test execution.
The problem is nondeterministic, but if it appears, test execution is suspended and in the log you will find information like this:
[11-18 11:15:49,WpfApplication2.exe(2244:1),Error] BrowserRemoteClient.Disconnect() : Could not join the listener thread even after two 1-second wait attempts and an interrupt.
Because of its nondeterministic character you have to retry the test several times.
In my case it usually occurs in ten to twenty repeats.
I reproduced this bug in two different computers (local and virtual) with different OS (Win7, WinServer2012), so it's not an environment problem. Therefore I hope you will find it also.
My version of Test Studio is 2015.3.1314.0
Hope to hear from you soon.
Regards,
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 18 Nov 2016, 02:23 PM
Sorry, I forgot to add the test files.
0
Hi Tomasz,
Thank you for your efforts and cooperation. I tested the provided sample app along with the test case you have shared. I ran the test more than 20 times and couldn't reproduce the same behavior as what you described.
Though the version of Test Studio I am using is the latest internal build R3.1117.2 and this might be the reason. Could you please upgrade to our latest release and test with it? It could be downloaded here.
Thank you once again for your assistance!
Regards,
Elena Tsvetkova
Telerik by Progress
Thank you for your efforts and cooperation. I tested the provided sample app along with the test case you have shared. I ran the test more than 20 times and couldn't reproduce the same behavior as what you described.
Though the version of Test Studio I am using is the latest internal build R3.1117.2 and this might be the reason. Could you please upgrade to our latest release and test with it? It could be downloaded here.
Thank you once again for your assistance!
Regards,
Elena Tsvetkova
Telerik by Progress
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 02 Jan 2017, 09:48 AM
Hi Elena,
Sorry for my delay. I had a little problem with my trial period extending.
I tested the sample app using the same version of Test Studio that you mentioned, and unfortunately the problem is still exist.
I have modified a little the test scenario. Elena please do some test with it. The best case is to set the test list with this scenario, 3 minutes recurring and 10 occurrences.
The problem not always appears in one schedule period, but if you repeat it a couple of times it should appear.
It is very important to me, to eliminate this problem. I will be very grateful for your persistence in searching it.
Sorry for my delay. I had a little problem with my trial period extending.
I tested the sample app using the same version of Test Studio that you mentioned, and unfortunately the problem is still exist.
I have modified a little the test scenario. Elena please do some test with it. The best case is to set the test list with this scenario, 3 minutes recurring and 10 occurrences.
The problem not always appears in one schedule period, but if you repeat it a couple of times it should appear.
It is very important to me, to eliminate this problem. I will be very grateful for your persistence in searching it.
0
Hi Tomasz,
Thanks for providing me another sample. Please allow me some time to test this and once I gather some observations I will get back to you.
Regards,
Elena Tsvetkova
Telerik by Progress
Thanks for providing me another sample. Please allow me some time to test this and once I gather some observations I will get back to you.
Regards,
Elena Tsvetkova
Telerik by Progress
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 07 Mar 2017, 10:36 AM
Hi Elena,
Do You have any results of tests? Let me know if I could help in some way.
Regards,
0
Hi Andreas,
Please excuse me for not getting back to you.
Unfortunately I was not able to reproduce the reported behavior even with the last provided modified test scenario. Could you record a short video demonstrating the issue on your end? It might bring additional details how to reproduce it. We recommend Jing as a free and easy to share online video recording app.
Regards,
Elena Tsvetkova
Telerik by Progress
Please excuse me for not getting back to you.
Unfortunately I was not able to reproduce the reported behavior even with the last provided modified test scenario. Could you record a short video demonstrating the issue on your end? It might bring additional details how to reproduce it. We recommend Jing as a free and easy to share online video recording app.
Regards,
Elena Tsvetkova
Telerik by Progress
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 23 Mar 2017, 02:52 PM
Hi Elena,
I have recorded two videos. In the long one I show the right behavior of the Test Studio in processing the scenario. The short one shows the error occurence - the same scenario is never ended and I have to terminate it manualy.
0
Hi Tomasz,
Thank you for contributing further details.
Unfortunately I wasn't able to play the shared videos for some reason. I see they are in a quite common format and tried few players with no luck.
Since you have submitted a support ticket on the same topic I would like to kindly ask you to continue the communication in that thread to not duplicate the effort from both sides.
Thanks for your understanding!
Regards,
Elena Tsvetkova
Telerik by Progress
Thank you for contributing further details.
Unfortunately I wasn't able to play the shared videos for some reason. I see they are in a quite common format and tried few players with no luck.
Since you have submitted a support ticket on the same topic I would like to kindly ask you to continue the communication in that thread to not duplicate the effort from both sides.
Thanks for your understanding!
Regards,
Elena Tsvetkova
Telerik by Progress
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 24 Mar 2017, 01:42 PM
Hi,
Elena - to play shared videos please use MPC-HC media player (https://mpc-hc.org/) and let mi know. At my side it's working and the videos are played.
What operating system (on which You have Test Studio installed) do You use to reproduce this bug?
Regards,
0
Hello Andreas,
Thank you for getting back to me. Unfortunately I am still not able to run the videos - the suggested video player reported that I might be missing some codecs. Would it be possible for you to re-record them using Jing?
I have tested the provided script and the application on a Windows 10 machine. What is the one you have in use? If you let me know I could try on a similar machine as well.
Regards,
Elena Tsvetkova
Telerik by Progress
Thank you for getting back to me. Unfortunately I am still not able to run the videos - the suggested video player reported that I might be missing some codecs. Would it be possible for you to re-record them using Jing?
I have tested the provided script and the application on a Windows 10 machine. What is the one you have in use? If you let me know I could try on a similar machine as well.
Regards,
Elena Tsvetkova
Telerik by Progress
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 04 Apr 2017, 01:01 PM
Hi Elena,
once again :)
I have recorded three videos. In the "without_error" one I show the right behavior of the Test Studio in processing the scenario. The "error1" and "error2" shows the error occurence (in diffierent test step of the same scenario) - the same scenario is never ended and I have to terminate it manualy.
I use Windows 7 and Windows Server 2012 R2 (two machines). If You have a possibility to try on similar machines, please check it. As You see on the videos I had no problem with reproducing this error.
0
Hi Tomasz,
Thank you for providing the videos. Please allow me some more time to try to reproduce the issue on Win 7 or Win Server 2012 R2 machines. I will get back to you once I have my observations.
Thank you again for your continuous cooperation! :)
Regards,
Elena Tsvetkova
Telerik by Progress
Thank you for providing the videos. Please allow me some more time to try to reproduce the issue on Win 7 or Win Server 2012 R2 machines. I will get back to you once I have my observations.
Thank you again for your continuous cooperation! :)
Regards,
Elena Tsvetkova
Telerik by Progress
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Hello Tomasz,
Thank you for you patience!
Please excuse me the investigation took so long. Unfortunately I am still not able to reproduce the issue you observe.
Though I would like to inform you our next major release is planned for end of this month. It would be nice if you could give it a try and hopefully the inconsistent behavior does not persist anymore.
Thank you once again!
Regards,
Elena Tsvetkova
Telerik by Progress
Thank you for you patience!
Please excuse me the investigation took so long. Unfortunately I am still not able to reproduce the issue you observe.
Though I would like to inform you our next major release is planned for end of this month. It would be nice if you could give it a try and hopefully the inconsistent behavior does not persist anymore.
Thank you once again!
Regards,
Elena Tsvetkova
Telerik by Progress
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 24 May 2017, 02:05 PM
Hi Elena,
Thanks for the information. Ofcourse I will do the tests with new release of Test Studio and give You a feedback with it. I hope this notdeteministic problem with connection to application window will no longer exist.
Regards
0
Hello Tomasz,
Thanks for your understanding and cooperation! I also hope the upcoming official release will satisfy your requirements!
Regards,
Elena Tsvetkova
Progress Telerik
Thanks for your understanding and cooperation! I also hope the upcoming official release will satisfy your requirements!
Regards,
Elena Tsvetkova
Progress Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 05 Sep 2017, 09:38 AM
Hi Elena,
After Your last release I checked the 2017.2.824.1 version on my testing machine. Unfortunately problem described above still exists. I'm wondering how can I help You in tracking this bug. If You have some idea, please let mi know.
0
Hello Tomasz,
Thank you for getting back to us.
I am sorry to hear the issue is still persistent on your end. The only way an issue could be investigated in depth is to be reproduced. As you know I was unable to and this complicates the research.
However, you could try on a different machine just for testing purposes. If the misbehavior does not appear it might be related to the used computer and OS.
Please excuse us for any caused inconvenience. Thank you for your understanding!
Regards,
Elena Tsvetkova
Progress Telerik
Thank you for getting back to us.
I am sorry to hear the issue is still persistent on your end. The only way an issue could be investigated in depth is to be reproduced. As you know I was unable to and this complicates the research.
However, you could try on a different machine just for testing purposes. If the misbehavior does not appear it might be related to the used computer and OS.
Please excuse us for any caused inconvenience. Thank you for your understanding!
Regards,
Elena Tsvetkova
Progress Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Matthias
Top achievements
Rank 1
Rank 1
answered on 11 Sep 2017, 03:11 PM
Hi Tomasz, Hi Elena,
As described in this bug report [1], we suffer from this issue already a couple of years since we are running more than 400 UI Tests every night and 2 - 3 will fail during each run.
Our test execution environment is Windows Server 2016.
@Tomasz: Could you already produce the issue on Windows 10 machines?
@Elena: What is the status of this bug [1]? Will this be fixed?
Thanks,
Matthias
[1] - https://feedback.telerik.com/Project/161/Feedback/Details/162947-intermittent-error-on-closing-wpf-window-causes-failure-to-find-window-in-open-cl
As described in this bug report [1], we suffer from this issue already a couple of years since we are running more than 400 UI Tests every night and 2 - 3 will fail during each run.
Our test execution environment is Windows Server 2016.
@Tomasz: Could you already produce the issue on Windows 10 machines?
@Elena: What is the status of this bug [1]? Will this be fixed?
Thanks,
Matthias
[1] - https://feedback.telerik.com/Project/161/Feedback/Details/162947-intermittent-error-on-closing-wpf-window-causes-failure-to-find-window-in-open-cl
0
Hi,
Thank you guys for your cooperation.
I would like to kindly ask for your assistance again since I am unable to reproduce the issue on my end with the provided sample.
Tomasz had shared plenty of details along with sample app and test to get the error. Though I still could not face the same misbehavior.
@Matthias: Could you please try to reproduce the issue on your end using the test and application that Tomasz had prepared? Please let me know if you would need these packed since the communication is quite long. Please share the observed results as well as some specifics for the used environment.
@Tomasz: Please also share some details about the environment on which you are able to reproduce the issue even for a single test quick run.
@Matthias: The bug report you have referred to is in testing stage but it actually relates to a custom project containing a sample app and a sample test. The issue with this project was how the application is being initialized in the code of the test. An important note would be that the project in question dates back in two years so I am not sure it could be referred to the current state.
Please let me know if I have to provide any further details. Thanks in advance for your cooperation!
Regards,
Elena Tsvetkova
Progress Telerik
Thank you guys for your cooperation.
I would like to kindly ask for your assistance again since I am unable to reproduce the issue on my end with the provided sample.
Tomasz had shared plenty of details along with sample app and test to get the error. Though I still could not face the same misbehavior.
@Matthias: Could you please try to reproduce the issue on your end using the test and application that Tomasz had prepared? Please let me know if you would need these packed since the communication is quite long. Please share the observed results as well as some specifics for the used environment.
@Tomasz: Please also share some details about the environment on which you are able to reproduce the issue even for a single test quick run.
@Matthias: The bug report you have referred to is in testing stage but it actually relates to a custom project containing a sample app and a sample test. The issue with this project was how the application is being initialized in the code of the test. An important note would be that the project in question dates back in two years so I am not sure it could be referred to the current state.
Please let me know if I have to provide any further details. Thanks in advance for your cooperation!
Regards,
Elena Tsvetkova
Progress Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings
0
Yaroslav
Top achievements
Rank 1
Rank 1
answered on 30 Oct 2017, 04:19 PM
Hello, All.
I had experimented with Telerik Testing Framework, and ran UI tests with NUnit.Console.Runner.
I have get the same problem as Tomasz and Matthias during last half year. My tests pass almost always but sometimes some of my tests can't find dialog window, which is present and visible. This problem is not deterministic and repeats unpredictably.
When I enable TraceLog it seems that this problem repeated less often but still present.
Thanks,
Yaroslav.
0
Tomasz
Top achievements
Rank 1
Rank 1
answered on 31 Oct 2017, 10:35 AM
Hi, All :)
@Elena, @Matthias: I have reproduce the issue on Windows 2012 Serwer and Windows 7 machines. Unfortuneately for the present moment I can't check it od Windows 10 machine.
@Yaroslav: Please check if Your logs contains "Could not join the listener thread even after two 1-second wait attempts and an interrupt" after the issue occure. Please write also what operation system You have.
Regards,
Tomasz
0
Hello,
Thank you all for contributing.
I will be glad to cooperate if there are further details on the case. Unfortunately last time I wasn't able to reproduce that despite all the efforts of Tomasz. Hopefully this time it could be reproduced and investigated.
I am looking forward to hearing from you all. Thanks!
Regards,
Elena Tsvetkova
Progress Telerik
Thank you all for contributing.
I will be glad to cooperate if there are further details on the case. Unfortunately last time I wasn't able to reproduce that despite all the efforts of Tomasz. Hopefully this time it could be reproduced and investigated.
I am looking forward to hearing from you all. Thanks!
Regards,
Elena Tsvetkova
Progress Telerik
Quickly become an expert in Test Studio, check out our new training sessions!
Test Studio Trainings
Test Studio Trainings