Handling Signal R

[Jon]

Hi guys

We've been experimenting with SignalR. I'm having trouble getting TS to detect page load completion, browser ready, and other similar states. I find it will just sit and time out. I've tested multiple transfer methods, the biggest problem being with forever frames which is also the most common method. 

A scenario example is we have a typical login page which contains a username, password, and submit button (of type submit), on submit the login page posts to another page that uses signalr.

The steps:

-Type Username in username field
-Type Password in password field
-Click Submit button <--- This is where our test fails

TS will time out saying that the button has timed out, though the page will be shown and displayed correctly. My train of thought leads me to believe that the forever frame is causing TS to not detect the page load completion.

If you have any tips or solutions I would love to hear them!

Cheers
Jon

[Jon]

Bump

[Mario]

Hello Jon,

Thank you for supplying so much detail regarding your application. I believe your reasoning is spot on, Test Studio is currently not working so nicely with how SignalR built applications can keep a constant connection open to receive data. This results in timeouts from the browser never indicating it is 'Ready'. From your various comments, I have created this Feature Request in our feedback portal.

You may still be successful in automating this with Test Studio by following some general guidelines, for example:

With Click or Enter text steps, enable the 'SimulateRealClick' or 'SimulateRealTyping' property respectively. Then convert the steps into code (right-click -> Customize step in code) so as to comment out the ScrollToVisible line with the intent of preventing Test Studio from needing to wait for the browser to indicate it is ready prior to performing the action itself.

When creating actions, you could try using desktop commands (eg. "blind" Mouse Clicks and Keyboard Typing) while making sure not to include any code which will wait for the browser (eg. ScrollToVisible)

Please let me know whether any of my suggestions help.

Kind regards,
Mario
the Telerik team

[Bob]

Getting the same issue with our app. Action completes, page is never ready. Works fine in Chrome / FF but breaks in IE9 / IE10

[Bob]

I'm concerned that this was submitted as a feature request and not a bug fix as Signal R is becoming a very common practice.

[Jon]

Robert, I agree. I realize that adding features for specific customers is not generally a good practice. I would like a milestone prediction or something along the lines for when we can expect this feature.

[Bob]

Another reason it's most likely a bug and not a feature is that it works just fine in Chrome. 

InnerException:
System.TimeoutException: Wait for condition has timed out
   at ArtOfTest.Common.WaitSync.CheckResult(WaitSync wait, String extraExceptionInfo, Object target)
   at ArtOfTest.Common.WaitSync.For[T](Predicate`1 predicate, T target, Boolean invertCondition, Int32 timeout, WaitResultType errorResultType)
   at ArtOfTest.Common.WaitSync.For[T](Predicate`1 predicate, T target, Boolean invertCondition, Int32 timeout)
   at ArtOfTest.WebAii.Core.Browser.WaitUntilReady()
   at ArtOfTest.WebAii.Core.Browser.ExecuteCommand(BrowserCommand request, Boolean performDomRefresh, Boolean waitUntilReady)
   at ArtOfTest.WebAii.Core.Browser.ExecuteCommand(BrowserCommand request)
   at ArtOfTest.WebAii.Core.Actions.Click(Element targetElement)
   at ArtOfTest.WebAii.Controls.HtmlControls.HtmlControl.Click()
   at ArtOfTest.WebAii.Controls.HtmlControls.HtmlControl.Click(Boolean isClosing)
   at ArtOfTest.WebAii.Design.IntrinsicTranslators.Descriptors.ClickActionDescriptor.Execute(Browser browser)
   at ArtOfTest.WebAii.Design.Extensibility.HtmlActionDescriptor.Execute(IAutomationHost autoHost)
   at ArtOfTest.WebAii.Design.Execution.ExecutionEngine.ExecuteStep(Int32 order)

[Bob]

Update.

After some research @

Basic SignalR Overview: http://stackoverflow.com/questions/7807066/how-signalr-works-internally/7874352#7874352
Force Specific Transport: http://stackoverflow.com/questions/9994776/signalr-how-do-i-disable-websockets

We were able to come up with a temporary fix. As part of implementing Signalr, we're going to have the specific user accounts for testing automation force SignalR into Long polling.

From the 2nd link above:
$.connection.hubs.start({ transport: 'longPolling' }, function() {

    console.log('connection started!');
});

[Cody]

Hi Robert and Jon,

The reason we classify this as a feature request is because Test Studio is functioning the way it was intended to function in this area... specifically automatically wait for the browser to set its "Ready" flag after every action test step (click, navigate, select, etc.).

Test Studio was not designed to handle applications like SignalR that keep the browser in a "busy" state. While SignalR is becoming more popular it is far from the defacto standard for building web applications. It is still only being used by a very small percentage of websites (though that percentage is growing). Websites using ASP.NET and Ajax are still by far the vast majority.

Since Test Studio was not designed to work with this type of web development framework we need to take the necessary time to study applications based on SignalR and determine the right way to implement and add support for it. We don't want to classify it as a "bug" and rush to implement something that won't work well in the long run. Such a classification would hurt both you and Telerik in the long run.

I hope that clarifies our view and approach to this problem.

P.S. @Robert, I am slightly confused at your one message "Another reason it's most likely a bug and not a feature is that it works just fine in Chrome" only because you follow it up with a stack trace of an error. Did this stack trace come from trying to run on Chrome?

My guess is it's working because SignalR probably detects which browser is being used and changes its behavior, not due to any fault of Test Studio.

Regards,
Cody
Telerik

[Bob]

The stack trace is from the browser that's broken - IE - not the one that works - Chrome.

As for why to call it a bug vs. feature, I just want to make sure this was getting enough attention if it was a major bug that was going to effect a lot of users, including ones who use Socket.io and other, real time connection frameworks. I even posted a work around for those having issues so that they would continue to use Test Studio if this was a show shopper. I hope it's obvious that by doing this, I'm trying to help Telerik, not hurt you.

It's disheartening that you would suggest I'm trying to hurt your company.

[Cody]

Hi Robert,

I am so sorry for giving you the wrong impression. I did not in any way feel you were trying to hurt Telerik! I appreciate all your feedback and suggestions.

It's also important for our customers to understand we need to take the appropriate amount of time to come out with the right long term solution to automation challenges to problems like working smoothly with websites using SignalR.

I hope that clears up this misunderstanding.

Regards,
Cody
Telerik

[E Rik]

Hey guys, 

If you're looking to do functional and performance tests on SignalR, you should try Tresi (http://www.wcfstorm.com/wcf/learn-more-tresi.aspx)