This is a migrated thread and some comments may be shown as answers.

[Solved] Always Fails? What?

30 Answers 379 Views

Telerik Testing Framework

This is a migrated thread and some comments may be shown as answers.

This question is locked. New answers and comments are not allowed.

Kori asked on 21 Jul 2009, 01:47 PM

I must be doing something wrong here.

I'm just trying to test a simple asp.net login form on my site. I extrapolated from the sample code and I can't get it to work.

I have the following test case:

                    [Test]         
            public void LoginTest()         
            {         
                 // Launch a browser instance         
                 Manager.LaunchNewBrowser(BrowserType.InternetExplorer);         
                     
                 // The active browser         
                 ActiveBrowser.NavigateTo("http://localhost/BillingSystem/Login.aspx");         
                     
                 // Enter the information         
                 Find.ById<HtmlInputText>("ctl00_cphMainContent_Login1_UserName").Text = "admin";         
                 Find.ById<HtmlInputPassword>("ctl00_cphMainContent_Login1_Password").Text = "password";         
                     
                 // Cause the page to login         
                 Find.ById<HtmlInputSubmit>("ctl00_cphMainContent_Login1_LoginButton").Click();         
                     
                 Assert.IsTrue(ActiveBrowser.Url.Contains("Start.aspx"));         
            }         

The login moves from Login.aspx to Start.aspx - No assert case really works, not looking for text (ActiveBrowser.ContainsText("Welcome") for instance. Making the thread sleep doesn't work, using the Wait.For's don't seem to work, nothing I have done let's me test this case. Why?

Am I doing something wrong? Help!

30 Answers, 1 is accepted

answered on 21 Jul 2009, 04:22 PM

Hello Kori,

Which testing framework are you trying to use? NUnit or Visual Studio Team Test? I see you have your test decorated with [Test] at the beginning. This is for NUnit. Visual Studio Team Test uses [TestMethod] when decorating tests. If you are using NUnit, are you running the test using the NUnit test runner? Some more information about your testing framework & environment will be helpful.

Kind regards,
Cody
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.

answered on 21 Jul 2009, 04:22 PM

Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.

answered on 22 Jul 2009, 06:21 AM

Hi Kori,

I've never used TestDriven.NET myself (one of my team-mates did and maybe I can get his help) so I'm afraid I'm going to be of little help on how to use that test running specifically. If it's a problem getting TestDriven.NET to respond properly, have you tried making something simple, something that doesn't use WebAii code, such as creating a test that should always pass, or always fail and see if you get the expected results?

But I'm also unclear on what exactly is going wrong. You say you're unable to test your particular test case, but I don't quite understand really what went wrong compared to what you were expecting to happen. Can you clarify better what you expected to see and what you actually see?

All the best,
Cody
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.

answered on 22 Jul 2009, 01:28 PM

Okay,

I've tested the following cases:

                    [Test]         
            public void FailTest()         
            {         
                 Assert.AreEqual(0, 1);         
            }         
                     
            [Test]         
            public void SuccessTest()         
            {         
                 Assert.AreEqual(0, 0);         
            }         

And both of those work just like they should (ie, fail test fails, success test succeeds)

What I'm trying to test is that my login redirects to the correct page essentially. I get the test to input the username and password, then I want to check that it has indeed moved from Login.aspx to Start.aspx. Not that I particularily need the check the login, as ASP.NET membership works just fine - but later on, I'll need to test form input and go from page to page to check that the result is what I'm expecting.

answered on 22 Jul 2009, 02:37 PM

Hi Kori,

Thanks for verifying you do get good pass/fail indication. Now given the code from your original post, was it supposed to pass or supposed to fail? If it's supposed to fail, what do you see happening in the browser window running your test? Does it set the user name and password fields correctly? After it clicks the login button does the server correctly respond (denying the login) and remain at the login.aspx page or does it ignore the bad logon credentails and moves to the start.aspx page anyway?

Greetings,
Cody
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.

answered on 22 Jul 2009, 04:14 PM

In my original test, it does exactly what it's supposed to do.

Set's the username and password correctly
Clicks the login button causing the page to automatically redirect to Start.aspx

And the actual test is to just double check that it is in-fact on the Start.aspx page. So I would expect that the test should pass, and have no clue as to why it doesn't. It doesn't matter how I try to identify if the browser is on the right page. I've tried a number of things, everything I've done leads me to assume that ActiveBrowser is still on the Login.aspx page - even though in the browser during the test it performs as it should.

If my test is something like:

                    Element e = Find.ByAttributes("class=title");         
            Assert.AreEqual("Welcome!", e.TextContent);         

I would expect that would pass, since the words "Welcome!" only appear on Start.aspx - but that test shows me that it's reading the value in the title element for Login.aspx in e.TextContent instead of Start.aspx.

answered on 22 Jul 2009, 07:07 PM

Hi Kori,

Thanks for the detailed explanation. I think I (finally) understand what's happening and why it's going wrong. The information I was missing is the fact that the code you posted is supposed to pass but is actually failing at the Assert.

What must be going on is during the redirect from Login.aspx to Start.aspx communication between the browser and the web server must have gone quiet just long enough for the framework to believe it's complete. The framework keeps a copy of the DOM & URL in memory and uses that for all of its testing. Some time later (maybe just a few milliseconds) communication to the web server starts up again and Start.aspx actually gets control. Unfortunately by this time the WebAii framework and the browser are now out of sync.

The fix is pretty easy. Add a WaitForUrl after your Click and your Assert such that the code now looks like this:

    
            // Cause the page to login  
        
            Find.ById<HtmlInputSubmit>("ctl00_cphMainContent_Login1_LoginButton").Click();  
        
            // Wait for the redirect to finish  
        
            ActiveBrowser.WaitForUrl("Start.aspx", true, 5000);  
        
            Assert.IsTrue(ActiveBrowser.Url.Contains("Start.aspx"));

All the best,
Cody
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.

Accepted

answered on 23 Jul 2009, 02:10 PM

Hi Kori,

First I must apologize because I made a small, but criticl mistake in the code I recommended. Hopefully this correction helps. Preceed the url with a '~' character. This means partial matching instead of exact matching. Like this:

    
            ActiveBrowser.WaitForUrl("~Start.aspx", true, 5000);

If that doesn't help then something very unusual is happening with your website and I'd like to take a look at it first hand myself. Is there any way you can make it publicly accessible and give me the URL with userID and password to study?

Kind regards,
Cody
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Check out the tips for optimizing your support resource searches.

answered on 04 Sep 2009, 02:36 PM

Hi,

I've similar problem, my test case sometime passed sometime failed without changing of anything. I tried to solve this with suported Wait methods but it seems that they don't work. The uncommented part of code below passed more times than commented but sometimes it failed too. Do you have any of idea why Waiting doesn't work as we expected and how to ensure that test cases will be passed every time they should?

[TestMethod]
        public void FirstLoginPageLabelsLanguage()
        {
            // Launch a browser instance
            Manager.LaunchNewBrowser(BrowserType.InternetExplorer);

            // The active browser
            ActiveBrowser.NavigateTo("http://localhost/DCMSv2/Default.aspx");

            ActiveBrowser.WaitForUrl("http://localhost/DCMSv2/Default.aspx", true, 5000);

            Assert.IsTrue(ActiveBrowser.Url.Contains("Default.aspx"));

            //Silverlight application instance of object
            var slApp = ActiveBrowser.SilverlightApps()[0];

            slApp.Find.Strategy = FindStrategy.WhenNotVisibleReturnNull;

            TextBlock obj = slApp.Find.ByName<TextBlock>("UsernameLabel");

            if (obj != null)
            {
                obj.Wait.ForExists(10000);
                Assert.IsTrue(obj.Text == "User name:","step 1");
            }
            else
            {
                Assert.IsTrue(false, "step 2");
                string ttt = "ttt";
            }

            //slApp.Find.Strategy = FindStrategy.WhenNotVisibleReturnElementProxy;

            //FrameworkElement usernameLabelProxy = slApp.Find.ByName("UsernameLabel");
            //usernameLabelProxy.Wait.ForExists(15000);
            //Assert.IsTrue(slApp.Find.ByName("UsernameLabel").TextContent.Equals("User name:"));
        }

The problem appears for "UsernameLabel" assert, not for "Default.aspx" assert. Also I used VSunit test.

answered on 04 Sep 2009, 04:53 PM

Hello Alexander,

When you say your test sometimes fails, in what manner does it fail? What is the specific error message, and which line of your code is it failing at? Once I know the specific error you are getting I can better assist how to overcome that error. I'd like to know what error you get when you try both methods of the code sample (the one currently uncommented versus the one you have commented out).

Also the lines:

    
            ActiveBrowser.WaitForUrl("http://localhost/DCMSv2/Default.aspx", true, 5000);  
        
            Assert.IsTrue(ActiveBrowser.Url.Contains("Default.aspx"));

should not be necessary. The NavigateTo call will not return until the browser is done loading that webpage and is ready for another operation. Do you have any problems if you remove these two lines of code?

All the best,
Cody
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

answered on 07 Sep 2009, 08:21 AM

Hi Cody,

Thanks for feedback.
The errors are just: Assert.IsTrue failed.
Code lines:
Assert.IsTrue(obj.Text == "User name:","step 1");
//Assert.IsTrue(slApp.Find.ByName("UsernameLabel").TextContent.Equals("User name:")); when is uncommented

The same problem exists if I remove two lines of code that you mentioned.

I think that assert fails because sometime Assert operation performs before browser is done loading that silverlight webpage. But I don't understand why supported Wait methods doesn't work, the assert operations performs without defined waiting.

answered on 07 Sep 2009, 09:14 AM

Alexander is tester in my team so i will try to help him explain his problems...

we did try two different approaches

    
            slApp.Find.Strategy = FindStrategy.WhenNotVisibleReturnNull;  
        
            TextBlock obj = slApp.Find.ByName<TextBlock>("UsernameLabel");             
        
            if (obj != null)  
        
            {  
        
                obj.Wait.ForExists(10000);  
        
                Assert.IsTrue(obj.Text == "User name:","step 1");  
        
            }  
        
            else{      Assert.IsTrue(false, "step 2");  }

The is the best option we found. again in let say 10% of the cases test will fail as assert with message "step 1" will fail.
It never fails with "step 2" msg.

2.
slApp.Find.Strategy = FindStrategy.AlwaysWaitForElementsVisible;
Assert.IsTrue(slApp.Find.ByName("UsernameLabel").TextContent.Equals("User name:"));

The assert above fails totally randomly...and very often.

we did try .Wait.ForExists(15000) on text box. Maybe % of successful asserts will be higher than in solution 2.
If it mean something this is login screen located in telerik window control. We are using latest internal telerik build.

answered on 07 Sep 2009, 10:58 AM

Hi guys,

Your scenario indeed sounds like a synchronization problem. You can actually wait until the text gets updated the same way as you assert that. Here is an example that should help you implement that for your scenarios:

    
            Wait.For<TextBlock>(block => block.Text.Equals("my text"), myTextBlock, 2000);

This Wait.For call will internally refresh the TextBlock element until the timeout passes. You may need to play with the timeout value as you will get exception if the condition (Wait.For() the first argument) is not met in time (the timeout is in milliseconds).

Also, I assume you need that to run your tests with WebAii Testing Framework, correct? That won't work in WebUI Test Studio (I checked our recorded and found you guys have tried the test studio as well) so let me know if you need it with the coded tests there too.

Hope this helps!

Kind regards,
Konstantin Petkov
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

answered on 07 Sep 2009, 12:11 PM

Hello Konstantin,

    
            TextBlock UsernameLabel = slApp.Find.ByName<TextBlock>("UsernameLabel");  
        
            Wait.For<TextBlock>(block => block.Text.Equals("User name:"), UsernameLabel, 20000);  
        
            Assert.IsTrue(UsernameLabel.Text == "User name:", "step 1");

Test above semas to work always. We will let you know how does it work when we have more tests and after more test runs.

Does the testing as described above seams little inconvenient and coding overkill?

We did sell nice story to the client about WebUI Test Studio and possibility to record test by just clicking and how that will play nice with your controls we are using…
As we are not able to show recording of functional tests (You promised support for that in q3 release, for SL) we must show something so we want to code few tests by hand in order to show at least something to the end client.
It would be nice if you have at least some unofficial / unsupported version of WebUI Test Studio for Silverlight... we could help you test it ;)

Dario Djurica
Senior Software Developer
Levi9 Global Sourcing

answered on 07 Sep 2009, 12:23 PM

Hello Dario,

I'm glad my suggestion helped. Let me know if you need further assistance with that.

I really hope we will manage to announce a Beta of WebUI Test Studio 2.0 (including the Silverlight recorder and Translators) earlier than Q3 (Q3 release target is at the beginning of November). As soon as we have a stable build of the Silverlight recording as well as all the other features Telerik and ArtOfTest currently work on, we will surely publish that.

We indeed realize it's always best to get customer Beta feedback and consider updates for the official version accordingly. Thanks for your interest!

Kind regards,
Konstantin Petkov
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

answered on 14 Nov 2009, 12:17 AM

This is my first Silverlight Test and
I am getting this exception ,

Test method WebAiiTest.WebAiiVSUnitTest1.FirstSilverlightTest threw exception: System.TimeoutException: Wait for condition has timed out.

My Test method:

       [TestMethod]
        public void FirstSilverlightTest()
        {
            //Launch New Browser
            Manager.LaunchNewBrowser(BrowserType.InternetExplorer);

            // navigate to the page
            ActiveBrowser.NavigateTo("http://localhost/SilverlightTestApp/SilverlightTestAppTestPage.aspx");

            // Get an Instance of Silverlight Application
            SilverlightApp app = ActiveBrowser.SilverlightApps()[0];

            // Set the Text of the Text Box
            app.FindName<TextBox>("txtYourName").Text = "Dude";

            // Click the Button
            app.FindName<Button>("btnClick").User.Click();

            //Now Verify The Text
            Assert.IsTrue(app.Find.ByText("p:Hello").Text.Equals("Hello Dude"));

        }

App Config:

<section name ="WebAii.Settings" type="ArtOfTest.WebAii.Core.SettingsConfigSectionHandler, ArtOfTest.WebAii, Version=2.0.5.0, Culture=neutral, PublicKeyToken=4fd5f65be123776c"/>
</configSections>
<WebAii.Settings enableSilverlight="true">

</WebAii.Settings>

answered on 16 Nov 2009, 08:07 AM

Hello Nikhil,

Can you please elaborate more on the issue. Which line from the below test throws the error? Does the test launch the browser and loads the Silverlight application or does it fail on the very fist action?

Have you enabled Silverlight automation via the appropriate setting as explained at the document here (check the bolded note at the bottom)? Please also review the list of limitation at the same document and let us know if you need further assistance.

All the best,
Konstantin Petkov
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

answered on 11 Dec 2009, 08:54 PM

Hi Nikhil,

As Konstantin mentioned, we will need more information: Which line from the below test throws the error? Does the test launch the browser and loads the Silverlight application or does it fail on the very fist action?

Also, have you tried using the period (".") after localhost, as in:

ActiveBrowser.NavigateTo("http://localhost./SilverlightTestApp/SilverlightTestAppTestPage.aspx");

And are you using your own custom Silverlight controls that inherit from the Silverlight 2/3 controls? And do you have more that one Silverlight app on the page or is the Silverlight app in a frame?

The last line in your test code might also need a refresh of the framework visual tree before the assert as in:

//Now Verify The Text
System.Threading.Thread.Sleep(5000); //Wait for your Silverlight app to update field
app.VisualTree.Refresh();
Assert.IsTrue(app.Find.ByText("p:Hello").Text.Equals("Hello Dude"));

Best wishes,
Nelson Sin
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

answered on 11 Dec 2009, 09:40 PM

Hi ,
Please find my answers inline

Hi Nikhil, As Konstantin mentioned, we will need more information: Which line from the below test throws the error? Does the test launch the browser and loads the Silverlight application or does it fail on the very fist action?

-- app.FindName<TextBox>("txtYourName").Text = "Dude"; throws the error
-- yes the test launches the browser and loads the sl app

Also, have you tried using the period (".") after localhost, as in:

ActiveBrowser.NavigateTo("http://localhost./SilverlightTestApp/SilverlightTestAppTestPage.aspx");
-- I tried this, "Wait for condition has timed out", occurs right over here when I apply the dot

And are you using your own custom Silverlight controls that inherit from the Silverlight 2/3 controls? And do you have more that one Silverlight app on the page or is the Silverlight app in a frame?
- No custom SL control, Just one SL App, SL App not in a frame

The last line in your test code might also need a refresh of the framework visual tree before the assert as in:

//Now Verify The Text
System.Threading.Thread.Sleep(5000); //Wait for your Silverlight app to update field
app.VisualTree.Refresh();
Assert.IsTrue(app.Find.ByText("p:Hello").Text.Equals("Hello Dude"));
-- It fails before hitting this code line

answered on 14 Dec 2009, 08:02 PM

Hello again Nikhil,

Can you loop through the list of text boxes found by using AllByType method and see if one of them is the text box you are looking for?

Going through the list may help you narrow down how the framework is finding the text box in question. The Text Box might be found just using part of the your XamlFindExpression as in either: new XamlFindExpression("Name=txtYourName") or new XamlFindExpression(""XamlTag=TextBox").

At the very least, if you can find the text box in the list and try setting the Text property to the string "Dude" and see if it sets the actual text in your app.

All the best,
Nelson
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.

answered on 14 Dec 2009, 08:28 PM

Hi Nelson,
IList<TextBox> l = app.Find.AllByType<TextBox>();
                l[0].Text = "Dude";
app.FindName<Button>("btnClick").User.Click();
TextBlock helloLabel = app.Find.ByName<TextBlock>("txtBlkDisplay");
                Wait.For<TextBlock>(block => block.Text.Equals("Hello Dude"), helloLabel, 50000);

                //Now Verify The Text
                System.Threading.Thread.Sleep(5000); //Wait for your Silverlight app to update field
                app.VisualTree.Refresh();
                Assert.IsTrue(app.Find.ByText("p:Hello").Text.Equals("Hello Dude"));

i am able to work with the above code, but the last line throws the error for wait condition
                Assert.IsTrue(app.Find.ByText("p:Hello").Text.Equals("Hello Dude")); , throws error

pp.FindName<Button>("btnClick").User.Click();

answered on 14 Dec 2009, 11:10 PM

Hello again Nikhil,

I'm glad to hear you got the code working. If you break after the list is populated and look at the TextBox at index 0, you can look at it's base Framework Element class.

There you can see the properties of the TextBox that the framework populates the object with, like Name, AutomationId, TagIndex, etc. Please craft your Find.By or XamlFindExpression according to these Properties.

Again, you can try partial find expressions like new XamlFindExpression("Name=txtYourName") or new XamlFindExpression("XamlTag=TextBox").

But if the framework does not populate the Name or AutomationID properties, then the framework cannot find these attributes for your Silverlight element. You can either use TagIndex or modifiy the element itself in order for the testing framework find the element using ByName or ById.

Regards,
Nelson
the Telerik team

Instantly find answers to your questions on the new Telerik Support Portal.
Watch a video on how to optimize your support resource searches and check out more tips on the blogs.