Load Testing and Performance Testing Results Analysis

[Mercedes]

Is there any documentation on how to analyze the load and performance testing results from the load and performance testing.

Thanks,
Nithin

[Byron]

Hi Brian,

Please review our Analyzing Results load testing guide. This document explains the different views in our load testing results and describes the meaning of each statistic included in those results. Please let us know if we can assist you further.

Regards,
Byron
the Telerik team

[Cairo]

Hello Byron,

Same question here. Do you have others documents, I mean a deeper walkthrough that will help analyze the results of test studio load test? Our project had just purchased a license of test studio which we will use in load testing our application. Right now, I'm facing challenges in analyzing the load test results.

One of the challenge I'm facing right now is that on the Page Specific Metrics of the result, there is only one URL that was returned wherein fact there are multiple URL that was captured during the recording of my user profile (webtest). Can you please explain why this is happening.

Thanks. Hope to hear from you soon.

Cairo

[Byron]

Hi Cairo,

I am working on creating a new in-depth document for analyzing load. Can you provide a screenshot of the load test behavior you are describing? 

Regards,
Byron
Telerik

[Cairo]

Hello Byron,

I've attached the two screenshots of the actual load test run. I've captured from an existing webtest and returned multiple URL responses (this is as expected,right? since my scenario does series of actions and transactions) - please refer to User Profile attachment. Then after executing the load test, I've ended with a single URL response - please refer to Page Specific Metrics attachment.
 
Why am I getting this behavior? Hope to hear from you soon.

Thanks,
Cairo

[Mark]

I just started using the Load testng aspect of the tool as well and noted the same issue when looking at the results.
I upgraded to the latest version of TTS and noted that the results now appear to contain averages for all of the pages that were hit.

[Byron]

Hi Cairo,

I'm sorry you're running into this behavior. Did the test create a large proportion of faulted users? You can select this as a metric in the chart view. If so, and if you enabled one or more dynamic targets for your load test, you may be experiencing a known issue with 2013 R1 load tests. To resolve this behavior, please install this custom build, and let us know if this resolves this behavior. 

If this does not describe your situation, please attach a copy of your test project  so we can attempt to reproduce your behavior.

Regards,
Byron
Telerik

[Cairo]

Hello Byron,

BTW, we are using 2012 R2 version.

Did the test create a large proportion of faulted users? To answer this question, YES - I've executed several load test runs and I'm still getting the same behavior wherein there is only a single URL response displayed under Page Specific Metrics and so with the number of faulted users. Am I missing something here?

Thanks,
Cairo

[Cody]

Hello Cairo,

I'm stepping in to assist with your load test problems so that Byron can return to his documentation duties.

Faulted users is a very bad sign. A faulted user is one that never finished running through all the steps of the user profile. It can explain why you only see one URL  in the page specific metrics. We need to find out why you're getting faulted users. The next step is to enable logging and view what the logs tell us. Please follow the steps at the last half of this page. Locate the log file and attach that to this ticket. Place it into a .zip file first.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Why am I getting large number of faulted users? What's the reason behind this behavior? I'm attaching the trace log generated when I executed a sample load test run wherein only a single URL response is displayed on Page Specific Metrics.

Thanks,
Cairo

[Cody]

Hello Cairo,

I apologize for the delay getting back to you on this.

I looked at the trace log but I don't see anything abnormal. All of the content indicates normal load testing. We need to dig deeper to figure out what's going on. To that end I'd like to look at this problem on your computer via GoToMeeting. Let me know what timezone you're in and your available and I'll setup the meeting. I'm located in Seattle, WA, which is GMT -8.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Thanks for looking at tracelog. By the way, I'm from Manila, Philippines (UTC/GMT +8). I'm available around 8AM Manila time onwards till 11PM. Please let me know of your preferred meeting date and time. Thanks again. I think the call will help us a lot in determining what causes the issue.

Thanks,
Cairo

[Cody]

Hi Cairo,

All right let's try your 10am, my 7pm.

1.  Please join my meeting, Wednesday, August 21, 2013 at 7:00 PM Pacific Daylight Time.
https://global.gotomeeting.com/join/611502077

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 611-502-077
Audio PIN: Shown after joining the meeting

Meeting ID: 611-502-077

Regards,
Cody
Telerik

[Cody]

Hello,

My apologies for not showing up on time. I accidentally got side tracked. I am online now waiting for you. I'll standby for a while hoping you get this message in soon.

Regards,
Cody
Telerik

[Cody]

Hi Cairo,

Sorry we were unable to connect today. Your 10am on most days works best for me. Let me know which day will work for you and I'll setup another meeting.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Apologies for not being able to join the meeting. I just read your email when I got to the office today. How about Tuesday, August 27, 2013 at 10AM Manila Time, that would be 7PM on your side. Please let me know if this will work on your side.

Thanks,
Cairo

[Cody]

Hi Cairo,

That's my Monday evening, your Tuesday morning. I can do that:

1.  Please join my meeting, Monday, August 26, 2013 at 7:00 PM Pacific Daylight Time.
https://global.gotomeeting.com/join/148078837

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 148-078-837
Audio PIN: Shown after joining the meeting

Meeting ID: 148-078-837

Regards,
Cody
Telerik

[Cairo]

Thanks Cody. Yes, we'll meet on Monday (your Monday).

[Cairo]

Hello Cody, Can we move the meeting at 8 PM(your time today)? Thanks, Cairo

[Cody]

Hello,

I just got your message. Yes that will work. I already starting the meeting and am waiting for you.

Regards,
Cody
Telerik

[Cody]

Hi Cairo,

I now believe you meant to move the meeting one hour later. I'm sorry you're not aware I'm now in Austin, TX, (GMT -6) such that one hour later would be my 10pm (I was in Seattle for only the last week). I am sorry but that's too late for me. Tomorrow evening will be hard for me. My 9pm, your 10am is the earliest I can meet tomorrow evening. Do you want to try then?

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

I'm sorry for the confusion, sure we can meet at 9PM (my 10 AM) tomorrow. Please send a meeting invite.

Thanks,
Cairo

[Cody]

Hi Cairo,

Sure here's the new invite:

1.  Please join my meeting, Tuesday, August 27, 2013 at 9:00 PM Central Daylight Time.
https://global.gotomeeting.com/join/752220989

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 752-220-989
Audio PIN: Shown after joining the meeting

Meeting ID: 752-220-989

Regards,
Cody
Telerik

[Cairo]

Hello Cody, it's almost 10 PM here but I'm still in the office, if you want we can jump into the meeting right now? Do you have a  personal email so I could send you a message thru email?

[Cody]

Hi,

You're in luck. I can do that. I just started the meeting and am online now waiting for you. Use the link from my last email to join.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

As discussed earlier, attached here are the fiddler result and tracelog of the load test execution.

Thanks,
Cairo

[Cody]

Hi Cairo,

We found what is causing the faulted users. Your website is setting this cookie:

ctl00_NavigationPaneItems={'SelectedItems':['1'],'ExpandedItems':[]}

We are using .NET for our cookie bag in our load test code, and .NET doesn't allow the , character in the values for cookies. Even though most browser do work with cookies like this, technically it is not valid according to accepted HTTP standards.

I'm afraid there is no workaround within Test Studio to this. Your only option to continue trying to use Test Studio load test is to "fix" your website in this area e.g. URL encode the cookie being set by the server.

I can still meet with you at your 10am.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Thanks again for your immediate response. I just have another question here, where can I find this cookie you are mentioning here?
ctl00_NavigationPaneItems={'SelectedItems':['1'],'ExpandedItems':[]}

Is it in the fiddler trace or the tracelog?

Thanks,
Cairo

[Cody]

Hi Cairo,

First I promised you the video from our first session. Here it is:
https://dl.dropboxusercontent.com/u/12886581/2013-08-27%2008.49%20654262%20Load%20Testing.wmv

To actually see the cookie being set is probably easiest using Fiddler. If you run Fiddler then access your application normally by launching a browser (not using Test Studio) you'll capture the HTTP traffic between the browser and your web server. All you need to do is capture the initial sign on screen plus logging on. Then look at the responses coming from the server. Look at the SetCookie's as shown in the attached screen shot.

P.S. We saw the cookie in the background communication of our load test running, which only we would be able to understand. Your web developers will better see and understand what Fiddler captures.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Thanks for sending the copy of our session.

First of all, I would like to validate if the screenshot you attached below is refering to SetCookie's you are mentioning?
Second, we've already communicated the fix you've suggested to our web developers and they implemented a code below on our application to address the issue.

<telerik:RadPanelBar runat="server" ID="NavigationPaneItems" Width="100%" PersistStateInCookie="True"

                                OnClientItemClicked="RadPanelBar1_OnClientItemClicked" OnClientItemClicking="RadPanelBar1_OnClientItemClicking"

                                OnItemClick="RadPanelBar1ItemClick" CausesValidation="false">

As per checking, we're still getting the same issue on the load testing result wherein a single URL is still returned.
I wasn't able to inform you also that our application will be completely hosted over HTTPS.

If you will be available today, I'm not sure though what time on your end, can we meet so that I can discuss the results again. We'll be willing to extend today just to catch up with you just leave me a message so I could reply asap.

Thanks a lot,
Cairo

[Cody]

Hello Cairo,

I would like to validate if the screenshot you attached below is refering to SetCookie's you are mentioning?

The screen shot I attached was just an example of where/how to find SetCookie in the traffic. It's not of the cookie we believe is causing the problem. For that we need a new Fiddler capture of running your application normally in a browser (without Test Studio). We won't see the problem cookie in Fiddler while the load test is running because our load agent is crashing at the time it's building the next HTTP request to be sent to the server. It never got sent. Since it wasn't sent there was no HTTP request for Fiddler to capture.

I am sorry I was not available to meet with you at the time you requested. I am busy late this evening and Monday is a national holiday in the US.

I think the easiest way to proceed, now that your developers have implemented a code change, is to ask you to capture a new Fiddler trace of your application running in just a browser and send us that trace. We'll study this new trace to see if there's still a cookie problem or not.

Based on the results of us studying the new Fiddler trace, we'll can decide what is the next step we need to take.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Thanks for your reply. Attached here is the new Fiddler trace of our application running in just a browser and please let us know of your findings. If you are available tomorrow or day after, can you please set up another meeting with us so we could discuss this in a call?

Thanks again.

Cairo

[Cody]

Hi Cairo,

Thank you for the new Fiddler trace. The changes made by your developers looks good. The , character no longer appears in the Set Cookie, so we should be good there.

You'll need to record a new load test now. The old load test still has the , character in it and it won't work, so go ahead and delete it.

I can meet in 45 minutes for a little while (no more than 30 minutes this time).

1.  Please join my meeting, Tuesday, September 03, 2013 at 8:30 PM Central Daylight Time.
https://global.gotomeeting.com/join/823444981

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 823-444-981
Audio PIN: Shown after joining the meeting

Meeting ID: 823-444-981

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Apologies for not meeting you today since I just read your email. Can we have the meeting tomorrow, around 9:00 PM Central Daylight Time)?
BTW, I re-executed another load test and found the issue is still occurring. Hopefully we could discuss this tomorrow on a call so I that I can show you the result of the load test run. I've also attached another Fiddler trace of the test execution today.

Thanks,
Cairo

[Cairo]

Hello Cody,

Apologies for not meeting you today since I just read your email. Can we have the meeting tomorrow, around 9:00 PM Central Daylight Time)?
BTW, I re-executed another load test and found the issue is still occurring. Hopefully we could discuss this tomorrow on a call so I that I can show you the result of the load test run. I've also attached another Fiddler trace of the test execution today.

Thanks,
Cairo

[Cody]

Hi Cairo,

I don't know why... the new Fiddler trace doesn't show any actual traffic being sent to your web server from the load test. There is only a lot of background overhead communication in the trace.

Can you also send me the application log like you did before (just like the Tracelog sent on 27-Aug-2013)?

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

The Fiddler trace I attached here is the result I got when I executed a sample load test run. However I am attaching the TraceLog you are requesting.

[Cairo]

Hello Cody,

The Fiddler trace I attached here is the result I got when I executed a sample load test run. However I am attaching the TraceLog you are requesting.

[Cody]

Hi Cairo,

Thank you for the trace log. I'm consulting with our load test developer now. I'll get back to you shortly with what our next step needs to be.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Do you have any updates on this?

Thanks,
Cairo

[Cairo]

Hello Cody,

If this will not be resolved, do you have any other alternatives on how we can capture the URL pages that failed on a load test? Are there any other ways on how we can pinpoint which pages responded the slowest?

Thanks,
Cairo

[Cody]

Hi Cairo,

I apologize for the delay. (it took me a while to get my software developer to study this with me).

The log indicates a problem with Dynamic Targets. Did you create a new load test and try running that, or did you run the old one you recorded before the website got changed? We would expect this if you wan the old trace. Make sure you have deleted it.

I can meet you online tonight:

1.  Please join my meeting, Tuesday, September 10, 2013 at 9:00 PM Central Daylight Time.
https://global.gotomeeting.com/join/185658117

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 185-658-117
Audio PIN: Shown after joining the meeting

Meeting ID: 185-658-117

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Can we have the meeting tomorrow, same time (Wednesday,September 11, 2013 at 9:00 PM Central Daylight Time).
And by the way, I've created a new load test, did not select any dynamic parameter during the design and after executing the loadtest I found out that is displaying the URL pages that were captured. However, the URL pages seems to have plenty of errors (almost all errors are 401 Unauthorized), you may refer to the attached screenshot. This issue is really really impeding us from performing the loadtest execution.

Please let me know if you are available tomorrow for the call.

Thanks,
Cairo

[Cody]

Hi Cairo,

A lost test with a lot of 401 HTTP response codes is normal and benign in a load test. They happen because of the way load tests work. Load tests do not keep any of the connections open when it makes a new HTTP requests. So effectively what happens over and over again is this sequence:

1. Open a new connection
2. Send the request to the server
3. Server responds "401" basically asking "who are you"
4. Load test responds with the Windows authentication credentials
5. Server responds 200 OK
6. Close connection

Because of step 6, the close connection step, the web server will insist on re-authenticating every time we open a new connection. We realize this is not the most efficient way of communicating to a web application that requires Windows authentication, but it does get the job done. The fact that you do get a new case created in your application is proof that the authentication is working.

We need to look past the 401's to determine what the real problem is.

1.  Please join my meeting, Wednesday, September 11, 2013 at 9:00 PM Central Daylight Time.
https://global.gotomeeting.com/join/977495021

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 977-495-021
Audio PIN: Shown after joining the meeting

Meeting ID: 977-495-021

Regards,
Cody
Telerik

[Cody]

Hi,

Any chance we can start early? I have somethings I'd like to do later if possible. I can start now if you're available.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

I am very sorry for not making to the call today. I really had to attend to very important personal matters this morning. However, I'll be staying up late tonight just so I could catch up with you as soon as go online tomorrow.

Thanks also for you reply. I'll be investigating the error today and will let you know of my findings. I'll be waiting for your reply.

Thanks,
Cairo

[Cody]

Hello Cairo,

I will let you know when our GoToMeeting account is available. At the moment it's being used by another support technician for another customer (we can only do one session at a time with our account).

Regards,
Cody
Telerik

[Cody]

Hello,

I am online now. I'll wait 15 minutes.
1.  Please join my meeting.
https://global.gotomeeting.com/join/397145757

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 397-145-757
Audio PIN: Shown after joining the meeting

Meeting ID: 397-145-757

GoToMeeting® 
Online Meetings Made Easy®

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Im joining the meeting now.

Thanks,
Cairo

[Cody]

Hello Cairo,

Do you want to try again on Monday? I can my 10am (your 11pm) or my 8pm (your Tuesday 9am). Let me know what you'd like and I'll send out a new invite.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

I would like to have the meeting right now but I'm having troubles connecting on our servers. My apologies for the delay on my reply.

Thanks,
Cairo

[Cody]

Hi Cairo,

I am so sorry we keep missing each other. Unfortunately my normal office working hours are 9am -5:30pm. I received your latest request at 8:10pm, and I just wasn't ready/watching for it. In the past I would be expecting to meet and would be sitting at home waiting for it.

Let's try a different tactic instead.

Describe to me the technical problem you seem to be running into, and send me the following pieces of information:

• A copy of the load test. This is a .tstest file on disk, found in the test projects folder.
• A Fiddler trace of the load test trying to run. Make sure you select Start at 1 user, end at 1 user.
• A Fiddler trace of the application running normally in a browser, no Test Studio involved.

Given all of the above information I should be able to diagnose what's causing the problem.

Regards,
Cody
Telerik

[Cairo]

Hello Cody,

Same here, my apologies too. BTW, Im still in the office and if you want we can have a short call now before I go home.

Thanks,
Cairo

[Cody]

Hi,

All right let's see if we can connect this time:
1.  Please join my meeting.
https://global.gotomeeting.com/join/544916445

2.  Use your microphone and speakers (VoIP) - a headset is recommended.  Or, call in using your telephone.

Dial [phone]
Access Code: 544-916-445
Audio PIN: Shown after joining the meeting

Meeting ID: 544-916-445

GoToMeeting® 
Online Meetings Made Easy®

Regards,
Cody
Telerik

[Cody]

Hello Cairo,

To summarize, you had just a few additional questions which I was able to answer. Your load testing seems to be running just fine.

I am closing this ticket as resolved. Please create a new one for any new unrelated problems.

Regards,
Cody
Telerik