3 Answers, 1 is accepted
0
Hi George,
The purpose of a load test is to give you metrics for how well your application is behaving under various user loads, both light and heavy loads. It doesn't really give you a "pass/fail" answer. The only except is if you set goals for your load test to test. If the measured metrics do not meet your set goals then the load test will be marked Fail. Otherwise the load test will always be marked Pass.
Now having said that, why do you ask? Are you having some problem you need assistance with?
Regards,
Cody
Telerik
The purpose of a load test is to give you metrics for how well your application is behaving under various user loads, both light and heavy loads. It doesn't really give you a "pass/fail" answer. The only except is if you set goals for your load test to test. If the measured metrics do not meet your set goals then the load test will be marked Fail. Otherwise the load test will always be marked Pass.
Now having said that, why do you ask? Are you having some problem you need assistance with?
Regards,
Cody
Telerik
0
George
Top achievements
Rank 1
Rank 1
answered on 09 Jul 2015, 10:42 AM
Oh okay. I have another question. Why is it that when I load test starting with 1 user and ended it with 1 user, ramp up user count to 0, and run until 5 minutes, then set the average response time goal to 2500 ms, the average response time result is bigger than the result of 10 users?
0
Hi George,
Let me reiterate your inquiry to make sure I understand the question:
You run a load test with 10 VU's and get an average response time of X. Then you run a load test with just 1 VU and get an average response time of Y. Why is Y bigger then X if X was with more users?
That's impossible to answer definitively without having a close look at the data and the results. The one case that comes to mind to explain this is that when you run the load test with just 1 VU the HTTP transactions are being accepted and processed normally by your web application resulting in relatively long response times. But when you run the same load test with 10 VU's for some yet to be determined reason the web application is NOT accepting the HTTP transactions and is returning error messages instead which results in very short response times, thus a much smaller average response time.
If my theory is correct, I would expect you to be seeing errors in the web application log. Test Studio does not have access to this log (since it's remote, on the server). It also is unable to determine if the responses coming back are normal or error responses. Error responses normally come back as HTML with an error message contained in the text. Test Studio can't tell the difference between HTML with normal data and HTML with an error message since only the HTML/text is different. Thus Test Studio still indicates the load test passed (after all it ran and successfully sent all the HTTP requests to the server) even when the web application is erring out.
My advice in this case is to:
1) Have a look at the web application log looking for application errors being logged
2) Have a look at the Execution Server log. We want to look for faulted users which indicate a problem with the load test running.
Regards,
Cody
Telerik
Let me reiterate your inquiry to make sure I understand the question:
You run a load test with 10 VU's and get an average response time of X. Then you run a load test with just 1 VU and get an average response time of Y. Why is Y bigger then X if X was with more users?
That's impossible to answer definitively without having a close look at the data and the results. The one case that comes to mind to explain this is that when you run the load test with just 1 VU the HTTP transactions are being accepted and processed normally by your web application resulting in relatively long response times. But when you run the same load test with 10 VU's for some yet to be determined reason the web application is NOT accepting the HTTP transactions and is returning error messages instead which results in very short response times, thus a much smaller average response time.
If my theory is correct, I would expect you to be seeing errors in the web application log. Test Studio does not have access to this log (since it's remote, on the server). It also is unable to determine if the responses coming back are normal or error responses. Error responses normally come back as HTML with an error message contained in the text. Test Studio can't tell the difference between HTML with normal data and HTML with an error message since only the HTML/text is different. Thus Test Studio still indicates the load test passed (after all it ran and successfully sent all the HTTP requests to the server) even when the web application is erring out.
My advice in this case is to:
1) Have a look at the web application log looking for application errors being logged
2) Have a look at the Execution Server log. We want to look for faulted users which indicate a problem with the load test running.
Regards,
Cody
Telerik