Load and Performance Testing; What’s the Difference?

The difference turns out to be a lot! To start with load and performance testing goals are very different.

Load testing goals usually seek to answer questions regarding system capacity, throughput, and client response times for multiple concurrent users. For example, how many concurrent users can the system successfully accommodate? Or how many transactions per second can the system successfully process? Or can the system successfully scale to 1,000 users? Whereas performance testing goals seek to answer questions regarding server processing, and client response times for a single user. The primary goal of performance testing is to tune the system to achieve maximum performance, [e.g. execute a given amount of work (transactions) in the least amount of time (transactions per second (TPS)].

Load and performance testing are both challenging; however, performance testing is the easier of the two because only one user is tested. Whereas in load testing there are hundreds or thousands of concurrent users tested.

Additionally the implementation of performance testing is much less complicated because, again of the fact that only one user is being tested versus hundreds or thousands for load testing. Also, each user type in load testing must have a scenario designed, implemented, and modified, to execute and simulate live users as accurately as possible.

When modifying and enhancing test code for load testing the test engineer must consider concurrent users and adjust the test code accordingly. This will mean modification of session IDs, user form data, and input data, whereas in performance testing there is only one user and typically no session IDs, etc., need to be handled.

Because the performance test only involves one user, it is typically a much shorter test than a load test. Load testing involves hundreds or thousands of concurrent users and is usually a much longer test and thus much more data to be collected and saved.

When it comes to results analysis and correlation of test data, the performance test data volume is much less then the volume of load test data. Again, because performance testing executes only one user, thus there is much less data to be analyzed for performance vs. load testing, even though the same or similar metrics are collected.

During the remediation phase of testing, performance testing is a much smaller problem space than load testing. With remediation of performance tests, typically metric ‘outliers’ are identified, then the engineer drills down to determine root cause. Load testing remediation is not that straight forward or easy. Load testing results may show errors. However, there may be bottlenecks which are more difficult to identify and more difficult yet to remediate. So, it turns out there is a great deal of difference between load and performance testing.

Test Studio, from Telerik, has long been capable of performance testing. Further, any functional test can also be used as a performance test. To execute a performance test in Test Studio, there are just three steps: 1) Create or select an existing web functional test that satisfies the test scenario criteria, (e.g. user actions, and or transactions to be executed), and open that test, 2) configure performance test properties such as location to store results then select the computer to collect performance metrics from, 3) execute (run) the performance test.

With the latest Test Studio 2012 R1 release in April, load testing has been added to the capabilities of Test Studio!

Through load and/or performance vulnerabilities of software comes financial vulnerabilities of the company, which both directly affects company financials and customer satisfaction. Test Studio can help mediate these risks and reduce vulnerabilities.