Have you wondered how the teams working on Telerik products test software? In the next chapter of our detailed guide, we give you deeper insight into the processes of our Business Services. You can find past chapters here.
You’re reading the fourth post in a series that’s intended to give you a behind-the-scenes look at how the Telerik teams, which are invested in Agile, DevOps, TDD development practices, go about ensuring product quality.
We’ll share insights around the practices and tools our product development and web presence teams employ to confirm we ship high quality software. We’ll go into detail so that you can learn not only the technical details of what we do but why we do them. This will give you the tools to become testing experts.
Important note: After its completion, this series will be gathered up, updated/polished and published as an eBook. Interested? Register to receive the eBook by sending us an email at email@example.com.
Register for the eBook Now
The Web team is the gatekeeper for all business-critical applications. They are responsible for applications related to billing, authentication, UI, web administration and the shopping cart. The failure of any of these systems can cost us business. Applications for which Business Services is responsible run across all of Progress, and include:
The QAs from Business Services begin participating in the test development processes as early as possible, taking part in all important requirements meetings. Time is set aside to refine and rework requirement documents, and clear up any misunderstandings. After refining the requirements, the team provides estimations for development and testing schedules, delivering a written Project Scope Statement.
The primary source control system of the team code is Microsoft Team Foundation Server (TFS). For code quality of production and tests, we use a static analyzer called “Style Cop.” Check-ins are not allowed until all warnings are fixed.
The next level of defense is the peer or guide code reviews; to enforce them, every member of the team has specific TFS check-in policies installed.
After that, the CI Builds get the source code from version control, updating binaries’ versions, compiling the code, running tests and outputting the resulting binaries to a set folder. The CI builds are TFS build definitions with a "Rolling Builds" trigger set without any additional restrictions, as they are independent of any other part of the release automation process. As such, they are designed to have the full capability to run as often as possible, so long as there are code changes to warrant starting a build, and available build and test agents.
Each of the Create Package (CP) jobs handles taking build output for a given application, creating a deployment package for that application and deploying it to the environment in its first stage. Each of the jobs runs a script with preset parameters. The script is a batch file that processes the contents of a specified folder and uses AppDirector's command line tool to create deployment packages for each subfolder. The job polls the contents of a pre-set folder on a schedule (every 30 minutes) and runs.
For application deployment, we use an internal tool called AppDirector. AppDirector manages deployment pipelines for multiple applications and spanning multiple environments. After a package is deployed, the AppDirector triggers a specified Jenkins build responsible for locking the environment (no other deployments are allowed until unlocking) and runs all E2E/System/UI tests. If there are failed tests, promotion to next environments is not permitted.
Continuous Delivery is about keeping your application in a state where it is always ready to be deployed into production. Here is a diagram that shows an example of the Continuous Delivery pipeline:
Here all the pieces of the application come together in a running instance of the application. The set of unit tests is regularly run with each check-in. If the tests fail, the build is discarded. Otherwise, it is automatically promoted to System Testing environment.
Here all integration tests are executed. If the tests fail, the build is discarded. In case all tests pass, the build is automatically promoted to systems integration testing.
During system integration testing, QAs perform exploratory testing, running all UI automated tests and test suites. Successful builds will be promoted to the user acceptance testing environment.
In this testing environment, the user stories are validated by project managers as to whether they pass their acceptance criteria. Acceptance tests/criteria are usually defined by business customers or business analysts, but written in the story by the responsible SCRUM team master and validated by PMs. These are high-level tests to verify the completeness of a user story or stories “played” during any sprint or iteration.
Ideally, these tests are created through collaboration between business customers, business analysts, testers and developers, and must include both business logic tests and UI validation elements. The business customers, or product owners, are the primary project stakeholders; as the user stories pass their acceptance criteria, business owners know developers are progressing in the right direction. The application then moves to the Production environment manually, and the package upload process is triggered by the release managers. We perform around 10 builds per day.
Once the application approval is done the application then becomes live. There’s a new upload once a week.
For test case management, Business Services uses Microsoft Test Manager. All test cases are stored as TFS items. Most of the time, we don’t use Microsoft Test Manager directly; rather, test cases are written/read through Test Case Manager, an application that extends the MS Test Manager. The tool speeds test case writing and maintenance. Over the last year and a half, we’ve created more than 4,700 test cases with more than 1,200 shared steps.
Test Cases Report through Test Case Manager:
Test Cases generated by test runs- e.g. Dynamic Test Cases:
Dynamic Test Cases Web Dashboard (powered by Kendo UI):
Next time, we'll talk about incident management and area of testing within the Telerik by Progress Business Services.
Did you find these tips helpful? Stay tuned for the next chapter and let us know your feedback. And don't forget to register to receive the eBook that will be put together when this series is complete.
If you are interested in trying the latest Test Studio, feel free to go and grab your free trial today.
Anton is a Quality Assurance Architect at Progress. He is passionate about automation testing and designing test harness and tools, having the best industry development practices in mind. He is an active blogger and the founder of Automate The Planet. Ardent about technologies such as C#, .NET Framework, T4, WPF, SQL Server, Selenium WebDriver and Jenkins, he won MVP status at Code Project (2016) and MVB (Most Valuable Blogger) at DZone.
Subscribe to be the first to get our expert-written articles and tutorials for developers!