Have you wondered how the teams working on Telerik products test software? In the next chapter of our detailed guide, we give you deeper insight into the processes of our web division. You can find past chapters here.
You’re reading the seventh post in a series that’s intended to give you a behind-the-scenes look at how the Telerik teams, which are invested in Agile, DevOps, TDD development practices, go about ensuring product quality.
We’ll share insights around the practices and tools our product development and web presence teams employ to confirm we ship high quality software. We’ll go into detail so that you can learn not only the technical details of what we do but why we do them. This will give you the tools to become testing experts.
Important note: After its completion, this series will be gathered up, updated/polished and published as an eBook. Interested? Register to receive the eBook.
Register for the eBook Now
Chapter Four: Telerik Developer Tools: Legendary Product Quality with Ever Faster Release Cycles
The Developer Tools Division comprises many teams, each with a specific product focus, including:
- WinForms: Responsible for development, support and quality assurance of UI for WinForms. The product goal is to allow companies and developers to rapidly build LOB applications with stunning UI.
- XAML: Responsible for development, support and quality assurance of UI for Silverlight and WPF. The team’s goal is to help developers build enterprise-grade XAML applications quickly.
- Kendo UI: Responsible for the development, support and quality assurance of the most complete UI framework for HTML/JS development for any platform, browser or device.
- NativeScript: Responsible for the development, support and quality assurance of the NativeScript framework, which helps you build native apps from a single code base.
- AppBuilder: Responsible for the development, support and quality assurance of AppBuilder development platform, which enables developers to create cross-platform mobile apps.
- AJAX: Responsible for the development, support and quality assurance of Telerik UI for ASP.NET AJAX, which enables developers to build web apps for any browser or device.
- iOS: Responsible for the development, support and quality assurance of native and customizable UI controls for building iOS apps.
- Android: Responsible for the development, support and quality assurance of native and responsive UI controls for building Android apps.
We use the Scrum methodology to plan, package and release software products. We ship a major version three times a year. Approximately a month after a major release we deliver a service pack with minor revisions. Additionally, some teams release internal builds weekly containing the latest bug fixes. This dynamic cycle determines our milestones before releasing a product and our testing cycle which guarantees the quality of our software.
For major releases we define two important timeframes:
- Feature freeze: Three weeks before the release date the development of new features and controls ends; QA starts testing functionality and logs bugs
- Code freeze: Two weeks before release date; developers fix only blocking and critical bugs related to the new features and logged from QA
For service packs, we define one milestone:
- One week before the release date is code freeze, during which QA tests the product and verifies that it can be released.
Definitions of “Complete”
An issue is resolved when the issue is logged correctly into TeamPulse/TFS. It must be set to “Ready for Test” by the developers and include short notes of tested cases during fixing. The code is then merged into all appropriate branches. A feature is complete when it’s logged correctly into TeamPulse/TFS and a functional documentation specification is present. Then the feature is implemented, and unit tests are performed if possible. A help article and/or SDK example should also be present. Test plans and functional tests are completed, if possible, by the QAs.
We have a policy to distribute an internal build at least once a week. Code reviews are performed per check-in and are mandatory for every team member. Every fix that will be included in the weekly internal build has the highest priority in the QA backlog for testing.
The robust CI process that we have implemented allow us to do this and guarantee an increase of quality with every new version.
We set up different environments to cover all major customer scenarios and cases. These include variations of OS, Browser, Visual Studio, CI tools, Source Control tools, TeamPulse, third-party integration tools, databases and so on.
Areas of Testing
The first verifications are developer-written unit tests. Developers write unit tests to ensure server-side code or is working as expected. They are triggered as part of the CI build for every supported framework, which happens after every single check-in.
Functional and Non-functional testing
We log and manage all found issues during testing. All issues from customer tickets and all feature requests are logged and managed in GitHub.
Functional tests are based on tasks created from developers and written in TeamPulse.
For example, for Kendo UI online demos we have a number of staging servers for the different platforms (JSP, MVC, PHP) and development versions (current version fixes or next version features). We also have a staging server with stubs (replication examples) of already fixed issues. Functional tests executed on these stubs make sure no regression will occur for already fixed issues. We maintain a set of functional tests of Kendo UI Mobile for Android that can run on a simulator or a real device. With the help of Test Studio team we develop and maintain Test Studio extensions for the Kendo UI framework that are of great help for Test Studio users who create functional tests for Kendo UI.
- Selenium tests: Selenium tests have been migrated to Test Studio, but we still have lots of them, as they are more suitable for testing some scenarios
- Regression testing: We maintain hundreds of automated tests, divided into subsets and executed on a daily or hourly basis.
Automated tools used include:
- QSF: Ensures every demo in our QSF is loaded properly without server errors and checks for XHTML errors on demos
- Fiddler: Runs over QSF examples and verifies that there are no errors and missing resources
- MSOCAF validation: Office 365 code analysis framework to validate the code before release
- Sikuli-based UI automation framework: Includes machine-friendly reports (for Jenkins integration) and human readable report, plus video log for easy analysis.
Manual Functional Testing
Not everything can be automated. In some cases, we perform manual functional testing. This is useful:
- Testing for new features, including random testing and exploratory testing for small features
- Finding UI glitches
- Testing install, uninstall, shortcuts, VS scenario templates in different environments such as every possible environment we have time for
Coming Up Next
Next time, we'll talk about continuous integration, API and stress testing within the Telerik Dev Tools division.
Register for the eBook
Did you find these tips helpful? Stay tuned for the next chapter and let us know your feedback. And don't forget to register to receive the eBook that will be put together when this series is complete.
If you are interested in trying the latest Test Studio, feel free to go and grab your free trial today.