Regulatory Testing—Differences in Approach and Scope

Regulatory testing requires specific planning, development, documentation and test case traceability. See how it differs in approach and scope from other testing, with medical device software as an example.

Software testing is a variable field involving learning and testing using a wide variety of techniques, disciplines and methods. Software applications impact our lives from providing entertainment in games, movies or sports to helping us drive or find transportation to managing our healthcare or keeping us alive by measuring and monitoring vitals.

Most software applications do not impact our lives as significantly as medical devices. Testing for software integrated with or as part of medical devices, or medical device software, requires extensive, planned and specific testing. Testing ensures the product meets or exceeds government regulatory requirements. Regulatory testing is required and means a deeper test dive into the inner workings of the software and the device.

In this article, we’ll discuss the differences in testing approach and scope regulatory testing entails. Additionally, we’ll cover the importance of design, development and testing documentation to enable regulatory compliance. Regulatory testing is unlike any other software testing adventure in its need for detailed, documented and thorough testing. What are the differences? Why is traceability critical for regulatory test case documentation? How does a testing team plan for or approach testing medical device software or integrated medical devices?

Key Takeaways

• What is regulatory testing?
• Why is testing different for software that’s deemed a medical device?
• Does the Agile methodology work for regulatory testing?
• How to approach testing medical device software?
• Find out why design and development documentation are critical for regulatory testing.
• Why is traceability for test cases important for regulatory testing?
• Learn how testing differs for software that must comply with FDA regulations.

This guide provides high-level information on regulatory testing for medical device software and how regulatory testing differs in approach and scope.

What Is Regulatory Testing?

Regulatory testing ensures that an application meets a legal set of standards and requirements.

For example, medical device software must meet U.S. government regulations from the Food & Drug Administration (FDA). The FDA is a government entity that exists to ensure the safety and security of drugs, medical devices, food and biological products. Not only are there specific government requirements to meet, but also international regulatory requirements. For example, one major regulatory standard is the IEC (International Electrotechnical Commission). Software application organizations use the IEC regulations as a benchmark standard to ensure products meet the established standards.

Regulatory standards and legal compliance rules exist to protect the public from bad software. Software technology continues to expand rapidly into all forms of modern healthcare. Software for modern healthcare systems includes simple forms such as step trackers up to critical life support systems and patient monitoring applications. The current market value for medical devices in the US is at or above $455 billion as of 2022.

Regulatory testing is critically important for the health and safety of all of us. Consider the increase in medical device software within the past five years. Many devices consist of only software and are often created by software development organizations without medical device industry experience. Organizations new to the medical device software industry are often unfamiliar or unaware of the governing regulations required. Regulatory requirements protect public safety by providing a level of regulatory testing to ensure software applications meet or exceed standards before products hit the market.

Why Is Regulatory Testing Conducted Differently for Medical Device Software?

Medical device software and medical devices with software fall into different categories of FDA regulations in the US. Regardless of which category the software falls into, the reason regulatory testing is conducted differently for medical device software is because poor quality software can harm patients directly or indirectly. Medical device software is expected to meet the minimum regulatory standards and requirements for patient safety.

Meeting the minimum legal requirements is typically not enough for an application to succeed. Healthcare software popularity is built on trust that develops over time. Trust enables doctors, nurses and other medical staff to use the medical software as intended. Medical personnel must trust the software works properly before they’ll go all in and use it.

Regulatory testing for medical device software must satisfy legal requirements but also the requirements of medical staff users. Satisfying medical staff users is a tougher task than it sounds. Medical personnel are responsible for patient safety and care. Software applications that have bugs or inconsistencies do not inspire trust regardless of the priority of the defect or bug.

Granted, most software application testing must satisfy users, but not to the extent and life or death reasons as medical software regulatory testing. Regulatory testing for software and devices ensures patient safety isn’t compromised by application errors.

The business reason for medical device and medical software application providers is undetected defects in software applications have reputational impacts. Creating trust in the medical provider market is crucial to application success. Without trust, the application is abandoned, unused and reviewed negatively by peers. A negative reputation for medical software adversely affects a business’s reputation and revenue.

Does the Agile Methodology Work for Regulatory Testing?

The short answer is no, Agile methodology does not work well for Regulatory testing. Why? Although it’s possible to use Agile boards to track work tasks effectively, the variances in tasks and timelines make it difficult to track at Agile speed.

Additionally, documentation of design, development and testing is typically minimal in Agile. Detailed and complete documentation of all aspects of medical device software is required to meet regulations and standards.

Most software development shops find themselves trying to meet compliance without credible documentation. Don’t have thorough and complete documentation? No deal. The application will not pass regulatory and compliance testing without re-engineering full documentation.

The Agile methodology typically involves development iterations of various lengths. Although the methodology works well for most software application development, it falls flat for regulatory testing.

Regulatory testing is an extremely detailed, specific set of testing techniques that explicitly test application functionality. The testing cannot be rushed, reduced, skipped altogether or substituted with coded unit tests. There are no shortcuts to regulatory testing. It is exact, detailed, thorough, and occurs in a regular cadence.

Regulatory testing also requires a complete product with integrated pieces and parts all in place before testing begins. Testers can’t test using simulated data sets or mocked API or database connections. The system must be in a fully developed state for regulatory testing.

How to Approach Testing Medical Device Software

Prepare, plan and strategize. It’s critical to understand which FDA guidelines the software falls under as the regulatory details are different for each. Additionally, the development team must understand what other regulations apply to the software including international standards. Before any coding or design occurs, it must be determined what the software classification is and then plan for meeting the appropriate regulatory goals.

Medical devices or medical device software cannot be used in any healthcare setting for information management, workflow automation or used in integrated medical devices unless the software has been approved and verified and is accompanied by complete documentation.

How to plan an approach for regulatory testing? Start with a project or strategic plan first. The project plan defines the software type and the required regulatory standards that must be met and the documentation needs. Next, ensure the development team understands the documentation required before coding begins. Keeping up with the documentation is essential to avoid re-engineering documentation or excessive re-work before the application is released.

Create a thorough test plan including a traceability method and define the required documentation needed. Traceability proves the requirements were tested and provides evidence for meeting regulatory requirements. Traceability is critical to proper and effective documentation for regulatory testing.

Next, decide on the approach for software requirement verification and test design implications. Include how testers conduct design and requirement analysis to ensure requirements are credible and testable. Always include developers in testing conversations and planning to create testing approaches for improved test accuracy.

Additionally, plan for developer code reviews and unit testing. Unit tests provide a handy form of repeatable testing that can be executed during development to detect bugs early in the process. Remember to track each execution and store the results as documentation.

Plan for testing software verification and validation testing of all functionality and requirements. Again, track all test execution and results for regulatory documentation. Plan and schedule full regression testing for each release candidate build or in specified time increments until the application code is released. After regression testing comes the specific regulatory compliance testing for the product based on the previously identified standards. Verifying the software application meets all specified regulatory requirements takes a significant amount of time and expertise.

Other testing to include and plan for:

• Medical data exchange interoperability and functionality
• Medical data exchange security
• Medical system interoperability for integrated applications
• Full system security testing

Regulatory testing requires planning, documentation and traceability. Regulatory testing for medical device software or medical devices proves a software application complies with legal regulations and operational standards and is not only ready to use but is safe to use. Approach regulatory testing with a project plan and thorough test plan. Traceability and complete documentation are required—make sure the project and test plan specifically include detailed instructions on providing accurate documentation and traceability.

Need help organizing test cases for regulatory testing? Consider tools to make managing regulatory testing both efficient and effective. Testing tools like Test Studio leverage the latest in testing technology for creating, managing and executing regulatory and compliance testing.