You know that you have to demonstrate Usability Testing for your In Vitro Diagnostic Device (IVD) according to the In Vitro Diagnostic Device Regulation (IVDR), but you don’t know yet how exactly you have to prepare for it? Then this article is right for you.
We answer the following questions.
- How can you best prepare for the Usability Testing of your IVD?
- What do you need to do in order to lay the necessary foundation for testing?
- What do you have to consider when recruiting the respective user group?
- What do you need to know about the correct documentation?
If you read this article, you will find clear answers to all these questions and at the end you will be able to assess what is essential for a pragmatic and goal-oriented implementation of Usability Testing for your IVD.
What is required of you?
Since the introduction of the IVDR many fundamental things have changed for you as a manufacturer of IVDs. Whereas under the IVDD, approximately 20% of IVDs had to be approved by a Notified Body, under the IVDR, approximately 80% of products now have to be approved (source). The requirements for your product have also been extensively expanded. One of the most important changes is the required, although not explicitly named, proof of the Usability of your IVD.
So, what are you required to do with regard to the Usability of your IVD? You must carry out Usability Testing and Usability Engineering and provide proof of both with a Usability Engineering File in order to be able to approve your product via a Notified Body. Even for products that are already on the market, you must provide evidence of a root cause analysis using a Usability Engineering File.
The objective
Summative Usability Testing is essential to demonstrate the safe Usability of your product for clearance. Accordingly, we focus on summative evaluation in this text. Of course, it is also possible to test and iteratively improve your IVD in formative evaluations.
Summative evaluation tests whether specific users can use a product, system or service effectively, efficiently and satisfactorily in a specific context of use. This must not result in any harm to users, patients or third parties. Only if this is achieved does your IVD meet the Usability requirements of the IVDR.
Security risks during use are identified during testing, documented, and then the product is redesigned through Usability Engineering to ensure safe use as much as possible.
You can get an overview of how we do Usability Testing here. How Usability Testing for IVDs can look like in the specific case of Covid19 lay tests, you can read in our article “Usability Testing for Corona Self-Tests“.
The following things are relevant for Usability Testing of your IVD:
- The right preparation
- The right user group
- The right use environment
- The right equipment
- The answer to the question “Can sample collection be simulated?”
- The completeness and comprehensibility of your instructions for use
- The correct documentation of the testing
- The specific requirements of the IVDR for your type of IVD
Let’s look at these points in detail.
The right preparation
What do you need to have done beforehand in order to start Usability Testing?
- You must have already defined the Use Specification. That is: you must have defined the purpose of your IVD, who it will be used by, the environment in which it will be used, and how it will be used.
- The performance evaluation testing of your IVD must be complete.
- Your Instructions for Use must be complete for summative testing.
- The product must be in a testable version.
The right user group
To prove that your product is safe to use, you need to show that the user group you specify is capable of using it without critical errors. Accordingly, you must test the product with real users or show that you have already successfully tested products with equivalent usage and the same target group. It is therefore imperative that the participants always correspond to the target group of your IVD.
If you want to get clearance for your product for medical professionals, then you need to test with users with that same professional background.
It is somewhat more difficult with the lay application. Here, a representative cross-section of society must be tested. Accordingly, you need a large number of participants for your Usability Tests here. However, there may also be restrictions on the test group here. For example, a pregnancy test can only be tested with women of a certain age range.
The following applies to all tests: Start recruiting test participants as early as possible. This will help you avoid time bottlenecks shortly before the start of the test. Some target groups are more difficult to recruit than others. Think about which participants you need at an early stage.
Also remember to recruit more participants than you need in order to test efficiently in case of cancellations and illness. If you want to save yourself the effort of recruiting, you can hire external agencies. They often have access to a large database of possible participants from all user groups.
The right environment for use
Where will your IVD be used? According to the answer to this question, you need to simulate a realistic use environment for your Usability Tests.
Possible impairments of the user’s concentration must be realistically simulated here. This means: If your IVD is to be used in noisy places, you must depict this during the tests.
The right equipment
You should be able to answer the following questions for yourself in advance:
- What facilities do you have for testing? What premises and resources can you provide?
- What does a realistic environment for use look like? Is it noisy and hectic, or are there many people in the room?
- What are the steps involved in using the IVD?
- How do you want to realistically represent the use of your product for Usability Testing?
You do not necessarily have to take samples from real people during your tests. For many cases, sample collection can be simulated. This can be done on a patient simulator, for example, or on one of many different and sometimes very expensive dummies.
The following question is important for you: what kind of dummy do you need to simulate the use correctly? Make sure that the product is suitable for demonstrating what you want to demonstrate. For example, a certain part of the dummy can be color-coded to show during a test whether the user has hit the right area when swabbing.
Can sample collection be simulated?
In vitro diagnostic devices for trained personnel often allow simulated sample collection. IVDs designed for lay use, on the other hand, must be tested with sufficient lay personnel. Here, sample collection cannot be simulated realistically in many cases. As a result, your Usability Test becomes a clinical study. Accordingly, you need the approval of an ethics committee. You should consult external experts here, as preparation and implementation can quickly become very time-consuming.
Instructions for Use
Safe use under realistic conditions also means that the participants receive no further assistance other than the package insert for your product. Instructions for Use on the packaging are also relevant here. If you cannot demonstrate that your product is safe to use with the information provided, your Instructions for Use may need to be adapted.
It must also be possible to interpret the test results correctly with the aid of the package insert. Every test participant must be able to interpret every possible result without error. Of course, you cannot have each participant take a large number of tests that always show a different result. To demonstrate that participants can interpret all results, you can print photos of previously performed tests on cards and then present them for interpretation.
The right documentation
Notified bodies require not only the performance of Usability Testing, but above all complete proof of everything you have done with regard to the Usability of your product. Here you are obliged to provide a coherent argumentation about your approach. In other words, you must demonstrate that the steps you have taken result in your IVD being efficient to use and as risk-free as possible.
Accordingly, you need to be clear from the start about how you want to document your tests without gaps. This includes the following points:
- You must be able to justify your choice of methods.
- The Usability Test data must be made available.
- The Usability Engineering activities must be demonstrably carried out by personnel qualified to do so.
Particular focus should be placed on the Usability Engineering File. This contains everything you have done with regard to the Usability of your product and thus provides the evidence for the Notified Bodies.
If you are working with a UX agency, the agency can also support you in creating the Usability Engineering File. If you have only requested support for the study execution, then you will usually receive a test report that documents all contents of the testing and can be added as a report to your Usability Engineering File.
The specific requirements for your type of IVD
Before you tackle Usability Testing, you should be clear about the specific requirements of the IVDR for your type of IVD and what that means for testing.
The same device can generate different testing requirements depending on the application. For example, if it is a near-patient IVD used by healthcare personnel, sample collection can be simulated with the right equipment. However, if the same product is designed for lay use, this results in a much larger target group where you can no longer simulate sample collection.
The risk class to which your IVD has been assigned is irrelevant for Usability Testing. This does not result in any special requirements for the test procedure. Possible requirements may, however, arise after completion of the testing when evaluating the observed errors with regard to the resulting risk.
For some IVDs, there are already detailed MDCG Guidance Documents that provide very clear specifications that also relate to Usability Testing. For example, a certain number of participants is required for Corona Self-Tests.
If you want to know ahead of time if there is a detailed MDCG Guidance Document for your type of IVD, you can check here. Many of the documents are still in development and new ones are being added all the time.
Conclusion
Usability Testing for IVDs should be carefully prepared and fully documented. This is the only way to provide the Notified Body with all the necessary evidence of the Usability of your product. There is no way around testing with the right user group if you want to secure the approval of your IVD. The cost of the right equipment and the time required to recruit test participants can quickly take up a lot of resources. Accordingly, it is worthwhile to bring an external partner on board here.
For your preparation for the Summative Evaluation according to IVDR, we have prepared a checklist including explanations for you here.
What is your experience with Usability Testing for IVDs? What is your specific case? Write us a comment or get in touch via our contact form. We would be happy to talk through your specific case together in a free get-to-know-you meeting. We look forward to hearing from you.
