When conducting a summative evaluation for medical devices, there are several mistakes that should be avoided to avoid jeopardizing the approval of your medical device and running into costly delays.
Here are some common mistakes that should be avoided. Enjoy reading!
Pitfall 1: No formative evaluations have been conducted
Without formative evaluations conducted during the development process, it can be difficult to identify all potential problems and risks associated with the use of the medical device. While formative evaluations are not mandatory, they allow you to get early feedback from users and iteratively improve the product before it hits the market. Without this preliminary work, there is a risk that unacceptable risks will be overlooked, which is essential for the approval of the medical device. Formative evaluations are therefore crucial to ensure that the product meets the required safety and performance standards.
Worst case: Without formative evaluations, there is a risk that the medical device does not meet the required safety and performance standards, which is only discovered in the summative evaluation. This results in unnecessary delays and costly re-design.
Pitfall 2: Insufficient sample size
A sample that is too small can affect the representativeness of the results and potentially overlook important security or performance issues. To avoid this pitfall, it is important to determine an appropriate sample size that covers a representative sample of potential users and usage scenarios. This requires careful planning and consideration of statistical methods to ensure that the results are reliable and meaningful.
Worst Case: A worst case scenario for insufficient sample size would be that important safety or performance issues with the medical device go undetected. This could lead to the product being placed on the market even though it is potentially dangerous or ineffective. This in turn could lead to legal consequences, such as recalls, claims for damages or even lawsuits. In addition, most regulatory authorities expect a minimum number of people for testing and may not accept usability studies with fewer participants.
Pitfall 3: Unclear or missing evaluation criteria
It is important to define clear criteria on the basis of which the medical device is assessed. Missing or vague criteria can lead to subjective or inaccurate assessments. To avoid this, the evaluation criteria should be carefully defined and documented before the evaluation begins. These criteria should cover all relevant aspects of the device, including safety, performance, usability and compliance.
Worst case: If the evaluation criteria are unclear or missing, the worst case scenario could be that the medical device is not evaluated appropriately. This could lead to incorrect assumptions about its safety, performance or usability, which could ultimately lead to negative effects on patients or users.
Pitfall 4: Lack of diversity in the participant group
A heterogeneous group of participants is important in order to cover the diversity of potential users and application contexts. A one-sided group can exclude important usage scenarios and lead to a biased evaluation. To ensure adequate diversity, you should find as many people and backgrounds from one group as possible (different ages, different practical experience, …).
Worst case: The worst case scenario for a lack of diversity in the participant group would be that important user groups or application contexts are not adequately considered. This could result in a medical device that is not suitable or safe for certain population groups, which could lead to serious ethical or legal issue.
Pitfall 5: Unrealistic test conditions
The test environment and conditions should be as realistic as possible to ensure that the evaluation results reflect the actual use of the medical device. This may mean that tests are conducted in a clinical environment or that specific usage scenarios are simulated to replicate the real conditions under which the product will be used.
Worst case: The worst case scenario for non-realistic test conditions would be that the evaluation results do not reflect the actual performance or safety of the medical device. There is a risk that the Notified Bodies will not accept this in the summative evaluation and refuse approval. The test then has to be carried out again at high additional cost and the approval is delayed at high opportunity cost.
Pitfall 6: Lack of consideration of potential risks
It is important to adequately consider potential risks of the medical device and include them in the assessment. This requires a thorough analysis of the product functions and characteristics as well as an assessment of the potential impact on the safety and health of users.
Worst case: The worst case scenario if potential risks are not taken into account would be that the medical device enters the market and causes unexpected harm or negative effects on users. This could lead to serious consequences, such as legal action, reputational damage or even withdrawal of approval.
Pitfall 7: Insufficient documentation
Comprehensive documentation of the assessment process and results is important to ensure transparency and enable the tracking of decisions and measures. The documentation should include all tests carried out, results obtained, problems identified and the measures and decisions derived from them. This not only facilitates internal tracking, but also supports regulatory compliance and enables other stakeholders to understand the assessment process.
Worst case: The worst case scenario in the absence of documentation would be that important decisions or results cannot be tracked or proven and the Notified Bodies therefore refuses approval.
Conclusion
Failure to avoid pitfalls in the summative evaluation for medical devices can cost you (temporary) approval and mean a costly re-execution including previous revision measures.
The following article shows you how to prepare and carry out the summative evaluation correctly: “Summative evaluation for medical devices: 10 best practices“.
In order to look at your specific case and meet your individual challenges, we offer you a free strategy discussion.
We look forward to hearing from you. Simply use our contact form to get in touch with us.
