You are about to obtain approval for a medical device and know that your product must undergo a summative evaluation/human factors validation? But how do you best organize the usability validation study? Internally with your own team? Or externally with a freelancer or a specialized agency?
One thing is clear: the solution must fit your needs. There is no universal answer to the question of whether usability validation should be carried out internally or externally. The choice depends on your specific situation and your specific requirements. A combination of internal and external approaches may even be appropriate.
In this article, we will therefore show you the advantages and disadvantages of the various options. This will enable you to make an informed decision as to whether you want to carry out the summative evaluation/human factors validation internally or externally and complete it successfully.
Advantages of carrying out a summative evaluation/human factors validation internally:
- Availability: Internal resources are continuously available and can be used flexibly.
- Deep product understanding: Through many years of collaboration, the internal team develops a deep understanding of your medical devices.
- More cost-effective: Existing internal resources can save costs.
- Flexibility: You have control over the use of internal resources and can act flexibly.
Disadvantages of conducting a summative evaluation/human factors validation internally:
- No availability: Internal resources for usability testing are often overloaded. Complex and possibly even international validation studies can be a particular problem.
- Lack of neutrality: Internal usability researchers may be biased. This is critical from a regulatory perspective, as the same people who are involved in the design decisions are then responsible for the study.
- Lack of expertise: If there is no internal department with the relevant knowledge, it is often the case that one person has to familiarize themselves with the subject matter. Even if this is done intensively, there is often a lack of experience in setting up and conducting usability validation in a manner that is compliant with regulations.
If there is no internal department or expertise, the following points must also be considered:
- Long start-up time: Setting up an internal department requires time and patience.
- Commitment for many years: Creating an internal department requires long-term commitment.
- Workload must be sufficient: Smaller and medium-sized companies in particular do not constantly launch new medical devices on the market. An internal department is then faced with the challenge of having too much or too little to do at certain times.
- Time-consuming recruiting: Hiring qualified employees can be a challenge.
- Work processes must be developed: Establishing suitable processes requires additional resources.
- Lack of scalability: Internal resources may not be flexible enough to meet unexpected demand.
Advantages of outsourcing summative evaluation/human factors validation to a freelancer:
- External perspective: A freelancer can offer a neutral perspective on your product.
- Speed: Freelancers can be available quickly and often require little preparation time.
- Flexibility: You can use freelancers as needed.
- Lower costs: Freelancers can be more cost-effective than the other two options if you only need specific services.
Disadvantages of outsourcing a summative evaluation / human factors validation to a freelancer:
- Lack of scalability: Freelancers cannot provide additional resources when demand increases.
- Single person: If a freelancer falls ill, their work may come to a standstill.
- Limited expertise: Freelancers typically specialize in specific areas. For example, they may be able to conduct a study, but not recruit participants or provide the test environment.
- Lack of work authorization for international studies: If you are planning an international study, a lack of work authorization in the target country can be a problem that cannot be solved quickly.
- Lack of quality assurance: There may not be enough information about the quality of a freelancer. A freelancer’s processes are typically not certified.
- Search for a freelancer: Finding a suitable freelancer takes time and effort.
- Training period: Freelancers need time to familiarize themselves with the product content.
Advantages of outsourcing summative evaluation / human factors validation to an agency:
- High scalability: Agencies can provide suitable personnel as needed.
- Speed: Agencies can deliver results quickly.
- Flexibility: Agencies have experts from various fields. This increases flexibility when other expertise is needed at short notice.
- High quality standards: Good agencies offer clear quality standards. For example, a TÜV-certified QM system in accordance with ISO 13485 (not mandatory, but often helpful for audits).
- Broad experience: Agencies have experience with different customers and user types, but also with the requirements of notified bodies/regulatory bodies.
- External perspective: Agencies can identify sources of error and offer a neutral perspective.
- Existing usability labs: Agencies typically have their own usability labs, such as operating rooms or doctors’ offices, so they can quickly replicate the required realistic test environments.
- Neutral user recruitment: Agencies can recruit neutrally.
- Internationality: Agencies specializing in medical devices often work internationally. This means that language barriers, work permits, international recruitment, etc. are already in place.
Disadvantages of conducting a summative evaluation / human factors validation externally with an agency:
- Higher costs: Agencies can be expensive, depending on your point of view.
- Search for an agency: Finding a suitable agency takes time and effort.
- Changing contact persons: Some agencies may involve different contact persons.
- Training period: Agencies need time to familiarize themselves with the product content.
Conclusion – internal or external?
As already mentioned, the best solution depends on your preferences and requirements. An internal department offers long-term cooperation, but may lack an external perspective. A freelancer can be deployed flexibly, but may not offer the same breadth of expertise. An external agency offers fast results, flexibility, and high quality standards, but requires a corresponding budget and time for the search.
With the advantages and disadvantages of each option in mind, you can make the right decision for you – whether internal or external. We are happy to answer any questions you may have and support you in conducting your usability validation study.
- If you would like to find out how we conduct a summative evaluation/human factors validation, you can find the answers here.
- Are you currently planning your usability validation study and need help with the perfect timeline? Then the webinar “Perfect Timeline for Summative Evaluations & Human Factors Validation Testings” will help you.
- If you would like to discuss your specific case with us directly, please contact us via our contact form. We look forward to hearing from you.
