In this article, you will find a summary of the individual points on human factors validation. Part 2 will be linked here shortly.
Enjoy reading!
The basis: Human Factors Engineering and its importance for medical device development?
The aim of human factors engineering is to enable the simple and safe use of medical devices. To this end, human capabilities, limitations, and characteristics are considered and incorporated into the development process.
This is to be achieved by requiring manufacturers to minimize possible application errors and the resulting damage through targeted product design. To prove this has been achieved, a final and evaluative test of the product is carried out at the end of the human factors engineering process. This test is called Human Factors Validation.
The results and the entire preceding process are collected in the Human Factors Report and submitted to the FDA.
And how do you implement everything? Assistance: Applying Human Factors
To implement the FDA requirements in a targeted manner, the FDA repeatedly provides guidance documents that provide essential assistance and create a better understanding of how implementation can succeed.
One such document is „Applying Human Factors and Usability Engineering to Medical Devices”. In chapter 8 you will find all the notes on Human Factors Validation Testing.
We have summarized the contents below.
General information on Human Factors Validation Testing:
The chapter on “Human Factors Validation Testing” deals with the performance of tests to ensure that a device can be used safely and effectively by its target users for the intended purposes and under real-life conditions. The tests should be able to detect any user errors caused by the design of the user interface and be realistic.
To summarize, the main points for the design of Human Factors Validation Testing are:
- Test participants should represent actual users.
- All critical tasks must be considered in the test.
- The user interface should be available in the final design.
- The test conditions must be realistic.
Human Factors Validation Testing must be sensitive enough to detect unconscious user errors and not overlook severe dangers due to design flaws. The results should be universally valid to confirm the safety and effectiveness of the device under real-world conditions. The test protocol must include a detailed discussion of all critical tasks and data collection methods, and the test results should provide a comprehensive analysis of all user errors encountered.
Although Human Factors Validation Testing is typically conducted in simulated environments, in some instances, it may be conducted under real-world conditions or as part of a clinical trial. Manufacturers are encouraged to submit their test protocols in advance to ensure acceptance of the planned methods.
(Source: Applying Human Factors and Usability Engineering to Medical Devices, chapter 8)
The simulated use:
The test conditions for simulated usage scenarios must be realistic enough to allow valid conclusions to be drawn for the actual application. The relevance of realism results from the risk analysis about the specific use of the device, the users, the application environments, and the user interface. Environmental factors such as poor lighting or distractions should be included in the simulated environment.
During human factors validation testing, participants should be able to use the device as independently and naturally as possible without being influenced by the test leader.
Labels and aids in actual use should also be available in the test, but their use should be optional for the participants. These elements can be assessed separately after the simulated test. If telephone help is available in real life, it can be simulated in the test but should not be offered in the same room and should not guide users through the test.
(Source: Applying Human Factors and Usability Engineering to Medical Devices, chapter 8)
The test participants:
For human factors validation tests, the test participants must represent the actual target group of the product. In general, there should be at least 15 participants per user group, with several different user groups (e.g., by age, profession, or role). Participants should cover a wide range of characteristics within their group, including possible functional limitations. Testing should preferably be conducted with US participants to ensure that the results are transferable to actual conditions in the US. Employees of the manufacturer should only serve as test participants in exceptional cases.
(Source: Applying Human Factors and Usability Engineering to Medical Devices, chapter 8.1.1)
Tasks and use scenarios:
Human Factors Validation Testing must cover all critical tasks from previous analyses and evaluations and represent these in logical, natural workflows as usage scenarios in the test protocol.
Depending on the complexity of the device and the number of critical tasks, several test sessions may be required. Successful user performance should be defined in advance for each task.
The test protocol should also justify to what extent and how often the participants should use the device.
Particular attention must be paid to infrequent tasks where errors can have serious consequences; these must be considered in the test according to the risk. Infrequent and high-risk use scenarios are critical for evaluating the safety and effectiveness of medical devices.
(Source: Applying Human Factors and Usability Engineering to Medical Devices, chapter 8.1.2)
Instructions for Use:
The design of the IFUs can already be assessed as part of the formative evaluation, but the labeling in the Human Factors Validation Testing should reflect the final design. This refers to all labels on the device and accessories, display information, packaging, associated labeling, instructions for use, user manuals, and quick start guides.
Although Human Factors Validation Testing can be used indirectly to evaluate the suitability of the instructions for use, this is done exclusively in actual device use, including the participants’ acquired knowledge of essential aspects of device use. The aim is to evaluate the extent to which the instructions for use support the safe and effective use of the device.
Deficiencies in the device labeling and IFUs become apparent through the performance of the participants or through subjective feedback. Suppose human factors validation testing reveals critical usage errors or indications of difficulties with essential tasks through participant feedback. In that case, it is unacceptable to claim in a pre-submission that the risks have been mitigated by changes to the instructions for use or other labeling elements unless additional test data is available to demonstrate that the changes made have contributed to reducing the risks to an acceptable level.
(Source: Applying Human Factors and Usability Engineering to Medical Devices, chapter 8.1.3)
Training of users:
The training provided in the Human Factors Validation Test should correspond to what actual users receive. If most users receive little or no training, the test participants should also be untrained.
Results from the Human Factors Test that show usage errors in or difficulties with critical tasks cannot be mitigated simply by promising “additional training” in a pre-submission unless data is provided to demonstrate the effectiveness of this measure.
Training content, formats, and methods should, as far as possible, match those received by actual users, and testing should not take place immediately after training to allow for the decay of what has been learned. The gap between training and testing and the training provided in the test should be accurately described in the test protocol.
The incorrect use of training for the human factors study is repeatedly a reason for rejection or criticism of the study in contact with the FDA. It thus leads to many delays and unnecessary additional costs in the approval process.
(Source: Applying Human Factors and Usability Engineering to Medical Devices, chapter 8.1.4)
Conclusion
Human Factors Validation Testing plays a crucial role in ensuring the safety and effectiveness of medical devices. It enables manufacturers to identify and correct potential user errors and design flaws before market launch, thus minimizing the risk to users and patients. Conducting such tests under realistic conditions ensures that the results are reliable and transferable.
Here are the key points that the FDA guidance document requires of you:
- Test participants should represent actual users.
- All critical tasks must be included in the test.
- The user interface should be available in the final design.
- The test conditions must be realistic.
The second part of this article, in which you will learn everything about data collection, data evaluation, and some exceptional cases of human factors validation testing, can be found here. We would be delighted if you read along again.
Do you need help with the planning, implementation and documentation of your Human Factors Validation? Please get in touch using our contact form. We look forward to hearing from you.
