---
title: Writing Usability Test Protocols for Medical Devices
description: How to write a usability test protocol that survives notified body review under EN 62366-1, covering objectives, tasks, success criteria, and data capture.
authors: Tibor Zechmeister, Felix Lenhard
category: Electrical Safety & Systems Engineering
primary_keyword: usability test protocol medical device
canonical_url: https://zechmeister-solutions.com/en/blog/writing-usability-test-protocols
source: zechmeister-solutions.com
license: All rights reserved. Content may be cited with attribution and a link to the canonical URL.
---

# Writing Usability Test Protocols for Medical Devices

*By Tibor Zechmeister (EU MDR Expert, Notified Body Lead Auditor) and Felix Lenhard.*

> **A usability test protocol that survives notified body review under EN 62366-1:2015+A1:2020 specifies objectives, participants, tasks tied to hazard-related use scenarios, measurable success criteria, data capture methods, and an analysis plan, all traceable back to the use specification and the risk management file. Everything else is padding.**

**By Tibor Zechmeister and Felix Lenhard.**

## TL;DR
- Clauses 5.7 to 5.9 of EN 62366-1:2015+A1:2020 require a documented usability evaluation plan. Your test protocol is the working-level instantiation of that plan.
- A defensible protocol covers eight sections: objectives, scope, participants, environment, tasks, success criteria, data capture, and analysis.
- Critical tasks come from the list of hazard-related use scenarios in the use specification and feed directly from the risk management file under EN ISO 14971:2019+A11:2021.
- Success criteria must be pre-defined and measurable. Writing them after you have seen the results defeats the purpose and notified bodies spot it instantly.
- Data capture combines observation notes, time stamps, video where ethically permitted, retrospective interviews, and post-task questionnaires.
- The analysis plan must describe how observed use errors will be classified, root-caused, and fed back into the risk file.

## Why this matters

Tibor has read usability protocols that were six pages long and contained the phrase "participants will use the device and feedback will be collected". That sentence is not a protocol. It is a placeholder where a protocol should have been.

The reason protocols fail is almost always the same. The protocol was written after the testing was already conceptually finished, as a retroactive document to satisfy the notified body. The task list was reverse-engineered from what the team happened to show participants. The success criteria were defined after the results were in and tuned to match. Notified body reviewers read hundreds of these files. They recognize retroactive protocols immediately and mark the whole usability file as unreliable.

A protocol written before testing, with pre-defined success criteria and a documented link back to the risk file, does two things. First, it makes the testing itself meaningful. Second, it makes the notified body conversation short. Felix has watched a startup close out their summative evaluation audit conversation in under 20 minutes because the protocol was clean and every observation had a traceable home.

## What MDR actually says

The MDR routes usability through Annex I §5 and Annex I §22 of Regulation (EU) 2017/745. The design must reduce risks linked to ergonomic features and the environment of intended use, and must account for the technical knowledge, experience, education, training, and use environment of the intended user. The operational standard is EN 62366-1:2015+A1:2020.

Clause 5.7 of the standard requires a usability evaluation plan. Clause 5.8 covers formative evaluation during design and clause 5.9 covers summative evaluation of the final user interface. The standard is explicit: summative evaluation must be based on critical tasks, conducted with representative users, in a realistic or simulated use environment, with pre-defined acceptance criteria.

The protocol is the document that operationalizes clause 5.9. It takes the use specification from clause 5.1, the user interface specification from clause 5.5, and the hazard-related use scenarios from clause 5.4, and turns them into a session-by-session plan that a moderator can execute and an auditor can verify. Without the protocol, the chain of evidence from MDR Annex I §22 to the final usability engineering file is broken.

## A worked example

Consider a Class IIb insulin titration app used by people with type 2 diabetes at home. The use specification defines one user group: adults aged 40 to 80, with a diagnosis of type 2 diabetes, with limited smartphone experience, using the device without direct clinical supervision.

The risk management file identifies three hazard-related use scenarios that become the critical tasks for summative testing. Task 1: interpreting the daily dose recommendation and entering it into an insulin pen. Task 2: recognizing and responding to a hypoglycemia warning. Task 3: reporting an unexpected symptom through the app.

The protocol for this test runs about 14 pages. The objectives section states that the goal is to demonstrate that residual use-related risks for the three hazard-related use scenarios are acceptable. The participants section specifies 15 users per group, recruited through two regional diabetes clinics, screened against the use specification profile. The environment section describes a quiet room in the clinic, with a realistic kitchen setup and an insulin pen trainer. The tasks section specifies the exact starting state of the app, the instructions read aloud by the moderator, and the stopping conditions.

The success criteria section lists, for each task, what counts as a use error, what counts as a close call, and what counts as successful completion. For Task 2, for example, the acceptance criterion is that at least 14 of 15 participants recognize the hypoglycemia warning within 10 seconds and take a documented corrective action. The data capture section specifies video recording with written consent, time stamps, moderator notes, and a structured post-task interview. The analysis section specifies the root cause classification method and the feedback loop into the risk file.

When the notified body reviewed this file, the only comment was a minor clarification request about the informed consent language. No finding was raised against the protocol structure or the success criteria. That is what a complete protocol buys a startup.

## The Subtract to Ship playbook

Writing a protocol is not a drafting exercise. It is the moment where the usability file, the risk file, and the design file have to line up. The steps below produce a protocol that passes notified body review and also makes the testing session itself meaningful.

**Step 1. Start from the hazard-related use scenarios.** Open the risk management file and list every hazard-related use scenario that has not been eliminated by inherent safety design. Each of these becomes a candidate critical task for summative evaluation. This is where Tibor sees the most drift. Teams write tasks based on what the app does, not based on what the risks are. The risk file is the starting point, not the feature list.

**Step 2. Write the objectives section in one paragraph.** The objectives are the purpose of the test, not a description of the device. State that the purpose is to generate evidence that residual use-related risks for the listed hazard-related use scenarios are acceptable under the conditions described in the use specification. Name the clause of EN 62366-1 being satisfied. Keep it short.

**Step 3. Specify participants with screening criteria.** Reference the post on recruitment for sample size and disqualification rules. The protocol must name the recruitment channel, the screening questionnaire, the sample size per group, and the exclusion criteria. A notified body reader should be able to reproduce the recruitment from this section alone.

**Step 4. Describe the environment concretely.** Do not write "a realistic use environment". Write what is in the room, what equipment is present, what distractors are present or absent, what lighting conditions apply, and what the moderator does and does not say. For home-use devices, describe the home simulation in enough detail that another team could reproduce it.

**Step 5. Write the task list with exact instructions.** For each critical task, specify the starting state of the device, the instructions given to the participant verbatim, the stopping condition, and what the moderator is allowed to do if the participant is clearly stuck. Ambiguity here creates session-to-session variance that contaminates the results.

**Step 6. Define success criteria before testing.** This is the most audited section. For each task, write what counts as success, what counts as a use error, and what counts as a close call. Use numbers where possible. "At least 14 of 15 participants complete the task without a use error" is better than "most participants complete the task". If a task has no pre-defined success criterion, it is not a summative task.

**Step 7. Specify data capture methods.** List the exact data captured in each session: video, audio, time stamps, moderator notes, structured observation form, post-task questionnaire, retrospective interview transcript. Reference the consent language for each type of recording. Notified bodies check data capture completeness against the protocol.

**Step 8. Write the analysis plan.** The protocol must state how observed use errors will be classified, how root cause analysis will be performed, how the feedback will be integrated into the risk management file under EN ISO 14971:2019+A11:2021, and who approves the conclusions. Without an analysis plan, the testing generates data that nobody knows how to interpret.

**Step 9. Review with the risk manager and the design lead before recruitment.** Tibor has seen protocols fail because the design lead had changed a screen flow the week before testing and nobody updated the protocol. A short joint review catches this. The protocol should be under formal document control in the QMS and approved before recruitment begins.

## Reality Check

Check each of the statements below against your current draft protocol. If you cannot tick every box, the protocol needs another pass.

1. Every critical task in the protocol traces back to a specific hazard-related use scenario in the risk management file.
2. The objectives section references the exact clause of EN 62366-1:2015+A1:2020 the test is satisfying.
3. The participants section specifies sample size, recruitment channel, screening questionnaire, and exclusion criteria.
4. The environment section describes the physical setup in enough detail to be reproducible by another team.
5. Every task has a verbatim instruction, a defined starting state, and a defined stopping condition.
6. Success criteria are numeric where possible and were written before any testing occurred.
7. The data capture section names every recording type and references the consent language.
8. The analysis plan describes how use errors feed back into the risk file and who signs off on the conclusions.
9. The protocol is under document control in the QMS and was approved before recruitment started.

## Frequently Asked Questions

**How long should a usability test protocol be?**
Complete protocols for moderate-complexity devices typically run 10 to 20 pages. Anything under five pages is usually missing one of the required sections. Anything over 40 pages is usually padding and will annoy the notified body reviewer.

**Can the same protocol cover formative and summative sessions?**
No. Formative evaluation under clause 5.8 and summative evaluation under clause 5.9 have different purposes, different participant rules, and different acceptance criteria. Write two protocols or two clearly separated sections within one master plan.

**Do we need to pre-register the success criteria somewhere external?**
External pre-registration is not required by EN 62366-1. What is required is that the success criteria are documented and approved under QMS document control before testing begins. The date stamp and version history in the QMS serves the same auditable purpose.

**What if a task was included in the protocol but nobody ever failed it?**
Record the result honestly. A task where all participants succeed is a positive finding about the design. Do not remove the task from the report to make the document shorter. The notified body checks the protocol task list against the report.

**Can we modify the protocol after the first few sessions?**
Formative protocols can be iterated between rounds. Summative protocols should not change once testing has started. If a serious problem is discovered, stop the test, re-plan, re-approve the protocol, and restart. Silent mid-test changes invalidate the evidence.

**Who should write the protocol?**
The human factors lead drafts it. The risk manager reviews it against the risk file. The design lead reviews it for technical accuracy. The regulatory lead reviews it for EN 62366-1 and MDR compliance. Four eyes minimum, all under QMS document control.

## Related reading
- [Usability test participants recruitment](/blog/usability-test-participants-recruitment). How to recruit the right users and how many of them you need.
- [Analyzing usability test results](/blog/analyzing-usability-test-results). Turning observed use errors into design changes and risk file updates.
- [Risk management and usability engineering link](/blog/risk-management-usability-engineering-link). How hazard-related use scenarios feed the usability protocol.

## Sources
1. Regulation (EU) 2017/745 on medical devices, consolidated text. Annex I §5, Annex I §22, Annex II.
2. EN 62366-1:2015+A1:2020, Medical devices, Part 1: Application of usability engineering to medical devices. Clauses 5.1, 5.4, 5.5, 5.7, 5.8, 5.9.
3. EN ISO 14971:2019+A11:2021, Medical devices, Application of risk management to medical devices.

---

*This post is part of the [Electrical Safety & Systems Engineering](https://zechmeister-solutions.com/en/blog/category/electrical-safety) cluster in the [Subtract to Ship: MDR Blog](https://zechmeister-solutions.com/en/blog). For EU MDR certification consulting, see [zechmeister-solutions.com](https://zechmeister-solutions.com).*