A benefit-risk analysis under MDR Annex I weighs the overall clinical benefit of a device against the totality of its residual risks. EN ISO 14971:2019+A11:2021 clause 8 provides the method. The trap most startups fall into: treating "benefit" as only the quantifiable headline number and ignoring the indirect benefits that qualitative framing is designed to capture.

By Tibor Zechmeister and Felix Lenhard.

TL;DR

  • MDR Annex I General Safety and Performance Requirement 1 demands that risks are acceptable when weighed against the benefits to the patient, and compatible with a high level of protection of health and safety.
  • EN ISO 14971:2019+A11:2021 clause 8 is the method the notified body expects for the overall residual risk evaluation.
  • Direct benefits are typically quantifiable. Diagnostic accuracy improvements, reduction in procedure time, measurable clinical outcome changes.
  • Indirect benefits are real but often qualitative. Reduced waiting time, improved triage, queue efficiency, reduced operator fatigue.
  • Auditors accept qualitative framing when the documentation transparently explains why quantitative data alone cannot capture the full benefit profile.
  • The benefit-risk conclusion flows into the technical documentation, the clinical evaluation report, and the risk management report.

Why benefit-risk is the hinge of MDR compliance

Benefit-risk is not a side document. It is the hinge that holds the entire MDR submission together. Every risk accepted during development, every residual risk disclosed in the instructions for use, every indication listed on the label, eventually has to answer one question: is the benefit to the patient worth it.

Tibor has reviewed benefit-risk sections in more than fifty MDR technical files. The pattern he sees again and again is founders treating benefit-risk as a checkbox exercise. A short paragraph at the back of the risk management report. A sentence that says the benefits outweigh the risks. No numbers. No reasoning. Nothing an auditor can challenge, because nothing is actually claimed.

The opposite failure mode is equally common. A founder shows up with a slide deck full of quantitative benefits. Sensitivity figures, specificity figures, time savings in seconds. And yet the device's real clinical value is about reducing emergency department queues, or letting a nurse triage faster in a rural clinic. The numbers on the slide do not capture the benefit the device actually delivers. The auditor reads the technical file, does not see the real benefit described anywhere, and concludes the benefit case is thin.

Both failure modes are avoidable. The fix is to treat benefit-risk as a genuine analytical exercise, with the right methodological honesty about which benefits are quantitative and which are qualitative.

What MDR actually says

MDR Annex I, General Safety and Performance Requirement 1 sets the legal bar. Devices shall achieve the performance intended by their manufacturer and be designed and manufactured in such a way that, during normal conditions of use, they are suitable for their intended purpose. Their known and foreseeable risks, and any undesirable side effects, shall be minimised and be acceptable when weighed against the evaluated benefits to the patient, and compatible with a high level of protection of health and safety, taking into account the generally acknowledged state of the art.

GSPR 2, 3, 4 and 8 then frame the specific obligations. GSPR 2 requires a risk management system. GSPR 3 requires that manufacturers establish, implement, document, and maintain a risk management process throughout the entire lifecycle of the device. GSPR 4 requires risk control measures, in the order of eliminating or reducing risks as far as possible through safe design, then adequate protection measures, then information for safety. GSPR 8 requires that all known and foreseeable risks and any undesirable side-effects shall be minimised and acceptable when weighed against the evaluated benefits.

EN ISO 14971:2019+A11:2021 clause 8 is the method that maps to these requirements. Clause 8 covers the evaluation of overall residual risk. It asks the manufacturer to consider whether the overall residual risk is acceptable in relation to the benefits of the intended use. Clause 8 does not mandate quantification. It mandates evaluation and documented rationale.

This point matters. EN ISO 14971 is explicit that acceptability criteria are defined by the manufacturer in the risk management policy, and that the benefit-risk evaluation takes into account both clinical and non-clinical factors.

Direct benefits and indirect benefits

In Tibor's experience, the cleanest way to structure a benefit-risk analysis is to start by decomposing benefit into two categories.

Direct benefits are the clinical effects attributable to the device's primary function. They are typically measurable. For a diagnostic device, direct benefit might be sensitivity, specificity, positive predictive value, or reduction in missed findings compared to the standard of care. For a therapeutic device, direct benefit might be reduction in symptom score, improvement in mobility, or reduction in hospital readmission. Direct benefits usually come out of the clinical evaluation and are supported by clinical data, performance data, or equivalence arguments.

Indirect benefits are everything else the device produces that still matters clinically or operationally. These are harder to quantify. Examples Tibor sees regularly in the startup files he reviews: reduced waiting time in outpatient clinics because the device enables faster triage, reduced operator fatigue because the device automates a repetitive task, reduced travel time for patients because a home device replaces a clinic visit, smoother workflow integration that frees staff for higher-value work, reduced cognitive load on junior staff because the device surfaces the relevant information at the point of decision.

Indirect benefits are not lesser benefits. They are often the actual reason a hospital buys the device. But they rarely fit cleanly into a quantitative framework. A diagnostic accuracy figure is a number. A reduction in triage friction is a situation. Trying to force the situation into a single number usually produces either a fake number or a number so narrow that it misses the real benefit.

EN ISO 14971 clause 8 and MDR Annex I do not require that every benefit be quantified. They require that the benefit be evaluated, documented, and weighed against the residual risks. Qualitative framing is a legitimate method when the benefit being described is qualitative in nature. What is not acceptable is hiding behind qualitative language to avoid showing data that could reasonably have been collected.

A worked example: an AI triage tool in an emergency department

Consider a hypothetical Class IIa software device that processes chest X-rays in an emergency department and flags suspected pneumothorax for urgent review. The founder has two clinical studies showing a sensitivity of 94% and a specificity of 88% compared to radiologist adjudication. That is the direct benefit. It is a number, and it goes into the clinical evaluation report and flows into the benefit-risk analysis as quantified evidence.

But the founder also claims the device reduces time-to-treatment for pneumothorax patients. This is the benefit the hospital actually cares about. How is it measured? One site measured a median time-to-treatment reduction of 18 minutes in a pilot. Another site reported qualitatively that the night shift triage workflow became noticeably less stressful. A third site reported that junior residents felt more confident because a second signal was available.

A thin benefit-risk analysis would list only the 94% and 88%. A transparent benefit-risk analysis would list the quantified direct benefit, then separately list the indirect benefits with the evidence available for each, explicitly flagging which evidence is quantitative and which is qualitative. The 18-minute figure from one site is weak quantitative evidence and should be labelled as such. The night shift stress reduction is purely qualitative and should be described with the user feedback verbatim, in context, not reframed as a pseudo-number.

On the risk side, the residual risks might include false negatives leading to missed pneumothoraces, false positives leading to unnecessary urgent review, automation bias, and alert fatigue. Each gets a severity, a probability, and a residual risk evaluation after controls. The overall residual risk evaluation under EN ISO 14971 clause 8 then explicitly weighs the acknowledged direct and indirect benefits against these residual risks. The conclusion is documented, signed, and referenced in the clinical evaluation report.

This structure survives audit because it is transparent about its own evidence base. The auditor can disagree with individual weights but cannot accuse the file of hiding the reasoning.

The Subtract to Ship playbook for benefit-risk analysis

Felix has coached 44 startups through their first MDR submission. The benefit-risk analysis is often the last thing the team writes and the first thing the auditor reads. The Subtract to Ship approach cuts this risk.

Step 1. Write the benefit statement before you write the risk file. Most teams write the hazard analysis first and then bolt benefits on at the end. Reverse the order. Start with a plain-language description of why the device exists and what it delivers to the patient, the user, and the healthcare system. This becomes the seed of the benefit side of the analysis and forces an honest conversation about what is actually known.

Step 2. Separate direct and indirect benefits explicitly. Create two lists. Direct benefits get quantitative evidence columns (study, endpoint, value, confidence). Indirect benefits get qualitative evidence columns (source, description, context, limitations). Do not mix them.

Step 3. For each indirect benefit, document why quantitative evidence is not available or not appropriate. This is the transparency clause. Auditors accept qualitative framing when the reasoning is visible. They reject qualitative framing when it looks like the team avoided collecting data.

Step 4. Link every residual risk in the risk file to the benefit-risk conclusion. The overall residual risk evaluation under EN ISO 14971 clause 8 is not a separate exercise. It is the aggregation of every accepted residual risk, weighed collectively against the documented benefit profile.

Step 5. Feed the benefit-risk conclusion into the clinical evaluation report and the risk management report. MDR Article 61 and Annex XIV Part A require the clinical evaluation to confirm the benefit-risk determination. If the benefit-risk analysis and the clinical evaluation contradict each other, the auditor will find it.

Step 6. Revisit the benefit-risk analysis as post-market data arrives. MDR Articles 83 to 86 require continuous post-market surveillance. The benefit-risk analysis is not a launch-day artefact. It is a living document that gets updated as real-world performance and complaint data accumulate.

Reality Check

  1. Can you state the direct and indirect benefits of your device in plain language without reaching for marketing claims?
  2. Does your benefit-risk analysis separate quantitative benefits from qualitative benefits, or does it blur them?
  3. For every qualitative benefit you claim, can you point to documented evidence (user research, site observations, pilot feedback) rather than assertion?
  4. Does your benefit-risk analysis document why quantitative evidence is not available for benefits where you use qualitative framing?
  5. Does the overall residual risk evaluation in your risk management report (EN ISO 14971 clause 8) reference the benefit-risk analysis explicitly?
  6. Is the benefit-risk conclusion consistent between the risk management report, the clinical evaluation report, and the technical documentation summary?
  7. Do you have a plan to update the benefit-risk analysis when post-market data arrives, or is it a launch-day static document?
  8. If an auditor asked you to justify a specific residual risk as acceptable, could you point to the exact line in the benefit-risk analysis that carries the weight?

Frequently Asked Questions

Is a qualitative benefit-risk analysis acceptable to notified bodies? Yes, when the benefits being described are genuinely qualitative and the documentation is transparent about the evidence base. EN ISO 14971 clause 8 does not require quantification. It requires evaluation and rationale. What notified bodies reject is hiding missing data behind qualitative language.

Where in the technical documentation does the benefit-risk analysis live? It lives in the risk management report (part of Annex II documentation) and is referenced from the clinical evaluation report. The conclusion appears in the technical documentation summary that goes to the notified body.

Can I use a benefit-risk matrix? A matrix can help structure the comparison, but a matrix alone is not an analysis. The matrix has to be accompanied by narrative reasoning that explains how direct and indirect benefits are weighed against the residual risks. Auditors read the narrative first.

What if my device has multiple intended uses with different benefit-risk profiles? Each intended use gets its own benefit-risk analysis. You cannot average them. MDR Annex I GSPR 1 requires acceptability for each intended use, not for the device as a whole.

How often should the benefit-risk analysis be updated? At minimum whenever the risk management file is updated, which under EN ISO 14971 and MDR Articles 83 to 86 should be continuous as post-market data flows in. A file last updated two years ago is a red flag in audit.

Sources

  1. Regulation (EU) 2017/745 on medical devices, consolidated text. Annex I General Safety and Performance Requirements 1, 2, 3, 4, 8. Article 61, Annex XIV Part A.
  2. EN ISO 14971:2019+A11:2021, Medical devices, Application of risk management to medical devices. Clauses 7 (Risk control), 8 (Evaluation of overall residual risk), 9 (Risk management review).
  3. EN ISO 14971:2019+A11:2021, Annex Z relating to MDR Annex I General Safety and Performance Requirements.