A startup can run a compliant usability engineering process under EN 62366-1:2015+A1:2020 at a fraction of the large-company cost, but only by being honest about which parts allow a simulated environment and which parts require real clinical or hospital conditions. Skipping the real conditions when they are required does not save money. It defers the cost to a full change control procedure after the device is on the market.

By Tibor Zechmeister and Felix Lenhard.

TL;DR

  • EN 62366-1:2015+A1:2020 is mandatory for demonstrating conformity with the use-related parts of MDR Annex I §5 and Annex I §22.
  • The dominant cost line in a usability engineering file is the summative evaluation, not the formative work that precedes it.
  • For mobile, handheld, and software-only devices, a well-justified simulated use environment can reduce summative cost by an order of magnitude without cutting corners.
  • For devices that depend on real clinical workflow, real hospital infrastructure, or real physiological response, no simulation is defensible.
  • Testing with the team's own engineers, sales staff, or friendly customers is not a summative evaluation, regardless of how it is labelled in the file.
  • The cost of skipping a real summative is measured in change control procedures, notified body re-engagement fees, and field safety corrective actions after market entry.

Why budget conversations belong in the usability engineering plan

Tibor has reviewed enough first-time technical documentation files to predict where the pressure point lands. The risk analysis has been done, the formative evaluation has been done, the use specification is in place, and then the founder discovers what a proper summative evaluation costs. At that moment a series of conversations starts inside the company about whether the summative is strictly required, whether the team's own engineers could stand in for recruited users, whether a desk review would suffice, or whether a notified body might accept a simplified version. Every one of those conversations is a warning sign.

Felix has seen the same conversation from inside startup teams. The engineering budget has been spent, the clinical evidence work is underway, the regulatory consultant has just quoted the summative budget, and the finance lead is trying to reconcile the number with the runway. The temptation to declare the summative done with internal users is enormous. It is also the single most common way startups manufacture a nonconformity that will resurface in the notified body audit.

The productive conversation is the opposite. It starts at the beginning of the development plan, before the money has been spent, and it asks a precise question: for this specific device, in this specific intended use, which parts of the summative evaluation can legitimately be run in a simulated environment and which parts cannot. That question has a real answer grounded in EN 62366-1:2015+A1:2020. It is not a negotiation and it is not a matter of interpretation. The standard gives manufacturers a framework for justification. Used properly, that framework lets a lean startup spend far less than a large company and still produce a file that a notified body will accept.

What EN 62366-1 allows and what it does not

EN 62366-1:2015+A1:2020 requires the summative evaluation to be performed under conditions that adequately simulate the intended use. The word "simulate" is the key. The standard does not require that every summative be run in a real hospital with real patients. It requires that the environment, the users, the context, and the procedures be representative enough that any hazard-related use scenarios surface during testing.

The standard also requires recruited users who represent the intended user population. A user recruited because they are a friend of the founder is not a representative user unless they also match the specified user profile from the use specification. An engineer on the development team is not a representative user under any circumstance, because their familiarity with the device invalidates the observation.

Within that framework, Tibor draws a practical line for budget purposes. Mobile, handheld, and software-only devices can often be evaluated in a simulated environment with recruited lay or professional users, a prepared room, a recording setup, and a moderated protocol. The cost is measured in tens of hours of clinical research organisation time, not hundreds. This is where startups should push for efficiency.

Devices that depend on real clinical infrastructure, on the other hand, cannot be simulated away. A sterilisable surgical tool in a real operating theatre, an imaging device that depends on a real scanner bed and a real patient body habitus, an infusion pump that must be tested against real patient monitoring systems, a device used only in the presence of a multidisciplinary clinical team: these cases require real clinical conditions. The simulation argument breaks down because the hazards the evaluation is trying to find only exist in the full clinical environment.

The midground is populated by home-use devices for lay users under MDR Annex I §22. These can often be evaluated in a simulated home environment with recruited users from the target demographic. The home does not have to be the user's actual home. A plausible domestic set-up with the right lighting, furniture, and distractions is usually sufficient. The critical constraint is that the users be recruited from the real intended population, including older users where applicable. Tibor has seen the failure mode repeatedly: a device intended for 70-year-old patients was tested with 25-year-old colleagues in a well-lit lab and declared intuitive. Real users in the kitchen at home at the end of a tiring day could not use it at all.

A worked example: two devices, two budgets

Consider two hypothetical startups, both applying EN 62366-1:2015+A1:2020 to their summative evaluation.

Startup A builds a smartphone-based medical calculator for professional clinical use. The device is a software application running on consumer hardware. The hazard-related use scenarios are around input errors, misreading outputs, and misapplying outputs to patient decisions. Startup A can justify a simulated environment: a prepared clinical consultation room, eight recruited clinicians from the specified user profile, a structured protocol covering each hazard-related scenario, screen recording, and a trained moderator. Budget for the summative portion comes in around a handful of working days of clinical research organisation time, plus the recruited participant compensation. The file is defensible. Tibor has seen comparable files clear notified body review without a single usability-related nonconformity.

Startup B builds an active surgical tool for minimally invasive procedures. The device is handled by an operating surgeon in a real operating theatre under real patient conditions. The hazard-related use scenarios include tissue damage from imprecise activation, confusion with adjacent instruments on the tray, misidentification of the working end during high-stress moments, and interaction with other devices in the sterile field. Startup B cannot simulate these scenarios away. A benchtop model does not reproduce the sterile field, the team dynamics, the time pressure, or the full instrument tray. The summative evaluation has to be run in a cadaveric lab with real surgeons and a real instrument tray, or in an animal lab with equivalent conditions, or as a clinical investigation under MDR Articles 62 to 82 with the appropriate authorisations. The budget is an order of magnitude larger. There is no compliant shortcut.

The lesson is not that Startup A is smarter than Startup B. The lesson is that the device itself determines where on the budget spectrum the summative has to land. A founder who tries to run Startup B's summative with Startup A's budget will fail the audit. A founder who runs Startup A's summative with Startup B's budget has wasted money.

The Subtract to Ship playbook

Felix's Subtract to Ship approach to lean usability engineering has five components that a startup can apply during the usability engineering planning phase under EN 62366-1:2015+A1:2020.

Component one: classify the device on the simulation spectrum early. Before any money is committed, ask the simulation question honestly. Would the hazard-related use scenarios surface in a simulated environment, or do they only exist in a real clinical setting? Document the answer in the usability engineering plan. The document is the contract with the notified body.

Component two: invest heavily in the formative phase. Formative evaluations are cheap. Running three or four iterative formative rounds with recruited users during development costs very little and consistently surfaces the design problems that would otherwise only appear in the summative. Every hazard fixed formatively is a hazard that does not require summative rework.

Component three: write the use specification with discipline. Tibor's insight is that the use specification is the most skipped and most important artefact. Decomposing the use into every real-world procedure, including cleaning, transport, installation, and edge cases, makes hazards visible while they can still be designed out. A rigorous use specification reduces the number of hazard-related scenarios the summative must cover and keeps the summative budget controlled.

Component four: if a simulated environment is defensible, document the justification in the usability engineering plan, not after the fact. The justification is what a notified body reviews. A post-hoc rationalisation added to the file after the summative has been run is far less credible than a justification written into the plan and referenced in the report.

Component five: if real clinical conditions are required, budget for them at the beginning of the development plan, not at the end. Tibor has watched startups discover the real-conditions summative requirement in the final quarter before CE marking. At that point, the budget shock triggers the shortcut temptation. The founders who budget for the real summative from day one never run into the shortcut conversation.

Felix adds a sixth component from the business side: treat the recruited users as a dual-purpose resource. The same users who participate in a summative evaluation can become early customers, early advocates, and early sources of post-market feedback if the engagement is handled ethically and transparently. The startups that plan the summative early often recover part of the cost through this channel.

Reality Check

  1. Has the development plan explicitly classified the device on the simulation spectrum, with a documented justification?
  2. Is the usability engineering plan dated before the first formative evaluation, and does it include the justification for simulated or real conditions?
  3. For a device requiring real clinical conditions, has the summative budget been allocated from the beginning of development rather than at the end?
  4. Are the recruited users for the summative drawn from the real intended user population, including realistic age ranges for lay-user devices?
  5. Have at least three iterative formative evaluations been run before the summative to reduce the number of hazards the summative must uncover?
  6. Is the use specification granular enough that hazard-related use scenarios for cleaning, transport, installation, and edge cases have been identified?
  7. Does the team understand that testing with engineers, sales staff, or internal users is not a summative, regardless of how the file labels it?

Frequently Asked Questions

Can a startup skip the summative evaluation entirely for a low-risk device? No. EN 62366-1:2015+A1:2020 applies across the classes of devices covered by MDR. The depth of the summative scales with the risk and complexity, but the obligation to perform one does not disappear.

Is a usability consultant or clinical research organisation mandatory? No, but an experienced moderator is effectively required. Startups that run their first summative without an experienced usability specialist typically miss observation categories the standard requires, and the file is pushed back during notified body review.

How many users are needed for a summative? EN 62366-1:2015+A1:2020 does not set a fixed number, but commonly accepted practice is a minimum of five to eight recruited users per distinct user group. Tibor's practical guidance is that startups often underestimate the number of distinct user groups the use specification actually identifies.

Can a clinical investigation double as a usability summative? Sometimes, if the clinical investigation protocol under MDR Articles 62 to 82 is designed from the beginning to collect usability data according to EN 62366-1:2015+A1:2020. Bolting usability onto an existing clinical protocol after the fact rarely works.

What is the worst thing a startup can put in a summative file? A summative evaluation run with the development team, the founders, or unnamed friendly users, labelled as representative. Tibor's audit experience is that this is the fastest path to a major nonconformity.

Does a simulated environment ever save money net of the justification work? For mobile, handheld, and software-only devices the simulation saves significant money even after the justification overhead. For devices that depend on real clinical infrastructure, the justification will not hold, and the attempt to build it wastes time on top of the eventual rework cost.

Sources

  1. Regulation (EU) 2017/745 on medical devices, consolidated text. Annex I §5, §22.
  2. EN 62366-1:2015+A1:2020, Medical devices, Part 1, Application of usability engineering to medical devices.
  3. EN ISO 14971:2019+A11:2021, Medical devices, Application of risk management to medical devices.