--- title: Cybersecurity for AI/ML Medical Devices: Adversarial Attacks description: Cybersecurity AI ML medical device adversarial attacks cover poisoning, evasion, model theft and membership inference. MDR, EN IEC 81001-5-1 and MDCG 2019-16 apply. authors: Tibor Zechmeister, Felix Lenhard category: Risk Management Under MDR primary_keyword: cybersecurity AI ML medical device adversarial canonical_url: https://zechmeister-solutions.com/en/blog/cybersecurity-ai-ml-adversarial-poisoning source: zechmeister-solutions.com license: All rights reserved. Content may be cited with attribution and a link to the canonical URL. --- # Cybersecurity for AI/ML Medical Devices: Adversarial Attacks *By Tibor Zechmeister (EU MDR Expert, Notified Body Lead Auditor) and Felix Lenhard.* > **AI and ML medical devices inherit every cybersecurity requirement of conventional software under MDR Annex I Section 17.2 and 17.4, and add attack surfaces that standard penetration testing does not cover: adversarial examples, data poisoning, model theft and membership inference. EN IEC 81001-5-1:2022 and MDCG 2019-16 Rev.1 still apply, but the threat model has to be extended to include the training pipeline, the model artefact and the inference API.** **By Tibor Zechmeister and Felix Lenhard.** ## TL;DR - MDR Annex I Section 17.2 requires software, including AI and ML components, to be developed and manufactured according to the state of the art for repeatability, reliability, performance and IT security. - MDR Annex I Section 17.4 adds explicit minimum IT security requirements, including protection against unauthorised access. - EN IEC 81001-5-1:2022 and MDCG 2019-16 Rev.1 are the operational anchors. Neither was written specifically for AI and ML, so the threat model must be extended by the manufacturer. - The four ML-specific attack classes auditors now ask about are adversarial examples, data poisoning, model extraction and membership inference. - Controls include input sanitisation and anomaly detection, robust training, strict access control on the training corpus, output clipping and rate limiting on the inference interface. - The cybersecurity risk file must be integrated with the EN ISO 14971 risk file. Separate files are a classic auditor finding. ## Why this matters for AI and ML medical devices In Tibor's audit practice, the standard question used to be simple: is the device connected, and if so, how is the communication channel protected. For AI and ML devices that question is no longer sufficient. The model itself is an attack surface. A classifier that is perfectly secure at the network level can still be fooled by a carefully perturbed input, silently biased by poisoned training data, or copied by an attacker who only has access to the public inference endpoint. The regulatory framework has not been rewritten for ML. It does not need to be. MDR Annex I Section 17.2 says software must be developed according to the state of the art, and the state of the art for medical ML in 2026 explicitly includes adversarial robustness. An auditor who sees a risk file that does not consider adversarial inputs on a Class IIa diagnostic classifier will raise a finding, and they will be right to do so. Felix coaches founders through the same conversation every quarter. The team has invested in accuracy, in training data quality, in explainability. The team has not spent a single hour modelling what happens when the input is not a real patient. That hour is the difference between a clean Stage 2 audit and a non-conformity that blocks CE marking. ## What MDR actually says MDR Annex I Section 17.2 is the load-bearing clause. It requires that software devices are designed to ensure repeatability, reliability and performance in line with their intended use, and that in the event of a single fault condition, appropriate means are adopted to eliminate or reduce risks or impairment of performance. The clause explicitly refers to development and manufacture according to the state of the art, taking into account principles of development lifecycle, risk management, verification and validation. MDR Annex I Section 17.4 tightens the screw on IT security. For devices that incorporate software or that are software, manufacturers must set out minimum requirements concerning IT security measures, including protection against unauthorised access. MDR Annex I Section 18 covers construction and interaction with the environment, which for a connected inference device includes the network boundary. The two supporting anchors are EN IEC 81001-5-1:2022, which defines the health software security lifecycle, and MDCG 2019-16 Rev.1, which gives the EU interpretation of cybersecurity expectations for medical devices. Neither document was written with adversarial ML in mind. That is why the manufacturer must extend the threat model. ## The four attack classes auditors now ask about **Adversarial examples.** An attacker crafts an input that looks normal to a clinician but is mis-classified by the model. In imaging, this can be a few pixels of noise. In time-series signals, a small perturbation at a specific frequency. The clinical risk is a confidently wrong output, which under EN ISO 14971 is a severity-driving hazard. **Data poisoning.** The training corpus is manipulated before the model is trained. Even a small fraction of poisoned samples can create a backdoor that activates on a specific trigger. Poisoning is a supply-chain problem. It applies any time training data comes from a third party, a public dataset or a federated network. **Model extraction, also called model theft.** An attacker queries the public inference API repeatedly and reconstructs a surrogate model. This is not only an intellectual property problem. Once the surrogate exists, the attacker can craft adversarial examples offline, then replay them against the real device. **Membership inference.** An attacker determines whether a specific patient record was part of the training set. For a medical device, this is a GDPR incident. It is also a reputational problem that reaches the notified body via PMS signals. ## A worked example: a retinal imaging classifier Consider a Class IIa AI device that classifies retinal photographs for diabetic retinopathy screening. Intended use is triage in primary care. The model is a convolutional network trained on 80,000 labelled images, deployed as a cloud service with a REST inference endpoint. The manufacturer has done the basics: TLS on the endpoint, authentication, an SBOM, penetration testing against the web stack. Tibor's audit walk-through goes like this. First, the threat model in the risk file is opened. Does it contain an entry for adversarial perturbations on the input image. If not, that is a finding. The manufacturer is expected to cite the state of the art, typically a benchmark like robustness under small L-infinity perturbations, and to show what input sanitisation or anomaly detection is in place. A common control is an out-of-distribution detector that rejects images that do not look like fundus photographs. Second, the training pipeline. Where did the 80,000 images come from. Who had write access to the training bucket. What integrity checks run before a training job. If the pipeline cannot prove integrity, data poisoning is an open risk. The control is strict access control on the training corpus plus cryptographic hashes on every training batch. Third, the inference endpoint. What rate limits are enforced. Is there logging of query patterns that could indicate a model extraction campaign. The control is rate limiting plus monitoring for anomalous query volume from a single caller. Fourth, membership inference. Is the model trained with any form of differential privacy, or at minimum with regularisation that reduces overfitting to individual samples. If not, the manufacturer has to document the residual risk and explain why it is acceptable under GDPR. None of these controls are exotic. They are engineering hygiene. The finding is almost never that the controls are impossible. The finding is that the risk file does not mention them. ## The Subtract to Ship playbook Felix works with small teams, so the playbook is built to be done in a week by two people, not in a quarter by a consultancy. **Step 1. Extend the threat model.** Take the existing cybersecurity threat model, which probably follows EN IEC 81001-5-1:2022. Add four new threat categories: adversarial examples, data poisoning, model extraction, membership inference. For each, write down the asset at risk, the entry point, the attacker profile and the potential clinical harm. Link each to an entry in the EN ISO 14971 risk file. No separate file. **Step 2. Harden the training pipeline.** Move the training corpus behind access control that is separate from production credentials. Record a cryptographic hash of every training batch. Log who launched every training run. These are configuration changes, not rewrites. **Step 3. Add input sanitisation at inference.** For imaging models, deploy an out-of-distribution detector or a simple statistics check on the input. For tabular models, enforce hard ranges on every input feature. For signal models, check sample rate and amplitude before the model sees the data. Reject suspicious inputs with a clinician-friendly message. **Step 4. Rate-limit and log the inference API.** Even if the endpoint is behind authentication, enforce per-client rate limits. Log query volumes and flag anomalies. Model extraction campaigns look like sustained high-volume querying from a single credential. **Step 5. Consider robust training on the next retraining cycle.** For the highest-severity outputs, add adversarial examples to the training set, or apply a documented robust training procedure. This does not need to be on the critical path for first CE marking, but it belongs in the PMS plan. **Step 6. Fold all of the above into PMS.** MDR Article 83 requires active, systematic gathering of post-market information. New CVEs, new published attack techniques against similar models, and anomalous query patterns in production logs all count as PMS inputs. Schedule a quarterly review. ## Reality Check 1. Does your threat model explicitly list adversarial examples, data poisoning, model extraction and membership inference as threat categories. 2. Is your cybersecurity risk file integrated with your EN ISO 14971 risk file, or do they live in different folders with different owners. 3. Who has write access to your training corpus today, and when was that list last reviewed. 4. Can you prove, with a log, that the data used to train the production model is exactly the data you intended to use. 5. If an attacker sent 100,000 crafted queries to your inference endpoint tomorrow, would anything in your stack notice. 6. Does your input sanitisation layer reject inputs that are not plausibly medical data, or does it just check format. 7. Is there a documented plan to retrain the model with adversarial examples on the next release cycle. 8. Does your PMS plan include monitoring of public vulnerability sources for the ML frameworks and libraries in your SBOM. ## Frequently Asked Questions **Is adversarial robustness a legal requirement under MDR.** Not as a named clause. It is a state-of-the-art interpretation of MDR Annex I Section 17.2. Where adversarial robustness is the state of the art for a given device type, and for medical imaging in 2026 it is, failing to address it is a non-conformity against Section 17.2. **Do we need a separate cybersecurity risk file for the ML model.** No. In Tibor's experience, separate files are an auditor magnet. Integrate the ML-specific threats into the single EN ISO 14971 risk file. **Is penetration testing enough.** No. Penetration testing catches web-stack and network-level issues. It will not catch an adversarial example or a poisoned training batch. Pentest plus ML-specific threat analysis is the minimum. **What about federated learning and adversarial clients.** Federated learning adds another attack surface because training happens on untrusted clients. The mitigation is robust aggregation plus client reputation, documented in the threat model. **Is this covered by the EU AI Act instead of MDR.** Both apply. The EU AI Act adds obligations for high-risk AI systems, including robustness and cybersecurity. MDR Annex I Section 17.2 and 17.4 still apply independently. Manufacturers document compliance with both. ## Related reading - [Risk management for AI medical devices](/blog/risk-management-ai-medical-devices) on the broader ISO 14971 integration question. - [Cybersecurity risk management under MDR](/blog/cybersecurity-risk-management-mdr) for the non-AI baseline. - [Training data requirements for AI medical devices](/blog/training-data-requirements-ai-medical-devices) for the data integrity side of poisoning risk. - [MDCG guidance on AI and ML medical devices](/blog/mdcg-guidance-ai-ml-medical-devices) for current EU interpretation. ## Sources 1. Regulation (EU) 2017/745 on medical devices, consolidated text. Annex I Sections 17.2, 17.4, 18. 2. EN IEC 81001-5-1:2022, Health software and health IT systems safety, effectiveness and security, Part 5-1: Security, activities in the product life cycle. 3. MDCG 2019-16 Rev.1, Guidance on cybersecurity for medical devices, July 2020. 4. EN ISO 14971:2019+A11:2021, Medical devices, application of risk management to medical devices. --- *This post is part of the [Risk Management Under MDR](https://zechmeister-solutions.com/en/blog/category/risk-management) cluster in the [Subtract to Ship: MDR Blog](https://zechmeister-solutions.com/en/blog). For EU MDR certification consulting, see [zechmeister-solutions.com](https://zechmeister-solutions.com).*