In November 2024, Team-NB — the association representing Europe's medical device Notified Bodies — published a questionnaire on artificial intelligence in medical devices. It contains 189 questions.
That is 189 questions a Notified Body auditor can now ask, one by one, when reviewing the technical file of any medical device that uses AI. They cover everything from training data sourcing to model validation to post-market monitoring — and they come on top of the existing MDR technical documentation requirements, not instead of them.
The questionnaire exists because there is no harmonised standard for AI in medical devices yet. CEN-CENELEC is working on it — EN 18286 entered public enquiry in late 2025 — but until it is published, Notified Bodies need something to audit against. So they wrote their own checklist. It is thorough. It is public. And it is already being used in audits.
Most of the 189 questions are manageable if your quality system is in order. But a handful require evidence that most technical files do not contain today. Here are six that consistently catch teams off guard.
The six questions
Can you reproduce your training and validation results?
Not "did you save the final model weights." Can someone re-run your training pipeline from raw data and arrive at the same model, with the same performance numbers? This requires version control on the data, the preprocessing steps, and the training configuration — not just the code. Most teams version the model. Very few version the full reproducibility chain.
How did you verify that your training data is representative of your intended patient population?
The questionnaire references ISO 13485 design input requirements. "We used a public dataset" is not an answer. The auditor wants documented evidence that you assessed demographic coverage, site-level variation, and known subgroup gaps — and that you made a conscious, recorded risk decision about whatever gaps remain.
What is your model's performance across clinically relevant subgroups?
Not overall accuracy. Stratified performance — broken out by the patient characteristics and operating conditions that matter for your intended use. If your device processes chest X-rays, what is the performance on portable versus fixed-unit acquisitions? On paediatric versus adult patients? One aggregate number does not satisfy this question, regardless of how high it is.
How do you detect that real-world input data has shifted away from what the model was trained on?
This is asking whether you have a drift detection mechanism in place. Most teams do not. The questionnaire expects a documented approach — not necessarily a fully automated system, but a defined method and monitoring plan. A vague statement about "ongoing monitoring" in the post-market surveillance plan does not cover this question.
What measures ensure the AI model operates safely within its validated scope?
This is the continuous-learning trap. The German Notified Bodies Alliance has stated explicitly that "dynamic AI" — systems that continue learning after deployment — are "not certifiable in principle" unless the manufacturer can validate safety at a defined development state. If your model updates in the field, this question requires serious, specific documentation of how safety is maintained between validation cycles.
Can you provide the Notified Body with access to your training, validation, and test datasets?
This one comes from Annex IV of the AI Act itself. The regulation states that the Notified Body shall be granted full access to training, validation, and testing datasets, including through API or other technical means enabling remote access. If your training data includes patient images, you need a documented plan for how to satisfy this requirement while maintaining data protection compliance. This is an open tension in the regulation — and "we did not think about it" is not a position you want to be in during an audit.
What this means in practice
None of these questions are unreasonable. All of them require evidence that takes time to produce — in some cases, time that requires going back to the data, re-running analyses, or implementing monitoring infrastructure that was not in scope during the original development cycle.
The Team-NB questionnaire is public. The gap analysis — identifying which of the 189 questions your current technical file cannot answer — takes an afternoon. The evidence production takes considerably longer, and that is the constraint that matters given where the calendar sits relative to December 2027.
The most useful thing you can do with the questionnaire is not read it as a compliance checklist to fill in. Read it as a window into what an auditor is actually looking for: reproducibility, representativeness, subgroup honesty, and a documented plan for what happens after deployment. Those four themes run through most of the questions that catch teams off guard.