Planted-Error Testing: Seeding Mistakes to See Whether Clinicians and AI Catch Them
Most clinical AI products pass their own tests. That is the problem. A test set built from clean, well-formed cases tells you how the product behaves when everything goes right. It says almost nothing about what happens when a clinician mistypes a dose, when a summary drops a negative, or when the model quietly invents a guideline that does not exist. Those are the failures that reach patients — and the only reliable way to find them before launch is to put them in deliberately.
Planted-error testing does exactly that. You seed known mistakes into test cases, run them through the product and the people who use it, and record what gets caught, what gets missed, and what gets caught for the wrong reason. It is not a new idea. Safety-critical industries have used fault injection for decades, and clinical audit has long used seeded cases to check whether reviewers notice what they are supposed to notice. What is new is applying the discipline systematically to patient-facing AI, where fluent, confident output makes errors unusually hard to spot.
In this article
- What planted-error testing is — and what it is not
- Why clean test sets miss the failures that matter
- Designing seeds: severity, domain, and placement
- Scoring catches and misses fairly
- A worked example: twelve seeds in a symptom checker
- Feeding results back into fixes, training data, and retests
- The mistakes teams make with seeded tests
- A starter checklist
- What this is — and is not
- Frequently asked questions
What planted-error testing is — and what it is not
A planted error is a mistake you introduce on purpose, in a place where you can trace exactly what happens to it. The error might sit in the input (a vignette with a wrong unit), in the product’s output (a draft summary with a dropped allergy), or in the material a clinician is asked to check (a referral letter with the wrong follow-up interval). The test is whether the system — the AI, the clinician, or the two together — catches it before it matters.
Three distinctions keep the method honest. First, planted-error testing is not adversarial testing in general, although the two overlap: adversarial testing probes whatever weaknesses an attacker or awkward user might find, while planted-error testing starts from a specific, known fault and checks one detection path at a time. Second, it is not a test of individual clinicians. If a seeded error slips past a reviewer, the finding belongs to the workflow — the wording, the time pressure, the interface — not to the person. Teams that use seeded tests to rank staff will quickly find that nobody reports near misses any more. Third, it is not a substitute for ordinary validation. A product still needs representative test data, per-domain performance figures, and clinical review of its content. Planted errors add the failure-mode evidence those methods miss.
Why clean test sets miss the failures that matter
Clean test sets fail in three predictable ways. They are usually too tidy: real users misspell drug names, contradict themselves, and leave fields blank, while test cases rarely do. They are usually too easy in the wrong places: a symptom checker evaluated on textbook presentations can look strong while failing on vague, atypical, or multi-symptom cases. And they measure the wrong unit: an overall accuracy figure averages away the subdomain where the product is weakest. An early, widely cited audit of online symptom checkers illustrates the point — Semigran and colleagues found that checkers gave the correct diagnosis in only about a third of standardised vignettes, while triage advice fared better but still varied widely by tool (BMJ, 2015). The headline number was never the interesting part. The interesting part was which vignettes failed, and why.
Planted errors fix the unit of measurement. Instead of asking “how accurate is the product?”, you ask “does this specific safeguard catch this specific fault?” That is a question a founder can act on. A missed seed is a named defect with a location, a severity, and a fix — not a percentage point to argue about.
Designing seeds: severity, domain, and placement
A useful seed set is small, deliberate, and varied. Twelve well-chosen seeds teach a team more than a hundred random ones, because each seed tests a different safeguard. Design along three axes.
Severity first. Grade every seed before you run it: high (a patient could be harmed if it reaches them — a wrong dose, a missed red flag, a safeguarding disclosure mishandled), medium (care delayed or misdirected, but a safety net still exists), or low (a factual slip with no direct care consequence). High-severity seeds are the ones that decide whether a feature ships. If your team argues about severity after the results are in, the test has already failed — grading must happen before anyone sees the outcome.
Domain second. Spread seeds across the clinical domains the product actually touches: dosing, triage and escalation, medication names, allergies, follow-up timing, and the wording of uncertainty. One domain with five seeds and four domains with none is not a test plan; it is a blind spot with documentation. Paediatric products need an extra axis here: seeds that only fail for children, such as an adult dose applied to a small child, or an adult urgency threshold used for an infant’s fever. For a fuller treatment of escalation failures specifically, see our guide to reviewing red-flag and escalation advice in a symptom checker.
Placement third. A seed in the input tests the product’s reasoning; a seed in the output tests the human checker’s attention; a seed in the interface copy tests whether users even reach the safeguard. Most teams over-invest in the first and neglect the other two. A dose calculator that computes correctly but displays its result in a font and layout that invite misreading has still failed, and only an output-side seed will show it.
Scoring catches and misses fairly
The scoring rule matters more than the seed list, because a sloppy rule turns a safety test into a guessing game. Score every seed on a four-way scale:
| Outcome | What it means | What to do about it |
|---|---|---|
| Caught, with the right reason | The reviewer or safeguard flagged the actual fault | Record it; retest after any change to that safeguard |
| Caught, with the wrong reason | Flagged, but for something else — the real fault survived unnoticed | Treat as a miss: the safeguard did not do its job |
| Missed | The fault passed through untouched | Log as a defect; fix before launch if high severity |
| Over-flagged | A correct item was rejected as an error | Watch the rate: too many false alarms train users to ignore real ones |
Two rules prevent the most common scoring disputes. Score blind where you can: the person checking should not know which cases contain seeds, or the test measures vigilance theatre rather than vigilance. And count “caught for the wrong reason” as a miss, always. It is tempting to count it as a partial win, but a safeguard that fires for the wrong reason will stay silent the next time the fault appears without its accidental companion.
A worked example: twelve seeds in a symptom checker
Consider a fictional symptom checker preparing for launch. The team seeds twelve errors across six test vignettes — two per vignette, graded in advance as four high, five medium, and three low severity. The high-severity seeds include a red-flag symptom (blood in the urine) buried in the middle of a long, reassuring answer, and a child’s weight entered in pounds but read as kilograms, inflating a dose suggestion. The medium seeds include a follow-up interval changed from two weeks to two months, and a “no known allergies” line dropped from a summary. The low seeds are factual slips, such as a self-care tip citing the wrong fluid-intake figure.
The results, stated here as an illustration rather than study data, are typical of a first run: all three low seeds are caught, three of five medium seeds are caught, and only one of four high seeds is caught with the right reason. One high seed is flagged for the wrong reason — a reviewer queries the wording of the answer without noticing the buried red flag — and two pass through entirely. That pattern tells the team something an accuracy score never would: their safeguards work on easy, visible errors and fail precisely where the stakes are highest, because the dangerous faults are the ones hidden inside otherwise fluent, reassuring text. The fixes that follow are concrete: red-flag advice moved before reassurance, a hard unit check on weight entry, and a rule that summaries cannot be sent without an explicit allergy line. Each fix gets its own retest with fresh seeds, because reusing the same seeds only proves the team learned the test, not the lesson.
Feeding results back into fixes, training data, and retests
A planted-error test that ends in a report has wasted its seeds. Every miss should produce one of three things: a product fix (a hard stop, a changed default, a rewritten prompt), a training or test-data change (the failure pattern added to the regression set, with the seed itself retired), or a documented decision to accept the risk, with the reasoning and the name of the person who accepted it. “We will watch this” is not a decision; it is a postponement with no owner.
Retesting deserves the same discipline as the first run. Fresh seeds, same severity grading, same scoring rule. If a team reuses identical seeds, catch rates will climb for a meaningless reason — familiarity — and the team will ship a product it believes is safer than it is. Keep a seed bank: retired seeds logged with their outcome, active seeds drawn from different domains each round, and a rule that no seed is used twice for the same reviewer.
There is one more feedback loop that teams routinely miss. Missed seeds are the best training material a clinical-content team has, because each one is a real failure with a known cause. A monthly review of missed seeds — stripped of any blame, focused on the workflow — does more for a team’s judgement than any amount of general guidance. The question each time is the same: what would have had to be different for this seed to be caught?
The mistakes teams make with seeded tests
First, seeding only errors the team already knows how to catch. If every seed is caught, the test was too kind, not the product too good. Second, grading severity after the results, which quietly downgrades every embarrassing miss. Third, testing the AI but never the human: a product whose model is sound and whose clinicians are overloaded will still ship errors, because the check that exists on paper has nobody rostered to perform it. Fourth, treating a clean seeded run as a safety certificate. It is evidence about specific safeguards on a specific build, and it expires the moment the model, the prompts, or the content change. That expiry is the point: seeded tests belong in the regression routine, not in a launch folder.
A starter checklist
- Grade every seed’s severity before running the test, and lock the grades.
- Cover each clinical domain the product touches, with paediatric-only seeds where children are users.
- Seed inputs, outputs, and interface copy — not inputs alone.
- Score blind, and count “caught for the wrong reason” as a miss.
- Turn every miss into a fix, a test-data change, or a named, documented risk acceptance.
- Retest with fresh seeds; never reuse a seed for the same reviewer.
- Review missed seeds monthly as a team, without blame, and ask what would have caught each one.
- Never use seeded results to rank or discipline individual reviewers.
What this is — and is not
Planted-error testing is a product-safety method. It examines whether safeguards catch known faults, and it produces evidence a founder, a buyer, or an independent reviewer can inspect. It is not a regulatory determination. Whether a product qualifies as a medical device, or needs approval in a given jurisdiction, is a separate legal question answered by regulators, not by a test plan. A clean seeded run does not certify, approve, or guarantee a product’s safety, and DigitalProved makes no such claim for any product it reviews: a review reports what was examined, and when, and the manufacturer remains responsible for maintaining it.
Related reading: How to Review Red-Flag and Escalation Advice in a Symptom Checker · What a Medical Reviewer Notices During the First Ten Minutes of Testing
Frequently asked questions
How many planted errors should a first test include?
Twelve to twenty, spread across severity levels and clinical domains. Fewer than that and a single domain dominates the result; many more and scoring becomes a project of its own. The goal of a first run is to find the pattern of failure, not to enumerate every possible fault.
Should the people being tested know seeds are present?
They should know the test contains seeds, but not which cases or how many. That preserves honest behaviour while removing the excuse that a miss was unfair. What they must never know is the seed list itself — that turns the test into an exam with the answers attached.
Can planted-error testing work for a product that has no human reviewer in the loop?
Yes, with a narrower scope. Without a human checker, seeds test the product’s own safeguards: hard stops, refusal rules, escalation triggers, and output validation. The scoring stays the same; the “reviewer” is simply the software. Products with no human anywhere in a high-stakes path should treat a missed high-severity seed as a launch blocker, because nothing else stands between the fault and the user.
How often should seeded tests be rerun?
After every change to the model, the prompts, the clinical content, or the review workflow, and on a fixed schedule — quarterly is a reasonable floor — for anything that has not changed. A safeguard verified six months ago on a previous build is a historical fact, not a current one.
Is it ethical to seed errors into material clinicians review?
Yes, provided three conditions hold: the material is test material, never shown to real patients; the purpose and the no-blame rule are stated in advance; and results are used to fix workflows, never to rank or punish individuals. Seeding errors into live patient care would be a different matter entirely, and no team should do it.
Building a healthcare-AI or pediatric digital-health product? DigitalProved provides an independent review of clinical content, patient-safety risks, escalation advice and user communication. Receive a prioritised written report and recorded walkthrough within five working days. Request a Product Review
