Per-Domain Safety Thresholds: Why One Overall Accuracy Score Hides Paediatric Failure
- The headline number is a hiding place
- Why one number misleads in paediatrics
- Step 1: Define the domains before you measure them
- Step 2: Set a floor for each domain, not a target for the average
- Step 3: Decide in advance what happens below the floor
- A worked example: 93% overall, 71% where it matters
- A founder’s checklist for per-domain thresholds
- Frequently asked questions
A founder shows you a slide: 93% accuracy. The test set is large, the methodology looks sound, and the number is better than the last release. It feels like a result worth shipping.
It is a real number. It is also the wrong number to make a safety decision with.
An overall accuracy score is an average, and averages hide exactly the thing that matters in a paediatric product: the small, high-stakes corner where the product is weak. A symptom checker can answer general health questions brilliantly and still miss the red flags in a feverish two-year-old. A dosing assistant can be right on routine calculations and wrong on the one weight band where the error harms a child. The headline figure stays high in both cases, because the failing corner is small and the easy cases are many.
This article explains how to stop judging a paediatric product by one number: define the domains that matter, set a minimum floor for each one, and decide in advance what happens when a domain falls below its floor.
The headline number is a hiding place
Overall accuracy answers one question: across everything the product was tested on, how often was it right? That is a reasonable question about general quality. It is a poor question about safety, for three reasons.
First, test sets reflect what is easy to collect, not what is dangerous to get wrong. General information questions and appointment queries are plentiful. Safeguarding disclosures, dosing edge cases and red-flag presentations in infants are rare — and they are the cases where a wrong answer does real harm.
Second, a strong result in a large, low-risk domain can cover a failing result in a small, high-risk one. If 750 of your 1,000 test cases are low-risk and the product scores 98% on them, the remaining 250 cases could be little better than chance and the overall score would still look respectable.
Third, a single number gives a team nothing to act on. “Improve accuracy from 93% to 95%” is not a plan. “Raise red-flag recognition in under-5s from 71% to 90% before the next release” is a plan — it names the domain, the gap and the work.
The point is not that overall accuracy is useless. It is a useful summary of general performance. The point is that it must never be the safety claim. Safety is judged domain by domain.
Why one number misleads in paediatrics
Children are not one population, and paediatric products do not do one job. Both facts make the aggregate trap worse than it is in adult products.
Age changes the task. A fever question about a six-week-old is a different clinical problem from the same question about a fourteen-year-old: different red flags, different thresholds for urgent care, different wording a parent needs. A product tested mostly on school-age children has barely been tested on infants at all, however good its headline score looks. We covered the risks adult-focused teams commonly miss in children in our article on seven paediatric risks adult-focused digital-health teams commonly miss.
Severity is uneven. Most of what a patient-facing product does is low-stakes: explaining a condition, booking an appointment, reminding a parent about a follow-up. A small share is high-stakes: recognising a red flag, giving a dose, responding to a disclosure of harm. Averages treat a correct appointment booking and a correct red-flag call as equal contributions. No parent, clinician or buyer would agree with that weighting.
Rare failures stay invisible longest. A domain with fifty test cases contributes little to the overall score, so a product can ship with a known weak spot in exactly the domain where cases are rare and consequences are severe. This is the same statistical pattern behind Simpson’s paradox: a trend that looks fine in the combined data reverses inside a subgroup. You only find it by looking inside the subgroups.
And paediatric data is thinner. There is less of it, it is harder to collect ethically, and it is split across age bands that behave differently. A test set that looks large overall can be very small once it is cut by age band and clinical task — small enough that a single percentage point swings on a handful of cases. That is an argument for more careful domain reporting, not for less testing.
Step 1: Define the domains before you measure them
A domain is a slice of the product’s work that can fail on its own. Define the slices before you run the tests, not after you see the results — defining them afterwards is how weak spots get folded into a flattering average.
Two cuts cover most paediatric products.
Cut by clinical task. What is the product actually doing? Typical domains: general health information, red-flag recognition and escalation advice, dosing and medication information, triage or disposition advice, safeguarding and disclosure handling, appointment and administrative help, and self-care guidance. Each of these can be right or wrong independently, and each carries a different level of risk.
Cut by age band. Neonates and infants, toddlers, school-age children, and adolescents are different users with different physiology, language and risk. A domain that passes for school-age children may fail for infants. If your product serves more than one age band, each band is a domain in its own right for the high-risk tasks at least.
A practical domain map for a symptom checker might look like this:
| Domain | Why it is separate |
|---|---|
| Red-flag recognition, under-5s | Highest-stakes judgement; age-specific signs |
| Red-flag recognition, 5 and over | Same task, different presentations |
| Dosing information | Weight-based arithmetic; unit traps |
| Safeguarding disclosures | Rare, severe, needs a human route |
| General health information | High volume, low stakes |
| Appointment and admin help | High volume, lowest stakes |
Keep the map small enough to maintain — six to ten domains is plenty for a small team. The test is whether each domain could fail while the others pass. If two domains always fail together, they are one domain.
Step 2: Set a floor for each domain, not a target for the average
A floor is a minimum acceptable result for one domain, agreed before testing. A product ships only if every domain clears its floor. One domain below its floor holds the release, however good the overall score is.
Floors are not uniform, and they should not be. The floor follows the harm of getting it wrong:
- Red-flag recognition and escalation advice: the highest floor. This is the domain where a miss delays urgent care. Set it with clinical input, and set it high.
- Dosing information: also a high floor, because a wrong dose is a wrong dose even when the arithmetic looks confident. Weight-based dosing is the highest-risk content a patient-facing product can produce, and a floor here should reflect that.
- Safeguarding disclosures: accuracy alone is the wrong measure here. The floor should cover recognition and routing — the product must not only spot the disclosure, it must get the user to a human and a proper channel. A product that recognises a disclosure and then handles it alone has failed the domain.
- General information and admin tasks: a lower floor is defensible. A wrong opening-hours answer is embarrassing; a missed red flag is dangerous. The floors should say so.
How to pick the numbers: start from the harm, not from what the model currently achieves. Ask a clinician what error rate they would accept in that domain for a product parents will use without supervision. Then check what a careful human comparator achieves on the same test set — a floor far above human performance is a wish, not a standard. And write the floors down, with the reasoning, before the test run. A floor set after seeing the results is a description of the results, not a standard.
One caution: a per-domain floor is a product-safety practice, not a regulatory approval. Clearing your own floors does not make a product a regulated medical device, and it does not mean a regulator has approved it. That distinction matters, and it is worth stating plainly to buyers: thresholds are how you hold yourselves to a standard, not a certificate. For the difference between clinical review, usability testing and regulatory assessment, see our comparison of clinical review, usability testing and regulatory assessment.
Regulators are moving in the same direction. The US Food and Drug Administration’s guidance on predetermined change control plans expects manufacturers of AI-enabled device software to pre-specify how a model will change and how each change will be validated — the same discipline of deciding the standard before the result. In the UK, NICE’s Evidence Standards Framework for digital health technologies sets evidence expectations by risk tier and, since its 2022 update, includes standards for technologies whose performance changes over time. Neither framework tells a founder what number to pick for a paediatric subdomain. Both point the same way: judge performance where the risk sits, not in aggregate.
Step 3: Decide in advance what happens below the floor
A floor without a consequence is a decoration. Before testing, agree what happens when a domain misses its floor. The decision ladder, in order of severity:
- Fix and retest. The domain is reworked — more training data, tighter prompts, a guardrail, a changed workflow — and retested on a fresh set before the release proceeds.
- Scope-limit the domain. If the product cannot yet handle dosing questions for infants safely, it stops answering them for infants and routes those users to a clinician or a safe default. A narrower product that knows its limits is safer than a broad one that does not.
- Escalate to a human. For the domains where a miss is dangerous, the fallback is a person. This only works if someone is actually rostered to respond — an escalation step that exists on paper but has nobody behind it is not a safety measure. Our article on human oversight in healthcare AI covers what that handoff has to look like in practice.
- Hold the release. If a high-risk domain is below its floor and cannot be fixed or safely scoped, the release waits. This is the hardest call, and it is the reason the floors and the consequences are agreed before testing, when the pressure to ship is lowest.
Record the decision and the reasoning. A buyer or a reviewer who asks “what happens when a domain fails?” should get a written answer, not a shrug. That record is also the beginning of the evidence file a buyer will ask for before a pilot — the same file we described in our article on why accurate medical content is not enough for a safe AI product.
A worked example: 93% overall, 71% where it matters
The numbers below are an illustrative example, built to show the arithmetic — not results from any real product. A symptom checker is tested on 1,000 cases, split across five domains:
| Domain | Cases | Correct | Score |
|---|---|---|---|
| General health information | 500 | 490 | 98% |
| Appointment and admin help | 250 | 245 | 98% |
| Red-flag recognition, under-5s | 120 | 85 | 71% |
| Dosing information | 80 | 63 | 79% |
| Safeguarding disclosures | 50 | 44 | 88% |
| Overall | 1,000 | 927 | 93% |
The headline is 93%. The product’s own floors were 90% for red-flag recognition, 90% for dosing, 90% for safeguarding, and 85% for general information and admin.
Two domains are below their floors: red-flag recognition in under-5s (71% against a 90% floor) and dosing (79% against a 90% floor). The overall score passes easily. The product does not ship.
Look at what the average was hiding. The two failing domains together are only 200 of the 1,000 cases — one in five. They are also the two domains where a wrong answer can harm a child. The 750 low-risk cases scored 98% and contributed most of the headline figure, while the 120 red-flag cases — the ones that decide whether a sick toddler gets seen tonight or tomorrow — were right barely seven times in ten.
The fix follows the decision ladder. Red-flag recognition in under-5s is reworked and retested on a fresh set, with a clinician reviewing the misses to find the pattern — in this kind of case, the pattern is often a specific presentation, such as fever in infants under three months, that the test set under-represents. Dosing is scope-limited while it is reworked: the product stops giving weight-based doses for children under two and routes those questions to a clinician, rather than answering them at 79%. The release proceeds only when both domains clear their floors on a fresh test set.
This is also where planted-error testing earns its keep: seeding known mistakes into the test set shows whether the review process catches the failures that clean test sets miss — particularly in the small, high-risk domains where a handful of misses matter most.
A founder’s checklist for per-domain thresholds
Use this before your next release. It mirrors the structure of a full safety self-check — domains, floors, consequences, records — without standing in for one.
- Map the domains first. List the product’s clinical tasks and the age bands it serves. Each combination that can fail on its own is a domain.
- Give every high-risk domain its own floor. Red-flag recognition, dosing and safeguarding each get a floor set by harm, not by what the model happens to score today.
- Write the floors down before testing. A floor agreed after the results is not a standard.
- Test each domain separately. Report the score, the case count and the floor for every domain — not just the overall figure.
- Count the small domains. A domain with fifty cases can still be the most important row in the report. Say so, and do not let its size excuse a low score.
- Agree the consequence in advance. Fix and retest, scope-limit, escalate to a human, or hold the release — decided before the pressure to ship arrives.
- Check the human route is real. If the fallback is a person, someone must be rostered, reachable and able to act. An unstaffed escalation step is not a safety measure.
- Keep the record. Floors, results, decisions and retest outcomes, in one place, so a buyer or reviewer can inspect the reasoning — not just the headline number.
Frequently asked questions
Q: Isn’t overall accuracy still a useful number?
A: Yes — as a summary of general performance and a way to track improvement between releases. It stops being useful when it is presented as the safety claim. Safety is judged domain by domain, because harm happens in specific domains, not in the average.
Q: How many domains should a small team track?
A: Six to ten is enough for most paediatric products. Cut by clinical task first — red-flag recognition, dosing, safeguarding, general information, admin — and then split the high-risk tasks by age band. If the list grows past a dozen, some domains can be merged; if a domain never fails, it can stay on the list with a light-touch check rather than being dropped.
Q: What if a domain has too few test cases to measure reliably?
A: Say so in the report, and treat the thin domain with more caution, not less. A small test set means the score is uncertain, so the floor should be applied conservatively — for example, requiring the domain to clear its floor with a margin, or to pass a clinician review of every miss. Collecting more cases for that domain becomes part of the next test plan.
Q: Should the floors be the same for every age band?
A: No. The floor follows the harm, and the harm differs by age. Red-flag recognition in infants carries a higher floor than the same task in adolescents, because the margin for delay is smaller and the signs are subtler. Set each floor with clinical input for that band.
Q: Does clearing all the floors mean the product is safe?
A: No. Floors are a minimum standard your team sets and holds itself to — they are not a regulatory approval, a certification, or a guarantee. A product can clear its floors and still have failure modes the test set did not cover. Thresholds are one part of safety work, alongside clinical review, oversight design, incident planning and post-launch monitoring.
Sources
- US Food and Drug Administration, Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions — final guidance, December 2024 (current version August 2025). https://www.fda.gov/media/166704/download
- National Institute for Health and Care Excellence, Evidence Standards Framework for Digital Health Technologies (ECD7) — published December 2018, last updated August 2022. https://www.nice.org.uk/corporate/ecd7/resources/evidence-standards-framework-for-digi-tal-health-technologies-pdf-1124017457605
Building a healthcare-AI or pediatric digital-health product? DigitalProved provides an independent review of clinical content, patient-safety risks, escalation advice and user communication. Receive a prioritised written report and recorded walkthrough within five working days. Request a Product Review
