Hospitals Can’t See Inside Your AI Tool — So the AHA Wants the FDA to Make You Prove It Works After Launch
On 19 October 2026, the public comment period closed on the FDA’s discussion paper on generative AI-enabled medical devices. It drew 541 submissions. Two of them matter more than most for anyone building healthcare AI: the letters from the American Hospital Association (AHA) and the American Medical Association (AMA). They were written independently, by organisations that represent hospitals and physicians respectively — and they land on the same point. The people who buy and use your tool cannot see inside it, so proving that it keeps working after launch is becoming your job, not theirs.
I write as a pediatrician by training who now reviews healthcare-AI products. This article is about what that convergence means in practice for founders and vendors, and what to build before a buyer — or a regulator — asks for it. It is principle-level only and is not legal advice.
Table of Contents
- What the AHA told the FDA
- The AMA said the same thing, differently
- Why this convergence matters for founders
- Practical founder checklist: post-market monitoring as part of the product
- Frequently asked questions
What the AHA told the FDA
The AHA’s letter to the FDA is dated 19 October 2026 and was reported the following day by Becker’s Hospital Review. Its central argument is about visibility. Hospitals, the AHA said, lack the technical visibility to independently assess these “black box” tools. A hospital can observe what a tool does in its own setting; it cannot inspect how the model underneath behaves, drifts or changes when the vendor updates it.
From that, the AHA draws a clear allocation of responsibility: vendors and developers — not hospitals — must be responsible for the ongoing performance, maintenance and integrity of the tools they market. It also asks the FDA to set post-market evaluation standards for developers, including performance metrics, thresholds that trigger additional evaluation, and requirements for communicating performance information to healthcare organisations.
One figure in the letter explains why hospitals are pushing this upwards. An AHA survey cited in the letter found that 74% of hospitals have multiple teams responsible for evaluating predictive AI. Evaluation inside hospitals is already fragmented across teams. Asking each of those teams to also reverse-engineer a vendor’s model is not a safety system; it is a gap dressed up as one.
The AMA said the same thing, differently
The AMA also wrote to the FDA, in a letter sent on 16 October and reported in October 2026. Its position, as covered by Medical Device Network and Medical Liability Monitor, is that regulatory flexibility must not compromise safety. On the role of clinicians, the AMA’s line is the one founders should read twice: physician expertise should “inform the standards, not serve as the safety net”.
That sentence rules out a design pattern we see often in review work — a product whose safety case quietly depends on a busy clinician catching its errors. We covered the practical side of this in our article on human oversight in healthcare AI: oversight only works when the person, the threshold and the authority to act are designed in advance. The AMA is now making the same point at policy level. It also argues that manufacturers must remain accountable for product performance, with stronger post-market surveillance than manufacturer self-reporting alone.
Why this convergence matters for founders
When the hospitals’ association and the physicians’ association tell the same regulator the same thing in the same week, the direction of travel stops being speculative. Post-market performance is moving from a buyer’s diligence question to a defined vendor obligation. Even before any rule is finalised, procurement teams read these letters too — expect the questions in your next enterprise evaluation to sound very much like the AHA’s list.
Three of DigitalProved’s long-standing positions line up directly with what the AHA is asking for. First, evaluate the deployed product as a whole — the model, the interface, the escalation routes and the setting it runs in — not a benchmark score from before launch. Our practical review framework for pediatric digital-health applications takes that whole-product view. Second, set minimum performance thresholds per domain, not one average across everything: a tool can perform well overall and still be unsafe in the one domain that matters, such as red-flag symptoms in children. Third, re-benchmark after any upstream model change. If your product sits on a foundation model that the provider updates, your launch evidence describes a product that may no longer exist — the pre-launch safety checks need a post-launch counterpart.
None of this replaces regulatory assessment, and an independent clinical review is not an approval. We set out the difference in clinical review, usability testing and regulatory assessment. What the AHA and AMA letters change is who is expected to hold the evidence — and the answer is the vendor.
Practical founder checklist: post-market monitoring as part of the product
- Write the monitoring plan before launch, not after the first incident. Name what you will measure in live use, how often, and who owns it.
- Set per-domain performance thresholds. Define the minimum acceptable performance for each clinical domain and user group your tool serves, not a single blended score.
- Define re-evaluation triggers in advance. State the threshold breaches, complaint patterns or error rates that trigger additional evaluation — the exact mechanism the AHA asks the FDA to require.
- Treat every upstream model change as a release. Re-run your benchmark set after any foundation-model update, version change or prompt change, and keep the before-and-after results.
- Build a buyer-ready evidence file. Keep your metrics, thresholds, trigger history and re-evaluation results in a form you can hand to a hospital evaluation team without a fortnight of preparation.
- Decide how you communicate performance to customers. The AHA wants requirements for communicating performance information to healthcare organisations — having a clear, regular format ready is a competitive advantage now, not just compliance later.
- Do not design clinicians in as the safety net. If your safety case needs a user to catch the tool’s mistakes, redesign the product; use clinician expertise to set the standards and thresholds instead.
None of these steps needs a regulation to be finalised before it is worth doing. Each one also makes an independent review faster and cheaper, because the evidence already exists.
Frequently asked questions
What did the AHA ask the FDA to do about AI tools?
In its letter dated 19 October 2026, the AHA argued that hospitals lack the technical visibility to independently assess “black box” AI tools, that vendors and developers must be responsible for the ongoing performance, maintenance and integrity of the tools they market, and that the FDA should set post-market evaluation standards for developers — including performance metrics, thresholds that trigger additional evaluation, and requirements for communicating performance information to healthcare organisations.
How many submissions did the FDA’s generative AI discussion paper receive?
The public comment period closed on 19 October 2026 with 541 submissions, including letters from the American Hospital Association and the American Medical Association.
What does “inform the standards, not serve as the safety net” mean?
It is the AMA’s formulation for the role of physician expertise: clinicians should help set the standards a product is judged against, rather than being relied on to catch the product’s errors in daily use. Manufacturers, the AMA argues, must remain accountable for product performance.
What should healthcare-AI founders do now?
Build post-market monitoring into the product: a written monitoring plan, per-domain performance thresholds, defined triggers for additional evaluation, re-benchmarking after any upstream model change, and a buyer-ready evidence file that a hospital team can inspect.
Building a healthcare-AI or pediatric digital-health product? DigitalProved provides an independent review of clinical content, patient-safety risks, escalation advice and user communication. Receive a prioritised written report and recorded walkthrough within five working days. Request a Product Review
