Most vendor demos show the chatbot working. Few show it failing safely. Use these questions to find out.
Scope
- What is the bot allowed to do, and what is it blocked from doing?
- Does it give medical advice, or general information only? Where is that line written?
Safety behaviour
- What happens with an emergency phrase?
- How does it handle children, pregnancy and medication questions?
- What does it say when it does not know?
Evidence
- Which clinician reviewed the content, and when?
- How are answers tested, with how many test cases, and are failures recorded?
- Can you see examples of answers it got wrong and what changed afterwards?
Operations
- Who monitors conversations, and how fast do they respond to a problem?
- How are updates tested before they go live?
- Where is conversation data stored, and who can read it?
- What is the process if something goes wrong: who is told, in what order?
Score each answer: clear, partly clear, or vague. Two or more vague answers in the safety section is a reason to pause.
Want a quick read on your own bot? Try the free Chatbot Safety Self-Check. It takes about five minutes.
General information, not legal or clinical advice.
Related reading: How a Healthcare Chatbot Should Hand Off to a Real Clinician in a Crisis · Why a Symptom Checker Needs Different Red Flags for Children · Ten Clinical-Safety Checks Before Launching a Healthcare Chatbot · Why Accurate Medical Content Is Not Enough for a Safe AI Product
