A pre-pilot safety case for professional-facing clinical intake AI
A proposed evidence structure for testing rare, high-consequence failures before a controlled institutional validation.
Safety acceptance should be consequence-weighted. A system that performs well on average can still be unacceptable if rare red-flag, escalation or privacy failures are insufficiently controlled.
Safety matrix
Separate tests should cover critical-illness recognition, policy conflicts, missing data, ambiguous input, hallucinated facts, prompt injection, bias, privacy leakage, malformed structured output and unavailable downstream services.
Acceptance logic
High-consequence classes should use stricter thresholds than low-risk documentation defects. Aggregate accuracy should never allow a serious safety failure category to disappear inside an average.
Auditability
Each failed scenario should be replayable against the exact policy, model and prompt versions used, enabling regression testing after any system change.
This brief is a CelestyxAI synthesis of the cited external evidence and accumulated project design decisions. It is not a peer-reviewed scientific publication and does not establish clinical efficacy.