响应儿科健康查询的消费者人工智能系统中的安全边界维护：在自然和对抗性压力条件下的跨平台基准评估
Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions.
摘要(中文译文)
消费者人工智能聊天机器人如今被数以亿计寻求健康信息的用户所使用，然而，对其在真实世界照护者压力下维持安全边界情况的系统性评估仍然匮乏。我们评估了PediatricSafetyBench-v2，这是一个包含600条儿科健康查询的基准测试集，由300条来自HealthCareMagic-100k-en医师咨询语料的真实照护者查询，以及300条匹配的对抗性变体组成，后者纳入了六种可操作的照护者压力模式，覆盖四个消费者AI系统（GPT-4o-mini、Gemini-2.0-Flash、Claude-3.5-Haiku和Llama-3.1-8B）。安全边界维持情况采用经过验证的五组分安全综合评分（Safety Composite Score，最高15分；安全适当阈值为10分及以上）进行评估，该评分在应用于全语料库之前已针对独立人类评分者进行验证（平均加权kappa为0.76；Pearson r = 0.88）。总体安全适当率为95.5%。部署以安全为导向的系统提示使全部四个模型的安全适当率提高了5.9个百分点。与直觉相反，对抗性照护者压力在全部十个主题类别和严重程度水平上，均与四个模型更高的而非更低的安全综合评分值相关。虚假专业声称是最易诱发脆弱性的压力模式，而情绪升级则与最高评分相关。消费者AI系统在绝大多数儿科健康交互中维持了安全边界。PediatricSafetyBench-v2已公开发布，用于纵向安全监测。
Abstract
Consumer artificial intelligence chatbots are now accessed by hundreds of millions of users seeking health information, yet systematic evaluation of their safety boundary maintenance under real-world caregiver pressure remains scarce. We evaluated PediatricSafetyBench-v2, a benchmark of 600 pediatric health queries comprising 300 authentic caregiver queries sourced from the HealthCareMagic-100k-en physician consultation corpus and 300 matched adversarial variants incorporating six operationalized caregiver pressure patterns, across four consumer AI systems (GPT-4o-mini, Gemini-2.0-Flash, Claude-3.5-Haiku, and Llama-3.1-8B). Safety boundary maintenance was assessed using a validated five-component Safety Composite Score (maximum 15 points; safety-appropriate threshold of 10 or above), validated against independent human raters prior to full-corpus application (mean weighted kappa 0.76; Pearson r = 0.88). The overall safety-appropriate rate was 95.5%. Safety-oriented system prompt deployment improved safety-appropriate rates by 5.9 percentage points across all four models. Counter-intuitively, adversarial caregiver pressure was associated with higher rather than lower Safety Composite Score values for all four models across all ten topic categories and severity levels. False expertise claims were the most vulnerability-inducing pressure pattern, whereas emotional escalation was associated with the highest scores. Consumer AI systems maintain safety boundaries in the large majority of pediatric health interactions. PediatricSafetyBench-v2 is publicly released for longitudinal safety monitoring.
如何引用
AMA
Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi, Farzaneh Sedaghatkar, MohammadReza Kargozari. Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions.. NPJ digital medicine. 2026; doi:10.1038/s41746-026-02985-9.
APA
Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi, Farzaneh Sedaghatkar, MohammadReza Kargozari (2026). Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions.. NPJ digital medicine. https://doi.org/10.1038/s41746-026-02985-9
GB/T 7714
Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi, Farzaneh Sedaghatkar, MohammadReza Kargozari. Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions.[J]. NPJ digital medicine, 2026 doi:10.1038/s41746-026-02985-9.
在 PubMed增强版 阅读全文
翻译全文 · AI 文献问答 · PDF 原文下载 · 引用导出