大语言模型与欧洲呼吸学会（ERS）急性呼吸衰竭高流量鼻导管指南一致性比较评估
A comparative evaluation of large language models in aligning with European Respiratory Society (ERS) guidelines for high-flow nasal cannula in acute respiratory failure.
摘要(中文译文)
大型语言模型（LLMs）在急性呼吸衰竭（ARF）临床决策标准化方面的潜力，尤其是在优化经鼻高流量氧疗（HFNC）应用方面，正日益受到关注，这源于该病症病因的复杂性。本研究旨在评估人工智能（AI）模型在与欧洲呼吸学会（ERS）关于HFNC用于急性呼吸衰竭（ARF）的临床实践指南保持一致方面的临床实用性。研究评估了八种先进LLMs在八个与ARF中HFNC使用相关的代表性临床场景中的表现。每位模型的回答由三名独立评审员在四个维度上独立评分：准确性、过度结论性、补充价值和完整性，采用5点李克特量表。评审者间信度采用Fleiss' Kappa进行测量。同时分析了可读性指标，包括Flesch阅读易读性（FRE）和Flesch-Kincaid年级水平（FKGL）。在准确性或过度结论性方面未检测到差异（均P>0.99）。各模型在补充价值方面出现显著差异（P<0.0001）。DeepSeek-V3.1排名最高，显著优于Claude和ChatGPT模型（P<0.05）。在完整性方面，ChatGPT-4o的得分低于DeepSeek-V3.1（P=0.006）和DeepSeek-R1（P=0.01）。可读性分析显示，DeepSeek-V3.1获得最高FRE评分，而DeepSeek-R1的FKGL最低，反映出更好的文本可及性。评审者间信度为中等至高度一致（κ=0.781）。对LLMs用于HFNC指南解读的评估表明，尽管所有模型均准确，但DeepSeek-V3.1和R1在完整性、补充细节和可读性方面表现优异，使其更适合用于ARF临床决策支持。将其整合至关键工作流程中仍需专家监督。
Abstract
The potential for large language models (LLMs) to standardize clinical decision-making in ARF, particularly in optimizing high-flow nasal cannula (HFNC) therapy application, is gaining interest due to the condition's complex etiology. This study aimed to evaluate the clinical utility of artificial intelligence (AI) models in aligning with the European Respiratory Society (ERS) clinical practice guidelines for the use of HFNC in acute respiratory failure (ARF). Eight advanced LLMs were assessed across eight representative clinical scenarios related to HFNC use in ARF. Each model's responses were independently scored by three independent reviewers across four domains: accuracy, overconclusiveness, supplementary value, and completeness, using a 5-point Likert scale. Inter-rater reliability was measured by Fleiss' Kappa. Readability metrics, including Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL), were also analyzed. No differences were detected in accuracy or overconclusiveness (both P>0.99). Significant differences emerged in supplementary value among models (P<0.0001). DeepSeek-V3.1 ranked highest, significantly outperforming Claude and ChatGPT models (P<0.05). For completeness, ChatGPT-4o scored lower than both DeepSeek-V3.1 (P=0.006) and DeepSeek-R1 (P=0.01). Readability analyses revealed that DeepSeek-V3.1 achieved the highest FRE score, while DeepSeek-R1 had the lowest FKGL, reflecting better textual accessibility. Inter-rater reliability was substantial (κ=0.781). Evaluations of LLMs for HFNC guideline interpretation show that while all are accurate, DeepSeek-V3.1 and R1 excel in completeness, supplemental detail, and readability, marking them as more suitable for ARF clinical decision support. Their integration into critical workflows still demands expert supervision.
如何引用
AMA
Hongxia Wu, Lumeng Lu, Yangyang Guo, Shuai Wang, Chengqi Dong, Xu Qiu, et al. A comparative evaluation of large language models in aligning with European Respiratory Society (ERS) guidelines for high-flow nasal cannula in acute respiratory failure.. Journal of thoracic disease. 2026; doi:10.21037/jtd-2026-1-0230.
APA
Hongxia Wu, Lumeng Lu, Yangyang Guo, Shuai Wang, Chengqi Dong, Xu Qiu, et al (2026). A comparative evaluation of large language models in aligning with European Respiratory Society (ERS) guidelines for high-flow nasal cannula in acute respiratory failure.. Journal of thoracic disease. https://doi.org/10.21037/jtd-2026-1-0230
GB/T 7714
Hongxia Wu, Lumeng Lu, Yangyang Guo, Shuai Wang, Chengqi Dong, Xu Qiu, et al. A comparative evaluation of large language models in aligning with European Respiratory Society (ERS) guidelines for high-flow nasal cannula in acute respiratory failure.[J]. Journal of thoracic disease, 2026 doi:10.21037/jtd-2026-1-0230.
在 PubMed增强版 阅读全文
翻译全文 · AI 文献问答 · PDF 原文下载 · 引用导出