评估食管癌和癌症患者生成性人工智能反应的准确性和可读性
Assessing the Accuracy and Readability of Generative Artificial Intelligence Responses for Esophageal and Gastric Cancer Patients.
摘要(中文译文)
背景：生成型人工智能（GenAI）模型因其可访问性和效率而越来越多地用于医学信息检索。然而，他们的反应的准确性和可读性，特别是对上消化道癌症的反应，仍然没有得到充分的评估。这一差距凸显了严格评估的必要性，以确保可靠的患者教育和临床整合。目的：本研究旨在评估四种著名的GenAI模型（Kimi、DeepSeek、ChatGPT和Gemini）在解决与食管癌和胃癌相关的以患者为中心的问题时产生的反应的准确性和可读性。方法：对每种模型提出20个关于食管和癌症的标准化医学问题，涉及疾病定义、治疗和管理领域。四名肿瘤学家通过5点李克特量表评估了反应的准确性，并使用Flesch-Kincaid易读性、Flesch-Kinkaid等级和SMOG指标分析了反应的可读性。通过问卷调查确定了患者的高兴趣问题。结果：比较GenAI生成的回答的准确性，DeepSeek在关于定义和治疗的问题上获得了最高的总体准确性得分，并优于其他模型，而ChatGPT在管理相关的询问中表现出色。在亚组分析中，与癌症治疗的问题相比，GenAI模型在回答患者更喜欢询问的定义和管理问题方面表现出更高的准确性。所有模型产生的答案都需要从11年级到大学水平的阅读能力。结论：本研究表明，在GenAI模型的比较评估应用中，DeepSeek为上消化道癌症关于定义和治疗的询问提供了最准确的回答，而ChatGPT在管理相关问题上表现出优势。然而，所有模型都会生成需要高级阅读水平的文本，这突显了在不影响准确性的情况下优化可读性的必要性。GenAI显示出对患者教育的承诺，但需要对临床整合进行严格验证。
Abstract
: Generative artificial intelligence (GenAI) models are increasingly used for medical information retrieval, due to their accessibility and efficiency. However, the accuracy and readability of their responses, specifically for upper gastrointestinal cancers, remain inadequately evaluated. This gap highlights the need for rigorous assessment to ensure reliable patient education and clinical integration. : This study aimed to assess the accuracy and readability of responses generated by four prominent GenAI models (Kimi, DeepSeek, ChatGPT, and Gemini) when addressing patient-focused questions related to esophageal and gastric cancers. : Twenty-five standardized medical questions about esophageal and gastric cancer covering domains of disease definition, treatment and management were posed to each model. Responses were assessed by four oncologists for accuracy by a 5-point Likert scale and analyzed for readability using Flesch-Kincaid Reading Ease, Flesch-Kincaid Grade Level, and SMOG metrics. High-interest questions for patients were identified via questionnaires. : Comparing the accuracy of GenAI-generated responses, DeepSeek achieved the highest overall accuracy score and outperformed other models in questions about definitions and treatments, while ChatGPT excelled in management-related inquiries. In subgroup analysis, GenAI models exhibited higher accuracy in answering definition and management questions, which patients preferred to inquire, compared with questions about cancer therapies. The responses produced by all models required a reading capacity from 11th-grade to college level. : This study revealed that in this comparative evaluation application of GenAI models, DeepSeek provides the most accurate responses for upper GI cancer inquiries about definition and treatment, while ChatGPT showed superiority in management-related questions. However, all models generate texts requiring advanced reading levels, highlighting a need for readability optimization without compromising accuracy. GenAI shows promise for patient education but requires rigorous validation for clinical integration.
如何引用
AMA
Shanhu Ran, Wenlong Guan, Ran Wei, Yukun Chen, Bo Zhang, Yating Wang, et al. Assessing the Accuracy and Readability of Generative Artificial Intelligence Responses for Esophageal and Gastric Cancer Patients.. Journal of clinical medicine. 2026; doi:10.3390/jcm15082958.
APA
Shanhu Ran, Wenlong Guan, Ran Wei, Yukun Chen, Bo Zhang, Yating Wang, et al (2026). Assessing the Accuracy and Readability of Generative Artificial Intelligence Responses for Esophageal and Gastric Cancer Patients.. Journal of clinical medicine. https://doi.org/10.3390/jcm15082958
GB/T 7714
Shanhu Ran, Wenlong Guan, Ran Wei, Yukun Chen, Bo Zhang, Yating Wang, et al. Assessing the Accuracy and Readability of Generative Artificial Intelligence Responses for Esophageal and Gastric Cancer Patients.[J]. Journal of clinical medicine, 2026 doi:10.3390/jcm15082958.
在 PubMed增强版 阅读全文
翻译全文 · AI 文献问答 · PDF 原文下载 · 引用导出