ASYM：通过曼巴增强注意力特征融合进行多模态抑郁识别

摘要(中文译文)

抑郁是一种普遍存在的精神障碍，具有严重的全球影响。传统的基于访谈的评估受到主观性、流程冗长和医疗资源获取不平等等方面的限制。尽管人工智能的进步促进了多模态抑郁检测模型的发展——利用视听数据作为生物信号的可及替代方案——但当前方法仍面临长时时间建模效率低下和多模态融合浅表化的挑战。此外，基于生物信号的方法受限于高成本和适用范围狭窄。这些挑战凸显了优化多模态解决方案的迫切需求。本文提出ASYM（Attentive Synergy Mamba），一种用于抑郁识别的新型多模态架构，包含三个核心模块：跨模态交互Mamba、多尺度门控并行融合和多模态增强Mamba。首先，利用卷积神经网络和双向Mamba模块对每种模态的特征进行交互增强。随后通过交叉注意力机制提取跨模态互补信息。双路径融合模块随后增强多尺度表征，并通过动态加权整合跨模态特征。最后，特征表征通过一系列双向Mamba模块进行精炼。在D-Vlog和LMVD数据集上使用准确率、精确率、召回率和F1分数进行评估，结果显示ASYM在D-Vlog上达到70.91%的准确率和77.13%的F1分数，在LMVD上达到74.68%的准确率和74.90%的F1分数。两个数据集的宏平均性能超越了所有对比的主流方法。消融研究证实了每个组件的必要性，移除任一模块均显著降低性能，凸显了所提出架构的有效性和关键贡献。尽管多模态抑郁检测已较单模态方法有所改进，但长序列处理中的计算效率低下和融合策略不充分等问题仍然存在。我们的模型通过多模态交互和多尺度特征融合解决了这些局限。未来工作将聚焦于跨多样化人群的临床验证，以弥合计算精神病学与临床实践之间的差距。

Abstract

Depression is a prevalent mental disorder with a severe global impact. Traditional interview-based assessments are limited by subjectivity, lengthy procedures, and unequal access to care. Although advances in AI have facilitated multimodal models for depression detection-using audiovisual data as an accessible alternative to biosignals-current approaches remain challenged by inefficient long-term temporal modeling and superficial multimodal fusion. Moreover, biosignal-based methods are constrained by high costs and narrow applicability. These challenges underscore the urgent need for optimized multimodal solutions. This paper proposes ASYM (Attentive Synergy Mamba), a novel multimodal architecture for depression recognition, comprising three core modules: a Cross-Modal Interactive Mamba, a Multi-Scale Gated Parallel Fusion, and a Multimodal Enhanced Mamba. First, features from each modality are interactively enhanced using convolutional neural network and Bi-Mamba blocks. Cross-modal complementary information is then extracted via a cross-attention mechanism. A dual-path fusion module subsequently augments multi-scale representations and integrates cross-modal features through dynamic weighting. Finally, the feature representations are refined by a series of Bi-Mamba blocks. Evaluations on the D-Vlog and LMVD datasets using accuracy, precision, recall, and F1-score showed that ASYM achieved an accuracy of 70.91% and an F1-score of 77.13% on D-Vlog, and 74.68% accuracy with a 74.90% F1-score on LMVD. The macro-average performance across both datasets surpassed all compared mainstream methods. Ablation studies confirmed the necessity of each component, as removing any module significantly degraded performance, underscoring the efficacy and critical contribution of the proposed architecture. While multimodal depression detection has improved upon single-modality approaches, issues such as computational inefficiency in long-sequence processing and inadequate fusion strategies persist. Our model addresses these limitations through multimodal interaction and multi-scale feature fusion. Future work will focus on clinical validation across diverse populations to bridge computational psychiatry and clinical practice.

如何引用

Frontiers in psychiatry 的其他文献

• 运动拖延与运动成瘾的社会生态相关性：一项基于单所大学样本的初步探索性研究
• 磷脂酰丝氨酸对注意缺陷多动障碍儿童行为问题的影响
• 与睡眠有关的勃起疼痛、勃起功能障碍和心理困扰
• 营养状况介导了中国社区老年人抑郁与轻度认知障碍之间的关系
• 青年男性儿童期情绪虐待经历和情绪智力
• 抑郁症青少年静息状态前额叶活动和网络拓扑结构的改变：一项fNIRS研究
• 皮质醇/脱氢表雄酮比值升高介导了药物初次发作强迫症患者症状严重程度与工作记忆障碍之间的关联
• {“_”：“基因突变支持GTP酶激活蛋白（RhoGAP家族）在特发性自闭症谱系障碍中的意义。”，“i”：[“从头开始”，“ARHGAP32”]}
• 内化的批评和缺乏关怀：从儿童虐待和忽视到通过自我同情造成双重伤害的不同途径
• 中国痴呆症患者照顾者的照顾者负担与家庭复原力之间的关系：相互性的中介作用