Hi-RAHrasp：一种用于任务导向抓取的循环中人类体验增强方法
Hi-RAGrasp: A Human-in-the-Loop Experience-Augmented Method for Task-Oriented Grasping.
摘要(中文译文)
随着老龄化社会对辅助机器人需求的不断增长，家庭环境中的任务导向抓取变得越来越重要。与结构化的工业环境相比，家庭场景具有物体多样、布局非结构化以及任务语义高度多变等特点。然而，传统方法侧重于几何稳定性，未能捕捉操作区域上与任务相关的语义约束，而现有方法存在推理不稳定且缺乏将人工干预纳入推理过程的有效机制等问题。为应对这些挑战，我们提出Hi-RAGrasp，一种集成了渐进式多阶段推理、人在回路（Human-in-the-Loop，HITL）交互和检索增强生成（Retrieval-Augmented Generation，RAG）的任务导向抓取框架。由粗到细的流水线从物体级定位逐步精化到部件级定位，实现了从人类指令到细粒度任务相关区域的鲁棒映射。同时，引入了HITL校正机制和结构化人类经验数据库，并与RAG相结合，形成统一范式：在有可用先验经验时与之对齐，否则回退到推理，从而无需重新训练即可实现经验复用和未来经验积累。此外，提出了一种几何启发式分割（Geometric Heuristic Segmentation，GHS）方法，以改善无纹理物体的任务相关区域定位。实验表明，我们的方法在评估数据集上取得了77.73%的分割成功率，在真实场景中取得了75%的抓取成功率，显著优于现有方法，在开放环境中展现出较强的有效性和实用性。
Abstract
With the growing demand for assistive robots in aging societies, task-oriented grasping in household environments has become increasingly important. Compared with structured industrial settings, household scenarios are characterized by diverse objects, unstructured layouts, and strong variability in task semantics. However, traditional methods focus on geometric stability and fail to capture task-relevant semantic constraints on manipulation regions, while existing approaches suffer from unstable reasoning and lack effective mechanisms for incorporating human intervention into the reasoning process. To address these challenges, we propose Hi-RAGrasp, a task-oriented grasping framework that integrates progressive multi-stage reasoning, Human-in-the-Loop (HITL) interaction, and Retrieval-Augmented Generation (RAG). A coarse-to-fine pipeline progressively refines predictions from object-level localization to part-level grounding, enabling robust mapping from human instructions to fine-grained task-relevant regions. Meanwhile, a HITL correction mechanism and a structured human experience database are introduced and combined with RAG to form a unified paradigm that aligns with prior experience when available and falls back to reasoning otherwise, enabling experience reuse and future experience accumulation without retraining. In addition, a Geometric Heuristic Segmentation (GHS) method is proposed to improve task-relevant region localization for textureless objects. Experiments show that our method achieves a segmentation success rate of 77.73% on the evaluation dataset and a grasp success rate of 75% in real-world scenarios, significantly outperforming existing methods and demonstrating strong effectiveness and practicality in open environments.
如何引用
AMA
Yaxin Liu, Yue Hu, Yan Liu, Ming Zhong. Hi-RAGrasp: A Human-in-the-Loop Experience-Augmented Method for Task-Oriented Grasping.. Sensors (Basel, Switzerland). 2026; doi:10.3390/s26103221.
APA
Yaxin Liu, Yue Hu, Yan Liu, Ming Zhong (2026). Hi-RAGrasp: A Human-in-the-Loop Experience-Augmented Method for Task-Oriented Grasping.. Sensors (Basel, Switzerland). https://doi.org/10.3390/s26103221
GB/T 7714
Yaxin Liu, Yue Hu, Yan Liu, Ming Zhong. Hi-RAGrasp: A Human-in-the-Loop Experience-Augmented Method for Task-Oriented Grasping.[J]. Sensors (Basel, Switzerland), 2026 doi:10.3390/s26103221.
在 PubMed增强版 阅读全文
翻译全文 · AI 文献问答 · PDF 原文下载 · 引用导出