Sensing contact pressure applied by a gripper is useful for autonomous and teleoperated robotic manipulation, but adding tactile sensing to a gripper's surface can be difficult or impractical. If a gripper visibly deforms when forces are applied, contact pressure can be visually estimated using images from an external camera that observes the gripper. While researchers have demonstrated this capability in controlled laboratory settings, prior work has not addressed challenges associated with visual pressure estimation in the wild, where lighting, surfaces, and other factors vary widely. We present a deep learning model and associated methods that enable visual pressure estimation under widely varying conditions. Our model, Visual Pressure Estimation for Robots (ViPER), takes an image from an eye-in-hand camera as input and outputs an image representing the pressure applied by a soft gripper. Our key insight is that force/torque sensing can be used as a weak label to efficiently collect training data in settings where pressure measurements would be difficult to obtain. When trained on this weakly labeled data combined with fully labeled data containing pressure measurements, ViPER outperforms prior methods, enables precision manipulation in cluttered settings, and provides accurate estimates for unseen conditions relevant to in-home use.
翻译:扶手应用的感测接触压力有助于自动和远程操作机器人操作操作,但将触觉感应器添加到扶手表面可能困难或不切实际。如果在施用力时抓手明显畸形,接触压力可以用外部摄像头的图像进行视觉估计。虽然研究人员在受控实验室环境中已经展示了这种能力,但先前的工作没有解决与野外视觉压力估计有关的挑战,因为野外的照明、表面和其他因素差异很大。我们提出了一个深层次的学习模型和相关方法,以便能够在广泛不同的条件下进行视觉压力估测。我们的模型,即机器人视觉压力动画(ViPER),从一对视手相机中提取图像,作为输入和输出一个代表软握手所施压的图像。我们的关键见解是,在难以获得压力测量的环境下,力/压力感测可用作有效收集培训数据的薄弱标签。当就这一标签不高的数据与包含压力测量的完整标签数据相结合时,ViPER比先前的方法要好,能够使在软的环境下精确地进行操纵,并且提供精确的图像。</s>