Dexterous manipulation with anthropomorphic robot hands remains a challenging problem in robotics because of the high-dimensional state and action spaces and complex contacts. Nevertheless, skillful closed-loop manipulation is required to enable humanoid robots to operate in unstructured real-world environments. Reinforcement learning (RL) has traditionally imposed enormous interaction data requirements for optimizing such complex control problems. We introduce a new framework that leverages recent advances in GPU-based simulation along with the strength of imitation learning in guiding policy search towards promising behaviors to make RL training feasible in these domains. To this end, we present an immersive virtual reality teleoperation interface designed for interactive human-like manipulation on contact rich tasks and a suite of manipulation environments inspired by tasks of daily living. Finally, we demonstrate the complementary strengths of massively parallel RL and imitation learning, yielding robust and natural behaviors. Videos of trained policies, our source code, and the collected demonstration datasets are available at https://maltemosbach.github.io/interactive_ human_like_manipulation/.
翻译:由于高度的状态和动作空间以及复杂的接触,对人体机器人进行不相干操纵仍然是机器人中一个具有挑战性的问题,然而,需要熟练的闭路操作,以使人体机器人能够在结构化的现实环境中操作。强化学习(RL)历来要求大量互动数据,以优化这种复杂的控制问题。我们引入了一个新的框架,利用基于GPU的模拟的最新进展,以及模拟学习在指导政策搜索方面的模仿力量,以引导有希望的行为使RL培训在上述领域可行。为此,我们展示了一种渗透式虚拟现实远程操作界面,目的是在接触的丰富任务和受日常生活任务启发的一套操纵环境上进行交互式的人类操作。最后,我们展示了大规模平行的RL和模拟学习的互补优势,产生强有力的自然行为。经过培训的政策的视频、我们的源代码和收集到的演示数据集可在https://maltemosbach.github.io/interactive_human_lish_manipulation/上查阅。