The incredible feats of athleticism demonstrated by humans are made possible in part by a vast repertoire of general-purpose motor skills, acquired through years of practice and experience. These skills not only enable humans to perform complex tasks, but also provide powerful priors for guiding their behaviors when learning new tasks. This is in stark contrast to what is common practice in physics-based character animation, where control policies are most typically trained from scratch for each task. In this work, we present a large-scale data-driven framework for learning versatile and reusable skill embeddings for physically simulated characters. Our approach combines techniques from adversarial imitation learning and unsupervised reinforcement learning to develop skill embeddings that produce life-like behaviors, while also providing an easy to control representation for use on new downstream tasks. Our models can be trained using large datasets of unstructured motion clips, without requiring any task-specific annotation or segmentation of the motion data. By leveraging a massively parallel GPU-based simulator, we are able to train skill embeddings using over a decade of simulated experiences, enabling our model to learn a rich and versatile repertoire of skills. We show that a single pre-trained model can be effectively applied to perform a diverse set of new tasks. Our system also allows users to specify tasks through simple reward functions, and the skill embedding then enables the character to automatically synthesize complex and naturalistic strategies in order to achieve the task objectives.
翻译:人类所展示的令人难以置信的田径运动的壮举之所以能够成为可能,部分是由于通过多年的实践和经验获得了大量通用运动技能。这些技能不仅使人类能够执行复杂的任务,而且还为在学习新任务时指导其行为提供了强有力的前科。这与物理学性格动画中常见的做法形成鲜明的对比,因为在这种动画中,控制政策通常都是从零开始对每项任务进行培训。在这项工作中,我们提出了一个大规模的数据驱动框架,用于学习用于物理模拟字符的多功能和可再应用技能嵌入。我们的方法结合了来自对抗模仿学习和不受监督的强化学习的技术,以发展产生类似生命行为的技能嵌入技术,同时也为新的下游任务的使用提供了易于控制的代表。我们的模式可以使用无结构运动剪辑的大型数据集来训练,而无需对运动数据作任何特定任务的说明或分解。在这项工作中,通过利用大规模平行的 GPU- 模拟模拟模拟模模模模模模模具模拟模拟模拟模拟,我们能够用十年来训练技能嵌入的技巧嵌套,在模拟的系统里,我们也可以学习一套精细的精细的精细的模型。