This paper presents a study on natural language to sign language translation with human-robot interaction application purposes. By means of the presented methodology, the humanoid robot TEO is expected to represent Spanish sign language automatically by converting text into movements, thanks to the performance of neural networks. Natural language to sign language translation presents several challenges to developers, such as the discordance between the length of input and output data and the use of non-manual markers. Therefore, neural networks and, consequently, sequence-to-sequence models, are selected as a data-driven system to avoid traditional expert system approaches or temporal dependencies limitations that lead to limited or too complex translation systems. To achieve these objectives, it is necessary to find a way to perform human skeleton acquisition in order to collect the signing input data. OpenPose and skeletonRetriever are proposed for this purpose and a 3D sensor specification study is developed to select the best acquisition hardware.
翻译:本文介绍了一项关于自然语言的研究,以人-机器人互动应用为目的,手语翻译手语的自然应用。通过介绍的方法,由于神经网络的性能,人类机器人TEO预计将通过将文字转换为运动而自动代表西班牙语手语。手语翻译自然语言对开发者提出了几项挑战,例如输入和输出数据长度与非人工标记的使用不一致。因此,神经网络以及随后的顺序模型被选为数据驱动系统,以避免传统的专家系统方法或时间依赖性限制导致有限的或过于复杂的翻译系统。为了实现这些目标,有必要找到一种获取人体骨骼的方法,以便收集签名输入数据。为此提出了OpenPose和骨架Retriever建议,并开发了3D传感器规格研究,以选择最佳的获取硬件。