利用深强化学习和模拟转让解决火箭联盟 (On the Verge of Solving Rocket League using Deep Reinforcement Learning and Sim-to-sim Transfer)

Marco Pleines,Konstantin Ramthun,Yannik Wegener,Hendrik Meyer,Matthias Pallasch,Sebastian Prior,Jannik Drögemüller,Leon Büttinghaus,Thilo Röthemeyer,Alexander Kaschwig,Oliver Chmurzynski,Frederik Rohkrähmer,Roman Kalkreuth,Frank Zimmer,Mike Preuss

from arxiv, Accepted at IEEE Conference on Games 2022, 8 pages, 6 figures, 3 tables

Autonomously trained agents that are supposed to play video games reasonably well rely either on fast simulation speeds or heavy parallelization across thousands of machines running concurrently. This work explores a third way that is established in robotics, namely sim-to-real transfer, or if the game is considered a simulation itself, sim-to-sim transfer. In the case of Rocket League, we demonstrate that single behaviors of goalies and strikers can be successfully learned using Deep Reinforcement Learning in the simulation environment and transferred back to the original game. Although the implemented training simulation is to some extent inaccurate, the goalkeeping agent saves nearly 100% of its faced shots once transferred, while the striking agent scores in about 75% of cases. Therefore, the trained agent is robust enough and able to generalize to the target domain of Rocket League.

翻译：自动培训的代理人本应玩电子游戏,他们应该合理地完全依赖快速模拟速度或对同时运行的数千台机器的高度平行。这项工作探索了机器人所建立的第三个方法,即模拟到现实的转移,或者如果游戏本身被视为模拟,即模拟到现实的转移。在火箭联盟的情况下,我们证明在模拟环境中利用深强化学习可以成功学习守门员和罢工者的单项行为,然后将其转回原游戏。尽管实施的培训模拟在某种程度上不准确,但目标维护代理人在转移后可以省下近100%的瞄准镜头,而打击代理人则在大约75%的案例中得分。因此,训练有素的代理人足够强大,能够推广到火箭联盟的目标领域。