Multiagent reinforcement learning algorithms (MARL) have been demonstrated on complex tasks that require the coordination of a team of multiple agents to complete. Existing works have focused on sharing information between agents via centralized critics to stabilize learning or through communication to increase performance, but do not generally look at how information can be shared between agents to address the curse of dimensionality in MARL. We posit that a multiagent problem can be decomposed into a multi-task problem where each agent explores a subset of the state space instead of exploring the entire state space. This paper introduces a multiagent actor-critic algorithm and method for combining knowledge from homogeneous agents through distillation and value-matching that outperforms policy distillation alone and allows further learning in both discrete and continuous action spaces.
翻译:多剂强化学习算法(MARL)在复杂的任务上已经得到证明,这些任务需要多个代理人组成的团队进行协调才能完成。现有工作的重点是通过集中的批评者共享信息,以稳定学习,或通过交流提高绩效。但一般不研究如何在代理人之间共享信息,以解决多剂强化学习算法(MARL)中的维度诅咒。我们认为,多剂问题可以分解成一个多任务问题,即每个代理人可以探索国家空间的一个子集,而不是探索整个国家空间。本文介绍了一种多剂行为者-化学算法和通过蒸馏和价值匹配将同质剂的知识融合在一起的方法,即单靠政策蒸馏来超越政策蒸馏,并允许在离散和连续的行动空间中进一步学习。