基于动态分层与自学习的多智能体自适应协作模型

项目名称： 基于动态分层与自学习的多智能体自适应协作模型

项目编号： No.60874042

项目类型： 面上项目

立项/批准年度： 2009

项目学科： 金属学与金属工艺

项目作者： 曹卫华

作者单位： 中南大学

项目金额： 30万元

中文摘要： 本项目针对复杂动态环境下的多智能体系统（Multi-Agent System,MAS）协作问题，研究并提出了一种新颖的基于动态分层与自学习的多智能体自适应协作模型。针对动态分层问题，对构成搜索空间的状态和动作空间分别抽象，提出了基于多智能体自适应聚类的状态抽象算法和基于有效执行频率的动作抽象算法，进而提出了一种融合状态抽象和动作抽象的MAS动态分层方法，实现了在动态未知环境下MAS分层结构的自动生成和动态优化；为了改善协作行为的准确性和适应性，通过对MAS中智能体的状态预测，缩小策略的搜索空间，并利用强化学习的自适应性，提出了基于值函数逼近和基于最优跟踪的多智能体强化学习算法；将动态分层技术和基于状态预测的多智能体强化学习算法有机结合，提出了基于探索信息自适应聚类的多智能体动态分层强化学习算法，建立了完整的MAS自适应协作模型。通过典型的分布式系统RoboCup救援仿真平台对所提算法进行了验证，所得结果进一步说明了模型的泛化性和有效性。研究成果为多智能体协作提供了一种有效和可行的新方法，促进了MAS理论和技术的发展，具有重要的科学意义。

中文关键词： 多智能体系统；动态分层；自学习；自适应协作模型；RoboCup救援仿真平台

英文摘要： To the question of cooperation for multi-agent system (MAS) in the complex and dynamic environment, this project researched and presented a new multi-agent cooperative model with adaptability based on dynamic hierarchy and self-learning. To the dynamic hierarchy, the technique of abstracting is adopted to deal with the search space, which includes state and action space, then, the algorithms, which are state abstraction based on multi-agent self-adaptive cluster and action abstraction based on valid execute frequency, are proposed. Furthermore, incorporating the state abstraction and action abstraction to generate hierarchy and optimize dynamically. To improve the veracity and adaptability of the cooperative behavior in MAS, state prediction is used to reduce the search space, and then, based on the adaptability of reinforcement learning, the algorithm of multi-agent reinforcement learning with joint state approximation and the framework of multi-agent reinforcement learning based on optimal tracking are presented. Moreover, based on the dynamic hierarchy and multi-agent reinforcement learning based on state prediction, multi-agent dynamic hierarchical reinforcement learning with adaptive clustering based on the exploration information is proposed, and the intact multi-agent cooperative model with adaptability is constructed. The results of proposed method, obtained from the classical cooperative system, RoboCup Rescue simulation show the validity and generalization of proposed method. It is believed that our research results of this project will provide an efficient and feasible method for the the cooperation of MAS, promote its development and have great significance in the field of MAS.

英文关键词： multi-agent system; dynamic hierarchy; self-learning; adaptability cooperative model; RobCupRescue