Quality Diversity (QD) algorithms like Novelty Search with Local Competition (NSLC) and MAP-Elites are a new class of population-based stochastic algorithms designed to generate a diverse collection of quality solutions. Meanwhile, variants of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) are among the best-performing derivative-free optimizers in single-objective continuous domains. This paper proposes a new QD algorithm called Covariance Matrix Adaptation MAP-Elites (CMA-ME). Our new algorithm combines the dynamic self-adaptation techniques of CMA-ES with archiving and mapping techniques for maintaining diversity in QD. Results from experiments with standard continuous optimization benchmarks show that CMA-ME finds better-quality solutions than MAP-Elites; similarly, results on the strategic game Hearthstone show that CMA-ME finds both a higher overall quality and broader diversity of strategies than both CMA-ES and MAP-Elites. Overall, CMA-ME more than doubles the performance of MAP-Elites using standard QD performance metrics. These results suggest that QD algorithms augmented by operators from state-of-the-art optimization algorithms can yield high-performing methods for simultaneously exploring and optimizing continuous search spaces, with significant applications to design, testing, and reinforcement learning among other domains. Code is available for both the continuous optimization benchmark (https://github.com/tehqin/QualDivBenchmark) and Hearthstone (https://github.com/tehqin/EvoStone) domains.
翻译:质量多样性(QD)算法,如Novellty Search with Local Competition(NSLC)和MAP-Elites(MAP-Elites),是新型基于人口的动态自适应技术与在QD中维持多样性的存档和绘图技术相结合的一种基于人口的随机算法,旨在产生多样化的优质解决方案。与此同时,Cavariance 矩阵适应进化战略(CMA-ES)的变异(CMA-ES)是单一目标连续域中最优秀的无衍生衍生物优化工具。本文建议采用一个新的QLAME(CMA-ME)和MA-ME(MA-ES)的新算法将CMA-ES的动态自适应技术与在QD中维持多样性的存档和绘图技术相结合。 标准连续优化基准基准的实验结果显示,CMA-ME(C-ME)找到质量优于MAP-E-ELE;同样, ES-ME发现比C-ES和MAP-Elites(C-Elites)更全面、更多样化的战略质量和更多的战略选择方法。总体而言,CMAMA/ME-stalvacialalal 使用标准的Supluplistryalalalalalalalalalalal-ass