Alert button

PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration

Add code
Alert button
Mar 16, 2022
Figure 1 for PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration
Figure 2 for PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration
Figure 3 for PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration
Figure 4 for PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration

Share this with someone who'll enjoy it:

Learning to collaborate is critical in multi-agent reinforcement learning (MARL). A number of previous works promote collaboration by maximizing the correlation of agents' behaviors, which is typically characterised by mutual information (MI) in different forms. However, in this paper, we reveal that strong correlation can emerge from sub-optimal collaborative behaviors, and simply maximizing the MI can, surprisingly, hinder the learning towards better collaboration. To address this issue, we propose a novel MARL framework, called Progressive Mutual Information Collaboration (PMIC), for more effective MI-driven collaboration. In PMIC, we use a new collaboration criterion measured by the MI between global states and joint actions. Based on the criterion, the key idea of PMIC is maximizing the MI associated with superior collaborative behaviors and minimizing the MI associated with inferior ones. The two MI objectives play complementary roles by facilitating learning towards better collaborations while avoiding falling into sub-optimal ones. Specifically, PMIC stores and progressively maintains sets of superior and inferior interaction experiences, from which dual MI neural estimators are established. Experiments on a wide range of MARL benchmarks show the superior performance of PMIC compared with other algorithms.

* A preliminary version has been accepted on the Cooperative AI Workshop at 35th Conference on Neural Information Processing Systems (NeurIPS 2021)  

Share this with someone who'll enjoy it: