This is a collection of Multi-Agent Reinforcement Learning (MARL) Resources. The purpose of this repository is to give beginners a better understanding of MARL and accelerate the learning process. Note that some of the resources are written in Chinese and only important papers that have a lot of citations were listed.
I will continually update this repository and I welcome suggestions. (missing important papers, missing important resources, invalid links, etc.) This is only a first draft so far and I'll add more resources in the next few months.
Update (2026): Added recent open-source environments/benchmarks (JaxMARL, SMACv2, Melting Pot 2.0, Gigastep, POGEMA, ...), a new MARL Libraries / Frameworks section, a new LLM-based Multi-Agent Systems section, and a Recent Surveys (2022–2026) list.
This repository is not for commercial purposes.
My email: [email protected]
- Courses
- Important Conferences
- Reviews
- Books
- Open Source Environments
- MARL Libraries / Frameworks
- LLM-based Multi-Agent Systems
- Research Groups
- Companies
- Paper List
- Talks
- Useful Resources
- TODO
- RLChina
- UCL Multi-agent AI
- SJTU Multi-Agent Reinforcement Learning Tutorial
- SJTU Reinforcement Learning
- AAMAS, AAAI, IJCAI, ICLR, ICML, NIPS
- Sorted by difficulty (roughly)
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective
- Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
- A Review of Cooperative Multi-Agent Deep Reinforcement Learning
- Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
- A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
- Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
- A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems
- Multi-agent Reinforcement Learning: A Comprehensive Survey
- A Survey of Progress on Cooperative Multi-Agent Reinforcement Learning in Open Environments
- A Survey of Multi-Agent Deep Reinforcement Learning with Communication (AAMAS 2024)
- A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives
- A Survey of Safe Reinforcement Learning and Constrained MDPs: Single-Agent and Multi-Agent Safety
- Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
- Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Survey
- A Survey on Large Language Model based Multi-Agent Systems: Recent Advances and New Frontiers in Application
- Multi-Agent Collaboration Mechanisms: A Survey of LLMs
- If multi-agent learning is the answer, what is the question?
- Multiagent learning is not the answer. It is the question
- Is multiagent deep reinforcement learning the answer or the question? A brief survey Note that A Survey and Critique of Multiagent Deep Reinforcement Learning is an updated version of this paper with the same authors.
- Evolutionary Dynamics of Multi-Agent Learning: A Survey
- (Worth reading although they're not recent reviews.)
- Multiagent systems: Algorithmic, game-theoretic, and logical foundations
- Multi‐Agent Machine Learning A Reinforcement Approach
- StarCraft Micromanagement Environment
- pymarl is the original environment mentioned in the paper The StarCraft Multi-Agent Challenge. Note that pymarl is based on SMAC.
- MARL-Algorithms is a simplified implementation of pymarl
- EPyMARL is a extended python MARL framework with more environments (Level Based Foraging, Multi-Robot Warehouse, Multi-Agent Particle Environment) and more algorithms. Paper
- pymarl2 added code-level tricks to the original pymarl. Paper
- Multi-Agent Particle Environment PyTorch Implementation
- Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents
- OpenSpiel: A Framework for Reinforcement Learning in Games
- Hanabi-learning-environment
- RoboCup 2D Half Field Offense
- Pommerman
- Multi-agent-emergence-environments
- Google Research Football
- MAgent Note that the original project is no longer maintained.
- DI-engine
- MARLlib is a MARL Extension for RLlib
- Multiagent Mujoco
- PettingZoo website
- Safe Policy Optimization (SafePO)
- (I personally recommend the first two environments for beginners, especially EPyMARL.)
- SMACv2 — an improved, procedurally-generated version of the StarCraft Multi-Agent Challenge. Paper
- JaxMARL — a large collection of GPU-accelerated MARL environments (including SMAX, an SMAC re-implementation) and baseline algorithms in JAX; extremely fast. Paper
- Melting Pot 2.0 — DeepMind's suite for evaluating generalization to novel social situations (cooperation, competition, deception). Paper
- Overcooked-AI — a benchmark for human-AI coordination; the de-facto standard for zero-shot coordination research. Paper
- Gigastep — one billion steps per second multi-agent RL; large-scale aerial/ground combat scenarios. Paper
- POGEMA — a fast benchmark platform for cooperative multi-agent pathfinding (MAPF). Paper
- MOMAland — Farama's benchmarks for multi-objective multi-agent RL. Paper
- MARBLER — standardized evaluation of multi-robot RL algorithms on the Robotarium. Paper
- MABIM / ReplenishmentEnv — a versatile MARL benchmark for inventory management. Paper
Training libraries and algorithm frameworks (as opposed to task environments).
- MARLlib — a Ray/RLlib-based library unifying many MARL algorithms across a wide range of environments through a standardized interface. Paper
- BenchMARL — a TorchRL-backed benchmarking library for reproducible, standardized comparison of MARL algorithms, tasks and models. Paper
- EPyMARL — Extended PyMARL adding more algorithms (IPPO, MAA2C, etc.) and environments.
- pymarlzooplus — a further-extended PyMARL/EPyMARL benchmarking suite for complex fully-cooperative tasks. Paper
- Mava — InstaDeep's research-friendly JAX framework for scalable MARL.
- JaxMARL — bundles both JAX environments and baseline MARL algorithms.
- TorchRL — PyTorch-native RL library with first-class multi-agent support (MAPPO, IPPO, QMIX, MADDPG); the backend of BenchMARL.
The intersection of Large Language Models and multi-agent systems has become a very active area. A few widely-used open-source frameworks:
- CAMEL — communicative agents / role-playing for LLM societies. Paper
- AutoGen — Microsoft's framework for building multi-agent LLM applications via conversation. Paper
- MetaGPT — a multi-agent framework encoding SOPs for collaborative software teams. Paper
- ChatDev — communicative agents that collaborate to develop software. Paper
- AgentVerse — a framework for multi-agent collaboration and emergent behavior. Paper
- Generative Agents — interactive simulacra of human behavior. Paper
- See also the LLM-based Multi-Agent section in Multi-Agent-Reinforcement-Learning-papers.
| Organization | Reaearcher | Lab homepage (if any) |
|---|---|---|
| Oxford | Shimon Whiteson, Jakob N. Foerster | link |
| University College London (UCL) | Jun Wang | |
| Tsinghua University (THU) | Chongjie Zhang | link |
| Tsinghua University (THU) | Yi Wu | |
| Peking University (PKU) | Zongqing Lu | |
| HUAWEI | Hangyu Mao | |
| Nanjing University (NJU) | Yang Yu | |
| Yuandong Tian | ||
| Tianjin University (TJU) | Jianye Hao | link |
| University of Illinois at Urbana-Champaign (UIUC) | Kaiqing Zhang | |
| Peking University (PKU) | Yaodong Yang | Link |
| Nanyang Technological University (NTU) | Bo An | |
| Shanghai Jiao Tong University (SJTU) | Weinan Zhang | link |
| University of Chinese Academy of Sciences (UCAS) | Haifeng Zhang | link |
| University of Edinburgh | Stefano V. Albrecht | link GitHub |
| University College London (UCL) | UCL Deciding, Acting, and Reasoning with Knowledge (DARK) Lab | Link |
| University of Maryland | Furong Huang | Link |
- https://github.com/TimeBreaker/Multi-Agent-Reinforcement-Learning-papers
- https://github.com/TimeBreaker/MARL-papers-with-code
- https://github.com/TimeBreaker/Adversarial-Reinforcement-Learning-Papers
- https://github.com/LantaoYu/MARL-Papers
- https://www.youtube.com/watch?v=W_9kcQmaWjo
- https://www.youtube.com/watch?v=TMTT2z8lifA
- https://www.youtube.com/watch?v=Yd6HNZnqjis
- https://www.youtube.com/watch?v=ufFue5_gR4c
- https://www.techbeat.net/talk-info?id=501
- https://www.bilibili.com/video/av457780236/
- https://space.bilibili.com/551888585/channel/detail?cid=167587
- https://www.bilibili.com/video/BV1ig4y1v7xU
- https://www.bilibili.com/video/BV18z411q7Kc
- https://www.bilibili.com/video/BV1k5411V7ue
- https://dblp.uni-trier.de/
- https://paperswithcode.com/
- https://www.connectedpapers.com
- https://deeplearn.org
- https://spinningup.openai.com/
- https://github.com/openai/spinningup
- https://github.com/Jinjiarui/hrl-papers
- http://www.neurondance.com/
- https://www.zhihu.com/question/376068768
- https://www.zhihu.com/question/323584412
- https://zhuanlan.zhihu.com/p/372558232
- https://space.bilibili.com/4801051?spm_id_from=333.788.b_765f7570696e666f.2
- https://www.zhihu.com/people/tian-yuan-dong
- https://www.zhihu.com/people/eyounx
- https://www.zhihu.com/people/wan-shang-zhu-ce-de
- Wechat public account: AIORHHC; RLCN
- https://www.bilibili.com/video/av925922430/
- https://www.bilibili.com/video/av626777400/
- https://github.com/NeuronDance/DeepRL
- The Research Groups part needs to be completed
- The Companies part needs to be completed
- The Useful Resources part needs to be perfected
If you find this repository useful, please cite our repo:
@misc{chen2021collection,
author={Chen, Hao},
title={A Collection of Multi-Agent Reinforcement Learning Resources},
year={2021}
publisher = {GitHub},
journal = {GitHub Repository},
howpublished = {\url{https://github.com/TimeBreaker/MARL-resources-collection}}
}