publications

2022

2021

1. Amortized Rejection Sampling in Universal Probabilistic Programming

   Naderiparizi, Saeid, Scibior, Adam, Munk, Andreas, Ghadiri, Mehrdad, Baydin, Atilim Gunes, Gram-Hansen, Bradley, Witt, Christian Schroeder, Zinkov, Robert, Torr, Philip H. S., Rainforth, Tom, Teh, Yee Whye, and Wood, Frank

   AISTATS 2021 2021
2. Revealing Robust Oil and Gas Company Macro-Strategies Using Deep Multi-Agent Reinforcement Learning

   Radovic, Dylan, Kruitwagen, Lucas, Witt, Christian, Caldecott, Ben, Tomlinson, Shane, and Workman, Mark

   SSRN 2021
3. Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS

   Witt, Christian Schroeder, Huang, Yongchao, Torr, Philip H. S., and Strohmeier, Martin

   arXiv:2111.12197 [cs] Nov 2021
4. Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning

   Iqbal, Shariq, Witt, Christian A. Schroeder De, Peng, Bei, Boehmer, Wendelin, Whiteson, Shimon, and Sha, Fei

   In Proceedings of the 38th International Conference on Machine Learning Jul 2021

   Abs

   Multi-agent settings in the real world often involve tasks with varying types and quantities of agents and non-agent entities; however, common patterns of behavior often emerge among these agents/entities. Our method aims to leverage these commonalities by asking the question: "What is the expected utility of each agent when only considering a randomly selected sub-group of its observed entities?" By posing this counterfactual question, we can recognize state-action trajectories within sub-groups of entities that we may have encountered in another task and use what we learned in that task to inform our prediction in the current one. We then reconstruct a prediction of the full returns as a combination of factors considering these disjoint groups of entities and train this "randomly factorized" value function as an auxiliary objective for value-based multi-agent reinforcement learning. By doing so, our model can recognize and leverage similarities across tasks to improve learning efficiency in a multi-task setting. Our approach, Randomized Entity-wise Factorization for Imagined Learning (REFIL), outperforms all strong baselines by a significant margin in challenging multi-task StarCraft micromanagement settings.

2020

1. Simulation-Based Inference for Global Health Decisions

   Witt, Christian, Gram-Hansen, Bradley, Nardelli, Nantas, Gambardella, Andrew, Zinkov, Rob, Dokania, Puneet, Siddharth, N., Espinosa-Gonzalez, Ana Belen, Darzi, Ara, Torr, Philip, and Baydin, Atilim Gunes

   ML for Global Health Workshop at ICML 2002 Jul 2020
2. FACMAC: Factored Multi-Agent Centralised Policy Gradients

   Peng, Bei, Rashid, Tabish, Witt, C. S. D., Kamienny, Pierre-Alexandre, Torr, Philip H. S., Bohmer, Wendelin, and Whiteson, S.

   In Advances in Neural Information Processing Systems Jul 2020
3. Deep Multi-Agent Reinforcement Learning for Decentralized Continuous Cooperative Control

   Witt, Christian, Peng, Bei, Kamienny, Pierre-Alexandre, Torr, Philip, Bohmer, Wendelin, and Whiteson, Shimon

   arXiv:2003.06709v4 [cs.LG] Mar 2020

   Abs

   Deep multi-agent reinforcement learning (MARL) holds the promise of automating many real-world cooperative robotic manipulation and transportation tasks. Nevertheless, decentralised cooperative robotic control has received less attention from the deep reinforcement learning community, as compared to single-agent robotics and multi-agent games with discrete actions. To address this gap, this paper introduces Multi-Agent Mujoco, an easily extensible multi-agent benchmark suite for robotic control in continuous action spaces. The benchmark tasks are diverse and admit easily configurable partially observable settings. Inspired by the success of single-agent continuous value-based algorithms in robotic control, we also introduce COMIX, a novel extension to a common discrete action multi-agent \Q\-learning algorithm. We show that COMIX significantly outperforms state-of-the-art MADDPG on a partially observable variant of a popular particle environment and matches or surpasses it on Multi-Agent Mujoco. Thanks to this new benchmark suite and method, we can now pose an interesting question: what is the key to performance in such settings, the use of value-based methods instead of policy gradients, or the factorisation of the joint \Q\-function? To answer this question, we propose a second new method, FacMADDPG, which factors MADDPG’s critic. Experimental results on Multi-Agent Mujoco suggest that factorisation is the key to performance.

4. Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?

   Witt, Christian Schroeder, Gupta, Tarun, Makoviichuk, Denys, Makoviychuk, Viktor, Torr, Philip H. S., Sun, Mingfei, and Whiteson, Shimon

   arXiv:2011.09533 [cs] Nov 2020

   Abs

   Most recently developed approaches to cooperative multi-agent reinforcement learning in the }emph{centralized training with decentralized execution} setting involve estimating a centralized, joint value function. In this paper, we demonstrate that, despite its various theoretical shortcomings, Independent PPO (IPPO), a form of independent learning in which each agent simply estimates its local value function, can perform just as well as or better than state-of-the-art joint learning approaches on popular multi-agent benchmark suite SMAC with little hyperparameter tuning. We also compare IPPO to several variants; the results suggest that IPPO’s strong performance may be due to its robustness to some forms of environment non-stationarity.

5. Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning

   Rashid, Tabish, Samvelyan, Mikayel, Schroeder de Witt, Christian, , Farquhar, Gregory, Foerster, Jakob, and Whiteson, Shimon

   Journal of Machine Learning Research Nov 2020

2019

1. Effective Approximate Inference for Nested Simulators

   Gram-Hansen, Bradley, Golinski, Adam, Witt, Christian, Naderiparizi, Saeid, Scibior, Adam, Munk, Andreas, Wood, Frank, Torr, Philip, Teh, Yee Whye, Baydin, Atilim Gunes, and Rainforth, Tom

   AABI 2019 Nov 2019
2. Hijacking Malaria Simulators with Probabilistic Programming

   Gram-Hansen, Bradley, Witt, Christian, Rainforth, Tom, Torr, Philip, Teh, Yee Whye, and Baydin, Atilim Gunes

   AI for Social Good Workshop at ICML 2019 Nov 2019
3. The StarCraft Multi-Agent Challenge

   Samvelyan, Mikayel, Rashid, Tabish, Witt, Christian Schroeder, Farquhar, Gregory, Nardelli, Nantas, Rudner, Tim G. J., Hung, Chia-Man, Torr, Philip H. S., Foerster, Jakob, and Whiteson, Shimon

   arXiv:1902.04043 [cs, stat] Dec 2019

   Abs

   In the last few years, deep multi-agent reinforcement learning (RL) has become a highly active area of research. A particularly challenging class of problems in this area is partially observable, cooperative, multi-agent learning, in which teams of agents must learn to coordinate their behaviour while conditioning only on their private observations. This is an attractive research area since such problems are relevant to a large number of real-world systems and are also more amenable to evaluation than general-sum problems. Standardised environments such as the ALE and MuJoCo have allowed single-agent RL to move beyond toy domains, such as grid worlds. However, there is no comparable benchmark for cooperative multi-agent RL. As a result, most papers in this field use one-off toy problems, making it difficult to measure real progress. In this paper, we propose the StarCraft Multi-Agent Challenge (SMAC) as a benchmark problem to fill this gap. SMAC is based on the popular real-time strategy game StarCraft II and focuses on micromanagement challenges where each unit is controlled by an independent agent that must act based on local observations. We offer a diverse set of challenge maps and recommendations for best practices in benchmarking and evaluations. We also open-source a deep multi-agent RL learning framework including state-of-the-art algorithms. We believe that SMAC can provide a standard benchmark environment for years to come. Videos of our best agents for several SMAC scenarios are available at: https://youtu.be/VZ7zmQ_obZ0.

4. Multi-Agent Common Knowledge Reinforcement Learning

   Schroeder de Witt*, Christian, Foerster*, Jakob, Farquhar, Gregory, Torr, Philip, Boehmer, Wendelin, and Whiteson, Shimon

   In Advances in Neural Information Processing Systems Dec 2019

2018

2017

2016

2015

2014