Building Multi Agent RL Baselines in Unity

Reinforcement Learning is a branch of machine learning where an agent preforms in an environment and gets rewarded for his action. The agent goal is to learn a behavior policy that will maximize his reward over time.

Some of the research done in this field, is about expending RL to multi agent settings. The goal of this research is to solve problems where many agents preforms in the same environment, at the same time, and each of them try to maximize his own reward. This type of problems raises challenges that don’t exist in single agent problems and requires different solutions.

Today, the development and testing environments for multi agent reinforcement learning are scarce, and the existing environments are ill-suited for solving some interesting problems.

In this project, we developed a new environment for multi agent reinforcement learning research, where one can implement and test our suggested algorithms or develop his own.

Our environment is a 3D realistic car track with up to 16 cars that can drive simultaneously.

We built our environment in Unity 3D, and implemented both DDPG and MADDPG using PyTorch packages. We provide some base line results for comparison and open source code for implementing other algorithms.