The goal of our project is leveraging a new representation of visual data, Deep Latent Particles (DLP), to train an agent based on Reinforcement Learning algorithm. For implementing and evaluating the project we used Pong, one of Atari games.
DLP decomposes the visual input into low-dimensional latent “particles”. Each particle is represented by its spatial location, scale, transparency, and visual features of its surrounding region.
Reinforcement learning is a machine learning method in which an agent interacts with the environment and learns to act within that. For training the agent, we use Deep Q-Network (DQN) algorithm that gets observations from the agent’s environment as input and maps them into the agent’s action space. In this project we focus on observations that are images and represent them using a DLP model.
At the first stage of the project, we created a dataset of observations from Pong.
At the second stage, we trained a DLP model using the dataset we created for producing meaningful particles for Pong.
At the last stage, we experimented with a new approach for implementing the DQN algorithm using a DLP representation model of the observations. For each observation, we produce a set of particles and use different architectures to map the produced particles to action space. We tried various architectures: MLP, Deep Sets and GNN.
Our method showed impressive performance for Pong, when used with a GNN architecture, exceeding the performance of the CNN-based baseline.