In this project, we demonstrate a visualization methodology and use it to analyze the learning process of A2C agents trained on the game Atari Breakout (Atari2600).
We present examples for using our method to explore how A2C agents learn to carve side-tunnels, avoid states of losing a game and other useful skill.
Our method uses a modified version of the popular nonlinear dimensionality reduction algorithm t-SNE. The modification includes two different kinds of priors that overcome the stochastic nature of t-SNE and intend to keep the low dimensional representations of the agent’s neural network latent space comparable along the learning process.
The project also includes a comparison between the original t-SNE and its modified version and prove that our priors exploit t-SNE’s capabilities to preserve high dimensional information in a low dimension map while adding the consistency property to it.
Using our new technique, we demonstrate how an agent’s perception of its environment changes during the training process, skills are being gained and semantic similarities between states are being formed. The implications of such insights range from an optimization of the training process to design a new agents networks architecture.