The goal of this project is to test a new continuous reinforcement learning algorithm in several different simulated environments.
The algorithm, SIMPLE, uses a neural net to simulate the reward function for the environment’s state, similarly to the DDPG algorithm. The project’s contribution comes from its unique approach to finding an optimal action in a continuous action space. Using the neural net as a model for the reward function, it uses the Stochastic Gradient Descent (SGD) algorithm to find the action that gives the maximum reward for a given state.
The network was trained by simulating several hundred playthroughs in the three AI gym environments – Lunar Lander Continuous, Half Cheetah, and Ant. A separate network was trained from scratch for each environment. The algorithm reached a high average reward relative to benchmarks in the Lunar Lander and Half Cheetah environments, though lower than state of the art. In addition, the algorithm outperformed the discretization approach in the Lunar Lander environment.
We conclude that the algorithm is a promising solution to the continuous reinforcement learning problem. Further testing on real and simulated environments might show the algorithm’s applicability in other areas.