One of the greatest long-lasting issues in the field of reinforcement learning is the ability to increase a model’s performance and training it on offline, partially observable data. This problem is most obvious in fields like autonomous cars and medical procedures, since access to a simulator is expensive- as it can jeopardize people’s lives, while at the same time massive amounts of data from drivers and doctors exists and is just waiting to be utilized.
In this project we aimed to utilize offline data by adding it to the algorithm’s learning buffer. We dealt with the downfaults of this method (caused by conflicting policies on the large replay buffer) by overhauling the sampling process using rejection sampling. This new sampling process utilizes a discriminator- a CNN capable of differentiating different policies. Our method enables the utilization of offline data while at the same time stabilizing the steady state distribution of our samples, enabling an increase in performance.