Our Goal:
Implement the Action Robust RL framework within the OpenAIbaselines codebase and evaluate the approach across multiple algorithms.
The process:
In order to deal with the research we wanted to do we started the project by learning the content relevant to the field. To this end, we first saw the RL Course lectures by David Silver.
After learning the required areas in RL we read several articles including our supervisor article ( Tessler, C.; Efroni, Y.; and Mannor, S. 2019. Action robust reinforcement learning and applications in continuous control), on this article we relied throughout the project procedure, our goal was to apply the algorithm described in it on the discrete plane.
After concluding that we wanted to change the Q-table Update and Action selection we wanted to implement the change in our first experiment: MountainCar framework.
We started by researching the relevant code and figuring out where to put the lines of code we wanted to add, after adding these lines we built CLI tools and created thousands of graphs, from which we selected the selected graphs that show the most information.
After showing that we can create a robust agent that improves the performance of the original agent in a test environment different from the training environment, we turned to the second experiment – Dopamine framework and there we applied the above in a similar way, but now the environment was more complicated and difficult to understand. Finally we managed to meet the task there as well and wrote a short article describing the project procedure and presented it.