Long term planning is a common problem being researched. It appears in a lot of fields such as robot motion planning and navigation which we worked on in this project.
In this project the goal was to create an agent that control the motors of a robot joints to walk from an arbitrary point of a maze to an arbitrary goal, in a simulation using SOTA RL algorithms and even trying to improve them.
This problem is particularly difficult because we must solve a navigation task using just motor control actions. It is hard to define a sequence of good actions for this task analytically or find one just with classic search algorithms, mainly due to the complex physics of walking and the long horizon of the problem. In some mazes a path from start to goal may take thousands, or even tens of thousands time steps.
In this work we used RL algorithms to solve this problem and proposed a hierarchy approach to break the problem into multiple shorter horizon problems.
We got some good results on several mazes with varying levels of difficulty and with two different robots.