Given a point robot in a two-dimensional maze environment, the goal is to train an agent that solves the maze consistently and efficiently. At the start of each run, the agent’s initial position and its goal in the maze are initialized randomly. The state and action spaces are continuous, and the reward for each episode is binary—the agent receives a reward of 1 if it reaches the goal, and 0 in any other situation. Such an environment presents a significant challenge for classical RL algorithms, and in fact, many classical algorithms struggle to learn any useful policy.
This environment simulates many real-world problems characterized by a success-or-failure metric. Modern algorithms require engineering a reward function tailored to both the environment and the learning algorithm, a requirement that is impractical for generalizing to different problems. Due to these circumstances, the natural solution is an algorithm that handles binary rewards, as proposed in [1], by integrating Hindsight Experience Replay (HER) into advanced RL algorithms.
The project’s goal is to implement a method for labeling states in the state space according to their “reachability” from a given initial state and to assess whether this labeling is beneficial to the learning process. This labeling is implemented by a classifier that assigns a selected goal a flag determining whether it is reachable, hindsight reachable, or non-reachable. To demonstrate the effectiveness of the above approach, we compare a classical RL algorithm, its implementation with HER, and its implementation with HER combined with the classifier.