Our goal was to dive into the GCSL RL algorithm, and expand it based on other concepts in the reinforcement learning field to get a better model, that learns faster, is more stable, and hold some knowledge over how important some states are over others given the goal the agent is trying to reach.
We investigated whether Q-values can be introduced into the GCSL algorithm to create a better model, with the benefits of both, considering next topics:
- Runtime complexity – the new algorithm won’t be no good if it will extend the runtime of the current solutions.
- Resources – we would like the new algorithm to have as smallest overhead space as possible since the current solutions already take a lot of memory space and energy to compute.
- Accuracy – Last, but most important, is the accuracy. The new algorithm must yield better results than the current solution, which will be measured as the success ratio of the new agent and as its ability to foresee the predictions of and existing GCSL trained model.
We used GCSL because
- GCSL is good for exploration of the environment
- Supports many goals
- More stable than many other algorithms
We used Epsilon Greedy Bellman Update because
- It alowes learning towards interesting final state
- Gives a valuation of each explored state
By combining the two:
- we expect to learn a policy fast and reliably
- We expect to be able to tell the agent what states are more interesting for the goal
The results were that we succeeded is creating a Q-net infrastructure and successfully trained it to have a small error on the training data set. Unfortunately, it seems that the trained networks over an environment that was better suited for GCSL algorithms does not generalize well to a larger state-goal-action space.
A DALL-E generated photo exploring new territories: