The RL framework requires to define reward function, which generates scalar reward per action,
which can be a hard task.
Therefore, many RL environments are described with a ‘Sparse reward’, in which most of the agent actions would receive no reward, except for an action that would lead the agent to the final goal.
A lot of RL algorithms can have difficulties getting to a good result in those kinds of environments
For some tasks, evaluating the performance for a sequence of actions can be easier than evaluating the performance per action.
For many RL problems an informative reward function can improve the learning process of a sparse reward environment, though it can be much harder to define.
RL With Trajectory Feedback is an addon for regular RL problems, which enables to use observer feedback on agent’s performance to learn an informative reward function.
First, we showed that an informative reward function (per state, action pair) can be learned using a sequence reward function, which was represented by labeled sequences.
In addition, we showed that by using merely 200 user labeled short sequences, we could learn a reward function that brings unsolvable sparse-reward environment to converge.