Reinforcement learning (RL) is a popular method for solving problems involving decision tasks in which the agent has only limited environmental feedback. On top of RL, curriculum learning aims to shorten the training process by training the agent on ascending difficulty environments in order to solve tasks that otherwise can’t be taught. Despite being an acceleration method, choosing the sequence of the environments is atrial and error procedure. In order to solve that and fasten the training procedure, we propose a novel method where a Multiarmed bandit (Coach) chooses the curriculum without human interference, which optimizes a multi-armed bandit problem with stochastic non-stationary rewards where the rewards are the average return of the agent from the training epoch. We observe that the Coach follows a logical environmental sequence along non-human like logic.