Halite by Two Sigma (“Halite”) is a resource management game where you build and control a small armada of ships. Your algorithms determine their movements to collect halite, a luminous energy source. The game combines both short-term tactics with long-term planning, making it very suited for many Reinforcements Learning \ planning algorithms available today, which is a perfect fit for our goal.
Solving the Halite game is not an easy task. However, this project will make us one step closer.
We will use DQN to train the model to collect Halite as much as possible for various situations (one ship, two ships and three ships), the model will be trained again and again to improve and become as strong as possible.
In our project, first we performed a simple task to get a basic understanding of how to use DQN. Then, a model with only one ship and no opponents was trained. We slowly increased the complexity of the tasks while dealing with the difficulties that we got, the model had to control two ships and then three. After that, the model had to learn how to interact with one opponent.
Due to the difficulties that we faced while trying to train our agent, changes need to be made to overcome these difficulties, hoping the new adjustments will make the learning more stable and less time consuming.
We proposed the following solution:
Step 1:
Rather than beginning with a random policy, start with a good policy that was acquired by supervised learning over data.
Step 2:
Instead of only having one model to learn how to control the movement of a various number of ships, many models will be trained, and not just one. Each model will learn how to control a constant number of ships, which will be easier to learn. After training each model and reaching satisfying results, a dataset will be built and used in step 3.
Step 3:
Each state is accompanied by a decision in the dataset. So, a function that maps a state to an action can be learned by supervised learning