Project Goal: Use tweets database, stocks data and machine learning algorithms to predict future stocks movement. Our solution: We achieved our solution by 6 major steps 1. Build a database of tweets using Tweeter developer’s API – We collected ~150k Tweets from 15.12.20 to 5.3.20 using a daily run python script – The tweets were associated with 19 major companies using relevant keywords – The keywords were categorized as “Safe” or “Unsafe” depend on possible irrelevant use of them in a sentence – The number of followers for every Tweet publisher were collected as well 2. Fetch the corresponding stock movement graph for each Tweet – Using “Yahoo Finance” we collected stock data for 30 days before and after each Tweet publish date 3. Produce labels based on the stock graph: – We calculated the deviation of the stock movement from the average movement in the days after the Tweet publication while giving a higher impact to the closest days: 𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿𝐿= 1 ∑ 𝑑𝑑 𝑁𝑁−1 𝑑𝑑=0 �(𝑁𝑁−𝑑𝑑𝐿𝐿𝑑𝑑)⋅𝐷𝐷𝐿𝐿𝐷𝐷𝐷𝐷𝐿𝐿𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷(𝑑𝑑𝐿𝐿𝑑𝑑) 𝑁𝑁−1 𝑑𝑑𝑑𝑑𝑑𝑑=0 4. Convert the Tweets to numeric representation using NLP: – Using pretrained BERT model English dictionary we converted the Tweets to numeric vectors 5. Train BERT model with the database – We used BERT model to compare Tweets using their words inner relations 6. Test the model and improve the results based on the test conclusions – We used a histogram to determine the majority opinion in each date for a given company and got 35.2% average deviation error per Tweet – We tried to filter our database based on: o Safe/Unsafe keywords o Number of followers o COVID 19 impact on the market (Using only Tweets from before the impact) We got the best result using the entire database, suggesting that even Tweets with lower impact (followers) give a good perspective for public opinion, and that most of the “Unsafe” keywords provided valuable information. – We tried a bigger BERT model but got better results on the basic one. Suggesting that training the bigger network requires a bigger database.