Predict Stocks with Tweet Sentiment

Question

Hello everyone,I am new to DS/ML and currently trying my head on this project. I would like to predict the stock price for one or two day ahead based on the historic price and the tweet sentiment for that particular stock.Currently, I have around 350,000 tweets dataset for a particular stock and did sentiment analysis using Vader Lexicon analysis, and the stock dataset from yfinance. Plotted both the charts to see some correlation and found out twitter sentiment does have affect on the stock price.I am confused with how to use this sentiment to predict the stock or how should i proceed further with this project or is this something useless if I put it on my Resume.Thanks

kratom_sandwich · Accepted Answer

Have you tried looking into academic literature? There's lots of papers on text analysis, including several surveys. Start with [1] for example, which creates a word list suited for applications in finance, and then work with forward citations.[1] DOI: 10.1111/j.1540-6261.2010.01625.x - "When Is a Liability NOT a Liability? Textual Analysis, Dictionaries, and 10-Ks" by Loughran and Mcdonald

anigbrowl · Answer

Why don't you just give it $1000 and let it trade for you, if the effect is strong maybe you won't need to care about your resume.

sigmaprimus · Answer

Possibly weighting sentiment against frequency of tweets?
I would say that You also need to include swings in the general marketplace. I would do this by observing comparables or simmilar stocks and analyze them using your system, if they all line up or one stands out that might be the key.
The more data points You can use the better, especially if you are using ML to create the model.
Robinhood might be a good test case, they are planning an IPO so You can watch it right from the beginning. It would be ironic if all the Gamestop investors took their profits and hedged Robinhood into the ground! If that happens I bet preceding Twitter tweets will be a harbinger of sorts.
Just don't get caught up in a "Red/Black" gambling system and lose your shirt!
Maybe looking 2 days into the future is a bit too ambitious of a target right now, a couple hours or even less time would be a huge advantage and should make it faster to verify.
Good luck!

zora_goron · Answer

I don't have any experience in this domain, but perhaps one thing to try is formulating this challenge as taking in Tweet sentiment and predicting the delta in stock price. For example, using the stock data, generate the % change in stock price every day compared to the prior day. Then, using Tweets from a date _n_ (or range of dates), try to predict the % change of the stock price on date _n+1_.

almostkorean · Answer

Sounds like a perfect fit for https://signals.numer.ai. If you have any questions there's an active chat community at https://community.numer.ai

giantg2 · Answer

Yes, it can be done. It likely won't produce the results you hope for because other people have already done it (the game has already changed to account for this product). It could still be a good resume booster.

visox · Answer

Maybe something interesting to you https://insightrend.com/?term=gme

yvino · Answer

can you make it online lets validate it.