Bitcoin News and Big Data

For this project, I wanted to see how the titles of news articles about bitcoin affect its price. I analyzed 500,000 words from 50,000 article titles related to bitcoin. In order to do this I needed a few things:

  • A list of the past three years of Bitcoin prices down to the minute
  • An RSS feed that had a long history of articles
  • A database to quickly summarize data

I found an awesome website (Kaggle) that had a CSV of all the past bitcoin prices minute-by-minute to the beginning of 2015. I also found a website that would give me the RSS feed required (BitNewz).

Screen Shot 2017-11-19 at 2.40.24 AM.png

Now that I had my dataset, I started by parsing all of the data from the CSV file into my program. After that, I parsed the RSS feed for the last 50,000 articles about bitcoin. The titles of all 50,000 were then divided up into their individual words.

gif

I used the timestamps of the articles in order to find the same timestamp in the parsed CSV file. Then I added specific amounts to the timestamp in order to find out the difference in price after the article was posted. I did this for 5-minute intervals all the way up to 55 minutes. This data was being appended to an excel file as it was running through the articles.

Screen Shot 2017-11-18 at 1.36.23 PM.png

Next, I opened up this excel file with over 500,000 records in Microsoft Access. I ran a query on this data in order to consolidate all of the words that were the same and get an average price change for each word. After running the query, I was left with about 26,000 records in the dataset. I immediately noticed some interesting numbers. For example, whenever the word “CEO” was mentioned in an article, the price of bitcoin went on a downward trend.

Screen Shot 2017-11-19 at 2.42.00 AM

After that, I exported these records back to excel. This is the excel file that I used in order to make price predictions for new articles.

The final python program gives users a few choices, they can:

  1. Turn on an RSS monitor that will notify them whenever a new article comes out, as well as give them a price prediction based on the title of that article.
  2. View the 15 most recent articles posted online about bitcoin, and see what the price predictions were for those.
  3. Enter their own title/keywords and see what the price predictions would be.

Screen Shot 2017-11-19 at 2.35.51 AMScreen Shot 2017-11-19 at 2.43.17 AMScreen Shot 2017-11-19 at 2.44.04 AMScreen Shot 2017-11-19 at 2.44.18 AM

View the full code on GitHub

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s