2016 US Presidential Election Tweets Crawler

A streaming tweets crawler for collecting tweets related to 2016 US presidential election.

How were the keywords generated?

  1. Count the frequency of 1-gram and 2-gram in the recently collected data.
  2. From the most frequent 1000 1-gram and the most frequent 1000 2-gram, manually select keywords that are related to the presidential election.
    • Remove the words that are obviously too general (e.g. vote, win, debate, etc.).
    • Search rest of the frequent keywords in Twitter, observe the search result to see if most of them are related to the election.
    • Search Twitter at https://twitter.com/search?f=tweets&vertical=news&q={query} (login required) – replace {query} with the keyword.

 

Downloads :

 

2016 US Presidential Election Tweets Crawler Python Script

 

I hope this tutorial will surely help you. If you have any questions or problems please let me know.

Happy Hadooping with Patrick..

Leave a Reply

Your email address will not be published. Required fields are marked *