Mining Tweet Data From Twitter

Twitter is Warehouse of data. It is trove of peoples Opinions and emotions. It contains lot of data of real people which is very useful to observe trends and build models. What makes Twitter really Attractive is it has some really nice API collection. This API’s Allows us to harness it data and build systems that are Useful like Real time Sentiment Analysis and Reputation analysis

we Will dive Deep into Twitter’s API’s collections. As Data mining Enthusiast we are interested in data collection and not Application Development so we will mainly concentrate on Certain API’s

But We should be careful as Rest API’s are rate limited. After retrieving data from these API’s we need to store them efficiently we will use a database so that analysis later on will be easy.

There are lot of option while choosing database SQL? NOSQL? SQL is fairly easy and doesn,t require that much effort So we will use MYSQL for this example. Alright lets Dig in

First of all for Using Twitter API you need a Twitter Devloper account So just sign up for a twitter account in Twitter and go to This Link and create a Twitter App . Type any app name and description and get the consumer key. You need to Create an Access Token in order to send requests to Twitter API. Click create access Token to create access token . Totally we need Four Keys for Accessing Twitter API

R is a choice of many Statisticians and have really good eco-system of libraries for Data Analysis. For Twitter Data collection We have “twitteR” library and for Data Storage “RMysql”. Here we use twitteR library to search trending topics in particular place and use Search API to store those tweets in a data base

Loading libraries required

Now we need to acess geometric data in-order to choose location in which we have to get the twitter trends

Now the above code can be simplified by “dplyr’s” Piping operator but to keep things Clean and simple we will do it in the old way. Now we need to iterate over the above list and get the tweets and store them in a data base .

Yet another Tool which is Highly Matured and flexible for data analysis. It has very good library for Twitter. Its “Tweepy”. Tweepy has many functions but we mainly concentrate on Data collection features

First Lets import the needed libraries and Access Keys

Next we need credentials of the MYSQL database in which we need to store our tweet data

We should have have a listener class to listen to the incoming stream of twitter data. Below we implement the listener class

In the above listener class we are listening to the data and extracting the tweet, username time-stamp and location of the tweeter to the database table

Unlike in the Previous Example in R We need to create the table before hand in the database server with the column’s we are using above . However We can Do it Through SQL Statements in Python. To do it just add below code after initiating connection

Then we can use the credentials and collect the Stream data of particular Tracking word of interest lets say “Batman” 😛

Now we have a Data base of tweets we can Use of Twitter Sentiment Analysis

Related posts:

You might be thinking, what is it all about? I bet we have come with the topic which you might have never thought about- How to do Mobile Keyword Research? You have never thought of this topic How to…

An innovation which is built on Sui blockchain. The ecosystem is getting bigger even Sui mainnet is not ready yet. Sui is the most premiere blockchain in the world. It’s technology is as innovative…

I was running an investment-backed startup in Washington, D.C. — the city that I called home for four years during and after college. I also had a girlfriend at the time who I met during my travels…