Category Archives: twitter_mining

Twitter mining project posts

twitter_mining: oauth with tweepy

Tweepy provides a module to authenticate with twitter using OAuth. The example below retrieves the auth credentials from a config file and creates a filter stream for the terms 'technical' and 'elvis'. You can get a CONSUMER_KEY and CONSUMER_SECRET by creating a twitter dev account at http://dev.twitter.com/apps/new. The access token and access token secret can be found under the "My Applications" link in your account.

import tweepy
import ConfigParser
import os

class Listener(tweepy.StreamListener):
    def on_status(self, status):
        print "screen_name='%s' tweet='%s'"%(status.author.screen_name, status.text)

def login():
    config = ConfigParser.RawConfigParser()
    fn = os.path.join(os.environ['HOME'],'conf', 'twitter_mining.cfg')
    config.read(fn)

    CONSUMER_KEY = config.get('auth','CONSUMER_KEY')
    CONSUMER_SECRET = config.get('auth','CONSUMER_SECRET')
    ACCESS_TOKEN = config.get('auth','ACCESS_TOKEN')
    ACCESS_TOKEN_SECRET = config.get('auth','ACCESS_TOKEN_SECRET')

    auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
    auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
    return auth

try:
    auth = login()
    streaming_api = tweepy.streaming.Stream(auth, Listener(), timeout=60)
    streaming_api.filter(follow=None, track=['technical','elvis'])
except KeyboardInterrupt:
    print "got keyboardinterrupt"

Find the complete codebase on github at: https://github.com/telvis07/twitter_mining

new project: twitter mining

I've started a new twitter mining project in a effort to blog regularly and play with interesting tech. The initial code is on github at https://github.com/telvis07/twitter_mining. The code will be updated often and this blog will be updated weekly (hopefully). The goal of this project is to develop novel ways find the MOST meaningful tweets and tweeters over a given interval. I will blog about the current code and any additions I make.

This project is inspired by my awesome wife Sharon who wants info relevant to her site MidtownSweets. This is also inspired by the book Mining The Social Web by Matthew A. Russell - which is a great book.