Code for retrieving followers from Twitter API results in al

This is the place for queries that don't fit in any of the other categories.

Code for retrieving followers from Twitter API results in al

Postby Clockchan » Wed Aug 07, 2013 9:05 pm

I would like to scrape the Twitter API to retrieve the follower ids for specific users, so that I can map their connections.

When I run the code below, the followerIds for each individual user are the same, which can't be right. Every time I run the program I get a different common follower id...every user has the same follower id, but the follower id that's repeated is different every time I run the program. How do I fix this problem?

Code: Select all
    try:
        import json
    except ImportError:
        import simplejson as json
        import urllib2
        import urllib
        import codecs
        import time
        import datetime
        import os
        import random
        import time
        import tweepy
    from tweepy.parsers import RawParser
        import sys

    fhLog = codecs.open("LOG.txt",'a','UTF-8')
    def logPrint(s):
    fhLog.write("%s\n"%s)
    print s

    #List of screennames of users whose followers we want to get
    users =["_AReichert",
    "_CindyWallace_",
    "_MahmoudAbdelal",
    "1939Ford9N",
    "1FAMILY2MAN",
    "8Amber8",
    "AboutTeaching",
    "AcamorAcademy",
    "acraftymom",
    "ActivNews",
    "ActuVideosPub",
    "ad_jonez",
    "adamsteaching",
    "ADHD_HELP",
    "AIHEHistory",
    "ajpodchaski",
    "ak2mn",
    "AkaMsCrowley",
    "AlanAwstyn",
    "albertateachers"]


     # == OAuth Authentication ==


    # The consumer keys can be found on your application's Details
    # page located at https://dev.twitter.com/apps (under "OAuth settings")
     consumer_key=""
     consumer_secret=""

    # After the step above, you will be redirected to your app's page.
    # Create an access token under the the "Your access token" section
    access_token=""
    access_token_secret=""


    auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_token, access_token_secret)

    rawParser = RawParser()
    api = tweepy.API(auth_handler=auth, parser=rawParser)


    #Will store ids of followers for each user in the user_output directory
    os.system("mkdir -p user_output") #Create directory if it does not exist

    userCnt=0
    fhOverall=None
    for user in users:
         userCnt+=1
         print("Getting user %s of %s"%(userCnt,len(users)))
         count=1
        nCursor=-1#First page
        while count>0:
            id_str=user

            try:
               fh=open("user_output/"+str(id_str)+"_" + str(count) + ".json","r")
               result=fh.read()
               fh.close()
               wait=0
            except:
               result=api.followers_ids(count=5000,user_id=id_str,cursor=nCursor)
               fh=open("user_output/"+str(id_str)+"_" + str(count) + ".json","w")
               fh.write(result)
               fh.close()
               wait=60


            result=json.loads(result)
            nCursor=result["next_cursor_str"]
            if nCursor=="0":
                count=-1
                nCursor=None
            else:
                count+=1
                print("Another page to get")

            time.sleep(wait)



    logPrint("\nDONE! Completed Successfully")
    fhLog.close()
Clockchan
 
Posts: 1
Joined: Wed Aug 07, 2013 8:51 pm

Re: Code for retrieving followers from Twitter API results i

Postby micseydel » Wed Aug 07, 2013 11:09 pm

Can you replicate your problem with 10 or less lines of code that we are able to run ourselves?
Join the #python-forum IRC channel on irc.freenode.net!
User avatar
micseydel
 
Posts: 929
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA


Return to General Coding Help

Who is online

Users browsing this forum: No registered users and 2 guests