Twitter Search Example in Python

This post is response to a few comments. Firstly, as some people had requested it, I’ve re-written the example from here in Python. I’ll be the first to admit, my Python skills aren’t fantastic, but I tested it against collecting 5k tweets and it appeared to work.

The library can be found at Similar to the Java one, it defines an abstract class you can extend from and write your own custom implementation for what you do with the Tweets on each call to Twitter.

It uses BeautifulSoup for the HTML parsing, and just the standard JSON library to parse the raw response from Twitter.

Secondly, if you want some ideas on the sorts of queries you can do, I would suggest you mess about here: Whatever options you select on that and click search, you should be able to copy the query it generates in the search box into the script, and it’ll perform the same search.

12 thoughts on “Twitter Search Example in Python

  1. Hi Tom! Thank you very much for doing this in Python as well.

    I’ve opened up an “Issue” on GitHub for some things with the code that I think are not working. I might be wrong. I’ll be glad to help you.


    • . . . . ok, ignore my message above, it seems it is my IP that is limited to 10 days. When I tried a different IP address the code works brilliantly!

    • I assume you mean CSV rather than CVS, and if so, I’d recommend having a look at

      Some quick basic pseudo code (this isn’t exact, but hopefully it’ll help) would be something like below.

      Assume ‘tweets’ are the list of tweets that was extracted.

      import csv
      with open("tweet_csv_file.csv","w") as f:
      writer = csv.writer(f)
      for tweet in tweets:
      writer.writerow([tweet['id'], tweet['text']])

      The writer.writerow method takes in a list of data, so you can put whatever you want into it.

      Hope that helps!

  2. Hi Mr. Tom..
    Great code, thanks for your sharing.
    But, i still have a problem. i am newbie on Python. i have run this code and it’s working but not like what i want.
    for example, if i want to search tweets contain ” Candi Borobudur” from 2016-10-20 until 2017-02-11 without retweet, what part of code that i should change? i’ve change in search_query (line 252) and select_tweets_since and until (261-262) but it doesn’t affects. The result still load from the current time.

    Please help me Mr. Tom..
    Thank you

  3. Do I need to be careful about how much data I am getting from Twitter? Is there something similar to rate limit for sending API calls here? I just want to prevent my IP from being banned. Thanks.

  4. Hi Tom, thank you very much for sharing this work!
    Yesterday I successfully collected more than 30k tweets related to “Botlibre Bot” using your python code. However, today when I attempted to do the same thing, the program stopped after collecting almost the most recent 3979 tweets. It stopped at line 52, “if len(tweets) == 0: break”. It seems that there are no more tweets. Do you know what the problem is? Does Twitter make some changes on their source code?

    Thanks again.

Leave a Reply

Your email address will not be published. Required fields are marked *