Skip to content

Twitter Search Example in Python

This post is response to a few comments. Firstly, as some people had requested it, I’ve re-written the example from here in Python. I’ll be the first to admit, my Python skills aren’t fantastic, but I tested it against collecting 5k tweets and it appeared to work.

The library can be found at https://github.com/tomkdickinson/Twitter-Search-API-Python. Similar to the Java one, it defines an abstract class you can extend from and write your own custom implementation for what you do with the Tweets on each call to Twitter.

It uses BeautifulSoup for the HTML parsing, and just the standard JSON library to parse the raw response from Twitter.

Secondly, if you want some ideas on the sorts of queries you can do, I would suggest you mess about here: https://twitter.com/search-advanced. Whatever options you select on that and click search, you should be able to copy the query it generates in the search box into the script, and it’ll perform the same search.

9 Comments

  1. Santiago Larrain Santiago Larrain

    Hi Tom! Thank you very much for doing this in Python as well.

    I’ve opened up an “Issue” on GitHub for some things with the code that I think are not working. I might be wrong. I’ll be glad to help you.

    Thanks!

  2. ikim ikim

    Thank you. this works well ^_^

  3. Reza Reza

    Hi Tom,

    This is amazing! The Python code works like a charm. Thanks a lot.

  4. Andrew Andrew

    It looks like this code now only allows 10 days of tweets to be scaped

    • Andrew Andrew

      . . . . ok, ignore my message above, it seems it is my IP that is limited to 10 days. When I tried a different IP address the code works brilliantly!

  5. awesome work, just what I was looking for. Thanks for putting together a python version!

  6. Folken Folken

    Hi Tom,
    great work!
    Sorry for this question but I’m a newby.
    How I can export data in cvs?

    • I assume you mean CSV rather than CVS, and if so, I’d recommend having a look at https://docs.python.org/3/library/csv.html.

      Some quick basic pseudo code (this isn’t exact, but hopefully it’ll help) would be something like below.

      Assume ‘tweets’ are the list of tweets that was extracted.


      import csv
      with open("tweet_csv_file.csv","w") as f:
      writer = csv.writer(f)
      for tweet in tweets:
      writer.writerow([tweet['id'], tweet['text']])

      The writer.writerow method takes in a list of data, so you can put whatever you want into it.

      Hope that helps!

Leave a Reply