Miscellany

TwitterMirror, private tool for mirroring Twitter timelines

For several apps I use Twitter to display news. Developers can register their app and receive authorisationcodes to be able to communicate with Twitter's servers.

But, Twitter enforces limits on how many requests an app may make per time period. Currently this is 180 requests per 15 minutes, after these 180, an error is returned instead of the requested content.

In order to circumvent this limitation, I wrote a python script that fetches several timelines, and hosts them on Google App Engine. This script handles all authentication and authorisation and allows mobile apps to fetch the mirrored content.

Currently this is used in Word Search Plus and Café Kleyn, but more will follow.

The script used the Tweepy library to communicate with the Twitter API. On App Engine a cron job runs every 3 minutes and calls a URL (available only to admin users) which starts the script.

In order to minimize App Engine datastore calls, the mirrored timelines are stored in memcache.