Tweetake: Back up your Tweets – if you aren’t running Tweetdeck, that is

Technology Blog

[Update: since Tweetake was “whitelisted” (granted higher request limits) by Twitter, this problem should no longer occur. You should be able to run Tweetake and Tweetdeck together now.]

Tweetake is a new application I’ve been working on with Nikki Pilkington. It lets you back up data from Twitter: tweets, followers, favourites and friends. We think it’s very useful and we have plans to make it even better.

Since it was released a few days ago it’s had mixed publicity largely due to an error message that some users such as pixelbits were getting with depressing regularity. I looked into the error and now can report some interesting findings.


The problem was partly with my code and partly, at least in some cases, with other Twitter applications. Here’s how I’ve summarised it to pixelbits:

Hi, I’m the developer of Tweetake and I’d like to apologize for this problem and explain what’s going on. After investigation – my co-Tweetaker Nikki was getting it as well – it would seem there’s a number of factors involved.

First up, there was a bug in Tweetake, which I fixed. But after that we were still seeing the error on Nikki’s account so I had a look at the actual numbers returned from the Twitter rate limiting interface. And the result was weird.

In case you aren’t aware, Twitter places a limit on the number of hits per hour per user account. Right now its 100 per hour. Every time you tweet, or refresh your timeline the number is reduced by 1, until you hit zero when the request is refused.

Tweetake needs to make a number of hits for its backup – exactly how many depends on the user and how many followers, friends and so on they have. If it gets the rate limit exceeded message, it gives up.

What I found when I looked at the numbers for Nikki was that she had a negative number of hits remaining (as many as minus 44 at one point), which didn’t make a lot of sense. But when she shut down Tweetdeck, things returned to normal and I was able to take a backup.

So it looks as if some clients including Tweetdeck get special treatment from Twitter. Which would be OK if they didn’t have such an unfair adverse effect on other applications, effectively locking them out completely.

So if you use Tweetdeck or any similar Twitter clients, please shut them down and wait a few minutes before using Tweetake. When our app’s a little more mature we’ll be asking Twitter if we can join the inner circle too but until then this measure is necessary.

I’d also ask Twitter to consider giving applications such as ours a more level playing field. One suggestion would be an API data item that tells us the user is running a client which is allowed to break the limit, preferably identifying it by name, so we can give back a more useful error message.

One Response to “Tweetake: Back up your Tweets – if you aren’t running Tweetdeck, that is”

Leave a Reply

  • (will not be published)

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>