Feeding Twitter links into Fever

Lately I’ve been investing some time into configuring Shaun Inman’s Fever feed reader. My aim has been to have its “hot” page become the thing I read when I want news, with the secondary aim being that this results in me wasting less time manually combing through sources like Twitter, HN, Reddit etc.

I’ve had some success with this, and in the process built a little tool which may be of use to others, hence this writeup.

Firstly, if you are unfamiliar with Fever or unsure of the functionality of “spark” feeds, read Anthony Drendel’s excellent post: Why and how I use Fever.

Twitter used to offer RSS output for certain bits of its API back in the v1.0 days. They discontinued it with v1.1 but in any case if you ever used it you’ll probably remember it being largely useless. It would output a list of RSS items all of which linked back to the tweets themselves on twitter.com. Any links the tweets may have contained would be written as <a> link elements inside the body of the RSS item. It was a terrible way to read tweets and nearly useless for extracting links.

Which is a shame because Twitter is an excellent source of what Fever refers to as “sparks”: corroborating links. Links shared/discussed by my friends and people I follow on Twitter are often of interest, so if they match up with links Fever is picking up from elsewhere I definitely want them to be highlighted and make the “hot” list.

twitter2atom

My solution to this problem was to knock up a little PHP/curl tool which queries the Twitter API, extracts only tweets containing links, optionally removes URL shortening & redirects, and outputs an Atom feed suitable for use with Fever. You can find it on github as twitter2atom. The README has install instructions and usage examples; basically you throw it on a protected bit of server space and point Fever (or any other feed reader) at it.

I would advise against making it publically available as it will be accessing Twitter via OAuth as you, thus it is privy to potentially restricted tweets. It would probably also contravene Twitter’s API usage policy.

I personally use it to harvest links from a couple of Twitter lists I follow plus one of my own, as well as my Timeline and some searches for specific keywords (I find a search for “infosec” to be a good source). It passes through most API options so you could get really fancy if you wanted (a search with logical operators filtered to a specific date range for instance).

Update: I’ve written up some notes about spark feeds in general.