Comments on: VoteReport mapping and data feeds

By: Andrew

Andrew — Fri, 07 Nov 2008 19:40:14 +0000

Sorry – for some reason my blog wasn’t emailing me with your comments. I apologize! Also, things were quite hectic around the day.

First off – the HTML page is using hAtom, so well structured if you wanted to parse that. Also, you could grab the KML files in count of 4000 over 3 pages: reports.kml?page=1&count=4000&live=1

Looks like the JSON is throwing an error. I’ll look at it.

By: James Baker

James Baker — Wed, 05 Nov 2008 22:33:32 +0000

I’m starting to feel a bit awkward about being in an echo chamber here… I guess I’ll finish out my little saga.

Found the count/page params from the OpenSearch xml, but there are still problems with the json response code path. The largest count= # I’ve gotten to work is 8400 (there are 11,000 tweets total) – anything above that seems to push the server beyond a timeout and it returns nothing.

Breaking up the request into several pages is also failing: For example, http://votereport.us/reports?count=1000&page=11 works while adding “.json” to that reports request returns nothing.

I’m posting here because this post is newer than anything on the wiki – are none of the project developers paying attention? I’m trying to do some visual analysis and then bring more attention to this project. I’d also like to redistribute the (flattened and abridged) CSV version of the data I’ll be analyzing – which seems to be in line with the spirit of this project, but it’d be nice to confirm that with someone.

By: James Baker

James Baker — Tue, 04 Nov 2008 23:04:00 +0000

Well, so much for day-of analysis. I’d love to do a day-after analysis, if a download of the entire tweet stream could be made available.

By: James Baker

James Baker — Tue, 04 Nov 2008 18:19:24 +0000

I’m trying to suck down the JSON data, and I’m not getting much. Sometimes I’ll get ~15K (5-10 tweets), sometimes an empty file. Is this supposed to be streaming data? I can’t seem to get anything using dtstart. I’m trying to download *everything* for analysis; is this intended, and the site just overwhelmed?

By: Sean Wohltman

Sean Wohltman — Sat, 01 Nov 2008 22:48:18 +0000

Andrew, fantastic job – putting the network link on a tour is something that should be done everywhere possible on election day, and I’ll make sure its done at the Reston Google Office’s lobby.

May want to tie in some sort of color-coded feed as the networks start to call states, and if a large volume of tweets are coming in, filter the top tweets not just by time, but by the geocoding result to the states of particular interest.

i.e , If Ohio is in a tight race for contention, start filtering the feed to show time-relevant Ohio tweets until the state is called.

Also, is it too late to try to tie the tweets to the actual polling locations?

Kind of odd to see tweets popping up in the middle of interstates because that’s where the geocoding returns.

~Sean