I just discovered a bug in Ruby’s open-uri library, caused by the Net::HTTP library, or at best really annoying behavior that will break when you try and use it. The problem lies in dealing with an HTTP request that returns multiple values for a field. This forum topic discusses that having multiple instances of a field is valid in HTTP1.1 (RFC2616).

So when you use Net::HTTP to fetch a document like this, it will create an array for every field, holding each value found for the field. That’s great. However, in open-uri#last_modified then gets this array and joins the values together to get a single string.

The example I ran into is the Last-Modified field. Given HTTP headers like this:

HTTP/1.1 200 OK
Server: Undisclosed-Webserver/0.1
Date: Tue, 14 Aug 2007 15:03:29 GMT
Last-modified: Tue, 14 Aug 2007 15:03:29 GMT
Content-type: application/vnd.google-earth.kml+xml
Last-modified: Wed, 08 Aug 2007 18:34:55 GMT
Connection: close

And then calling response.each_header {|f,v| puts "#{f}: #{v}"} will result in Last-Modified: "Tue, 14 Aug 2007 15:21:22 GMT, Wed, 08 Aug 2007 18:34:55 GMT". Go ahead and try throwing Time.httpdate at that. You’ll get ArgumentError: not RFC 2616 compliant date:.

To simply recreate this bug, try the following ruby code:

require 'open-uri'
res = open("http://popsci.com/popsci/kml/popsci_future_environment.kml")
res.last_modified

A simple solution, or workaround, is to just get the hash and deal with it yourself: response.to_hash.each {|f,v| puts "#{f}: #{v}"} # => last-modified: "Tue, 14 Aug 2007 15:21:22 GMT

I’m not going to mention any names on this particular offending party that was silly to include 2 Last-Modified dates, but you know who you are.

I’m posting this in hopes that search engines grab it and put it at the top of their list when other poor soul’s run into this problem.

If you’re trying to grab a web resource using Open-URI, and you are using basic authentication (username/password) then you’ll need to make sure to require 'open-uri' or you’ll get:

open("http://example.com/site", :http_basic_authentication => ["username", "password"])
TypeError: can't convert Hash into String

Of course, then you toss some yummy Hpricot into the mix for parsing/scraping the good bits of the HTML.

Sunlight Labs has released a public API, their Sunlight Datakit. It’s a straight-forward, simple API for getting access to their Civic data, like Congressional Representatives, zipcodes, timezones, and some geographic information.

There is some basic information about elected representatives that makes politico mashups easier: the ability to tie a name to a state, the district and zip codes that they represent, their office telephone number, and so on. We have put together a simple labs “datakit” that does this for us, drawing from several publicaly available data sources. We are making this fully available and have provided a fully documented API for the methods we have developed for those sources. Find out about the datakit here.

Of course, any API needs a nice little client to tie it into your applications. Here is my Ruby client. It’s very simple, because is uses the fallback method_missing to handle any function passed to the class. This also allows the class to be extended by implementing specific methods if more processing of the response is needed.

require 'open-uri'
require 'rexml/document'
require 'cgi'
SUNLIGHT_HOST = 'http://sunlightlabs.com/datakit/'

class Sunlight
  def self.method_missing(service_method, *args)
    params = args[0].collect {|k,v| CGI.escape(k.to_s) + '=' + CGI.escape(v.to_s)}.join('&')
    url = SUNLIGHT_HOST + service_method.to_s + "?" + params
    open(url).read.split("|")
  end
end

resp = Sunlight.getDistrictFromZip5({:zip => 20740})
puts resp.inspect
  # MD:5
  # MD:4

resp = Sunlight.getRepresentativeNameFromCityState({:city => 'Detroit', :state => "MI"})
puts resp.inspect
  # Kilpatrick, Carolyn C.
  # Conyers, John  Jr.
  # Levin, Sander M.
  # McCotter, Thaddeus G.
  # Dingell, John D.

The Sunlight Datakit currently offers the following functions. Check out the documentation for information on the parameters and returned values.

• getDistrictFromZip5
• getStateFromZip5
• getDistrictFromZip9
• getStateFromZip9.php
• getRepresentativeNameFromDistrict
• getRepresentativePhoneNumberFromDistrict
• getRepresentativeRoomNumberFromDistrict
• getCityFromZip5
• getCityStateFromZip5
• getLatitudeFromCityState
• getLongitudeFromCityState
• getZipCodesFromCityState
• getTimezoneFromCityState
• getRepresentativeNameFromCityState
• getRepresentativeNameFromState
• getStateAbbreviationFromStateName
• getStateNameFromStateAbbreviation

Leopard will officially support Ruby & Python bindings in Cocoa which is terrific news. I’m a big fan of interpreted languages when appropriate.

Of course, there are about a dozen other very cool technologies for developers in Leopard that should really make application development, faster, better, and more fun. I’ve gotten a chance to play some with Leopard previews, and the actual User-facing improvements are minimal. But what users will eventually get will be lots of better third-party applications. (via Theocacoa and Michael McCracken)

It’s great to see how much effort is going into not just creating slicker UI’s, but better supporting the people that really make or break an operating environment – developers. Microsoft has had a lot of developer support for quite awhile, and I constantly hear about various .NET meetings/presentations/technologies etc. Apple really kicked it off by releasing XCode for free (no ‘professional versions’ need apply) and then having terrific Developer Documentation and examples.

The Leopard developer page makes some odd claims:

Mac users love to exchange quick messages, have video conferences, and collaborate on each other’s desktops across the network.

um… right, ok. I just love collaborating on your desktop. Anyways, I can’t wait to dive in and start developing on Leopard.

On Wednesday at the SouthEast Michigan Ruby Brigade I gave a ‘lightning talk’ on Ruby & Hardware. It is a quick succession of slides (written using S5) on some of the cool hardware devices you can control with Ruby, or hardware you can control your computer with.

It has includes examples and links to using Ruby with:

• NabazTag
• Symbian Mobile Phones
• Nokia 770
• Roomba vacuum cleaners
• Apple Remotes (the kind you get with your MacBook/MacMini)
• SlimDevices SqueezeBox and SlimServer
• Lego Mindstorms NXT

There are definitely more devices out there that can or should work with Ruby. Given ruby-serialport it’s possible to control just about anything.