Each of these projects evolves the platform, adding new customization capabilities, more input types, browser support, and more. These modifications may be happening in rapid succession without the main Ushahidi development team even being aware of these changes. And the system may even be running in a remote area with little connectivity.

Traditionally, this has meant that a deployment would download a copy of the current release version, or maybe a development snapshot if there was some emminent new feature that was very useful, and then go off, make modifications – probably on a live server, and maybe email these changes back way after the event in hopes that some of the changes are accepted back into the platform. Updates to the main code base wouldn’t be easily applied to these heavily modified derivatives – so essentially every deploy is a fork of the code.

How Git can save the day

Fortunately, Ushahidi chose Git as the code repository server, although the installation instructions still suggest that you download the code. Git is meant to support just this kind of distributed workflow and collaboration.

If you’re not familiar with Git, I highly recommend checking out Git Internals. But to summarize, it is a versioning control system that is fast, efficient, has local indexes (stored in a local .git directory) and can reference any number of remote indexes to share commits, branches, and files. Where in traditional systems there is an ‘official’ host repository – in Git all repositories are equal and can quickly connect and syncronize.

What this means in a system like Usahidi is that any deployment would first just get a local clone of the official Ushahidi repository to their local system and setup and get running. If they make changes to this code they can just do a commit into their local index. In order to share this code with other developers on this same deployment they could just provide them with the Git link to this repository, or make a branch and add a deployment specific “remote” index that multiple developers could all push into.

Along the way as new code is released in the Ushahidi repository, these deployments could merge in these changes to their local branches without losing their local modifications. And conversely, local modifications could be merged and pushed back into the Ushahidi master index very easily.

Moving sideways

Now lets think about some really powerful uses of the Git architecture. Since the entire index is stored locally in a .git folder – it is easy to put an Ushahidi deploy on a USB stick or archived folder, send it around, make modifications in the field and continue to commit these changes to a local repository even while offline. Then when connectivity is restored, or the USB stick can be brought back to a networked computer, the modifications that had been made, and tracked, could be pushed back to a deployment or Ushahidi instance.

And with arbitrary remote indexes – individual deployments could share code and modifications between themselves without having to go through an Ushahidi instance. Local networks around an incident, culture, language, or feature set could easily collaborate and iterate the code. Imagine if in Gaza, the Al Jazeera instance could have shared code to other local organizations running similar systems.

I think there are even more potential applications of Git to distributed architectures that would be useful for document and database sharing that occur in fast paced situations. However, Git itself will have to work on some of the usability and interface design issues that make it a difficult tool for novice users.

I’ve been playing around with the very complete examples over the last couple of days and am really impressed with how easy it is to build Cocoa apps in just a few lines of Ruby code and using the new InterfaceBuilder. You get full-fledged support of all OS X’s goodness without having to worry about Objective-C.

However, I hadn’t fully considered some of the implications of this until I was stumbled across the open-source OmniFocus-UI. It is a Rails app written as an iPhone interface to the Productivity application (GTD) OmniFocus. It essentially provides a very lightweight web interface to the application by way of the AppleScript interface. rb-appscript and RubyOSA are two Ruby AppleScript bridges that make AppleScript actually really nice to work with.

The fact that it’s a Rails app is almost silly. There is no database connection to speak of. All storage and primary functionality is handled by the app itself. OmniFocus-UI could probably be rewritten as a Camping or Merb application very easily with a much smaller footprint.

The way the Rails app works is that the controllers just route commands via rb-appscript. So when you want a list of contexts (organization of todos), the Rails app just needs to do the following in Ruby:

require 'appscript'
doc = Appscript.app("OmniFocus").default_document
doc.contexts.get.length

Or to see how many Inbox items you have:

doc.inbox_tasks.count

If instead you want to use RubyOSA, it’s really just the Class name that changes (and some more underlying bits):

require 'rbosa'
doc = OSA.app("OmniFocus").default_document

What this means is, it’s very easy to build web interfaces on top of any AppleScript-able Mac application. The one difficulty, however, is configuring your Mac to make it easily externally accessible from anywhere on the Web. If you’re handy on IT, you can always setup port-forwarding on your router.

However, if you just want setup & forget, you should check out Prism (http://goprism.com) – which creates secure, private web accessible connections to your home computer.

Tags: apple, macosx, leopard, rails, ruby, rubyosa, scripting, bridge, prism

So when you use Net::HTTP to fetch a document like this, it will create an array for every field, holding each value found for the field. That’s great. However, in open-uri#last_modified then gets this array and joins the values together to get a single string.

The example I ran into is the Last-Modified field. Given HTTP headers like this:

HTTP/1.1 200 OK
Server: Undisclosed-Webserver/0.1
Date: Tue, 14 Aug 2007 15:03:29 GMT
Last-modified: Tue, 14 Aug 2007 15:03:29 GMT
Content-type: application/vnd.google-earth.kml+xml
Last-modified: Wed, 08 Aug 2007 18:34:55 GMT
Connection: close

And then calling response.each_header {|f,v| puts "#{f}: #{v}"} will result in Last-Modified: "Tue, 14 Aug 2007 15:21:22 GMT, Wed, 08 Aug 2007 18:34:55 GMT". Go ahead and try throwing Time.httpdate at that. You’ll get ArgumentError: not RFC 2616 compliant date:.

To simply recreate this bug, try the following ruby code:

require 'open-uri'
res = open("http://popsci.com/popsci/kml/popsci_future_environment.kml")
res.last_modified

A simple solution, or workaround, is to just get the hash and deal with it yourself: response.to_hash.each {|f,v| puts "#{f}: #{v}"} # => last-modified: "Tue, 14 Aug 2007 15:21:22 GMT

I’m not going to mention any names on this particular offending party that was silly to include 2 Last-Modified dates, but you know who you are.

The TwitterVision API looks like it was built on Rails, which means it is probably a very slick/simple addition for him to expose the data in XML, YAML, and JSON.

No sign of GeoRSS explicitly yet, so you can’t subscribe to just a feed of your friends’ locations, but that would again be something simple to add (and he does hint there is more coming), or for someone else to build on using his API.

One issue that still remains is if forcing a user to specify the “l:” prefix to a location is a usable/effective mechansism. Typing a colon (:) on a mobile phone isn’t the easiest thing to do, but at the same time using other characters such as a tilde (~) are often stripped out or munged by SMS gateways.

A more natural language parsing option would be interesting, but processing is intense with potentially very unclear results (’Sitting at the bar’, ‘Leaving home to go into town’) and also privacy or desire of the user to sometimes not share their location.

Using simple symbols, like parenths: Caffeine good (Sweetwaters Cafe)
would be a good option, since the location may not actually be part of my message, but I do want to share it.

As for parsing, something like Geocodr would provide a very robust geocoding system where the system could parse both odd areas (Soma), buildings/areas (Empire State Building), or even events (Web2.0 Expo).

So the main points are to allow opt-in based on my desire to share location (or destination), simple to enter on a mobile/laptop/brain implant, and optional location specification outside of the actual primary message content. And the issue applies outside of just Twitter, which really just serves as a good context. How would this be done in voice command for directions or location?

But again, needs to be simple and understandable. Don’t you like simple problems?

The article does a full walk-through, from concept, through development, and to scalability, deliverability, and maintainability, of a Ruby on Rails site/service.

Discussions (arguments) between the various languages and their frameworks are common. It’s helpful to see such a readable and poignant presentation of the entire process to address the various issues other developers bring up about the platform.

I have definitely seen the same effects he talks about in the article. Development and updates occur much faster. I can be more productive creating my ideas and getting them to a working prototype. The code is easy to read and small, so its also easy to maintain the code to optimize or extend with new features.

I’m not saying that the other languages or frameworks fail at this, I know that for me, it feels like its the fastest development I can do short of natural language requirements into code (”Make me a Map” )

Tags: rubyonrails, casestudy, business, webdevelopment, infoq, changingthepresent

If you’re trying to grab a web resource using Open-URI, and you are using basic authentication (username/password) then you’ll need to make sure to require 'open-uri' or you’ll get:

open("http://example.com/site", :http_basic_authentication => ["username", "password"])
TypeError: can't convert Hash into String

Of course, then you toss some yummy Hpricot into the mix for parsing/scraping the good bits of the HTML.

There is some basic information about elected representatives that makes politico mashups easier: the ability to tie a name to a state, the district and zip codes that they represent, their office telephone number, and so on. We have put together a simple labs “datakit” that does this for us, drawing from several publicaly available data sources. We are making this fully available and have provided a fully documented API for the methods we have developed for those sources. Find out about the datakit here.

Of course, any API needs a nice little client to tie it into your applications. Here is my Ruby client. It’s very simple, because is uses the fallback method_missing to handle any function passed to the class. This also allows the class to be extended by implementing specific methods if more processing of the response is needed.

require 'open-uri'
require 'rexml/document'
require 'cgi'
SUNLIGHT_HOST = 'http://sunlightlabs.com/datakit/'

class Sunlight
  def self.method_missing(service_method, *args)
    params = args[0].collect {|k,v| CGI.escape(k.to_s) + '=' + CGI.escape(v.to_s)}.join('&')
    url = SUNLIGHT_HOST + service_method.to_s + "?" + params
    open(url).read.split("|")
  end
end

resp = Sunlight.getDistrictFromZip5({:zip => 20740})
puts resp.inspect
  # MD:5
  # MD:4

resp = Sunlight.getRepresentativeNameFromCityState({:city => 'Detroit', :state => "MI"})
puts resp.inspect
  # Kilpatrick, Carolyn C.
  # Conyers, John  Jr.
  # Levin, Sander M.
  # McCotter, Thaddeus G.
  # Dingell, John D.

The Sunlight Datakit currently offers the following functions. Check out the documentation for information on the parameters and returned values.

• getDistrictFromZip5
• getStateFromZip5
• getDistrictFromZip9
• getStateFromZip9.php
• getRepresentativeNameFromDistrict
• getRepresentativePhoneNumberFromDistrict
• getRepresentativeRoomNumberFromDistrict
• getCityFromZip5
• getCityStateFromZip5
• getLatitudeFromCityState
• getLongitudeFromCityState
• getZipCodesFromCityState
• getTimezoneFromCityState
• getRepresentativeNameFromCityState
• getRepresentativeNameFromState
• getStateAbbreviationFromStateName
• getStateNameFromStateAbbreviation

Flickr announces Machine Tags, (via Mikel) which is formal support within Flickr’s database and API for what are also known as TripleTags, namespace:predicate=value.

Example:

geotagged
geo:lat=43.245
geo:lon=-83.211


The idea was to add some specific meaning to a set of data (photo) in a way that was accessible to the user through the user interface. Savvy types would add keywords to the EXIF Metadata, but this is tedious, and this metadata is often stripped out when using some image conversion utilities for resizing/cropping/etc. Therefore, soon after the popularization of folksonomy, users found a way to also add ’semantic’ data.

However, this usage, while working, was not inherently supported. People used it as a way to find all their ‘geotagged’ photos, or photos of forests, and so on. But parsing them and dealing with them was sometimes a pain from the developer perspective, and it also made a mess of your tag clouds or tag listings as you had all these one-time-use tags for the specific lat/lon point.

Now that Flickr internally supports machine tags in a good way, developers can really start using this as a way to attach arbitrary metadata to any data item (photo, etc.) Dan Catt gives a good discussion on what it all means and how it can be used.

I wonder if a Microformats-like impromptu standards body will pop-up to help formalize the namespaces and predicates. Why ‘geotagged’ and not ‘geolocated’, ‘geo’, ‘geomarked’, etc? While free-form naming is nice – so very extensible, being able to use a common vocabulary would also be nice.

Perhaps just accomodating current standards and applying them to the machine tags would be good without requiring any additional work. I’m thinking like hcard:fn=andrewturner, hcard:url=http://highearthorbit.com, hcard:phone=555-1212, etc.

You the developer

So, the end result to the developer is to check out the additional parameters to the Flickr search method in the API. There is now a machine_tags optional parameter that allows you to search by namespace, predicate, or value and use boolean constructions for these parameters.

Dan Catt even references Spimes as an excellent example of why machine tags can really empower devices, especially sensored/automated ones. And now that Flickr has no limits on uploads, you can create frequent update-rate sensored images.

Tags: flickr, semantic, geo, tags, photography,

Of course, there are about a dozen other very cool technologies for developers in Leopard that should really make application development, faster, better, and more fun. I’ve gotten a chance to play some with Leopard previews, and the actual User-facing improvements are minimal. But what users will eventually get will be lots of better third-party applications. (via Theocacoa and Michael McCracken)

It’s great to see how much effort is going into not just creating slicker UI’s, but better supporting the people that really make or break an operating environment – developers. Microsoft has had a lot of developer support for quite awhile, and I constantly hear about various .NET meetings/presentations/technologies etc. Apple really kicked it off by releasing XCode for free (no ‘professional versions’ need apply) and then having terrific Developer Documentation and examples.

The Leopard developer page makes some odd claims:

Mac users love to exchange quick messages, have video conferences, and collaborate on each other’s desktops across the network.

um… right, ok. I just love collaborating on your desktop. Anyways, I can’t wait to dive in and start developing on Leopard.

Tags: macosx, apple, leopard, ruby, python, cocoa, bindings, objective-c, programming, microsoft, .net, theocacoa, michaelmccracken

It has includes examples and links to using Ruby with:

• NabazTag
• Symbian Mobile Phones
• Nokia 770
• Roomba vacuum cleaners
• Apple Remotes (the kind you get with your MacBook/MacMini)
• SlimDevices SqueezeBox and SlimServer
• Lego Mindstorms NXT

There are definitely more devices out there that can or should work with Ruby. Given ruby-serialport it’s possible to control just about anything.

Tags: ruby, programming, roomba, nabaztag, symbian, nokia770, apple, squeezebox, slimserver, slimdevices, brigade, rubymi, rubybrigade

Ajaxian discusses Adobe’s new “Apollo”:

Apollo is client-based software that will run Flash applications separately from a browser, whether online or offline

The image shows an example travel application developed in Flash, and deployed to a desktop via Apollo. (via Digital Backcountry)

I also saw that Chris Messina is helping out on a project WebKit on Rails, whose goal is to make it easier to deploy Apple’s WebKit and also to “come up with new ideas and practices that leverage the WebKit platform”. WebKit is an excellent platform to develop desktop web apps, as it can be baked straight into a Cocoa application, but be accessing a “web application” that may be running locally on the users’ machine.

rails-app-installer allows you to bundle and install/uninstall a Rails application, including required gems.

$ gem install my_app
$ my_app install /some/path


Tags: web3.0, desktop, deployment, rails, webkit, chrismessina, mattwebb, tixe, adobe, apollo, railsappinstaller

It’s a really good how-to for making your own maps from your own data, and also an excellent idea for brain-dumping after a big project. Whenever I do a large project, I learn a lot of useful tips, strategies, what worked/didn’t work, etc. that is probably useful to a larger group of developers and users (for knowing what’s going on underneath). Not only for my own notes, but also anyone wanting to improve upon my work, or go and fix parts of it.

Documentation in engineering and software development is always emphasized and always a battle to get done. Inline documentation using something like Doxygen really helps, but then also putting together a single, simple overall report, especially on an editable format like a Wiki, can really make a big difference.

Tags: nestoria, osm, tiles, howto, geo, mapping, programming, documentation, doxygen, mikelmaron, brainoff,

Using Tesly Jr., its very easy to keep your development team up to date with the status of the project. Combine this with runing tests on deploy with Capistrano (via Jamis Buck) and you have a that much better integration of testing into your development and release cycle.

Tesly Jr. is free when using the hosted service, and available for just $24 if you want your own copy to run on your own systems.

Tags: rails, ruby, programming, rubyonrails, teslyjr, bencurtis, testing, capistrano, jamisbuck

I first noticed this when deciding what I wanted to do in Undergraduate studies. Computer Science was hot stuff in the late ’90’s, but it really seemed like companies weren’t looking for Computer Scientists, they were really looking for Software Engineers, Developers, and Programmers (all different jobs with different skills, personalities, and types of work).

Computer Scientists really should be developing “formal specifications of a programming language”, whereas Software Engineers should be figuring out how to scale out a database, or apply domain specific solutions.

Of course, I ended up doing Aerospace Engineering, but with a Computer Science Minor. The CS minor let me take C++ instead of Fortran, and also have courses where I actually learned how to do requirements documentation, work in a team, design and build large scale systems in a single semester – all skills I didn’t learn in the Aerospace department but have since applied to my work.

I firmly believe that in the near future, all engineers/scientists should/will learn how to program. It is becoming a basic skill necessary to do proper analysis. Given a little bit of programming ability, an engineer can relieve themselves of copying & pasting from Excel to MatLab or other such silly things. Teach them good programming and software design techniques, in a modern and easy to use language with good tools, like Python (and then SciPy and NumPy) and they can be much more effective designers and analyzers. Also, if they learn general, good technique, they can apply that ability to learning other languages that may be specific to their domain.

And no, learning Fortran (or Java) is not going to cut it.

Check out the presentation here: Ruby & Hardware.

I’ll be talking about them at the November AnnArbor.rb/RubyMI meeting if you want to come by.

Tags: ruby, gadgets, hardware, presentation, nokia, slimserver, squeezebox, nabaztag, roomba, michigan

Out of the Ruby realm, BlogHelper has a good set of tools and howtos for Using Microformats in WordPress.

Tags: ruby, microformatsstandards, , hatom, assafarkin, cheatsheet, wordpress, bloghelper

Yahoo has really impressed me with their change from “annoying portal with too many links” to “doing lots of cool stuff and telling people how to use it too”. Their developer network is a great resource for both just learning about a language/tool/design pattern, as well as how to leverage really deep and useful Yahoo resources within your own application.

Providing these resources is definitely in their own interests, such as Cache Yahoo! Web Service Calls using Ruby, but is also just a great service to the community. Their Gallery is also a good place for interesting sites and project ideas.

Tags: ruby, yahoo, programming

You can get the hCalendar here and the iCal link here.

The Problem

Here is the current HTML of the schedule. As you can see, this is an absolute mess of DOM. This table is in fact already the 4th embedded table (tables-within-tables-within-tables oh my!)

In the middle there was some actual interesting bits, such as presentation title, author, times, etc. So what we need to do is walk through all this and build up a conference.
The Solution
Employing some slick Ruby scripting – and using the very useful  scrAPI from Assaf we can define scrapers to walk over the multiple days, and then within those days grab each of the sessions. These are then output into proper hCalendar format like: 
<span class="vevent">
 <a class="url" href="http://www.foss4g2006.org/contributionDisplay.py?contribId=189&amp;sessionId=46&amp;confId=1">
  <span class="summary">Enabling Users to Produce personalized Geodata</span>
  <span class="details"><span class="vcard"><span class="fn">Mr. Andrew TURNER</span><span class="org">HighEarthOrbit</span></span></span>
  <abbr class="dtstart" title="2006-09-15T10:30:00Z">Friday,  15 September 2006 from 10:30</abbr>-
  <abbr class="dtend" title="2006-09-15T11:00:00Z">11:00</abbr>,
 at the <span class="location">Amphimax MAX 350</span>
 </a>
</span>
The code below makes parsing the nightmare above fairly simple, but due to the lack of any proper classes or id’s (each presentation is id="entry" – eep!), we have to find the bits we want by their current markup attributes. Not suggested, but at least this is nicer than trying to figure out the 10-levels of DOM starting at the root.
You can see the parser here.
]]>http://highearthorbit.com/converting-table-based-calendars-to-hcalendar/feed/5
		
		http://highearthorbit.com/rubyconfmi-the-word-is-testing/
		http://highearthorbit.com/rubyconfmi-the-word-is-testing/#comments
		Mon, 28 Aug 2006 03:19:13 +0000
		Andrew
				
		

		http://highearthorbit.com/rubyconfmi-the-word-is-testing/
		
			I went to my fourth conference this summer. Conferences, especially small ones, are a great chance to get quick insights into new ideas or technologies, but most importantly to share ideas with a lot of other people.

The presentations were good, many revolved around testing, with some talking about performance and deployment. There were very few presentations of new novel ideas or techniques. That’s where discussions come in.
I’m a new Ruby programmer who came to the party by the way of the popular gateway drug, Rails. So I’m still behind on proper and good Ruby tools and techniques. Pat Eyler gave an overview of various Ruby libraries for testing and performance such as test/unit, autotest, unit_diff, rspec, rcov, ruby-prof, and benchmark. 
He sequed very well into SouthEast Michigan’s own Patrick Hurley’s talk on C optimization. Coming from a C/C++ background, knowing how easy it is to add small chunks of fast C code to dramatically speed up code (Patrick’s example showed a 20-times increase). 
Back in Rails (esque) land, Zach Dennis talked about his ActiveRecord:Extensions (AR:E). ActiveRecord isn’t Rails specific, but makes a big part of the functionality behind Rails. AR:E adds a lot of very convenient functionality as well as speed improvements to database operations. 
And of course, we made another pilgrimage to the Grand Rapids Brewing Comany.
Tags: ruby, conference, rubyconfmi, optimization, programming, pateyler, patrickhurley, activerecord, activerecordextensions, rcov, testunit, autotest, rspec, benchmark
]]>
			http://highearthorbit.com/rubyconfmi-the-word-is-testing/feed/
		3
		


Tuesday, 12 September 2006







07:00

    Registration(Amphipôle (niv. 3): 07:00 - 09:00)
  













08:00













09:00




   [20] Getting Started with MapServer
   by Mr. Jeff MCKENNA (DM Solutions Group)
 

  paper