Progress still continues with the broad adoption of the OpenSearch-Geo extensions written at WhereCamp just about 3 years ago. The current OGC OWS-7 testbed is considering many ways in which to integrate OpenSearch-Geo into catalog services and part of the official OGC architecture.

Specifically, Pedro and Jo have done great work getting the community supported specification written into OGC format. You can read his excellent article outlining the proposed modifications. Simple modifications include changing the geo:locationString to geo:name and also adding a geo:geometry to use more complex geometries than a simple polygon.

If you want to chime in on the conversation, join the OpenSearch mailing list, leave a comment here, or come to a WhereCamp, such as the one in Mountain View next weekend, to discuss it with the community.

Some of the sites I use most don’t support the very nice feature of OpenSearch discovery links. Among these are Flickr and Google Maps – so I first have to navigate to Flickr, then search rather than doing it straight from the browser quick search area.

Fortunately, someone has provided this for me. Just go to http://mvinetwork.co.uk/opensearch/ and click open your OpenSearch discovery and add whatever engines you want!

Hopefully more services that already can support OpenSearch templating add the necessary and simple descriptions and discovery links.

For extra credit, you could extend the OpenSearch definition for GMaps to support location name: http://maps.google.com/maps?q={searchTerms}&hnear={geo:locationString?} – and even Flickr supports (and was the model for) OpenSearch-Geo box search.

Over the past two weeks I’ve been working with a great team of people helping to build VoteReport – an open public reporting system to be used during the 2008 US Election to track the situation as citizens cast their ballots. The simple goal is to make it easy for anyone to send in a report describing the wait time, overall rating and any complications that are impairing their ability to participate in the election. For more information check out http://twittervotereport.com.

Dave Troy has put together a solid backend that is aggregating together Twitter, SMS, voice, iPhone and Android native applications, and even YouTube. Others have built the iPhone specific applications. I’ve been working on the mapping and data sharing side of the project. The first goal was to provide a number of mechanisms to share the data that we’re gathering with everyone. Additional mashups and visualizations are free to use the data streams to pull all the data that VoteReport itself has – so definitely go wild with your ideas. A quick breakdown of what’s available:

OpenSearch – http://votereport.us/opensearch.xml

This is the OpenSearch description document that outlines all of the feeds and various filters that you can use when getting to the data. Always check this as we’ll update it with new parameters or data streams. In addition, the various responses discussed below include OpenSearch styling pagination so you can walk through the entire database of reports without having to drink right from the firehose. This also includes the OpenSearch-Time extension.

KML – http://votereport.us/reports.kml

Getting the reports.kml will give a Network Link – this is useful for GoogleEarth and other KML clients to automatically update every 60 seconds with new reports. You can append live=1 to get the full KML document. I have included all the useful attributes in the ExtendedData element of all the Placemarks. Each Placemark also has an id for easy reference.

GeoRSS-Atom – http://votereport.us/reports.atom

Just want to subscribe to the feed in your RSS reader, this feed is useful for getting updates.

GeoJSON – http://votereport.us/reports.json

JSON is super nice for doing client-side mashups and visualization. This is what the VoteReport Map itself is using. It includes a lot of information for each report, including reporter, icon, location.

All of these feeds even can take a dtstart= with an ISO-8601 date for getting reports after a certain time (and optionally dtend= for getting time-bounds of reports). A useful geographic filter is to use state= with the capitalized two-letter state code to just get reports within a state. So for example http://votereport.us/reports.atom?state=VA is a GeoRSS feed of reports in Virginia. As I mentioned, I did build a quick map that you can view at http://votereport.us/reports/map.

We’re continuing to build it out with new features as more data comes in. You can easily embed the map in your site using (and optionally remove the state=):

The difficulty with creating more visualizations is the lack of pre-election data. This system has been built to primarily capture a huge amount of valuable information for one day. We’re not sure before hand what this data will look like, coverage or attributes. Typically visualizations are made by exploring and playing with the data to see what emerges. In this case, we’re making estimates (and guiding via the tutorials) on what data we’d like. Therefore, the map itself has simple mechanisms for styling markers based on the user-supplied report. But the data is far to dispersed so far for something like a heatmap.

Fortunately, the team consists of a large number of public advocates that are spreading the word which should encourage more citizens to use the system and contribute both good and bad reports. Andy Carvin of NPR put together this NPR coverage, and we’ve also received coverage from Time, Huffington Post, New York Times, TechCrunch and even Craig Newmark. Check out the TVR press page for more coverage links.

And if you would like to help contribute to the project, check out the VoteReport Wiki. I imagine there will also be a number of post-election visualizations and analysis to come out of the reports.

Last Monday I participated in a Geospatial Search Summit hosted by the OGC as part of the quarterly Technical Committee (TC) meetings. The TC’s are primarily about various working groups discussing progress and status of standards or interoperability demos.

By comparison, this summit was meant as a brainstorming around geo and search interfaces and responses. Pulling from the announcement:

We would as much as possible like to bound the discussion to: 1) common ground for geospatial search for web resources and 2) integrating spatial search into search protocols. As part of the discussion we would also like to get advice from the other communities about which catalog/registry search protocol is the ‘mainstream’ one (or more?) that we (OGC) should align with and in turn, be sure that spatial search is supported in a thoughtful but not cumbersome way by the broader IT standards community.

You can see a partial list of attendees here.

There was a good overview of existing, albeit often quite complex, search interfaces. As is potential in meetings like this where attendees have their own history, investments, and beliefs in standards, the discussion can become difficult to easily resolve.

A couple of interesting agreements came out of the meeting. Foremost was the understanding for guidance of using simple, common formats as they already exist when appropriate. This means using OpenSearch as a base URI templating mechanism and follow GeoRSS-Simple specification for geographic data. Of course, a format can expand upon this and offer more complex formats that conform to more complex specs. But by at least providing a common baseline means that almost any service can easily interconnect with another service.

One difficult mechanism that is missing is a way for geographic search to specify the type of spatial operation. Typically most services assume a “within” or intersects”. For example, what restaurants are within a 5-mile radius of my position. However, it’s apparent that this can be confused based on assumptions and also does not provide for any other type of operations. Again, for example, find me all the hospitals that are not within the hurricane path.

A long-standing model for this is called the DE-9IM spatial operation set. It was presented by Eliseo Clementini, and also frequently attributed to Egenhofer. You can read more about it. Granted, a majority of geospatially-capable search interfaces may not require this, but it’s nice that there is a relatively straight-forward model that everyone can agree on.

I hope more attendees share their thoughts and outcomes. There are definitely many who point out the problems of designing standards in a smoke filled room, and I much rather bringing the discussion out into the open where more people can chime in and contribute.

At WhereCamp we finally sat down and hashed out a Geo-extension to OpenSearch.

If you’re not familiar with OpenSearch, the general concept is a simple format for specifying how to query a web resource, and additional metadata in the results to support syndicating these results. What does that mean to you, the developer/user?

If you didn’t realize it, the Firefox search bar is powered by OpenSearch. Watch for the area to glow on sites that include an OpenSearch definition. Once it’s added you can then quickly search the service directly from Firefox.

Take for example, a popular search engine – let’s call it yoogel. yoogel has a simple URL for a query: http://yoogel.biz?q=flumbits. This will search all of yoogel for ‘flumbits’. OpenSearch provides a way to define where that search term goes. Essentially it would look like: http://yoogel.biz?q={searchTerms}, where {searchTerms} would be replaced by any general string.

Aggregators and applications now have a way to simply define a search service and let a user just type in their terms, but then search N search engines. Of course, there are problem millions of hits for ‘flumbits’. So how would I know how to paginate and aggregate these results? OpenSearch lets me do http://yoogel.biz?q={searchTerms}&page={startPage?}

Ok, so now you know how to request a page. And the question mark, ?, lets me know this is an optional parameter. yoogel won’t yell at me if I don’t include it. I can know if I need to go to more pages (or how many pages to show my user) by using the nice OpenSearch response elements. These would actually go into the HTML page (or RSS, or other formats) response:

 <opensearch:totalResults>4230000</opensearch:totalResults>
<opensearch:startIndex>1</opensearch:startIndex>
<opensearch:itemsPerPage>10</opensearch:itemsPerPage>

Now I know there were in fact more than 4 million ‘flumbits’ responses and each page gives me 10 results.

OpenSearch has a lot more functionality and is especially useful when requesting syndication formats such as Atom or RSS.

Now onto the Geo

So that was a little introduction to OpenSearch – just so you could get the gist of the Geo-extensions. You’ve seen how you can search for ‘flumbits’, but what if I only want ‘flumbits’ in my neck of the woods.

OpenSearch Geo extensions let me add that type of search definition. Here is how to specify an optional bounding box search:

http://yoogel.biz/?q={searchTerms}&page={startPage?}&bbox={opensearchgeo:box?}

which would actually look like:
http://yoogel.biz/?q=flumbits&bbox=-111.032,42.943,-119.856,43.039

If you’re paying attention, you’ll notice the bounding box format is: west, south, east, north. That may not seem special (in fact, it’s different than what GeoRSS uses – shucks). But what’s really cool about that is it’s the order already used by a bunch of services you may have heard of such as Flickr, KML Network links (as used by Google Earth).

The OpenSearch Geo extension also includes definitions by: point/distance, polygon, location string (up to each service to do it’s own geocoding).

If you want to see them in action, check out the Mapufacture OpenSearch description.

When deciding the format of OpenSearch Geo, the guiding principle was to keep it inline with other emerging, open, geospatial standards such as GeoRSS and GeoJSON. That way, developers and users can learn the concepts and formats in parallel, and then easily switch between them as necessary (they’re all mostly just syntaxes and have appropriate uses in the appropriate places).

Your choice of flavors

To illustrate the point of interoperability and formats playing together, let me show you how an OpenSearch description would work for an example site that provided results in a multitude of formats.

<OpenSearchDescription>
<ShortName>Mapufacture</ShortName>
<Description>Search for geographic items in Mapufacture</Description>
<Tags>geo georss location kml aggregation geosearch</Tags>
<Contact>robot@mapufacture.com</Contact>
<Image width="16" height="16" type="image/x-icon">http://mapufacture.com/favicon.ico</Image>
<Url type="text/html" template="http://mapufacture.com/search?keyword={searchTerms}&location={opensearchgeo:locationString?}"/>
<Url type="application/rss+xml" template="http://mapufacture.com/search.rss?keyword={searchTerms}&location={opensearchgeo:locationString?}"/>
<Url type="application/vnd.googleearth.xml+kml" template="http://mapufacture.com/search.kml?keyword={searchTerms}&location={opensearchgeo:locationString?}"/>
<Url type="application/json" template="http://mapufacture.com/search.js?keyword={searchTerms}&location={opensearchgeo:locationString?}"/>
</OpenSearchDescription>

That is almost too easy – in one definition I’ve specified I can return results in 4 different formats. Internally, it would be easy to write out my results in these formats because again, they’re mostly just syntax. (note: of course my text/html would include Microformat geo, though it has a long way to come in handling anything more than point locations)

By the way – it’s tempting to think that this could be reduced to a single definition (I had that thought writing out the definition myself) that included a term for “format”. However, there isn’t a standard way to refer to formats, so that would almost require a dictionary linking MIME-type to format string, and specifying MIME-type in the URL is ugly and prone to errors. Perhaps something for the future.

But it’s just ‘neogeographers’ reinventing the wheel

Yeah, I know – those in the geo-world are asking, “Isn’t this like WFS Simple?” (or any number of other geo-search formats).

Yes, it is like them. But different too. For one, OpenSearch already has good market share and implementation (you probably don’t realize you’re using it often), so it makes sense to add Geospatial capability to it. Also, like GeoRSS-Simple is to GML, it’s more readily understandable by non-geospatial experts. This means more use and implementation. And hopefully OpenSearch Geo is kept parallel enough that as developers want more complex capabilities, they’ll ‘graduate’ to more complex formats like WFS.

But I don’t have the time

We have your time right here. It’s not part of Geo (yeah, yeah, I know about Spacetime and the fact that movement through time and space are indistinguishable), so it is it’s own extension, OpenSearch Time extension.

With OpenSearch Time you can specify time start, finish, and slices for searching data. This would be useful, for oh, Upcoming, Calendar applications, or Blogs.

For posterity, here is the original geowanking mailing list – call for feedback. This is still just a draft specification, so if you have thoughts, ideas, suggestions, now is the time to chime in before the web becomes rampant with OpenSearch geo.