MapQuest uses an “Outie” tab (highlighted in the screenshot below). The design choice was clearly to make it very explicit for users to show and hide the sidebar as it protrudes into the map interface. The pan and zoom controls are on the right-hand side, so when you toggle the sidebar, the controls stay in the same location. Another interesting aspect is how the map resizes. In MapQuest, the same geographic center and extents remain in the screen center – so as the sidebar closes the map shifts to the left and expands slightly.

Curious about how this varies, I checked in Google Maps. They chose to be much more subtle about their sidebar toggle. It is an “innie” that is subtly hidden within the header. Closing the sidebar turns the selection to an “outie”, but still remains out of the way in the header. A particularly interesting decision is that the map remains in the same location – so the zoom pan controls move but new areas of the map are exposed. So while the user doesn’t have a context shift (points on the map remain in the same area of the screen) the map now needs to be recentered so that the focus area can be kept in the center.

Lastly, looking at Bing maps it’s a bit of a hybrid between the two. The sidebar tab is in the header like Google, but hiding the sidebar re-centers the map like MapQuest. The controls in Bing are in the header, so they don’t need to shift when the sidebar is toggled. What’s perhaps a little confusing is there is also an “X” close button next to the sidebar tab that clears the search results. It’s not really clear why you would want to clear results – and instead there should be an option to go back to the “table of contents” or similar concept that shows simple links for directions and more.

Much like the emergence of Pan-Zoom bars have become the defacto standard in web mapping interfaces – the sidebar has also become nearly ubiquitous. So it’s interesting to see the slight variations as interaction designers experiment with what users will find easy to understand.

It’s almost half-way between Where2.0 and WhereCamp5280 is stacked to be an interesting discussion of the current state of affairs in what has been called “the year of location”. And given the cadre of people that will be coming to WhereCamp5280, such as Waze, MapQuest, WeoGeo, Safe, Google, USGS, ESRI, numerous other geo-geniuses, and of course, a cadre of FortiusOne engineers – we definitely should have some fascinating discussions. I hope if you’re nearby you can make it too!

I found the world of crisis supply chain operations fascinating in the complexities of moving something like tents to remote areas of China or even locally like Louisiana. There is a very complex landscape of Federal, State, and Local government, VOADs (Voluntary Organizations Acting in Disaster) such as Red Cross, FBOs (Faith Based Organizations) such as Salvation Army, and the Military. And that’s not even considering the complex organizational and operational processes within these organizations. Clearly the effect is a working, but highly inefficient and potential fragile operational capacity in responding to disasters. The flood of unwanted in-kind donations (as high as 90% of donated goods need to be discarded because they are unusable), competing interests, conflicting operations, and communications issues result in frustration and a concern that in a catastrophic disaster – particularly within the United States – that we would be ill-prepared to respond effectively.

However, these organizations are very interested in understanding how they can better coordinate and collaborate. There is a clear realization of the need to put in place better plans before a disaster occurs. The entire purpose of the workshop was to convene the different communities of government, military, NGO, private industry, and academia in order to share difficulties and brainstorm solutions.

Emerging Trends

In particular, my talk shared the emerging drivers, trends, and issues in information sharing and collaboration in humanitarian activities. Major events from Katrina, through Haiti earthquake and reconstruction have highlighted that citizen engagement through digital media is dramatically changing the on-the-ground needs sharing and response capabilities. Traditional crisis response organizations currently utilize a very top-down approach, be that at the “local” level of first responders in the country or region – but also through national efforts led by FEMA – that is being faced with these trends but currently not clear on how to incorporate the data. “Social Media” is currently primarily supported through external affairs and is considered a publicity mechanism. However, as was made clear in the recent American Red Cross Survey, 74% of the polled adults expect less than a 1-hour response to their need when published through a service such as Twitter or Facebook.

Open Sharing

The internet has provided a global, connected network that dramatically lowers the barrier to free exchange of data. Administrative policies focused on open-government, combined with general acceptance that shared data improves the quality and grows value is leading organizations to more readily share their data – particularly with open-standards.

Realtime Data

Inexpensive, connected, and prevalent mobile devices are dramatically increasing the number of ’sensor nodes’ that are publishing data continuously to the web. Social media, resource tracking, news, weather and climate sensors are all providing continous streams of data that have a huge value in providing situational awareness and communications.

Analytics

In order to understand the deluge of information, analysis tools are being put closer to users – particularly domain experts and locally situated groups that

Social Networks

People are connecting and collaborating through online networks, bridging social, family, professional and local communities. They’re able to communicate in real-time about issues they care about.

Crisis Crowds

Around any crisis, communities of interest – diaspora, family, and general good will – is causing people to want to actively participate in helping the survivors.

New sensor platforms

Mobile phones, Texting, broadband internet allow anyone, anywhere to be sharing data and providing information and feedback. In addition, inexpensive digital devices are allowing people to build ad-hoc balloon imaging and other sensing platforms.

‘Citizen’ Engagement

Combined, all of these capabilities are actually allowing the local, affected populations to have an immediate, positive impact on their response. Neighbors and communities are able to assist one another and coordinate with official response organizations.

Work we’re doing

Groups like CrisisCommons have a lot to offer as it combines members of these response organizations with technologists, private industry, and citizens in developing agile and supportive capabilities. In the workshop it became clear of the potential and growing need to utilize digital media as part of operational support and not just as public outreach. Integrating aggregation, analysis, and curation tools of the huge flows of data are vital to organizations so that they can understand their own operational picture as well as the broader ‘common operating picture’ across the entire disaster.

At FortiusOne, we’ve built GeoIQ to integrate dynamic data such as Twitter and Flickr with logistics information of shelters, hospitals and other infrastructure to provide these common operating pictures both within organizations as well as on the ground through field-deployed systems. GeoCommons has served as a tremendous repository of data and information analysis that augments these operations by providing to the general public the capability to contribute and share these analyses.

Disaster response is changing quickly – information technology playing a key role in quickly augmenting local and remote capabilities. The future is in combining these with actual logistics of materials through the international and national responders to be more effective and supportive.

These are all great advancements, and are blurring the lines between the online digital data and our interaction with the real world. However it’s vital that we realize the real potential application of these technologies and what our legacy is on the entire world. How can we engage with global citizens, understand their needs and desires, and collaborate on building channels of information and tools that serve our individual and collective goals.

Almost two years ago I moved from Michigan, with stints in California, to Washington, DC. I moved at an auspicious time in our nation as the highly contentious presidential election approached at the same time concerns on transparent monitoring of democratic elections and process loomed. Social media and streams such as twitter, smartphones, voice technology and visualization provided the components to demonstrate how we can enable citizens to share their experiences, their problems, and for us to openly see problems and victories as they occurred.

This same concept applies just a well around the world. Open platforms such as Ushahidi have helped bring citizen reporting in elections in India, Libya, Iraq, and Afghanistan – each to different outcomes – but still in a way that harbinges a more open and transparent government process.

Now through my experiences with CrisisCommons, working with multinational organizations such as the World Bank and the United Nations, and the federal and local governments, it’s clear to see how the leading edge of the Where2.0 community can have an amazing and unparalleled impact in providing understanding and change in global and local issues: Environment change, food security, humanitarian development, education, and disaster response.

In looking at the various open government initiatives, the questions arise in looking past the press release to the realized value of sharing data with businesses and citizens. I was struck my the foresight of the Arkansas AGIO team in the realization of how sharing data as broad and wide as possibly helps mitigate their vulnerability to disaster by enabling responders open access to vital information that would assist in response.

This concept is apparent in how OpenStreetMap was successful in Haiti. With the lack of official, government supplied data the best solution was to crowd-source the information from varied sources and rebuild the national data infrastructure, external to the government itself. While it has been unpredictably successful, the value continues to be the open access of the data by any and all organizations, and the eventual adoption by the government itself in rebuilding its capacity. The hope is that the government continues to openly collaborate with the global community in managing and maintaining this data so that the situation doesn’t need to reoccur.

In summary, the community is making a difference. The tools we develop in WhereCamp, IRC, open-source communities, and from companies are changing the capabilities of crisis response and development. My message is to urge the larger community to continue to think how their solutions can have a more broad impact.

If your technology can help a consumer find a great $4 latte, that’s good for your business. If it can also help a child find clean water near their village, that’s good for the world.

I remember being impressed by Platial and the goal of providing a way for anyone to easily annotate places that mattered to them.When I originally pitched the idea of a “Neogeography” book to O’Reilly it was with the inspiration of Di-Ann’s drive to citizen access to geospatial tools that I considered how people should be able to map their genealogy and share their trips.

As Mikel and I built Mapufacture, we partnered with Platial on several projects. Platial had attempted to make a local information aggregator that never really took off, and so we discussed how to utilize the geospatial data aggregation platform in Mapufacture to provide and aggregate content for Platial. I even helped build and test the Platial developer API using the first iterations of AtomPub and OpenSearch, the results of which can now be seen in Mapufacture’s and GeoCommons’ APIs.

In looking at specifically the GeoWeb landscape, Platial definitely provided a necessary capability of easily allowing people to annotate and share locations. It is the more explicit version of more recent location-sharing tools such as FourSquare, BrightKite, or Latitude that merely ask where you are, not what’s important to you. When Mapufacture was acquired by FortiusOne, the combination of the large head of geographic data in GeoCommons, combined with the very long-tail of aggregated sensor and streaming information provided for mixing disparate datasources and understanding of context and relevance. Users want to collaborate around all types of data, and share insights, find out relevant information, share this with friends, family, coworkers, and their government.

Clearly geographic data is not merely limited to traditional map sources or cartographic outputs. Location is being integrated across all platforms and recognized as a primary component of any data. What differs is the means by which users will interact, create, and use this information depending on their needs, context, and capabilities.

As has been widely reported by the news, GeoCommons is archiving the Platial user data and maps. Users can find their data by visiting the GeoCommons Platial Source page and searching for their username or maps and freely download them or build new maps and widgets. Along the way, perhaps users will also realize the capability of combining their personal information with relevant geographic data – because for example, you should know great surfing spots combined with wave heights and approved recreation areas.

Where to Surf? View full map

Di-Ann, Chris, Jason, Jake, and the rest of the tremendous Platial team have provided an amazing lead in the future of user contributed mapping – and while Platial itself is currently on hiatus, we’re excited that GeoCommons can provide a role in continuing open access to Platial users’ data and easy to use tools for them to visualize, analyze, and share their experiences and insights.

Mashable and James, amongst many others, have excellent predictions that will and won’t happen in 2010. Generally they are good insight into trends in the geo and mobile space, although I will take up counterpoint to some of his suppositions on File Formats, Interfaces, OpenStreetMap and Augmented Reality.

File Formats and Interfaces

Geo is definitely becoming mainstream – everyone in my family has a PND, uses Google Maps, and are asking about various location sharing applications. In the next year we’ll see geo become part of the assumed infrastructure, like the timestamp on a post or article, the location will be embedded.

I don’t think TAG (Twitter, Apple Google), as James puts it, will be the only location sharing services. They, along with even more used Facebook, will definitely be the general public interface to location query and sharing – but just because of this reason alone they will have to be very generic, leaving room for specialized location based services to still thrive in niches. FourSquare offers ‘gaming’ or Flickr visual media, and others for music, drinking, sight-seeing, and house finding. They will leverage TAG, or at least TG.

Apple is like the Nintendo of consumer technology – more interested in providing an integrated, compelling experience, and privacy, before full open-ness and engaging with the developer or geek. They’ll still have API’s, but not something like OpenSocial, GeoRSS, or FireEagle integration.

The iPhone, and to lesser extent Android, have been revolutionizing mobile devices. They are truly providing windows into the rest of the web of data combined with the real world. It’s natural for geopatial tools to move into these interfaces, but like any good user experience it won’t be the same capabilities you find on a desktop or browser application. The utilities will be specialized for the small screens, finger inputs, and out-and-about tasks.

For file formats, the Shapefile, unfortunately, isn’t near EOL. Too many tools only speak shapefile, and there is numerous legacy data that is still only available in Shapefile. Sites like GeoCommons offer alternate formats for all the data, but that still won’t remove this basic format. Only when there is a truly open, license free, API to File GeoDatabases (FGDB), and every off the shelf tool can talk that API or Spatialite, will Shapefiles begin disappearing out.

GeoRSS and/or KML, on the other hand, will be in every service that does anything Geo. Looking at any iPhone App review that includes KML (or doesn’t) brings up this point. Near enough everyone has Google Earth on their desktop, and Google is making big pushes in the utilization of Google Earth Plugin for in-browser virtual globes.

Visualization Technologies

To date, we’ve been stuck with either Flash or JavaScript DOM magic (and yes, Silverlight is out there too) in order to do data and geospatial visualization in the browser. As I mentioned, Google has been pushing Google Earth Browser, but also more generally they released O3D, a modern incarnation of X3D, providing for more general capabilities for creating 3D browser experiences. VRML lives!

More recently, there has been a resurgence in vector graphics that don’t rely on proprietary technologies or additional plugins. SVG and Canvas support is pretty widely supported except in the infamous Internet Explorer (which I hear is still being used even today). Examples such as ProtoVis, Cartagen and Tom Carden’s experiments definitely demonstrate that SVG is just on the cusp of being able to do a majority of compelling visualizations capabilities.

Another driver for alternative visualization platforms is the drive to mobile device integration. I don’t see Apple allowing Adobe onto the iPhone anytime soon, and even Android doesn’t have support. What types of visualization make sense is still a very open question – but whatever they are will be done with something like SVG.

Geo Data Skirmishes

James suggests that OpenStreetMap “won’t dominate”. While it won’t dominate, I disagree that it won’t continue to be extremely successful.

Google has recently moved to gathering their own data. They still have a long way to go, with many, many errors in roads, areas, addresses, and businesses and they’re using the crowd to help clean it up. Google is in fact proving the crowd-sourced model. It will be successful. Google is doing it with Google’s data, so there is no positive external benefit to that work – so to the industry it just looks like another data provider. However, with this proven model OpenStreetMap will succeed since any effort built into OSM has a positive benefit to anyone else.

However, there is a major difference in the trajectory OpenStreetMap is taking in the United States compared with Europe and other regions. In most other countries, the governments had very draconian licensing and as such OpenStreetMap was creating data from blank areas – starting from scratch, and building a community of volunteers along the way.

By contrast, in the US a vast majority of the data is free, and becoming more available everyday under the new administration. Therefore the US has a broad coverage of decent data without having first built the user community. So the difficulty here is both in building out community, as well as engaging companies that can do the same thing on their own while retaining proprietary rights to the data.

What’s fascinating, and what signals the ultimate long term success of OpenStreetMap, is that US state, local, and federal government agencies themselves are engaging with OpenStreetMap. They are investigating how to put their data directly into OSM, and possibly even re-incorporate updates and modifications back to their own infrastructures. Some are even considering using OSM toolset as their infrastructure. OpenStreetMap is going through some growing pains with respect to licensing, maintenance, and community – but all necessary steps in moving from a small cadre of hackers to a global, public project.

As we see an increase in open government, specifically driven by the US Administration’s directives, as well as other initiatives such as INSPIRE, this embrace and utilization of open platforms, and repositories, for sharing, federation, and syncronization of data will increase.

And as for augmented reality, it won’t be as big as you think… yet.

I was looking at a certain area in North East part of India ( State called “Arunachal Pradesh”) which is integral part of India.

Both URL’s have different take. [Google Maps] shows it as Disputed Territory ( with Dash lines) and [Google Ditu] shows it altogether as part of China!

So, that got me unruffled and to question validity of both these sources. How does Ditu differ from Google maps? Whats association between the two and does Ditu has autonomy to change the boundary of the maps as per its wish.

Arunachal Pradesh is a border region between China and India – with 70% of the land being claimed by the Chinese as South Tibet. The border in question was decided in 1914 and called the McMahon Line, but never agreed upon by the Chinese. The Google Ditu vs. Google Map views.

Territorial disputes are definitely not a new thing – however what is perhaps alarming is that there are two different representations of reality from the same vendor and data providers. So this is entirely a representational decision that is most likely driven by business and government pressures.

What’s particularly interesting here is that primarily these definitions of boundaries derive from the data providers. You can look in the bottom right corner for who the data providers are. For both versions the providers are the same: TerraMetrics, Mapabc, and Europa Technologies.

So it seems that the cartographic designers at Google Ditu have decided to represent it a certain way. Unfortunately, the map has no additional metadata. As broad consumption of maps increases, there is a commensurate interest in the why and what behind them. Who said these are the boundaries, when were they set, and why are they shown in this language?

And I don’t mean the long reams of unreadable metadata that are the current standards in the geospatial community, I mean human understandable descriptions of the various aspects of the data, while allowing additional discovery to deeper data.

One place that you can look at the data behind the source of the map is in OpenStreetMap. Arunachal Pradesh is shown similar to Google Maps version, and a user could optionally download the data to see the attributes, edit history and sources. Alternatively I can look in GeoCommons for the GADM Admin boundaries of India and see pertinent data on who provided the data, sources, and so on.

Boundary disputes in a bi-directional medium

The representation of boundaries is obviously a very contentious issue in mapping. Maps are perceived, and often do inform, territory. There is a long history of map representation being used to influence, coerce, and force land rights.

Unfortunately, even in a “Web2.0″ world of bi-directional sharing and collaboration, with maps we’re still often forced to accept a particular viewpoint. They have on-the-ground meaning and political impact. A well known example of this were the first “edit wars” in OpenStreetMap with the names of places in Cyprus. The resolution was to by default abide by the on the ground signage, but also store both versions and allow users to provide their own personalized perspective.

Understanding, awareness, and discussion about these issues is the reason for projects like OpenStreetMap or GeoCommons where you can download the information, build and share your own maps that represent your perspective.There

There isn’t an easy answer here – with companies such as Google there are obviously market, and government, forces that direct how to represent contentious issues. The best solution is to offer background, open data, and alternative perspectives. Without a voice, citizens are relegated to discussions by officials they may, or may not, have elected – and no meaningful way to illustrate their interpretation.

This Wednesday, I will be giving the GISDay plenary talk at the University of Virginia in Charlottesville. My talk, “Neogeography: from Tower to Town Hall” will discuss how the movement to broad, public engagement and collaboration, particularly around geographic contexts through web maps, mobile devices, and open data can build stronger communities, improved research, representative government and better livelihoods of people. (link to the UVA Calendar)

Around Washington, DC you can join the OpenStreetMapping party in Bethesda, MD on Saturday, or you can check out the large list other activities at GISVirginia. Whatever you do, spread the word and encourage people to go out and map something.

So of course, like any village, it needed to be mapped. I went down Saturday afternoon and captured the locations and names of all the buildings and paths that will be up for the week. These are then loaded into OpenStreetMap with start_date and end_date tags that notify the renderer when the features should be visible. It’s a similar model to how Burning Man is mapped year after year as it walks along the Black Rock Desert.

It’s ephemeral mapping – objects that exist in real place, but just for small slices of time. Important as any other building, yet typically relegated to flyers or verbal descriptions.

The fascinating part of projects like this is that OpenStreetMap allowed me to create a map that was useful and immediate. Within minutes of uploading the data, it was available as rendered tiles, vector data, and downloadable to GPS units and iPhones. People on the mall could immediately view the local map with this new information.

It’s a nice demonstration of how community projects like OpenStreetMap will continue to innovate faster, and more openly, then other ‘crowd-sourcing’ options.

There are many possible solutions, and a few that are within easy grasp that we can build our tools to encompass, and develop practices that encourage utilization of these solutions while still moving forward onto better ones as the GeoWeb matures. So we’ll take a few articles to look at specific solutions.

Discoverability

Perhaps the most prevalent issue, and the one that is most easily addressable, is the findability and discovery of geodata on the web. Mano Marks reflected this same sentiment in his blog post on standards.

In thinking about discoverability, there are several primary use cases to consider: Machine crawling, Human discovery, and Tool discovery. Providing data via just a single mechanism means that it doesn’t get utilized and consumed to it’s potential and so somewhere along the chain of utilization it will be a burden to actually incorporate into a workflow.

Think of the machines

Machine crawling is the ability for any spider to walk links, find data, metadata, and formats automatically. It’s what Google, or GeoNetwork does to find and register data sources.

There was recently a discussion on auto-discovery in the GeoWeb suggesting the use of robots.txt, sitemaps, or embedded META tags in HTML pages.

Consider how a spider would get to a site: it follows a link to a geospatial portal from some blog, resource, or directly entered as a good place to get data. It does a GET request on the root homepage, “/” which most likely returns the index.html equivalent. The program then parses through that for links or additional information.

If the spider knows about them, then it may ask for a sitemap.xml or robots.txt. But nothing in the original page request noted that this potentially very complete listing of data was there. This problem is the equivalent of an application having to know that it needs to ask for a GetCapabilities or other method to even discover what is available. Too much implicit knowledge of the specification is required for a program to easily discover new data and services.

What the program does see are these links that can contain information such as a link to a list of available resources. The simplest is a link to the Atom or RSS feed that can simply be a paginated list of all the resources available to the application. Within Atom, there is then the ability to link to various representations of that data in different formats. So applications are able to take the most appropriate format based on what they can consume.

Several years ago I first proposed how KML and GeoRSS could easily support one another via cross-links and with HTML documents. Atom has very nice rel and type attributes that allow for linking to all sorts of different representations. You can even link to OGC services like WMS and WFS using atom links.

Of particular interest here are looking at the currently approved list of Atom link relation types that provide basic semantics for telling you how what this link means. Is it another page? just related? It’s a limited set, but one that covers an approachable majority case for developers to begin using.

For example, mechanisms like OpenSearch, specified in a rel="search", simply notify the application that here is a service that it can query to get at additional resources. And with OpenSearch-Geo, a geoweb crawler can query information within a specific location or bounding area.

Humans need data too

Crawlers are great, they provide a say to pull together information into various other sites and tools to provide customized interfaces to users. However, within any site or tool, how should we expose geodata in a way that humans can easily use for whatever purposes the may have.

Again, links have become a very well understood concept on the Web. That underlined blue line states “beyond me lies an unspecified amount of information about this topic“. However, these links typically imply that they will open another human readable HTML page in the browser. A problem caused by links to media such as geospatial data is that the content behind a link may not be just text, it could be an image, audio, movie, KML, database, or a service. Clicking on that link relies on the browser interpreting the MIME-type (remember the point about how vital mime-types are?) and opening the application the user has specified, or left as default.

So consider what this means for generic media. Clicking open a link to an image probably just opens the image in your browser, or opening a movie loads an embedded video player. Geodata browsers, however, probably doesn’t have the same install base as say, Quicktime. Except perhaps GoogleEarth. The Web has become much more comfortable with clicking on a KML link and seeing Google Earth open up and show the data on a globe.

But something very vital often exists with a link to KML data – a recognizable icon that notifies the user (as they learn) that it is a file that will open in Google Earth, or another KML viewer. This is the same as the very widely used RSS icon.

I discussed this idea before about the geotag icon showing various other formats – and now sites like Data.gov actually show the various data format options.

So what we need for GeoWeb standards are some visual representation to people that they are can clink on this link and open a spatial relational database, or an OGC service, and perhaps have some confidence that there is an application that will provide them a useful way to access the data. (and I’m still waiting for Sean Gillies’ ISO and Dublin Core icons)

Of course, we should also employ emergent interfaces that show users the type of data links that are appropriate for them based on their profile or registered MIME-type handlers.

Man-Machine hybrids

So we have discovery links for machine crawlers to register and harvest geodata, and links for humans to click on to follow to data and within data. However, this can easily become overwhelming to need to click through to every link. Imagine if browsing Flickr through lynx.

Browsers already do a lot to assist users in finding relevant extra pieces of data in a page. RSS autodisovery links show up in URL bars notifying our feed readers that we can subscribe to this page. OpenSearch allows someone to embed this search into their browsers (most of them at least) to easily search the repository later.

The decreasing cost of links

These various approaches for different needs and use cases are all very well aligned. They don’t rely on additional external files that we need to make sure stay up to date or that tools are built to just know that the file can be found at a pre-defined location. Links cost next to nothing, mostly measured in bandwidth sizes, but provide a wealth of accessibility and discovery of geospatial data. Especially data in formats that make sense depending on the tools and use cases for different problems.

Of course, links alone don’t address all the needs of the evolving GeoWeb, they merely provide for the integration of geospatial data with the rest of the web. An important, necessary, but not entirely sufficient first step. We need to consider the actual uses and interfaces of these standards, archival, synchronization, conflation and more.

1. Introduction
2. Where We Are
3. Problems
4. Where We Need to Go
5. Solutions: Discoverability

Too often standards and tools are being by domain experts and technologists that lead to overly complex, and irrelevant formats that become a burden and introduce as many problems as they are trying to solve.

What they’re often not considering are the end user experiences. Who are the users, what are they trying to achieve, and how can these formats make for better, and easier utilization of these tools.

Granted, there are expert users. People who really want to make intricately related, projected, spatially and spectrally bounded queries into data and utilize them in advanced analytics engines. But these are not the majority and they’re not what is driving the long-term demand on the GeoWeb (you can use ‘long-tail’ here if you would like). Who are the users that want to engage with this information on a daily basis in their personal lives, businesses, family, safety, governance, and goals.

Grassroots is an option

I’m a very big fan of grassroots organization and emergent structures. The needs tend to grow from real demand, and solutions are built through actual demonstrated benefit and impact. They are agile, evolutionary, and garner broad support amongst users and developers. These are all aspects that are beneficial to achieving standards that meet the needs of end users and provide good experiences.

However, it is not the only solution. Grassroots tends to look at the immediate needs and may not incorporate more distant issues and expected needs. They seek for broad appeal, and “good enough” rather than totally encompassing all potential aspects of all interested domains. Top-down, industry derived, committee driven standards provide more directed needs and objectives that can serve different types of users.

So the solution is a hybrid – where grassroots solutions are encouraged as demonstrators and emergent needs – that are then accepted and supported by more formal organizations.

Conversation is required

But we also need to open up the conversation beyond just technologists and experts. We need to be engaging and understanding users – and not merely from the “how do I sell them more of my coffee”, but “what can I do to make their lives better”? And actually asking and engaging with them in dialogues.

This technique of user stories, and engagement is not new or unused. However it appears to be missing from the GeoWeb standards developments. We’ve been designing standards for ourselves first, and then foisting these upon others. Instead, we need to understand their needs and issues, and then apply our expert knowledge in how to approach solutions properly.

Other articles in the GeoWeb Standards series:

1. Introduction
2. Where We Are
3. Problems
4. Where We Need to Go
5. Solutions: Discoverability

Part 3 in looking at the current state of GeoWeb Standards. See the introduction here.

It’s time to take a hard look across the board at where we’re coming up short and issues that need to be addressed. One way to summarize:

GeoRSS, KML, and GeoJSON are the itching powder, squirting ink pen, and dribble cup of geodata formats.
– Sean Gillies

Sean is definitely known for his candor, and his viewpoint definitely has merit. Overall the various formats and standards fulfill various needs, but still don’t provide for all use cases, align well with best practices, or make sense to users and developers.

The simplest overall problem with many of these formats, and how they fit into the Web, is that they lack proper web-type descriptions. One primary mechanism that Web clients know how to present data is through the use of MIME-Types. MIME types provide a way for the server to notify clients that the data is in a format such as XML, Text, a PNG Image, and so on. These must be formally registered, but also ad-hoc, or vendor specific, types are commons.

In addition, MIME types allow crawlers and registries to easily record the type of the file in the metadata.

Looking over our list of various GeoWeb standards, it’s very easy to identify which formats abide by this and which don’t.

Atom, JSON, HTML, and SQLite all provide format specific MIME-Types, allowing clients to easily employ the proper applications. However, none provide a special mechanism for notifying that the data includes geospatial markup. Not necessarily a problem, geo shouldn’t be that special.

KML is perhaps the only format that has a geospatial specific MIME-type. However, despite it now being an OGC standard, the MIME Type is still the vendor specific: application/vnd.google-earth.kml+xml. However, KML was particulary ingenious in also providing for the compressed, or zipped, format as a unique MIME-type: vnd.google-earth.kmz.

GML is just XML, so that is entirely not useful in notifying a client that it should try and pass this onto a geo-enabled application. And Shapefiles are agglomeration of multiple files, and even zipped up are only marked as compressed files.

More broadly in services, the OGC has a mime type for service descriptions and responses: application/vnd.ogc.wms_xml, though errors have their own MIME-types: application/vnd.ogc.se_xml.

OpenSearch has a special MIME Type, and obviously Tiles and Image files have MIME-types.

Doesn’t matter if you can’t download it

Another major issues facing many of the GeoWeb formats is their file size. Generally, the web bounces back and forth between disregarding sizes due to assumed, ubiquitous high-speed and reliable connectivity, and trying to speed up pages. But even more important is the fact that many potential users don’t have access to high-speed internet and so their is a huge difference between 10k and 100k or 1MB of data.

To compare the sizes, I took a relatively large dataset from GeoCommons, Statistics Canada, Land and freshwater area, Canada, 2005 and exported it in a variety of formats, both uncompressed, and compressed via standard zip algorithms.

CSV just includes latitude and longitude columns of the centroid – so obviously not fully representative. An option would be to include the EWKB in a column for the full geometry – but that is far from any kind of ’standard’ that other tools would know how to intepret.

Perhaps most surprising from these results are that JSON is so large. Unfortunately, the syntax for complex geometries requires a lot of syntax that adds up in representing polygonal data.

Linkability, Durability, and Discoverability

Moving past purely file format and data type specifications brings up the issue of discoverability and linkability in GeoWeb standards. The Web is more than a list of documents that mention resources, but that they can actually link to durable endpoints that can be resolved, queried, accessed, and parsed.

Non-web native formats have no concept of linking. CSV, Shapefiles, and SQLite contain data, but no links. By contrast, Atom, GML, and KML are chock-full of links, although not always used to great effect. JSON can contain links, but without a schema, who knows what the link means.

Obviously the best model to follow here is HTML, which provides automatic links to feeds, OpenSearch description documents, pages, media, styles, and scripts.

However, what happens when a resource disappears and is no longer resolvable? How do you know where else to get another version of the same data, and is it the same data? This is becoming a big problem in the larger web, made more problematic by the use of URL shorteners, but also especially disconcerting when it affects the provenance and accuracy of geospatial data.

But without Complexity

While linkability, durability, and discoverability are vital to GeoWeb standards, the cost of complexity inhibits adoption and probability of support.

This is a long argument in many circles – often made more difficult by practitioners that have been working in a field for years or decades and consider the most opaque formats or concepts commonplace. Look to the OWL/RDF/SemanticWeb space for an example of how there is a mismatch between proponents and the general public.

A standard needs to have clear value to developers and users for it to even begin to be considered. No one is going to dive into a dense specification of a format without even knowing why they would want to use it or how it fits into workflows and architecture.

And complexity can also surface in small ways – inconsistant capitalization of element names (you know who you are KML), or by supporting a plethora of similar, but different flavors making it unclear which to use (GeoRSS).

Tools

In this last section of the overall problems we’re facing with GeoWeb standards, the most prevalent, and easy to address, is the lack of tools that interact and convert between these formats. Really, formats don’t matter to users – they have data from one source such as their camera, PND, blog posts, Government agency, etc. and they want to do something with it like understand what’s going on around them, find their favorite restaurant, save the rainforest, provide services, get their car fixed, or just share stories with their family.

Easy to use, engaging, and data agnostic tools are vital for adoption of any formats. Again you only have to look as far as KML’s meteoric rise from application specific format to perhaps the most ubiquitous, and growing, GeoWeb standard due to the compelling reason of “I want to see my house and things going on around the world”.

Why do none of the major RSS news readers really support GeoRSS? Every site should offer KML and Atom output of their data. Mobile devices should allow me to open in whatever mapping interface or app any of my data from any of my services.

Missing Middle Ground

Amongst the plethora of formats, we’re really missing some middle ground. Each of these formats are quite independent and unique of one another, with little cross pollination and linking occuring.

• Why can’t my KML file link to Atom updates and also to other formats?
• Can OpenSearch describe my tile pyramid?
• How do I describe my path through life, media, events, places I’ve lived, worked, and people I’ve known?

We too easily get caught either in this “this format must solve all possible problems”, or “it’s good enough so why change it”. In between we need to converge to understand use cases, and how these formats and specifications can cross various barriers – connecting the experts with the amateurs, the citizens and the authorities, one with another.

GeoWeb Standards Series

1. Introduction
2. Where We Are
3. Problems

I started in the introduction by talking about the general Web and some considerations of how geospatial data standards face unique challenges in resolving to broader data interoperability.

In evaluating the current status of standards, it’s useful to give an overview of the current standards, and brief thoughts on where they are working and need specific addressing.

I will also note that this discussion is focused on web-oriented geospatial data standards. There are many other geospatial data formats that exist – but are either too esoteric, proprietary, or not Web-aligned, to be useful in considering their application to utilization in the broad Web.

Shapefiles

Of course, with the first example I will slightly bend the statement above. Shapefiles are the bastard geodata denizens of the web. They are annoying in multiple ways. Foremost being that they are a proprietary data standard that is found entirely too common across geodata portals – especially government portals. However, there is too much information shared, and open tools that can use them, to ignore as serving a place in the GeoWeb.

Shapefiles are difficult to work with in the web. They are like portable databases, but actually consist of several files: datastore (dbf), geometries (shp). data to geometry join (shx), and optionally a projection definition (prj). Deriving from the usage as a binary data definition for desktop software storage developed by ESRI, they have historic shortcomings such as 12-character limit on attribute names, and restriction to a single geometry type (i.e. can’t mix lines, points, and polygons).

In addition with respect to web standards, they obviously have to deal with the multiple files, lack of a Mime-Type, and no web characteristics such as linking.

Microformat-Geo & Adr

Microformats are basic attempt at embedding data within generic HTML markup. The geospatial formats include simple 2-D coordinates, or an address; geo and adr respectively.

Microformats are nice because they align well within a prevalent data format and allow non-geographic expert users to easily embed information, either directly or via simple tools. Google and Yahoo both openly provide support for Microformats through improved search reliability and even some basic data manipulation tools via APIs. Other tools such as libraries and extensions also provide compelling use of Microformats with geospatial documents.

However, basic limits are that geo only allows for latitude and longitude, without any support for a height. Adr at least can provide more complete information, but neight geo nor adr allow for linking to external geometries – a common shortcoming of most the formats discussed.

Another problem with Microformats are that they don’t allow linking to context within a document. So while you can include location information in a paragraph, it is not possible to express how this location relates to the rest of an article or narrative.

So, for example, while it’s possible to markup the location of the White House, one can’t easily denote if this was the location of a press conference, or just that the U.S. President was there, or whatever else may have occured.

GeoRSS

GeoRSS arose out of the simple desire to include location in the increasingly prevalent RSS and Atom feeds from blogs and news sources. It’s another community driven and owned standard, like Microformats, that met existing needs from a bottom-up approach.

GeoRSS over the past few years has become increasingly common amongst web sites using maps and geospatial data. Google Maps, Yahoo Maps, Microsoft Bing all support exporting and importing data via GeoRSS, and major news outlets such as Reuters, and Al-Jazeera output GeoRSS.

Despite it’s widespread adoption, GeoRSS has some complexities that arose out of it’s development. There are 9 potential “flavors” of GeoRSS, although this is largely due to the 3 different formats of feeds: RDF, RSS, and Atom. There are still 3 formats of GeoRSS itself that can be utilized in any of the 3 feed formats: W3C, Simple, and GML. This causes confusion for developers, especially since W3C format is deprecated but still widely used. Perhaps this is one reason that despite GeoRSS being a simple extension to existing feed formats, there still is not GeoRSS support in any of the major news feed readers, except perhaps limited support in FriendFeed.

In addition, GeoRSS hasn’t really advanced in quite awhile despite multiple requests and discussions of extensions for multiple locations, time spans, external geometries, and feature identification.

KML

KML, or Keyhole Markup Language, became a defacto standard out of the popularization of Google Earth, formerly Keyhole Earth, and the wide creation and sharing of geographic data to use inside of this compelling 3D geobrowser.

KML offers a rich markup supporting feature locations, attributes, visual styling, 3D models, addresses, and even Atom links. In addition, it is now an OGC standard, and recently Google announced there were more than 500,000 KML files and 2 billion KML placemarks, or features (making an average of 4 placemarks per KML file).

However, KML is very clearly a direct object representation of the Google Earth application. Attribute names follow a rough camel-case convention based on parent or child classes, but sometimes this simple rule is broken in unclear ways, making it difficult for tool developers to create compliant tools. In addition, the styling capability is rudimentary with little true cascading support and no attribute or class styling capabilities.

Google continues to push forward the KML specification with vendor specific, gx:, extensions. The rest of the geospatial community has yet to attempt to influence the spec in any way despite these apparent problems.

GeoJSON

A much more recent community standard, GeoJSON merely adds geographic markup to the JSON format. This is primarily targeted to client to server communication and takes advantage of the compact size, and quick evaluation of JSON data.

GeoJSON nominally followed the GeoRSS definitions, making it easy to understand and leverage existing tools and knowledge. However, JSON itself does not provide for any actual format specification or schema definition, leaving clients to determine the layout of the JSON to agreed upon documentation rather than actual standards. This is becoming especially problematic as more services expose GeoJSON via APIs to third-party developers. It is really little more than arbitrary, unique XML without the extensive syntax.

GML

In response to the history of arbitrary, unique XML, Geography Markup Language, GML, was developed. GML follows a very strict and feature-rich mechanism for creating geographic schemas and domain specific semantics. It is used for very precise data interchange, typically over OGC services like WFS. GML is targeted to bridge the span from 1-D to 4-D geometries, multiple domains, and entirely customizable profiles, or versions, depending on a user or developer’s needs.

With GML’s power comes much complexity. Developers are typically required to devise and include their own unique schema definitions when using GML. The scope of writing a generic GML client is akin to writing a Ruby script interpreter and is daunting to general web developers that only want to include simple geographic capabilities to their general services. This complexity hampers it’s widespread adoption.

Other formats

There are a variety of other formats that are beginning to emerge on the broader web through a variety of fronts.

Spatialite is the set of spatial extensions to the open, portable SQLite format. SQLite is a file database that provides for full relational capabilities in a single file. Spatialite therefore adds geographic columns and rudimentary geospatial query support.

SQLite is already used by a variety of tools such as Google Gears for offline support and the Google Maps on the iPhone for storing tiles. We chose Spatialite for our Geocoder due to it’s compact nature and deployability. Spatialite makes for a very compelling option when you need to have access to an entire geographic database and perform operations on the data.

GeoPDF is working to become an open format. There is a pending OGC adoption of the georegistration embedding, and Adobe is pushing the ISO 32000 spec that includes how to embed vector and geographic drawing. There is still however a very fragmented ecosystem of tools and interoperability that threatens the format as a mechanism for disseminating geographic data.

CAP, Common Alerting Protocol, is a realtime focused format for sharing out alerts such as emergency news, earthquakes, or municipal signals. It is still an XML format with no real mechanism for ensuring delivery or timliness, and it is not clear the advantages over more broadly used and extended formats such as Atom.

There are still even more formats that are used in the GeoWeb such as CSV, GPX, RDF, and even OpenStreetMap (OSM). However, it is not really worth discussing these here as they are either too generic (CSV), or still too nascent (OSM) to really consider as an existing GeoWeb standard. They will, however, be discussed later in looking to the future of geodata formats.

Also, Semantic data such as Linked Data, RDF, or OWL are continuing to bubble beneath the surface. I will go in depth later on the potentials of semantic geospatial data standards.

Services

Beyond just data formats, there are a number of GeoWeb service, or interface, standards. Open Geospatial Consortium (OGC) dominates this landscape and provides various querying specifications such as Web Feature Service (WFS) and Web Map Service (WMS) in addition to other cataloguing and location-based service interfaces.

WFS and WMS both provide very full-featured capabilities, but also follow older paradigms of interfaces. Fortunately neither of them are SOAP-based, but instead rely on simple Query parameters for specifying bounding boxes, layers, formats, projections. Perhaps the biggest difficulty is that the service description is at the same endpoint as the service itself and often servers use the wrong MIME-types for the documents and errors.

More recently, general web standards organizations such as the World Wide Web Consortium (W3C) have been adopting geospatial additions for browser DOM geolocation, HTTP location information an privacy. ISO and OASIS are looking at OpenSearch-Geo for possible integration into their harvesting and cataloging standards.

OpenSearch-Geo follows the same concepts as GeoJSON and GeoRSS, providing a simple extension to a broadly adopted interface and merely adding geospatial components to it. In addition, by being only a templating specification, it can easily apply to describing general API’s such as Flickr, KML network links, or even WFS when applying the appropriate template markup.

However, while OpenSearch-Geo has garnered a lot of interest, it’s actually prevalent use isn’t clear. There are limited services that offer an explicit, compliant description of their geospatial search interfaces.

The View is Mixed

So the current state of GeoWeb data standards is quite mixed. There is no denying that they have becoming mainstream. We’re seeing some emergence, and divergence of more popular formats in Mapufacture and GeoCommons, both on upload as well as downloads or links. Google has released their figures for KML they’ve crawled on the web. By contrast, GeoCommons and Mapufacture rely on users to vet and register data sources, providing a different viewpoint into the utility of geospatial data formats on the web.

The above charts show the composition of data uploads, links, and entire composition of geodata uploaded to GeoCommons and Mapufacture. As an interesting comparison, downloads are definitely trending towards lighter weight standards: KML downloads account for 67.8% of all downloads, with CSV’s at 25.7% and Shapefiles for merely 6.3%. It is worth noting that this is merely a narrow viewpoint in the larger web – not accounting for OGC standards, raster data, and services. However it is still an enlightening consideration in looking at how people are actively engaging with the GeoWeb.

The different formats have all been used extensively, but when which format is most appropriate isn’t clear. This leads many applications to include multiple formats, an easy and appropriate solution but also one that can confuse users and provide for duplication. We’ll dive into more general problems in the next article.

1. Introduction
2. Where We Are
3. Problems
4. Where We Need to Go
5. Solutions: Discoverability

However, the state of the geospatial data and interoperability standards is quite mixed. We have both rigorous, standards body driven specifications that address formal needs, as well as community driven standards that have been emergent and lightweight by comparison. There has only been cursory cross integration between these methods, and looking forward there are still many unmet needs in current applications and new domains such as augmented reality, realtime sensors, narratives, external gazetteers, and general digital media questions of durability and archivability.

In order to consider these various aspects, of where we are and where we need to be going, I will be doing a series of articles looking at the various aspects of the Web and Geospatial data. From a general overview and then diving into considerations of utilization, complexity, size, and finally suggestions for moving into the future of geographic standards.

These notes are from my talk at GeoWeb 2009 on “GeoWeb Standards, How Far We’ve Come, How Far we need to go.” They reflect a very active time in developing standards that accommodate some of the unique aspects of geospatial data, as well as the convergence of the geospatial community with the broader data and web communities.

Part of the Web

In looking at GeoWeb standards, it’s worthwhile to consider what simple features has made the web so effective and powerful. The first sentence of the Wikipedia article on The Web states:

The World Wide Web is a system of interlinked hypertext documents accessed via the Internet.

This is a surprisingly concise and poignant definition. The key components can be summarized as: links, documents, accessibility, and open. I am slightly altering the words to give credence to the important meanings underlying such ideas as “the Internet”. The Internet is the broadly open, and universally accessible network of computer systems that allows anyone to access and publish information – a key component of the thriving, single, Web.

The GeoWeb, a term that is being utilized in order to direct ideas and conversation specifically towards geospatial concerns, is still an integral component of the general Web in the same way as the Semantic Web, Realtime Web, and Participatory Web, are also different aspects of the same entity. It is both special, and not unique.

However, it is still valuable to consider the role and mechanisms of integrating into the Web. Links and accessibility are themselves still suited and necessary for geographic information. Though this is often surprisingly missing or argued against. A common concern is that place is inherently more sensitive and therefore should be kept private and secure.

Additional unique considerations include findability, discovery, collaboration, and unification. Geographic data is both inherently sortable due to the mathematical nature of it’s construction in dimensional space, but also continuous in ways that textual or categorical information is not.

I’ve spoken before at the value of geography as a common context through which we can combine and compare disparate domains of data, but this also leads to difficulty in using web constructs to link to data such as “weather near Bermuda last week”, or “place of performance versus vendor of contracts”. Is this information shared through geographic interfaces of place, bounding boxes, or pagination of tiles?

And geography also has the benefit and difficulty of having unique place and identification. This means linking together data is more possible, such as when describing a building through business information, government zoning, weather, and user location. But it becomes more difficult when determining conflation of floors or offices within buildings, within larger regions, that change over time.

In the next article, I will do a quick survey of where we are with GeoWeb standards.

1. Introduction
2. Where We Are
3. Problems
4. Where We Need to Go
5. Solutions: Discoverability

Later this week I’m speaking at GeoWeb about the current progress of GeoWeb standards, how far we have to go, and how to get there. We have KML and GeoRSS leading the way in searchable, linkable formats, but still a plethora of Shapefiles strewn about. There are questions of findability, semantic ontologies, durability, and expressiveness. What are the adoption rate of these formats and their utility in the future real-time, mobile, linked, open web?

What else do you think is the good and bad of GeoWeb standards?

The geocoder was built as part of our FGDC CAP Grant to help GeoEnable Government Tabular Data and utilizes the free and open TIGER/Line street data as well as various address parsing and metaphone components for US level address parsing. Also, not everyone can call to a web-service, abide by the terms of service, or be limited by the speed and amount of geocoding queries.

The reason we’re open-sourcing it because primarily an open-source geocoder has been a sorely missing piece of the open-source geospatial stack. You have storage, analysis, rendering, geolocation, and even routing – but not geocoding, at least not in an active project way. GeoCoder::US has been around for a long-time and well built, in Perl, and despite it’s long-standing solid service at geocoder.us, it didn’t fit our needs.

So instead we worked closely with Schuyler Erle, one of the original developers of GeoCoder::US, to rebuild it in a modular way (in fact he finished it once and promptly rebuilt it again), and also in a popular, modern language, Ruby(that we happen to use as well).

We’re also hoping to engage the community in building out the Geocoder. Right now it has components for the United States – but we hope that others will add components for their countries. OpenStreetMap is coming along very well with adding both ranged, and even parcel level, address data. So a good first task would be to build out an OpenStreetMap data importer.

Feel free to check out the code on GitHub – fork it, let us know what you’re working on, any issues you run into, and how we can make the best, and open-source, geocoder out there. Look forward to more detailed posts on how we built it and how we’re using it in GeoCommons and GeoIQ.

Later this week I’ll head back over to Europe for State of the Map (SOTM), the annual OpenStreetMap conference. Three days of talks, demonstrations, brainstorming, demos, and camaraderie. In fact, GeoCommons is a Sponsor again this year (all three years and counting) with a very exciting and interesting surprise on how we’re supporting the conference.

Of all the upcoming conferences (Open Gov Innovations, GeoWeb, FooCamp) I have to admit I think SOTM is the most exciting. All the conferences are about change – incredible advancements that have come about in the past few years – but State of the Map has gone from a nascent concept, even an activist movement against the complex, and onerous licensing requirments of geospatial data in the UK, to a global phenomenon that is being leveraged by individuals, companies, governments, and global NGO’s.

For verification, take a glance at the OSM statistics. Two years ago there were just 8,000 registered users, last year there were 40,000, and today there are more than 124,000 users! The “Year of Edits” video never fails to leave an audience speechless and amazed. The US WhiteHouse is using OpenStreetMap and projects like WikiProject Palestine Gaza show that OSM is the tool people now turn to in a time of crisis and for data.

On Sunday I’m giving a talk about how we’re using OpenStreetMap in GeoCommons and our private GeoIQ servers: “Enterprise and Government Visualisation Analytics using OpenStreetMap”. It’s just one example of many about the power open and crowd-sourced data has in supporting and growing businesses and serving customer and citizen needs. Other companies such as CloudMade, DevelopmentSeed, and itoWorld are also building out the ecosystem that is necessary for open, community projects to have a longevity.

There is an entire suite of tools that has been given form and purpose because of the huge amount of open data. Mapnik and other map rendering engines have data attributes to style; JOSM, Potlatch, and other vector editing tools are beginning to provide more compelling, and non-expert interfaces for modifying topological, geographic data; GPS export, data licensing, navigation and routing are more problems that have been explored and solved through the OpenStreetMap community.

So I’m excited about State of the Map because it means a result of thousands of individuals hard work and aspirations culminating in a meeting to celebrate what has been accomplished and also set goals to much higher, and diverse peaks. It’s proof that a crazy idea of people running around with GPS receivers can make a real impact.

thinking of geofeeds where feed paging does not take you back in time, but increases the search radius. but how to specify paging semantics?

My first feeling is that ‘zooming out’ is not really a link. Pagination is just a crutch to deal with returning the full set of a single query in meaningful window sizes due to server response, bandwidth, client parsing, and maybe human interface. Zooming out implies actually performing a different search and would be a function of a client interface.

This mechanism is provided by OpenSearch-Geo, which communicates how a client would use a bounding box or polygon search. Therefore a client interface could choose how to zoom out or in and has the capability to query the system this way.

Feed Clusters

However, I still could imagine more specific uses for such a concept. Erik’s original idea is perhaps thinking more about using geography as a way to indicate limiting search set results. This might be done using clustering mechanism, such as k-means, similar to how one might view dense data on a map in clusters, but in search result feeds. The link elements would provide looking into any of these clusters, or zooming in.

<feed>
    <title>Search for 'coffee'</title>
    <georss:box>38.87,-77.2,38.89,-77.0</georss:box>
    <entry>
        <title>8 results</title>
        <georss:box>38.87,-77.2,38.91,-77.1</georss:box>
        <link rel="self"
          href="http://server/search.atom?q=coffee&amp;bbox=38.87,-77.2,38.91,-77.1<"/>
    </entry>
    <entry>
        <title>15 results</title>
        <georss:box>38.87,-77.1,38.9,-77.05</georss:box>
        <link rel="self"
          href="http://server/search.atom?q=coffee&amp;bbox=38.87,-77.1,38.9,-77.05"/>
    </entry>
</feed>

Structured clustering

Another way this concept could work more literally would be to utilize a hierarchy, or several hierarchies, that a client may be able to easily snap to in order to query larger or smaller geographic areas. These hierarchies are not apparent in a simple search template and can provide more semantics to indicate the larger area rather than just simply “zoomed out”.

For example, using the GeoNames GeoTree or GeoPlanet woeid’s, a search result could provide links from the specific bounding box query up to regional or districts that contain this query – as well as perhaps subsets contained within the bounding box.

<feed>
    <title>Search for 'coffee'</title>
    <georss:box>38.87,-77.2,38.89,-77.0</georss:box>
    <link rel="up"
        href="http://server/search.atom?q=coffee&amp;woeid=2347605"
        title="Search Virginia for 'coffee'" />
    <link rel="down"
        href="http://server/search.atom?q=coffee&amp;woeid=12590311"
        title="Search Arlington County for 'coffee'" />
    <link rel="down"
        href="http://server/search.atom?q=coffee&amp;woeid=12590343"
        title="Search Fairfax County for 'coffee'" />
    <entry>...</entry>
</feed>

Written on no wifi, sitting in the Detroit airport after a red-eye flight from WhereCamp.

Whether you were already serving your country, or are responding to the President’s call, share how you are delivering on change in your community.Whether it is an hour per month helping those struggling in the current economy, tutoring kids in your neighborhood every day, or anything else, we want to highlight what Americans are doing to strengthen our country.

This is very interesting on several levels. Foremost is the use of government provided (TIGER/Line) and crowd-sourced data (OpenStreetMap) in an official US Government Site. This is definitely an indicator that what were cutting edge tools have reached a critical mass to provide broad usability and appeal. Open Source? check

Looking underneath the hood, the data is provided via a KML feed (), so you can pull the data out and upload or map it however you want. Open Data? check

The site itself, Delivering on Change, is asking citizens to contribute stories and media about their personal engagement with change. This is an incredibly exciting step to ask for people to contribute to national storytelling and character. Citizen-sourced data? check

The new US administration is continually doing amazing, and open, initiatives. There is incredible excitement around Recovery.gov as a testbed for the next generation of transparency and embrace of technology and open data feeds.

Small next steps

My thoughts on interesting applications wouldn’t be complete without pointing out a couple of suggestions. While many defend the default OpenLayers controls – I personally think that implementors should take that next step and apply minor customization to better integrate the look and feel of the map controls into their site. I’ve talked before about how easy it is to change some CSS to replace the controls. Perhaps even just a darker blue background to match the White House blue in the logo. Customized?

Another, less highlighted but very important for Government sites is the integration of accessibility controls. OpenLayers supports map navigation using keyboard inputs – which provides for alternative interfaces to navigate the map. It’s not clear if this is official “508 compliant”, but at least demonstrates the potential. Accessible?

How you can help

So do you want to help make Change, especially with mapping data and technology? Come join us at the Washington, DC mapping party – currently planned for June 20 + 21, 2009 somewhere in DC (details coming soon). Or join a mapping party near you.

Crowd-sourcing geospatial information has definitely become a common component of recent breaking news stories. Flickr, Blogs, Maps, Twitter, YouTube, et. al. are all normal channels that people are turning to in order to share, follow, and re-broadcast reports, information, tidbits, thoughts, and actions. Most prevalently in this incarnation beginning with the San Diego fires, and more recently the Mumbai attacks, and now swine flu.

For example, in the Mumbai Attacks there was an widely retweeted (rebroadcast) report that the government had asked people to stop twittering – when it hadn’t. In addition, there were numerous other questionable outcomes of the use of this crowd-sourced data.

Emergency information flows

Today with Swine Flu, we’re seeing the same issue even through georeferncing of traditional media that carries multiple editions of the same report. The result makes a single case in a High School in New York City look like an outbreak of numerous cases. There is no easy way to cluster these reports and either validate or mark them as duplicate in a way that has longevity and can feed back into the flowing stream of data.

Previously, with minimal citizen access to data streams and reporting – the information coming through these channels had a limited reach. But with increased connectivity in a crisis – either on location or via remote observation – the information is moving faster and further. The result has been an increasing concern on the potential negative impacts on reporting of invalid information, duplicate reports, or inducing panic. However, it also has the potential for incredible impact through these information patterns – we’re awash in bits of data that can inform and coallesce to provide us a full picture of the emerging scenario.

A case study of a solution

When we were working on TwitterVoteReport, this was raised as a concern with the aggregation of the twitter, mobile, media, and voice reports. The result was the implementation of a “Sweeper Interface” where volunteers could go through submitted voting conditions and mark them as “Approve”, “Deny” or “Modify”. “Approve” had the effect of bumping-up a report and validating it; “Deny” marked the report as questionable and pulled it out of the stream; and “Modify” allowed a sweeper to correct the automatically extraced metadata such as wait time, condition rating, and polling location.

The concept is simple: if you can crowd-source the information, you can also crowd-source the filter.

Retweeting is an observable measure of the willingness and desire of the crowd to disseminate and curate information. Retweeting and reblogging is an active action (mostly), but also has the effect of muddying the actual stream of information (unless a verbatim wording is used, which makes it easier to automatically cluster). This energy can be used in a way that more directly helps filter the flow of reports and news.

Chris Blow and Kaushul Jhalla brainstormed around this in the fall after the Mumbai Attacks evaluation, the immediate and longer term potentials and called the idea “SwiftRiver“. You can read Erik Hersman’s overview of the concept and also a video from the Ushahidi technical workshop in Orlando in March.

The status of the filter

We’re utilizing the open-source VoteReport platform that was so well engineered by David Troy, and slightly generalized for InaugurationReport and we’re now building out more of the concepts on reporting, automated filtering, and human sweeping.

Swift is now sitting behind VoteReport.in, which is actually using Ushahidi as the front-end reporting system. Reports from Ushahidi, as well as aggregegated media, twitter, and other news are passed through and people will be able to help tag, and curate the reports in order to provide cultivated, important reports and information. You can check out the Swift codebase.

The future of the filter

While there is good progress in beginning to have user annotation and curation of crowd-sourced data, it still requires potentially pulling users out of the flow of the data itself. There could be more inline interfaces for having users provide this moderation within their twitter client as posts flow past – or even on a mobile device based on geolocation and on-the-ground verification of a questionable report.

And ultimately, this information needs to then get into the appropriate end-user’s workspace. That could be a first-responder, an organization, or even a person caught in the middle of a crisis that needs accurate, up-to-date information and how to act.

This will make for a very interesting discussion at the upcoming CrisisCamp.