As always, each new year brings a refreshed feeling of excitement. Perhaps its the long holidays and copious amounts of food, family and fun, or seeing a magic new number on the calendar that makes it feel like “The Future!”, or just a desire to take advantage of an allowed re-emergence of self and goal setting. Of course, time isn’t discontinous, so 2010 isn’t disconnected from the current continuum of development and trends – but it’s still worthwhile to take the time to step back and consider where we are and where we’re going.

Mashable and James, amongst many others, have excellent predictions that will and won’t happen in 2010. Generally they are good insight into trends in the geo and mobile space, although I will take up counterpoint to some of his suppositions on File Formats, Interfaces, OpenStreetMap and Augmented Reality.

File Formats and Interfaces

Geo is definitely becoming mainstream – everyone in my family has a PND, uses Google Maps, and are asking about various location sharing applications. In the next year we’ll see geo become part of the assumed infrastructure, like the timestamp on a post or article, the location will be embedded.

I don’t think TAG (Twitter, Apple Google), as James puts it, will be the only location sharing services. They, along with even more used Facebook, will definitely be the general public interface to location query and sharing – but just because of this reason alone they will have to be very generic, leaving room for specialized location based services to still thrive in niches. FourSquare offers ‘gaming’ or Flickr visual media, and others for music, drinking, sight-seeing, and house finding. They will leverage TAG, or at least TG.

Apple is like the Nintendo of consumer technology – more interested in providing an integrated, compelling experience, and privacy, before full open-ness and engaging with the developer or geek. They’ll still have API’s, but not something like OpenSocial, GeoRSS, or FireEagle integration.

The iPhone, and to lesser extent Android, have been revolutionizing mobile devices. They are truly providing windows into the rest of the web of data combined with the real world. It’s natural for geopatial tools to move into these interfaces, but like any good user experience it won’t be the same capabilities you find on a desktop or browser application. The utilities will be specialized for the small screens, finger inputs, and out-and-about tasks.

For file formats, the Shapefile, unfortunately, isn’t near EOL. Too many tools only speak shapefile, and there is numerous legacy data that is still only available in Shapefile. Sites like GeoCommons offer alternate formats for all the data, but that still won’t remove this basic format. Only when there is a truly open, license free, API to File GeoDatabases (FGDB), and every off the shelf tool can talk that API or Spatialite, will Shapefiles begin disappearing out.

GeoRSS and/or KML, on the other hand, will be in every service that does anything Geo. Looking at any iPhone App review that includes KML (or doesn’t) brings up this point. Near enough everyone has Google Earth on their desktop, and Google is making big pushes in the utilization of Google Earth Plugin for in-browser virtual globes.

Visualization Technologies

To date, we’ve been stuck with either Flash or JavaScript DOM magic (and yes, Silverlight is out there too) in order to do data and geospatial visualization in the browser. As I mentioned, Google has been pushing Google Earth Browser, but also more generally they released O3D, a modern incarnation of X3D, providing for more general capabilities for creating 3D browser experiences. VRML lives!

More recently, there has been a resurgence in vector graphics that don’t rely on proprietary technologies or additional plugins. SVG and Canvas support is pretty widely supported except in the infamous Internet Explorer (which I hear is still being used even today). Examples such as ProtoVis, Cartagen and Tom Carden’s experiments definitely demonstrate that SVG is just on the cusp of being able to do a majority of compelling visualizations capabilities.

Another driver for alternative visualization platforms is the drive to mobile device integration. I don’t see Apple allowing Adobe onto the iPhone anytime soon, and even Android doesn’t have support. What types of visualization make sense is still a very open question – but whatever they are will be done with something like SVG.

Geo Data Skirmishes

James suggests that OpenStreetMap “won’t dominate”. While it won’t dominate, I disagree that it won’t continue to be extremely successful.

Google has recently moved to gathering their own data. They still have a long way to go, with many, many errors in roads, areas, addresses, and businesses and they’re using the crowd to help clean it up. Google is in fact proving the crowd-sourced model. It will be successful. Google is doing it with Google’s data, so there is no positive external benefit to that work – so to the industry it just looks like another data provider. However, with this proven model OpenStreetMap will succeed since any effort built into OSM has a positive benefit to anyone else.

However, there is a major difference in the trajectory OpenStreetMap is taking in the United States compared with Europe and other regions. In most other countries, the governments had very draconian licensing and as such OpenStreetMap was creating data from blank areas – starting from scratch, and building a community of volunteers along the way.

By contrast, in the US a vast majority of the data is free, and becoming more available everyday under the new administration. Therefore the US has a broad coverage of decent data without having first built the user community. So the difficulty here is both in building out community, as well as engaging companies that can do the same thing on their own while retaining proprietary rights to the data.

What’s fascinating, and what signals the ultimate long term success of OpenStreetMap, is that US state, local, and federal government agencies themselves are engaging with OpenStreetMap. They are investigating how to put their data directly into OSM, and possibly even re-incorporate updates and modifications back to their own infrastructures. Some are even considering using OSM toolset as their infrastructure. OpenStreetMap is going through some growing pains with respect to licensing, maintenance, and community – but all necessary steps in moving from a small cadre of hackers to a global, public project.

As we see an increase in open government, specifically driven by the US Administration’s directives, as well as other initiatives such as INSPIRE, this embrace and utilization of open platforms, and repositories, for sharing, federation, and syncronization of data will increase.

And as for augmented reality, it won’t be as big as you think… yet.

I received an email the other day from a reader of my blog with a very interesting question:

I was looking at a certain area in North East part of India ( State called “Arunachal Pradesh”) which is integral part of India.

Both URL’s have different take. [Google Maps] shows it as Disputed Territory ( with Dash lines) and [Google Ditu] shows it altogether as part of China!

So, that got me unruffled and to question validity of both these sources. How does Ditu differ from Google maps? Whats association between the two and does Ditu has autonomy to change the boundary of the maps as per its wish.

Arunachal Pradesh is a border region between China and India – with 70% of the land being claimed by the Chinese as South Tibet. The border in question was decided in 1914 and called the McMahon Line, but never agreed upon by the Chinese. The Google Ditu vs. Google Map views.

Territorial disputes are definitely not a new thing – however what is perhaps alarming is that there are two different representations of reality from the same vendor and data providers. So this is entirely a representational decision that is most likely driven by business and government pressures.

What’s particularly interesting here is that primarily these definitions of boundaries derive from the data providers. You can look in the bottom right corner for who the data providers are. For both versions the providers are the same: TerraMetrics, Mapabc, and Europa Technologies.

So it seems that the cartographic designers at Google Ditu have decided to represent it a certain way. Unfortunately, the map has no additional metadata. As broad consumption of maps increases, there is a commensurate interest in the why and what behind them. Who said these are the boundaries, when were they set, and why are they shown in this language?

And I don’t mean the long reams of unreadable metadata that are the current standards in the geospatial community, I mean human understandable descriptions of the various aspects of the data, while allowing additional discovery to deeper data.

One place that you can look at the data behind the source of the map is in OpenStreetMap. Arunachal Pradesh is shown similar to Google Maps version, and a user could optionally download the data to see the attributes, edit history and sources. Alternatively I can look in GeoCommons for the GADM Admin boundaries of India and see pertinent data on who provided the data, sources, and so on.

Boundary disputes in a bi-directional medium

The representation of boundaries is obviously a very contentious issue in mapping. Maps are perceived, and often do inform, territory. There is a long history of map representation being used to influence, coerce, and force land rights.

Unfortunately, even in a “Web2.0″ world of bi-directional sharing and collaboration, with maps we’re still often forced to accept a particular viewpoint. They have on-the-ground meaning and political impact. A well known example of this were the first “edit wars” in OpenStreetMap with the names of places in Cyprus. The resolution was to by default abide by the on the ground signage, but also store both versions and allow users to provide their own personalized perspective.

Understanding, awareness, and discussion about these issues is the reason for projects like OpenStreetMap or GeoCommons where you can download the information, build and share your own maps that represent your perspective.There

There isn’t an easy answer here – with companies such as Google there are obviously market, and government, forces that direct how to represent contentious issues. The best solution is to offer background, open data, and alternative perspectives. Without a voice, citizens are relegated to discussions by officials they may, or may not, have elected – and no meaningful way to illustrate their interpretation.

This week is National Geography Awareness Week. Hopefully you’re celebrating it your own way by enjoying a map, thanking a cartographer, or even doing some mapping yourself! It’s clear that mapping and geo have entered the mainstream – everyone is engaging with maps through navigation systems, friend location finders, and virtual globes. The next step is to make people aware of the potential for them to personally engage with place and location for personal interests, business uses, and community building.

This Wednesday, I will be giving the GISDay plenary talk at the University of Virginia in Charlottesville. My talk, “Neogeography: from Tower to Town Hall” will discuss how the movement to broad, public engagement and collaboration, particularly around geographic contexts through web maps, mobile devices, and open data can build stronger communities, improved research, representative government and better livelihoods of people. (link to the UVA Calendar)

Around Washington, DC you can join the OpenStreetMapping party in Bethesda, MD on Saturday, or you can check out the large list other activities at GISVirginia. Whatever you do, spread the word and encourage people to go out and map something.

This week on the DC National Mall there is the 2009 Solar Decathlon. It’s a contest between 20 student groups from around the world that build, on the mall, sustainable, energy efficient, and modern houses. The competition measures their efficiency, quality, resource usage, and design. It’s a one week miny village.

So of course, like any village, it needed to be mapped. I went down Saturday afternoon and captured the locations and names of all the buildings and paths that will be up for the week. These are then loaded into OpenStreetMap with start_date and end_date tags that notify the renderer when the features should be visible. It’s a similar model to how Burning Man is mapped year after year as it walks along the Black Rock Desert.

It’s ephemeral mapping – objects that exist in real place, but just for small slices of time. Important as any other building, yet typically relegated to flyers or verbal descriptions.

The fascinating part of projects like this is that OpenStreetMap allowed me to create a map that was useful and immediate. Within minutes of uploading the data, it was available as rendered tiles, vector data, and downloadable to GPS units and iPhones. People on the mall could immediately view the local map with this new information.

It’s a nice demonstration of how community projects like OpenStreetMap will continue to innovate faster, and more openly, then other ‘crowd-sourcing’ options.

We have a rich history of geography, cartography and GIS that is currently tucked away in top drawers, intranets, and repositories that may not stay online when we most need the data. How do we expose these huge troves of data in a way that can be utilized across various domains. The GeoWeb is all part of the same web, semantic, sensor, social, (and interplanetary). So it is vital at the GeoWeb align itself with the web and the multitude of sources and endpoints that the web is reaching into.

There are many possible solutions, and a few that are within easy grasp that we can build our tools to encompass, and develop practices that encourage utilization of these solutions while still moving forward onto better ones as the GeoWeb matures. So we’ll take a few articles to look at specific solutions.

Discoverability

Perhaps the most prevalent issue, and the one that is most easily addressable, is the findability and discovery of geodata on the web. Mano Marks reflected this same sentiment in his blog post on standards.

In thinking about discoverability, there are several primary use cases to consider: Machine crawling, Human discovery, and Tool discovery. Providing data via just a single mechanism means that it doesn’t get utilized and consumed to it’s potential and so somewhere along the chain of utilization it will be a burden to actually incorporate into a workflow.

Think of the machines

Machine crawling is the ability for any spider to walk links, find data, metadata, and formats automatically. It’s what Google, or GeoNetwork does to find and register data sources.

There was recently a discussion on auto-discovery in the GeoWeb suggesting the use of robots.txt, sitemaps, or embedded META tags in HTML pages.

Consider how a spider would get to a site: it follows a link to a geospatial portal from some blog, resource, or directly entered as a good place to get data. It does a GET request on the root homepage, “/” which most likely returns the index.html equivalent. The program then parses through that for links or additional information.

If the spider knows about them, then it may ask for a sitemap.xml or robots.txt. But nothing in the original page request noted that this potentially very complete listing of data was there. This problem is the equivalent of an application having to know that it needs to ask for a GetCapabilities or other method to even discover what is available. Too much implicit knowledge of the specification is required for a program to easily discover new data and services.

What the program does see are these links that can contain information such as a link to a list of available resources. The simplest is a link to the Atom or RSS feed that can simply be a paginated list of all the resources available to the application. Within Atom, there is then the ability to link to various representations of that data in different formats. So applications are able to take the most appropriate format based on what they can consume.

Several years ago I first proposed how KML and GeoRSS could easily support one another via cross-links and with HTML documents. Atom has very nice rel and type attributes that allow for linking to all sorts of different representations. You can even link to OGC services like WMS and WFS using atom links.

Of particular interest here are looking at the currently approved list of Atom link relation types that provide basic semantics for telling you how what this link means. Is it another page? just related? It’s a limited set, but one that covers an approachable majority case for developers to begin using.

For example, mechanisms like OpenSearch, specified in a rel="search", simply notify the application that here is a service that it can query to get at additional resources. And with OpenSearch-Geo, a geoweb crawler can query information within a specific location or bounding area.

Humans need data too

Crawlers are great, they provide a say to pull together information into various other sites and tools to provide customized interfaces to users. However, within any site or tool, how should we expose geodata in a way that humans can easily use for whatever purposes the may have.

Again, links have become a very well understood concept on the Web. That underlined blue line states “beyond me lies an unspecified amount of information about this topic“. However, these links typically imply that they will open another human readable HTML page in the browser. A problem caused by links to media such as geospatial data is that the content behind a link may not be just text, it could be an image, audio, movie, KML, database, or a service. Clicking on that link relies on the browser interpreting the MIME-type (remember the point about how vital mime-types are?) and opening the application the user has specified, or left as default.

So consider what this means for generic media. Clicking open a link to an image probably just opens the image in your browser, or opening a movie loads an embedded video player. Geodata browsers, however, probably doesn’t have the same install base as say, Quicktime. Except perhaps GoogleEarth. The Web has become much more comfortable with clicking on a KML link and seeing Google Earth open up and show the data on a globe.

But something very vital often exists with a link to KML data – a recognizable icon that notifies the user (as they learn) that it is a file that will open in Google Earth, or another KML viewer. This is the same as the very widely used RSS icon.

I discussed this idea before about the geotag icon showing various other formats – and now sites like Data.gov actually show the various data format options.

So what we need for GeoWeb standards are some visual representation to people that they are can clink on this link and open a spatial relational database, or an OGC service, and perhaps have some confidence that there is an application that will provide them a useful way to access the data. (and I’m still waiting for Sean Gillies’ ISO and Dublin Core icons)

Of course, we should also employ emergent interfaces that show users the type of data links that are appropriate for them based on their profile or registered MIME-type handlers.

Man-Machine hybrids

So we have discovery links for machine crawlers to register and harvest geodata, and links for humans to click on to follow to data and within data. However, this can easily become overwhelming to need to click through to every link. Imagine if browsing Flickr through lynx.

Browsers already do a lot to assist users in finding relevant extra pieces of data in a page. RSS autodisovery links show up in URL bars notifying our feed readers that we can subscribe to this page. OpenSearch allows someone to embed this search into their browsers (most of them at least) to easily search the repository later.

The decreasing cost of links

These various approaches for different needs and use cases are all very well aligned. They don’t rely on additional external files that we need to make sure stay up to date or that tools are built to just know that the file can be found at a pre-defined location. Links cost next to nothing, mostly measured in bandwidth sizes, but provide a wealth of accessibility and discovery of geospatial data. Especially data in formats that make sense depending on the tools and use cases for different problems.

Of course, links alone don’t address all the needs of the evolving GeoWeb, they merely provide for the integration of geospatial data with the rest of the web. An important, necessary, but not entirely sufficient first step. We need to consider the actual uses and interfaces of these standards, archival, synchronization, conflation and more.

1. Introduction
2. Where We Are
3. Problems
4. Where We Need to Go
5. Solutions: Discoverability