The GeoCommons team is expanding and we’re looking for some cutting-edge developers and designers to join us. We’re using a wide range of technologies to build an easy-to-use and incredibly powerful geodata sharing, visualization, and collaboration platform that is being used in organizations from the government, to enterprise, to international NGO’s, to local communities and groups.

With GeoCommons, we’re integrating Neogeography with GIS to provide powerful tools to users: if you can make it fun on the web where users aren’t required to stay, then customers will love you. And by integrating with other tools that each user is comfortable with, whether it is Excel, Notepad, GoogleEarth, or ArcGIS Desktop and QGIS; we help bring GeoCommons to them rather than making them come to GeoCommons. We’re also pushing the next generation of GeoWeb standards: KML, GeoRSS, GeoJSON, and making them more powerful and supported. These are ideas we started with Mapufacture and are quickly integrating with Finder!, Maker! and the rest of the GeoCommons suite.

As a part of our team, you would investigate large-scale data sharing and linking, geospatial and data visualization mechanisms and tool development, web native API integration and community building. We’re working with many other groups in the open-source as well as GIS communities to help integrate data and tools to broadly disseminate all this quality data that has otherwise been inaccessible and make it easy to visualize and use in decision-making.

We’re looking for developers with real programming chops – you should be comfortable considering Mongrel and Nginx versus Passenger, know when to use unobtrusive Javascript or call ActionScript Flash hooks, have played with ActiveMQ and Stomp, beanstalkd, Starling or other queueing systems, read technology news and blogs and preferably have a site yourself where you share your experiences and code with the world. We’re looking for community members and developers that like working in teams, attending programming groups, and are comfortable sharing their ideas. We encourage you to have hobbies and side projects – we’ve built quite a few ‘lab’ tools ourselves such as context-free music and touchscreen whiteboards. And you don’t have to be an Apple user, but it helps.

Welcome to Washington, DC

FortiusOne is located in Arlington, VA – directly above the Courthouse Metro on the Orange line into DC, and a short walk into the district directly. The DC area is on an incredible spike of growing technology community. Where else can you live in a “metro area” that encompasses at least 3 states, all of which are metro accessible? The area is also renowned for it’s bike accessibility. The recent election has cast a spotlight on the future of technology in the government with President-Elect Obama’s Change.gov initiative. The upcoming inauguration is sure to be an incredibly historic event and you could be here to help map it.

As for the community, there are at least three Ruby-specific groups, a NOVALang where learning new programming languages is the prime objective, RefreshDC, TwinTech, and one of the most open governments to geodata standards and sharing. We’re also quite big fans of the local beer selection and hard to beat the food variety.

Let us know

So if this sounds exciting to you, and you’re interested in joining the team – please let us know! You can also check out the formal listing.

Dave Troy has put together a solid backend that is aggregating together Twitter, SMS, voice, iPhone and Android native applications, and even YouTube. Others have built the iPhone specific applications. I’ve been working on the mapping and data sharing side of the project. The first goal was to provide a number of mechanisms to share the data that we’re gathering with everyone. Additional mashups and visualizations are free to use the data streams to pull all the data that VoteReport itself has – so definitely go wild with your ideas. A quick breakdown of what’s available:

OpenSearch – http://votereport.us/opensearch.xml

This is the OpenSearch description document that outlines all of the feeds and various filters that you can use when getting to the data. Always check this as we’ll update it with new parameters or data streams. In addition, the various responses discussed below include OpenSearch styling pagination so you can walk through the entire database of reports without having to drink right from the firehose. This also includes the OpenSearch-Time extension.

KML – http://votereport.us/reports.kml

Getting the reports.kml will give a Network Link – this is useful for GoogleEarth and other KML clients to automatically update every 60 seconds with new reports. You can append live=1 to get the full KML document. I have included all the useful attributes in the ExtendedData element of all the Placemarks. Each Placemark also has an id for easy reference.

GeoRSS-Atom – http://votereport.us/reports.atom

Just want to subscribe to the feed in your RSS reader, this feed is useful for getting updates.

GeoJSON – http://votereport.us/reports.json

JSON is super nice for doing client-side mashups and visualization. This is what the VoteReport Map itself is using. It includes a lot of information for each report, including reporter, icon, location.

All of these feeds even can take a dtstart= with an ISO-8601 date for getting reports after a certain time (and optionally dtend= for getting time-bounds of reports). A useful geographic filter is to use state= with the capitalized two-letter state code to just get reports within a state. So for example http://votereport.us/reports.atom?state=VA is a GeoRSS feed of reports in Virginia. As I mentioned, I did build a quick map that you can view at http://votereport.us/reports/map.

We’re continuing to build it out with new features as more data comes in. You can easily embed the map in your site using (and optionally remove the state=):

The difficulty with creating more visualizations is the lack of pre-election data. This system has been built to primarily capture a huge amount of valuable information for one day. We’re not sure before hand what this data will look like, coverage or attributes. Typically visualizations are made by exploring and playing with the data to see what emerges. In this case, we’re making estimates (and guiding via the tutorials) on what data we’d like. Therefore, the map itself has simple mechanisms for styling markers based on the user-supplied report. But the data is far to dispersed so far for something like a heatmap.

Fortunately, the team consists of a large number of public advocates that are spreading the word which should encourage more citizens to use the system and contribute both good and bad reports. Andy Carvin of NPR put together this NPR coverage, and we’ve also received coverage from Time, Huffington Post, New York Times, TechCrunch and even Craig Newmark. Check out the TVR press page for more coverage links.

And if you would like to help contribute to the project, check out the VoteReport Wiki. I imagine there will also be a number of post-election visualizations and analysis to come out of the reports.

The very first session I attended was moderated by Peter Corbett from iStrategyLabs and Dmitry Kachaev of the Office of the CTO for DC talking about open data from the government.

The Princeton research paper “Government Data and the Invisible Hand” proclaims the need for government agencies to first and foremost share their data via open and broadly used standards (via Ars Technica).

The DC government is already sharing a large amount of data via a variety for formats: GeoRSS, KML, CSV, Shapefiles. Check it out at http://data.octo.dc.gov and also CapStat: Building a City That Works

Information. Knowledge. Progress. Welcome to CapStat, your resource as a District of Columbia resident to track how District Government is working for you. You can use these pages to track the performance of individual agencies, find neighborhood statistics and learn how your government is responding to the city’s most pressing challenges. As the District of Columbia works to become a world-class city, visit this page to follow its progress and find out how you can become part of the solution.

When the Office of the CTO for DC wanted to update their site and services they brainstormed how to do this – the typical method of hiring a very expensive contractor to build a complicated tool that would be poorly implemented – or open up the system for the community and see what emerges.

Fortunately they chose the latter and recently launched Apps for Democracy – a contest that anyone can submit an entry for a mashup or application using the DC data services.

Data stream of data

One comment that came up was the desire for a feed of the available data and updates. Right now there is a web page and some of the data is available as GeoRSS – so continually updated. However, what would really be great and facilitate federation would be a published GeoRSS feed of datasets that links to each of the available formats, updated times, filesizes, metadata, etc.

What is particularly funny about this project is its use of the term “GIS Technologies” to describe a very archaic map with stand-up pins. A GoogleMap is not GIS. The original site does allow users to search by specific attributes, but not really in a dynamic way.

The map is created by directly building markers up in Javascript. Assumedly the back-end is powered by a database, and it would have been nice of them to share out an actual listing of the underlying data. This is a perfect example of the study .gov web sites should focus on RSS, XML—not redesigns.

So instead I just created a KML version that uses Extended Data to call out the various attributes on the features such as LEED rating and number of buildings. You can see my own mashup and grab the KML file.

As I said before, this map I made isn’t “GIS”. But consider I spent about 5 minutes with the original Javascript and a bunch of RegExps to create the KML. It’s up to you to make something useful with it.

There has been a lot of dialog. Sean discusses a lot of his thoughts about semantic interoperability and formats. There has also been a number of discussions on the design itself – everything from the color, to the pushpin being indicative of points only – maybe reinforcing the “red dot fever” that plagues many maps.

These are really minor quibbles. Overall I think it’s a decent design that gives some simple meaning to what the icon conveys. However, the problem I do have is the Usage guidelines & examples. Essentially, they are saying it should be used for all geospatial formats.

Example from the site:

Bruce defends this:

Whereas the Geotag Icon describes a general concept (”This item is geotagged”) the KML icon and GeoRSS favicon each proclaim a file format. This is analogous to the Feed Icon: can you imagine having a different orange icon for each web feed format? There’s no reason why the Geotag Icon can’t sit side-by-side with file format icons if that’s what folk wish to do. But a well-recognized Geotag Icon (in time!) adjacent to the text description “Download KML file (opens in Google Earth)” could well be more informative to the majority users than what is otherwise sure to be a growing set of vaguely-related file format icons with which to become familiar. The power of de facto standard icons is in instant recognition—and the fewer the merrier!

I disagree, he’s proposing this one icon should be used for a multitude of different formats that each have different capabilities and uses. It’s not like the difference between RSS and Atom, it’s the difference between HTML and RSS or CSS. Or a Video and a Photo. Sure, they’re both images, but they’re also very different in what they do.

He’s creating additional confusion by using the Geotag icon for GeoRSS. GeoRSS isn’t even a file format, it’s an extension to another file format: RSS / Atom, and they already have a recognizable icon that has meaning to users. I wouldn’t want to put yet another icon in front of them that meant something slightly different. And KML is a visualization format, similar to HTML + CSS. GPX is a very specific format that works for handheld GPS units and PND’s. I’m surprised the Geotag Icon wasn’t proposed to be used for Geo and Adr Microformats, since it matches this formula of all things geo.

This is the follie of the greater GIS community – assuming something is primarily geo first, and general information second. I’m surprised this is idea is also followed by people outside the GIS world.

So I only ask that the Usage guidelines of the Geotag icon be scaled back. It’s interesting that it’s been incorporated into Minimap Sidebar – good idea, but perhaps again confusing application with format? Using it in a photograph or video is nice because it’s clear to me that the format is a video (and I don’t care if it’s mov, fla, et al.) and useful to be alerted that it has geocoded content inside. I also think it could be useful as a link to a page of Geospatial formats. Why not even use it like the Share this on… on the Geotag project page itself?

GPX icon is from Garmin’s Communicator Plugin. You could optionally replace the format names (like KML) with suggestion applications, but I find this a little to vendor specific. Don’t you dislike it when people say things like “I opened the Internet Explorer page…”?

I think this set of links is how I would do it in GeoPress. But don’t suggest that Geotag Icon become the over-arching marker for other formats that happen to contain geo-data. Otherwise, I’ll be suggesting a family of icons like Timetag, and Titletag.

libkml was originally “announced” about 6 months ago as part of the kick-off of the standardization of KML within the OGC.

libkml is interesting in several ways. KML itself is just an XML specification for geographic data. Nothing really special compared to other XML formats. However, as I’ve championed there is a big difference between types of developers that use and read schemas, and those that use libraries or simple examples and documentation to implement parsers or tools. This is justified in that developers (both consumers and producers as discussed here) are usually trying to solve some other problem and want to use a format like KML merely as a mechanism to publish and visualize their information. By providing a stable and full-featured library, developers are free to build tools around the library without having to deal with the intricacies and issues of the format itself.

Similarly to the effect of opening the standardization of KML to the OGC effected other organizations like Microsoft to embrace the format – an open-source library also encourages other implementations, or competitors, of KML applications. Google is primarily in the business of data organization and search – so the more tools that publish or utilize a format they can then index is a win.

Another implication of libkml is that a single library can grow with versions and features, again freeing the developer from having to track future versions or bug fixes to the format.

Lastly, libkml is written to be fast – which is essential for handling large KML documents, realtime visualization, and potentially even mobile/limited-resource clients. However, how small libkml can be made is left to be seen.

As Michael Ashbridge pointed out, this is a very “alpha release, not Beta in the Google sense”. In fact, in the documentation there is the very clear disclaimer: “THIS IS ALHPA SOFTWARE. Expect changes. We do not yet recommend use in production code.”

There are still a number of features that are not yet implemented that are forthcoming, or can be accomplished by the broader community. They’re looking for feedback from developers on the interface and functionality. The library is C++, with SWIG bindings currently in Ruby, Java, Python, Perl and PHP. There are examples for developers to get up and running quickly.

It’s released under the new BSD license. It is meant to be as open as possible for developers to use in both open-source and closed-source projects without worrying about interference with other licenses.

It’s great to see Google pushing on the open-{source,format} in geospatial. They’ve obviously done a lot to raise public awareness of placemarking and geospatial data with GoogleMaps and GoogleEarth – they’re now engaging the GIS community and helping them.

Hopefully people, at least developers and users in the know, can soon stop referring to KML strictly as “GoogleEarth format” or “GoogleEarth Layer”.

Dealing with Reality

An issue we commonly run into is the reality that there are a lot of KML and other data sources in the wild that are malformed. There is the common response “it works in GoogleMaps, why doesn’t it work elsewhere?”

libkml is able to handle, to some extext, ‘bad’ KML, but is very strict in outputting KML that is generated using the DOM API in the library. Hopefully this generally raises the quality of available KML.

lib{geo}

A potential extension to libkml that excites me would be the ability ingest a KML document and publish it out as other formats such as GeoRSS or GML. Especially if a higher-level interface was built onto libkml that abstracted away the specifics of KML and instead provided an interface for general geometry (and feature) creation and manipulation.

Unfortunately since my laptop hard drive died last week, I don’t have a development machine to build and play with this yet. But I expect to use this library in a number of projects.

Google Code Project: libkml.

There are several commonly encountered use cases. News reports typically mention several locations. Bloggers using GeoPress may tell a story about a trip and want to reference several spots along their trip – especially if they are documenting a tour that includes a path and sites along that path such as in EveryTrail. Dan Schultz talks about why “One Location Doesn’t Cut It”, citing other examples from news journalism.

Adrian and I recently sat down together to quickly brainstorm on what this may look like. The features we were looking to add were: multiple geometries, excerpt for that geometry, and toponym for that location (venue, city, etc.) Additionally, we didn’t want to break current compatibility.

Other services are already including multiple locations in different ways. Flickr outputs a single location in two different formats of GeoRSS: Simple, and some odd form of deprecated W3C. MetaCarta’s RSS-to-GeoRSS converted currently just dumps multiple locations into the entry, but without identifying if these are unique locations, or just variations in format type or hierarchy.

We wanted to call out that this is in fact a different type of geometry – a multi-geometry. Both KML and WKT support multi-geometry, but without being able to reference what the points are individually about. That’s useful if you are, say, marking all the holes in a field, but not for narratives.

Another feature we wanted to try and support was to be able to reference geometries stored elsewhere. Currently in GeoRSS feeds you’ll typically see references to a City or Country just include a point to the center of that geography. Not really indicative of what the article was about, or useful when trying to find all the geographic data about an area. So it’s important to include lines and areas as appropriate. However, including huge outlines of states or nations, potentially multiple times within a single feed, can have drastically bad consequences of increasing feed file size and complexity.

Here is a snippet of what we are proposing:

The first part to notice is that we wrapped the multiple geometries in a georss:collection. This allows current parsers to not be confused by encountering multiple georss elements unwrapped and being unclear if they are multiple representations of the same geometry, or different geometries.

We also included a excerpt attribute that allows you to include some text referencing what this location is specifically about. This can be text from the article itself, or some other useful information. One concept we had considered was using some reference to the text wrapped in the article itself, but this seemed burdensome and prone to problems using an attribute of one element to embedded text in another element.

The second element is a georss:polygon that includes a src reference to the geometry stored elsewhere. The rel tag specifies that it is the geometry of this element, and the type helps the tool know what the representation is of the stored geometry. This way a tool that is consuming the GeoRSS can go and fetch the geometry if it wants, or if it already has a cached version, say referenced elsewhere in this same feed, then it doesn’t have to request it again.

Of course, with a standards development, it is useful to consider how a user interface might provide for including multiple locations in an entry. Here is a mockup of how I imagine a simple interface would appear, and probably how we’d implement it in something like GeoPress:

Article: We went to visit downtown Cedarburg before the conference. Had some great sandwiches at Joe’s. If you haven’t been to Cedarburg, Wisconsin, then you haven’t really experienced the MidWest…

Locations:
– Excerpt: Went to visit downtown Cedarburg…
– Type: Point
– Geometry: 43.296700 -87.987500
– Name: Cedarburg, Wis.

To promote ideas and discussion around these and other proposals, I’ve created proposals at GeoRSS.org on multiple location and referencing external geometry. Please let us know what you think about the idea and format. We know that we can’t please everyone, but like the origins of GeoRSS, we’re just trying to address a real need with a simple format.

This is a really great step. I’ve looked at the EPA data and started writing various scripts to massage it into a nicer format than their CSV. A lot of work, a lot of data, and a pain to maintain. So it’s great that institutions are exposing their data in easy to consume and view formats.

However, it also illustrates the typical ‘munging’ of styling and data that users, even professional users like the EPA, will do with KML. In the Air Emission KML they are using Altitude to display some measure of chemical emission. It isn’t clear what the number really means, and seems really bad to have a data measurement in the geometry.

And especially other clients that will consume this data. How are they to know whether the height is really the altitude of a facility, or some other representative number? In fact, you can export the facilities KML with altitude representing: NOx, Lead, Particulate Matter, Sulfur Dioxide, or Volatile Organic Compounds and there isn’t a way to tell once you’ve exported. Unfortunately KML doesn’t currently support the ability to style based on ExtendedData, other than for the Balloon text. But it would still be useful to put this data there.

For example, in Mapufacture we pull in and store the arbitrary data with an KML or RSS feed. So then a user could add, for example, all Emission facilities with a high NOx emission rate within 5 miles of their house to the map of their community.

An answer could be a link for each element to the data in a different markup such as GML, but that seems slightly convoluted and difficult for tool developers. That would still be a good solution for advanced data and metadata specifying how and when the data was collected. But the point here is to publish a base level of information in a light-weight, broadly used data format.

A couple of other small niggles with their KML: the title is just “Temporary Places” and doesn’t have a meaningful title of what the data really is, or where it came from. A title that is representative of the data, so when I leave it in my KML viewer or ingest it into another tool, then I can remember what the data is about. KML 2.2 also supports attribution and atom links that should point back to the EPA site. Also, all facility names are all caps, which is how the data was stored in their old CSV files (when I looked before).

By Example

By way of specific discussion, lets do a simple example of modifying the EPA’s current KML to a more useful KML.

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.2">
  <Folder>
    <name>Temporary Places</name>
    <open>1</open>
    <Document>
      <name>Petroleum_facilities.kml</name>
      <Placemark>
        <name>SAM HILL OIL CO:&lt;br&gt;&lt;/br&gt; 725 S MAIN ST BRIGHTON CO 80601-3047</name>
        <description><![CDATA[<p>SIC: 2911<br />
          Petroleum Refining And Related Industries Petroleum Refining Petroleum Refining </p>
          <p>NAICS:       <br />
          </p>
          <img src="http://www.epa.gov/cgi-bin/broker?_service=data&_program=dataprog.dw_emisplot_epad8_facility_caps.sas&year1=2002&year2=2002&debug=0&site=15121"width="460" height="360">
          <hr><p><b>Petroleum Facilities Emissions (258  facilities)</b>]]></description>
        <styleUrl>#A</styleUrl>
        <Point>
          <extrude>1</extrude>
          <altitudeMode>relativeToGround</altitudeMode>
          <coordinates>-104.8257,39.9754,13.102642872</coordinates>
        </Point>
      </Placemark>
    </Document>
  </Folder>
</kml>

So let’s clean this up a little by adding some better titles, links to more information, and putting the data in a data location instead of with geometry.

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.2">
  <Folder>
    <name>EPA Air Emission Sources</name>
    <description>Under the Clean Air Act, EPA establishes air quality standards to protect public health, including the health of "sensitive" populations such as people with asthma, children, and older adults. EPA also sets limits to protect public welfare. This includes protecting ecosystems, including plants and animals, from harm, as well as protecting against decreased visibility and damage to crops, vegetation, and buildings.</description>
    <atom:link href="http://www.epa.gov/air/emissions/" />
    <atom:author>
      <atom:name>US Environmental Protection Agency (EPA)</atom:name>
    </atom:author>
    <open>1</open>
    <Document>
      <name>Petroleum Facilities</name>
      <atom:link rel="self" type="application/kml+xml" href="http://www.epa.gov/mxplorer/Petroleum_facilities_US.kmz"/>
      <Placemark id="sic:2911">
        <name>Sam Hill Oil Co</name>
        <description><![CDATA[
          <img src="http://www.epa.gov/cgi-bin/broker?_service=data&_program=dataprog.dw_emisplot_epad8_facility_caps.sas&year1=2002&year2=2002&debug=0&site=15121"width="460" height="360">]]></description>
        <styleUrl>#petroleum_facility</styleUrl>
        <address>725 S Main St, Brighton, CO 80601-3047</address>
        <ExtendedData>
          <Data name="SIC">
            <value>2911</value>
          </Data>
          <Data name="industryDescription">
            <value>Petroleum Refining And Related Industries Petroleum Refining Petroleum Refining</value>
          </Data>
          <Data name="facilityType">
            <name>Petroleum</name>
          </Data>
          <Data name="NOx (ppm)">
            <value>13.102642872</value>
          </Data>
        </ExtendedData>
        <Point>
          <coordinates>-104.8257,39.9754</coordinates>
        </Point>
      </Placemark>
    </Document>
  </Folder>
</kml>
		

To summarize, the EPA releasing their data in KML is a really great step to leading the way on information transparency and public awareness, however there should just a tiny bit more effort to demonstrate some better behavior in their shared data.

It’s also available via Mapufacture if you want to embed the map in your blog, get the GeoRSS, or mix in with another map you may have for say, other events and sights in Chicago.

I imported the KML file into my GPS receiver – which is pretty cool, and Mapufacture Mobile lets me use my N95 to find nearby locations. The original KML even has the address and phone number as KML elements, although no KML parser I’ve run across makes us of that info… yet. The iPhone Maps application loads ‘most’ of the markers, but doesn’t pull out the phoneNumber elements either.

Unfortunately, neither MGMaps (using OpenStreetMap tiles) nor GoogleMaps Mobile let me pull in KML – MGMaps complains and GoogleMaps Mobile just doesn’t have that bit of functionality. Perhaps they’re both waiting for OGC-KML ‘Mobile Profile’

Of course, a Dash would be a really good fit for this KML as well.

The big ‘announcement’ was that Google will be releasing an open-source KML library in C++ that implements and tracks the standard as it progresses. By providing a reference library it allows developers to more easily keep up to date with KML without having to maintain their own library and track standards changes.

Assumedly bindings could be written to make use of libkml in other languages like Ruby or Python.

This is smart for Google, and any publisher/consumer, as it helps ensure that clients and servers are properly using KML and there aren’t various mixed versions beings published.

The library is expected to be available the first quarter of 2008 – and available under an as yet to be determined open-source license.

Tags: kml, opensource, libraries, google, ogc

So if you want to continue playing along, track on over to the Mapufacture blog, or just toss the RSS or Atom feed into your reader and you’ll probably not notice the difference.

Tags: ogckml, kml, kml3, mapufacture, blog,

• Subscribe to del.icio.us, technorati, or your other aggregator of choice using the tag ‘ogckml’
• If you don’t want to clog your RSS Reader, then just check out the OGC Network KML aggregator that is already collecting all these links
• Tag your own blog posts or bookmarks to demos, stories, posts, or ideas using ‘ogckml’
• Join the #kml IRC channel on irc.freenode.net. IRC – it’s really cool (again)
• Join the GeoWeb REST mailing list/group for discussion on KML, and building a RESTful GeoWeb

Also – I’ll be moving my particular commentary over to the Mapufacture Blog, since that’s the company that I’m actually participating as a member of, and also where we’ll be implementing the new specs. My next posts will discuss using KML to link to external services such as OpenSearch-Geo, WFS, WMS, and more.

To date, metadata in KML has been somewhat confusing, ill-defined, and just kind of “wishy-washy”. This is the effect of several causes. First, KML is ‘lightweight’ and has been used by many mappers that haven’t had an interest in adding richer information to their pretty maps – yet. This is changing, as the concept of mapping, geography, and information is becoming easier to do and the concept more clear, more users are looking for how to include extra information about their geodata.

Another problem is the term Metadata, which has different meanings in different spheres. In GIS, Metadata are the supporting information that describe the data set: when it was created, who did the collecting, how complete is it, what are the license terms, keywords, coverage, and so on. The Federal Geographic Data Committee (FGDC) gives definition. For an example, check out the Baton Rouge GIS Metadata for Lots, 2004, East Baton Rouge Parish, Louisiana.

By contrast, programmers and general users have learned to refer to Metadata that is more generically “data about data”. This typically means just extra information that goes with the data, instead of primarily about the data. Examples include categories, tags, number of lanes for road, building type, and so on. GIS practitioners refer to these as attributes instead of metadata.

Anyways, the result has been that people often just become confused when talking with one another because of a different vocabulary. In the end, everyone just wants to be able to describe their data and add more information to it than just the strict geometry. See what the Dublin Core Metadata Initiative has done for HTML and RSS.

What’s the current situation?

Many developers have started pushing the envelope of what is possible in KML, and more directly, what Google Earth supports natively. Mano Marks at the Google Developer Knowledge Database provided a good document on how to possibly use XML in the Metadata tag.

Chris at FortiusOne has been blogging a lot about the how it is currently possible to use Metadata in KML 2.1 for thematic mapping. Jason Birch has found some really intriguing uses of the Schema tags that Google Earth actually uses for styling balloons.

Getting back to KML 3 and the OGC Technical Committee, one thing that seems difficult about the current KML Schema and Metadata support is that it allows (requires) users to define and use their own tags, or elements. Using Jason Birch’s example, he’s defined a schema for newts:

<Newt>
   <name>Sammy G</name>
   <styleUrl>http://www.jasonbirch.com/files/kml_schema_style.kml#newt_style</styleUrl>
   <Point>
     <coordinates>1.75,1.75,0</coordinates>
   </Point>
   <newt_id>36</newt_id>
   <newt_breed>Common Orange Slitherer</newt_breed>
   <newt_slime_factor>7.2</newt_slime_factor>
   <newt_tail_length>6</newt_tail_length>
 </Newt>

This is really powerful, but what this now requires is that any KML client or library that wishes to support metadata now has to delve into some fairly complex XML creation and parsing. The was created on the fly in the schema definition, which the client will have to parse and then use to parse the rest of the document to extract any information at all. A client couldn’t simply say “I don’t understand Metadata” and still successfully get the geometry out of the KML document.

This is especially important using the concept of modules that allow KML implementations to only include pertinent areas of functionality. If a mobile phone client want to display markers, but isn’t going to use Metadata it shouldn’t be required to implement the complex Schema parsing just to get at the markers.

Don’t remove my schema tags!

I personally have heard this a couple of times, and fear not, we’re not talking about doing away with Metadata or ’schema tags’. Instead, we’re considering just simplifying how they’re used. Primarily, we want to support lightweight addition of additional data without requiring you to use that data to understand the underlying geometry.

To illustrate, check out the example document at the OGC Network. Returning to Jason’s example, we’re going to define the Schema (to provide for more ’strictness’ to make XML-heads happy)

<Schema name="newt_data">
    <SimpleField type="int" name="newt_id"></SimpleField>
    <SimpleField type="string" name="newt_breed"></SimpleField>
    <SimpleField type="decimal" name="newt_slime_factor"></SimpleField>
    <SimpleField type="decimal" name="newt_tail_length"></SimpleField>
</Schema>

And then our actual placemark can use these attributes:

<Placemark class="newt" id="sammyg_newt">
  <name>Sammy G</name>
  <gml:Point>
    <gml:pos>1.75 1.75 0</gml:pos>
  </gml:Point>
  <link type="application/html" href="http://mynewtfarm.com/list#sammyg_newt"/>
  <Metadata class="newt_data">
    <property name="newt_id">36</property>
    <property name="newt_breed">Common Orange Slitherer</property>
    <property name="newt_slime_factor">7.2</property>
    <property name="newt_tail_length">6</property>
  </Metadata>
</Placemark>

What this now allows is for someone to easily add metadata, or attributes, without requiring a client to use it to just show a marker where ‘Sammy’ currently is located. In addition, the metadata could be more complex than just a simple string or numeric value if required.

But I want my real Metadata

While KML is a lightweight geospatial interchange format, it shouldn’t restrict users from referencing more formal data formats as necessary. In fact, it would be straight-forward to publish a simple KML file referencing all of your more complex geodata and then let users choose which data they want to investigate.

Using the , we can easily reference a related document that includes a full GML Metadata description.

<Placemark id="newt_10">
  <atom:link rel="related" type="application/xml+gml" href="http://mygisserver.com/lizards/newt/10"/>
  <Point><pos>43 82</pos></Point>
</Placemark>

(Note: application/xml+gml isn’t a valid MIME type that I’m aware of, I’m just using it as an example of how you can indicate to an application what is at the endpoint)

This way, a developer can include lightweight attributes, or Metadata, but also reference richer Metadata as necessary.

We still have to look at how this affects styling – we want to allow for templating marker descriptions (BalloonStyle) and also possibly apply styling based on values of the Metadata. If you have some suggestions – we’d love to hear them.

Sidebar: W[F,M]S

This also leads to interesting discussions about how KML fits into the OGC services. Two of the prevalent OGC OWS, WFS (Web Feature Service) and WMS (Web Mapping Service) differ in that WFS is supposed to supply just features, and WMS supplies visualizations. Typically, this has meant WMS supplies raster images – though at least one format, SVG, is still just ‘text’. Therefore, one may assume KML should be part of WFS – but instead it is being made part of WMS.

Personally, I think this distinction is tedious, and serves to confuse users. I understand that separating the two services makes it easier for developers, but this is an inverted approach and demonstrates that OGC standards are derived by developers and not many users.

Back to styling

Anyways, it is still apparent that styling is a key element of KML, and one that is desired to be solidified and strengthened in the future. There have been comments that while styling is useful, KML currently mingles features and style too closely. For example, to style a line in KML, you first define the style in your KML document head, but then have to explicitly reference that style from within the Placemark element.

KML does allow you to refer to another KML document for the style – so you could have a mapstyles.kml that you reference from your KML Placemarks. However you are fairly limited in what you can do with this styling and have to explicitly define it. Additionally, you can’t give cascading styling to your KML entities that might be grouped into Folders.

Therefore, the committee discussed looking at existing styling formats for ideas and possible use in KML. SLD (Styled Layer Descriptor) is the OGC standard for specifying the visualization and styling of features from OGC web services.

However, the thread is “Agile Geography”, and SLD+GML is not really consumable by the types of developers that are interested in KML. Another option that was discussed is adopting CSS3.

CSS offers a lot of very intriguing features that would benefit KML. The simple effect would be separating styling from the geometry. In KML 2.1, any Feature (abstract element for other KML elements) can have an id. Subsequently, KML3 could add a class attribute, like in HTML, that could loosely be used to give some categorization of the Feature (road, highway, animal, newt). Then, the CSS could reference these id, class, or even use selectors to apply styling to elements.

By way of example. Note, this is completely rough and is only an example of what might be possible with some sort of cascading-styling

roadstyles.kss

<kstyle>
    Placemark {
      font-size: 8px;
    }
    #roads {
      color: black;
      width: 2px;
    }
    .primary {
        color: blue;
        width: 4px;
    }
    .secondary {
      width: 1px;
    }
    Placemark[num_lanes=2] {
      width: 2px;
    }
</kstyle>

virginia_roads.kml

<kml>
  <styleUrl>http://mygeosite.com/roadstyles.kss</styleUrl>
  <Folder id="roads">
    <atom:author>
        <atom:name>VDOT</atom:name>
    </atom:author>
    <Placemark class="primary" id="I66">
      <gml:Point>
        <gml:pos>42, -83, 100</gml:pos>
      </gml:Point>
    </Placemark>
  </Folder>
</kml>

With my “web developer” hat, this seems really attractive. Using a language and markup I already know and have tools for. It also provides some really powerful capabilities for applying hierarchical styling, both through selectors, and also the ability to apply multiple stylesheets to a KML document. So there could be a stylesheet that is linked to by the document for roads, another for POI’s, and then I could apply my own default in my KML client application.

You may notice the Placemark[num_lanes=2]. CSS3 supports applying styles based on evaluating attributes of elements. So a CSS like this may allow you to easily create thematic maps based on the value of the metadata. However, we still need to figure out exactly how that would mesh with the Metadata module concept (coming up in future blog post).

There is still work to do evaluating the existing KML as well as SLD and some sort of CSS. Now would be a good time to speak up with your thoughts and use cases for how you would to style your data.

kml, Document, atom, Folder, Placemark, Link (and NetworkLink?), ScreenOverlay, TimeStamp, Geometry (see below)

This list was derived from the current KML 2.1 tags and signifies the general concepts we’re hoping to have in KML Core – allow a user to express location, time, and possibly link to another KML document or service. In addition, providing the current Atom support allows for attribution and other interesting things for linking to HTML and GeoRSS.

Geometry Markup

A couple of nearly given plans were laid out for for revisions to KML. The first is the markup of the geometry. OGC is well known for GML, and there are many tools that publish and consume GML. Having yet another geometry markup seems to be redundant and confusing. Therefore, KML 3 will use a minimal version of GML, GML Simple Features Profile. You can follow that link to the specification – but good luck finding an actual simple example or description of what it is.

To summarize, here is an example of a KML 2.2 point, and a GML Simple point:

<Point><coordinates>lon, lat[, alt]</coordinates></Point>        

<gml:Point><gml:pos>lat lon [alt]</gml:pos></gml:Point>

It’s mostly a simple markup difference. There is also the possibility that the gml: namespace prefix would be dropped and just be part of KML. GML provides for specifying the coordinate reference system (CRS), but for KML it is currently assumed to be EPSG4326 – that is to say, what you probably normally use for your map applications and off of your GPS receiver. Later I’ll put up a full comparison between all the current KML geometry markup and its comparable markup in GML.

Being a fan of simple markups myself (see GeoRSS Simple), as a developer I also understand that having a single (and simple) markup makes implementation and maintenance easier. KML already had embedded tags for locations and used a GML 2 like markup – so switching to GML 3 is fairly straight-forward.

Tags: ogc, kml, kml3, standards, formats, gml, features, kmlogc, ogckml

Committee example documents will be posted on the OGC Network page in the “Mass Market” (now known as Agile Geography) section. The KML page lists current examples that we’re working on, so you can follow along there in addition to this blog.

First, the title of this post is “KML 3″, however, it’s not clear yet if this will be 3, or 2.x. Really, the point is moot, and probably in the end be more reflective of how much a change this first revision is. So don’t digress too much on my use of “KML 3″.

Breaking down to build up

One of the primary topics discussed, and nominally agreed upon, was how to define KML such that developers can build capabilities for certain parts of KML without having to implement support for the entire specification. This is important based on the OGC’s normal method of operation and use. By providing a spec, developers are expected to implement that entire spec and then get a stamp of approval that they meet the spec requirements. Then, users of the clients or libraries can feel confident that the features they expect are in fact implemented, and implemented correctly.

KML has a large number of features, from simple geometry markup, to 3-D models, image pyramids, lookAt, and so on. Expecting all KML clients to implement all parts is unreasonable. The simplest demonstration of that is comparing current KML clients: mobile, web map, 3D ’spinny globe’. There are certain aspects of KML that make sense on some of the clients, but not on others.

For other formats, such as GML, the OGC has first defined a full super-set of the spec, and then defined profiles, which are defined subsets of the full spec. From my (and the committees) perspective this can be confusing to developers and users because the first time they go and read up on GML, they get the 400+ page specifications document, balk, and walk-away (or have mild breakdowns).

Instead, for KML, we’re planning on defining a core minimal set for KML, and then put the other functionality into modules. Applications and clients can then use modules that are appropriate to their application.

To illustrate, a simple phone may just want locations and some simple metadata (or attributes) of that location. For example, nearby restaurants and their opening hours. As the client (or library) becomes richer then additional modules can be implemented. A Weblog extension may support styling (icons, line colors) for publishing KML from posts.

So, applications will be able to simply define what modules they support so that users can clearly understand the type of functionality they can expect. In addition, these libraries and applications can achieve OGC verification by implementing the defined feature set in the modules they use.

So what’s in each of these modules? I’ll discuss each of them in successive blog posts (to make them all more readable and keep any discussion grouped together).

In the talk announcing KML submission to OGC, the phrase “KML is HTML for the geospatial web” was used. I think this is a perfectly apt analogy. To complete the analogy, (taking you back to your SAT days)

HTML : RSS :: KML : GeoRSS

The purpose of the following examples is to illustrate the use of GeoRSS to syndicate KML content, similar to RSS support in HTML. GeoRSS merely adds a geographic component to the common RSS that syndicates web content. KML similarly offers a geospatial alternative representation of HTML content.

Syndication

Therefore, as appropriate, GeoRSS should be used based on existing usages of RSS. This means providing a rel=”alternate” link in both HTML and KML files. Conversely, an Atom or RSS feed requires use of a ‘link’ element to reference the originating content of the syndication. A rel=”alternate” can then be used in a GeoRSS feed to reference the KML version of that content.

Within a GeoRSS file, the entire feed, or channel, can link to the content in type=”text/html” and type=”application/vnd.google-earth.kml+xml”. Within a single entry, the GeoRSS feed can then link to the specific anchor, or element, within the original HTML or KML content using an anchor in the href.

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>KML GeoRSS Example</title>
  <link href="http://example.org/"/>
  <updated>2007-06-04T12:34:02Z</updated>
  <author>
    <name>Andrew Turner</name>
  </author>
  <id>urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6</id>
  <link rel="alternate" type="test/html"
    href="http://example.org/"/>
  <link rel="alternate" type="application/vnd.google-earth.kml+xml"
    href="http://example.org/kml_georss.kml"/>

  <entry>
    <title>This is my first article</title>
    <link href="http://example.org/2007/06/04/article"/>
    <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
    <updated>2003-12-13T18:30:02Z</updated>
    <summary>
      KML is the HTML of the geospatial web, used to visualize content, and GeoRSS is the syndication of that content. 

      While you're here, make sure to check out my <a href="http://example.org/myspots.kml#favoritepark" type="application/vnd.google-earth.kml+xml kml">favorite park</a>.
    </summary>
    <link rel="alternate" type="application/vnd.google-earth.kml+xml"
     href="http://example.org/kml_georss.kml#article1"/>
  </entry>
</feed>

Documentation

Within an HTML file, the <head> element can contain alternate links to the KML representation as well as the GeoRSS syndication. Similarly, KML 2.2 can contain an <atom:link> alternate representation back to the original HTML content and GeoRSS syndication of the geospatial information. By using these links, a resource can offer multiple formats as appropriate, syndication for alerting users of updates to the content, or visualization for a richer geospatial view.

For a specific KML Placemark, an <atom:link> can be used to link to the original HTML element using the anchor support in the href. Conversely, the HTML element, similar to a GeoRSS item, can use an <a> element to link to the specific KML Placemark representation. This similarly allows for the documentation element to reference the richer visualization of the feature. Clicking this link would either launch a KML viewer, or the original HTML renderer could render the KML element inline, within the browser.

<html>
<head>
  <title>KML GeoRSS Example</title>
  <link rel="alternate" type="application/rss+xml"
    href="http://example.org/kml_georss.rss" title="GeoRSS feed for My Page">
  <link rel="alternate" type="application/vnd.google-earth.kml+xml"
    href="http://example.org/kml_georss.kml"/>
</head>

<body id="kml_georss">
    <h1>Example of KML, GeoRSS, and HTML being used together</h1>

    <h2 id="article1">This is my first article</h2>
    <p>KML is the HTML of the geospatial web, used to visualize content, and GeoRSS is the syndication of that content. 

      While you're here, make sure to check out my <a href="http://example.org/myspots.kml#favoritepark"    type="application/vnd.google-earth.kml+xml">favorite park</a>.
    </p>

</body>
</html>

Notice that by using the type attribute in the <a> we can link to other KML elements, similar to an HTML document linking to elements in other HTML documents.

Visualization

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.2" xmlns:atom="http://www.w3.org/2005/Atom">
<Document>
  <name>kml_georss.kml</name>
  <atom:link type="text/html"
    href="http://example.com" title="My Page"></atom:link>
  <atom:link rel="alternate" type="application/rss+xml"
    href="http://example.com/kml_georss.rss" title="GeoRSS feed for My Page"></atom:link>
  <Placemark id="article1">
    <name>This is my first article</name>
    <atom:author>
      <atom:name>Andrew Turner</atom:name>
    </atom:author>
    <atom:link href="http://example.org/kml_georss.html#article1"></atom:link>
    <description>
      KML is the HTML of the geospatial web, used to visualize content, and GeoRSS is the syndication of that content. 

      While you're here, make sure to check out my <a href="http://example.org/myspots.kml#favoritepark" type="application/vnd.google-earth.kml+xml">favorite park</a>.
    </description>
    <Point>
      <coordinates>-122.370533,37.823842,0</coordinates>
    </Point>
  </Placemark>
</Document>
</kml>

Together, these suggestions allow for parallel use of KML and HTML for content, and GeoRSS for syndication (notification of updates) to that content. It keeps the GeoRSS as a pure markup of the new information, and keeps styling information in HTML and KML.

Here is a spiffy diagram to help illustrate linking the documents and articles together:

A last consideration about linking HTML content and KML content. Both formats are merely markups for storing and rendering information and not bound to a specific renderer (e.g. Firefox or GoogleEarth). However, it is still beneficial to link to a richer geospatial markup from an HTML file. <a> elements can optionally utilize the type attribute to specify the link follows to a KML file. It should explicitly noted that opening a link to a KML type should not necessarily open in GoogleEarth. Again, KML is just a richer geospatial markup of information.

For example, if a user is viewing a location from a GeoRSS file loaded in a 2D map representation, and link in the content points to a KML type, clicking this link should first open the location in the same 2D map representation. If the link is clicked in an HTML file, the appropriate MIME-type handler should be invoked. This is similar to a user clicking a link in an HTML document and opening the link in the HTML renderer.

Depending on the 2D map rendered, an optional "view in 3D" link could be offered to open the Placemark in GoogleEarth or other KML renderer.

OpenSearch

This proposal is just a suggestion, and doesn't require a change to any of the included formats. In fact, it is just a recommendation for a common usage of these formats to provide for better user experience and linking together your content. Additionally, my previous discussion on OpenSearch Geo fits perfectly with this as you can specify search mechanism for the same content-types. So if you just want the recent changes, request the RSS. Then you could use those results (page) to query specific KML rich placemarks for visualizing.