Washington, DC
Subscribe to GeoRSS Subscribe to KML


Open Data needs Local Analysis

Published in Government


Open Data exists for a purpose. From point of capture, to publication and analysis, data seek to be used to make better decisions. By making the data open, more people can participate in that analysis and decision making process. Particular to government and community, the more people can understand, collaborate and reach consensus, the better the likely outcome.

To support this goal of open data a few of us are starting a new group in DC that will focus on very specific initiatives and issues related to our city and use data and analysis to gain insight and hopefully provide effective solutions. Each month we will choose a particular issue and dive deep into that issue to understand the current state, historical precedence, objectives, policies and start a data-driven dialogue. To guide our explorations, a government representative will introduce the topic, share current work and plans, and be available to answer questions from the group during and after the gathering.

Our first meeting is this Thursday, October 30 and we are focusing on DC’s VisionZero plan to reduce pedestrian and cyclist fatalities to none by 2017. Jonathan Rogers from DC Department of Transportation (DDOT) will be our government and data expert. In preparation for this meeting, DC OCTO has made available a number of open datasets such as the last 8 years of bicycle crashes, bike lanes, bike routes, and crowd-sourced locations of unsafe cycle conditions around the city. You can find the list and contributions in our github repository

If you are interested in data as a hobby or profession, want to apply your analysis skills to data science the heck of this data to help your fellow citizens, please join us! After this initial meeting we plan to migrate through the city, hosting events in each Ward at the local library so that we can make this collaboration inclusive to all citizens and focus on local community issues.

And if you live in another city or town, I encourage you to start something similar focusing on local issues. Ideally, working at the convergence of government, analysts, technologists, and citizens means we can give open data the purpose it wants to achieve.

Professional Civic Engineering

Published in Government, Technology, Web

“Unruly Nature”

Roads are particular engineering feat that deeply affect our regular lives yet pass by largely unnoticed. So effectively designed and implemented, we typically only notice them through relative minor, but annoying, failures such as potholes or flooding. Periodically, a major catastrophe reminds us the nature and importance of these infrastructure components and the imperative to design well, maintain regularly, and replace when necessary.

There are extremely well established practices for the design of roads, bridges, and nearly all physical infrastructure that compose our built environments in cities and communities. Centuries of practice, wisdom and science have been boiled down to codes and standards that prescribe the design of a road. Civil engineers rarely have the opportunity to truly design a road; more often they receive guidelines to be accomplished based on traffic volumes, load limits, environmental conditions, and available materials. Using processes of checklists, tables, and formulas, they crank through these to determine the basic characteristics of bed depth, width, rebar size and density, curb heights, etc.

Bridge Blueprint

At the beginning of the 20th Century as expansion thrived across the western United States, government agencies were dealing with a multitude of varied permits, maps and engineering plans for new infrastructure. A land surveyor, having just started as a state engineer of Wyoming in 1903 “was immediately confronted by the unruly nature of engineering and land surveying in this vast, largely undeveloped state where hungry prospectors and developers were rushing to gain access to state water for irrigation purposes.”

To address this issue, they developed the National Society of Professional Engineers and the Licensed Professional Engineer.

A century ago, anyone could work as an engineer without proof of competency. In order to protect the public health, safety, and welfare, the first engineering licensure law was enacted in 1907 in Wyoming. Now every state regulates the practice of engineering to ensure public safety by granting only Professional Engineers (PEs) the authority to sign and seal engineering plans and offer their services to the public.

Qualities of a Professional Engineer

PE StampThe Professional Engineer, or PE, is a certification that requires practitioners to prove their capabilities to design within standards, abide by a code of ethics, and have the experience and mentorship of another PE. After at least four years as an active engineer working on projects and passing a rigorous proficiency test, people are then qualified to approve engineering projects.

Today there isn’t a road, building, bridge, or airplane built that has not met with the approval and stamp of a PE. As such, the engineer is putting their name and qualifications on the line that the design and construction meet industry standards and is reliable enough for the public to safely use. This responsibility is important to ensure that our local and national infrastructure incorporate the long history of engineering wisdom to protect safety and ensure operation.

  • Only a licensed engineer may prepare, sign and seal, and submit engineering plans and drawings to a public authority for approval, or seal engineering work for public and private clients.
  • PEs shoulder the responsibility for not only their work, but also for the lives affected by that work and must hold themselves to high ethical standards of practice.

Open Civic Engineering

As I mentioned in my previous post, the government information architecture is a new type of civic infrastructure that citizens and communities increasingly rely upon. Through web and mobile applications, news feeds, and online forms we use the internet as a primary tool for engaging with city services. Fortunately API’s and other open information architecture developers can create unique and novel applications to improve community livelihood, visiting tourists, and growing business opportunities.

However, beyond novel applications, as a civic technology community we are now building new tools that support basic services such as 311, housing, emergency response and safety. If these services are an integral part of our society, should we now expect the same level of quality and stability we do of our roads and buildings?

Recent history is replete with application contests, prototype apps, local civic hacks, and even startup or large-scale companies that built technology that did not sustainably scale to the region it was meant to serve. Is this something to be expected or should we be developing a code of conduct and patterns by which we guide and even enforce quality and long-term maintenance of this new class of infrastructure?

Civic Imagination

“The Street finds its own uses for things— uses the manufacturers never imagined.” – William Gibson

CrisisCommons HackingThis is not meant to curb the surge of energy and innovation that has also entered government through the rapid pace of technology development. The potential for new ideas to quickly emerge and evolve can dramatically improve civil society across every level of government. If government is the platform, then it is imperative to leverage this platform to build applications into the hands of real users. Government serves the long-term requirements of citizens but technology has the capability to address emergent needs and interfaces. By contrast we’ve likely all experienced attempting to use a government website that states our browser is ‘too new’ and may not work with the application.

So we need to balance this unmitigated rush with consideration for the impact and expectations it will have from these very real people with real expectations. What is a good balance between “cowboy coding” and “stagnant kludge”.

Towards a code of conduct

Fortunately, I believe that the field of open citizen development is maturing. Organizations such as Code for America, Sunlight Foundation, OKFN, and agencies like the new 18F and UK GDS provide the forms to professionalize ‘civic hacking’ and develop codes of conduct for volunteers and companies that can address the issues noted above.


(All the jokes aside) using standards is the first necessary step to an operational and sustainable civil application. This means more than just JSON (which is an encoding) and more about the actual schema and structure. There are numerous existing standards, and increasing a set of commonly used and understood standards. Civic organizations should develop guidance on the baseline required standards and the optional additional standards to use.

In some case it makes sense to develop or evolve standards – but consider how this impacts existing tools and how to build community and adoption with new and existing applications.


Few things are better than real experience. The wisdom gained from successes and failures provide the best guidance for building reliable applications. Each day a new developer joins the community, excited to build tools for themselves and their neighbor. Beyond simple “how to program” we should be providing the mentorship on technology architecture, testing, usability, government processes and accessibility. The Professional Engineer requires at least four years of mentorship and their approval for your acceptance of the license. What is the equivalent “merit badge” to demonstrate the acquired skills to appropriately design, develop and operate these technologies?

Operational Plans

We build applications with the awareness that they have a lifespan. We will conceive, launch, grow and finally retire everything that we build. Nothing is eternal. At the beginning we should design into our applications the long-term maintenance and final transition plan. What is the plan to scale or grow to meet new requirements? How does data gathered be preserved and migrate to the next generation application so that we don’t lose years of valuable information and history. These don’t need to be the ultimate plan, but should include the consideration and plan that will evolve as much as the application does itself. But it is a maturity to plan for that final obsolesce on day one and as part of the entire engineering design.

TechCamp Ramllah.jpg

TechCamp Ramallah – mentoring the next generation of Civic Engineers

Today, anyone with a computer and a bright idea can build an application to improve the lives of citizens. As a community, we should work together to ensure their idea meets the needs of those citizens and can grow to become part of the broader civic technical platform. If you’re at this week’s Code for America Summit I would love to chat about this topic – or feel free to reach me directly here, on Twitter or via email.

Desire Paths to Open Data

Published in Government, Technology

Last week I made a quick statement sharing my concern for civic organizations promoting ETL – Extract Transform and Load – of open data instead of developing APIs. I felt it warranted a more thorough response than the terseness of microbursts.

Desire Lines and Road Surfaces

Walk through most parks and any college campus you will quickly notice that dirt worn pathways that connect between the sidewalks indicating pedestrian shortcuts. These desire lines indicate an initial and repeated optimization that lay outside the paved paths. Often these ad-hoc networks are a type of ‘footstream’ that are adopted and paved – or they are left to individual use, muddy in rain and undocumented or supported through groundskeeping. At scale this often explains entire city and county or even national road networks that started as ‘cowpaths’ and through continued and growing usage became official infrastructure – roads and highways – which are relied upon as a matter of business.

Desire Paths - Virginia Tech - DigitalGlobe.png

This road network is the infrastructure that government develops and promises to support as a necessary mechanism for citizens to build communities and businesses to operate commerce. Information infrastructure is the next generation that government is developing which increasingly becomes the relied upon and required tools for community and commerce. Tim O’Reilly has referred to “Government as a Platform” which means that we must be able to rely on these services as a durable backbone where which we build our numerous and diverse applications.

Opening Data: Prototype or Infrastructure

Open data started as simple file sharing. In my own city the data catalog was a large and easy to read list of datasets with metadata, links to common formats, and updated dates. Through a series of public contests, developers used these file downloads to build some compelling applications to highlight the future of Government IT. In 2008 Apps for Democracy was (link dead) and (link dead) and again in 2009, Apps for America winner was (n.b. link is dead).

It should be apparent that these contests and applications were interesting desire lines that did not provide or sustain a platform of information which citizens could rely on. Albeit just simple examples, they are indicative of the tendency to build simple one-time applications which unfortunately miss that next step of becoming part of the platform they seek to improve. I’ve heard similar examples from other cities where civic hackers created well-meaning and well-built applications but that sit so far outside the existing government operations that they require continued manual maintenance by these unpaid volunteers with the common outcome that the service stops updating (maybe while even still operating, arguably a worse condition than simply shutting down).

Unfortunately, even the original data catalog has slowly atrophied. Based on my own experience when looking for more recent crime data (the data catalog stops about September 2013) I learned that the internal system was being migrated and the transformation process had faltered and it just wasn’t a priority to get the system back online. It was too removed from their actual job of analyzing and responding to crime to make a separate feed available with any defined timeline.

Federal Highway Intersection.jpeg

Which is a stark reminder that despite the amazing capabilities technology can deliver, Government is foremost responsible for serving people, not serving technology. Everything it does in the end is to serve the communities that elect, fund, and generally are employed by, these governments. When most civil engineers are designing roads they don’t apply grandiose design aesthetics and creativity. They pull open their codes & standards, determine the appropriate concrete mixture, depth and rebars based on specs, and get to work developing the road that fits the expected and reliable operations that citizens need.

Operational Open Data is Sustainable Open Data

There are numerous studies, reports, case studies and general community practice have made the case that open data has a great potential benefits to civilian and business communities. Not least of which is the ability for government agencies to more easily share data between one another in addition to improving business efficiency and consumer decision making.

Government has a difficult and extremely important job. As an entity, it is not enamored with new techniques or formats. Attempting to create unidirectional bifurcations of the data create strain which will eventually give way when there is any pressure: time, fiscal, or personnel. New technologies need to understand these processes and costs in order to align themselves if they are to ever become part of the government platform.

For open data to move from a shortcut to part of the stable infrastructure, we need to design it from the beginning to be practical, sustainable and ultimately operational. Open data needs to be the way government operates, and it needs to be part of the living systems that manage and process the data as part of day-to-day business.

To the original question, generalized techniques such as ETL – extract, transform and load – have tremendous flexibility to explore new paths and opportunities. By enabling freedom to explore applications, new formats, and communities, government can observe and understand these desire lines. It can then make the decision if these paths become part of the supported network or if they indicate a necessary redesign of the large system to accommodate these concepts. Personally I’ve seen, and built, many ETL tools and community applications that worked on the outside of government. While fast moving an extremely agile, they ultimately are untenable to provide ongoing and durable platforms for information access.

By comparison we should be encouraging and working directly with government technical staff to specify and prototype API – application programming interface as they provide an excellent mechanism for this prototype to adoption. By developing an external interface to a service the provider is making a contract with end-users that is independent of the implementation details. This enables a developer to use their tools of choice with the intent that it could be rebuilt within government infrastructure while maintaining the promised interface that applications already rely upon. And finally like the observing the increasing depth and width of a dirt path the measured analytics behind and API help prioritize incorporation and operationalization.

Exemplar of this has most recently been the DCAT distributed catalog specification. Neither new nor novel as far as federated data catalogs are concerned, it was an API that was created in conjunction with technologists and government agencies and adopted independent of any technology implementation that is now poised to easily share links to data between numerous national and local government agencies, all in the public. Instead of building more data harvesters, an API means that anyone can both participate in production as well as usage of the open data however best fits their needs.

Desire to Collaboratively Craft

Perhaps the most exciting thing I have observed in my six years living in DC and watching the Open Government movement surge has been the positive growth and excitement of people within government to actively and publicly collaborate. More than merely publishing a catalog and running a competition, government representatives are eager to talk about ideas, share code and data, and hear where they can open their infrastructure for these types of creative developments.

While much of the commercial web is becoming ‘appified’ (and often eschewing access via common or open APIs), perhaps this is one case where it’s superb the government moves more slowly and is just now entering the time of the programmable web. For many of us who volunteer our time and expertise hoping to improve the civil societies in which we live, the best thing we can do is work closely to advise and create the best platform possible.

Where2.0 that matters

Published in Government, Neogeography, Where2.0

Last night I spoke at Ignite Where2.0. The community and ecosystem of Where2.0 continues to utilize cutting-edge technology to provide consumer and business services and needs. You can locate activities, friends, stores, media and more and have it integrated into mobile lives and online personas.

These are all great advancements, and are blurring the lines between the online digital data and our interaction with the real world. However it’s vital that we realize the real potential application of these technologies and what our legacy is on the entire world. How can we engage with global citizens, understand their needs and desires, and collaborate on building channels of information and tools that serve our individual and collective goals.

Almost two years ago I moved from Michigan, with stints in California, to Washington, DC. I moved at an auspicious time in our nation as the highly contentious presidential election approached at the same time concerns on transparent monitoring of democratic elections and process loomed. Social media and streams such as twitter, smartphones, voice technology and visualization provided the components to demonstrate how we can enable citizens to share their experiences, their problems, and for us to openly see problems and victories as they occurred.

This same concept applies just a well around the world. Open platforms such as Ushahidi have helped bring citizen reporting in elections in India, Libya, Iraq, and Afghanistan – each to different outcomes – but still in a way that harbinges a more open and transparent government process.

Now through my experiences with CrisisCommons, working with multinational organizations such as the World Bank and the United Nations, and the federal and local governments, it’s clear to see how the leading edge of the Where2.0 community can have an amazing and unparalleled impact in providing understanding and change in global and local issues: Environment change, food security, humanitarian development, education, and disaster response.

In looking at the various open government initiatives, the questions arise in looking past the press release to the realized value of sharing data with businesses and citizens. I was struck my the foresight of the Arkansas AGIO team in the realization of how sharing data as broad and wide as possibly helps mitigate their vulnerability to disaster by enabling responders open access to vital information that would assist in response.

This concept is apparent in how OpenStreetMap was successful in Haiti. With the lack of official, government supplied data the best solution was to crowd-source the information from varied sources and rebuild the national data infrastructure, external to the government itself. While it has been unpredictably successful, the value continues to be the open access of the data by any and all organizations, and the eventual adoption by the government itself in rebuilding its capacity. The hope is that the government continues to openly collaborate with the global community in managing and maintaining this data so that the situation doesn’t need to reoccur.

In summary, the community is making a difference. The tools we develop in WhereCamp, IRC, open-source communities, and from companies are changing the capabilities of crisis response and development. My message is to urge the larger community to continue to think how their solutions can have a more broad impact.

If your technology can help a consumer find a great $4 latte, that’s good for your business. If it can also help a child find clean water near their village, that’s good for the world.

Afghanistan Election Monitoring

Published in Government

Today the democratic elections are occurring in Afghanistan. There is a FrontLine SMS on the ground pulling in data into Alive in Afghanistan, which is running Ushahidi.

At FortiusOne, we’re pulling together demographic, violence, and polling information into a dashboard of maps.

Estimated Number of Voters by Polling Location vs Average Risk Level of Polling Centers, 2009 Afghanistan Presidential Election, Eastern Region of Afghanistan (View full atlas)

You can read more about it on USAID’s Global Development Commons, or how we’re partnering with numerous other projects to build out solutions like this for deploying to the field.

The compelling example here is the combination of demographic information with contextually relevant information (polling locations), realtime feeds of data, and user-generated, on the ground observations. Together they are helping provide a common picture into the complex relationships, consequences, and potential outcomes of the election might look like.