Some of those who read my blog may have noticed that, while I have been covering the developments around the Open Government Directive until its last deadline (April 7), I’ve been remarkably silent afterwards.
What I have been doing over the last couple of weeks has been to try to assess the many open government plans and judge their alignment with the respective agencies’ business as well as their longer term sustainability (i.e. rather than the mere compliance with what required by the directive).
During this effort – I reviewed about ten plans, coming up with my very personal evaluation grid – I found out that the US federal CIO and CTO at the White House were doing something very similar, by defining a list of 30 criteria for agencies to self assess against. Although their list is different, it is a good list and, besides few of the usual “have you read the directive” kind of criteria, it tries to look at the right elements. More recently, the White House published an updated open government scorecard that includes these self assessments: details about each are available in their respective open government pages.
We all know that self assessments leave ample room for discretion and interpretation. However the very first criteria I picked to see how it had been applied by any two agencies revealed how wide the gap is between different interpretations.
Let me share a concrete example .
One criteria that intrigues me a lot concerns the process to select high-value data, something I have been covering in earlier posts.
I picked the following criteria from the White House list
9. Does the plan identify key audiences for information and their needs, and the agency endeavors to publish high-value information for each of those audiences in the most accessible forms and formats?
which contributes to the category “Transparency Strategic Action Plan”.
I checked this for two different agencies: Department of Agriculture (USDA) and Department of Transportation (DOT). Let me state very clarify that I picked these two organizations for no particular reason (actually, both have pretty decent open government plans). They only happened to have respectively a “red” and a “green” status for that particular category in their self assessment.
The USDA plan says:
A calendar of new datasets and tools through December 2010 is under development. The calendar of complete datasets and tools will be available by the end of June 2010. USDA will also continue to solicit input from the public, federal agencies, and stakeholders on identification of high-value data and information that can be posted as high-value datasets.
So yes, that’s true, they do not really “identify key audiences etc”, so they gave themselves a “red”.
The DOT rates itself as “green”, but – although there are several pages on the open data process – the only sentences that really addresses the criteria above are the following:
Some examples of high-value data sets that are publicly available now, but not in open formats, include data on safety defects, car recalls, transit ridership, selected air carrier data, and selected transportation fatality data. These data sets will be considered in the data inventory prioritization process that is currently underway.
The DOT will complete a comprehensive Department-wide data inventory, to support the data set selection and release process, by September 30, 2010. After completing this inventory, the DOT will establish timelines for publication of appropriate information not yet available for download in open formats and set specific target dates for release. Once those target dates are formalized, they will be included in the next iteration of the DOT Open Government Plan.
In addition to creating a process for releasing data from the Department and developing a data inventory, the Department will also develop a method to prioritize data sets for release. We are considering enhancing usability by also indicating whether a high-value data set was previously unavailable, available only with a FOIA request, available only for purchase, or available but in a less user-friendly format.
In other terms, the DOT plan (dated April 7) recognizes that a process needs to be established, but says nothing about how that process will look like. More, the first sentence says that there is a prioritization process in place, while the third sentence says that it will be developed. Don’t get me wrong: the DOT plan is very specific about how to deal with open data (see the entire section 3.1), but when it comes to identify in the key audience and prioritize, it falls as short as the USDA plan. yet, it is self-rated as “green”.
[NOTE: The DOT published a revised plan dated April 30 - same as this post and later than the White House assessment - which addresses this particular criteria].
As the US CIO and CTO stated in a recent blog post, “more work remains to be done”. They based their judgment on the self assessments and in fact pointed to only three agencies – including DOT – as exemplary for having a green status in each criteria. However, as my little example just showed, one can read results and assessments in very different ways.
My personal assessment of open government plans was looking more at the potential for a sustainable implementation rather than completeness. in an area as new as open government, sometimes less is better than more, and a high-level plan with very little process may have better chances than one that looks fully fledged, and may not prove flexible enough to adapt to the many uncertainties ahead (such as continuous management support, actual use of misuse of open data, and so forth).
There is no doubt that the directive has spurred a lot of very good activity. It is time to share practices but also recognize that agencies are different and pursue different missions and goals: openness must have different flavors and emphasis to add rather than detract from their ability to deliver on their mission.
I am pretty sure that the US CIO and CTO have this very clear in their mind. In fact the best words in that post are those that are nor written. While the URL for the blog post is http://www.whitehouse.gov/blog/2010/04/26/honest-assessment-open-government-initiatives, the title is “An Initial Assessment of Open Government Plans”. The difference between “honest” and “initial” is a very important one, and looks like a Freudian slip. URLs usual carry the initial title with which the post was created. Therefore somebody at the White House realized that this is just an initial assessment, and honesty will come later as all agencies as well as the White House gain more experience with all this.