Andrea DiMaio

A member of the Gartner Blog Network

Andrea Di Maio
Managing VP
15 years at Gartner
28 years IT industry

Andrea Di Maio is a managing vice president for public sector in Gartner Research, covering government and education. His personal research focus is on digital government strategies strategies, Web 2.0, open government, cloud computing, the business value of IT, smart cities, and the impact of technology on the future of government Read Full Bio

Coverage Areas:

A Freudian Slip in Assessing Open Government Plans

by Andrea Di Maio  |  April 30, 2010  |  4 Comments

Some of those who read my blog may have noticed that, while I have been covering the developments around the Open Government Directive until its last deadline (April 7), I’ve been remarkably silent afterwards.

What I have been doing over the last couple of weeks has been to try to assess the many open government plans and judge their alignment with the respective agencies’ business as well as their longer term sustainability (i.e. rather than the mere compliance with what required by the directive).

During this effort – I reviewed about ten plans, coming up with my very personal evaluation grid – I found out that the US federal CIO and CTO at the White House were doing something very similar, by defining a list of 30 criteria for agencies to self assess against. Although their list is different, it is a good list and, besides few of the usual “have you read the directive” kind of criteria, it tries to look at the right elements. More recently, the White House published an updated open government scorecard that includes these self assessments: details about each are available in their respective open government pages.

We all know that self assessments leave ample room for discretion and interpretation. However the very first criteria I picked to see how it had been applied by any two agencies revealed how wide the gap is between different interpretations.

Let me share a concrete example .

One criteria that intrigues me a lot concerns the process to select high-value data, something I have been covering in earlier posts.

I picked the following criteria from the White House list

9. Does the plan identify key audiences for information and their needs, and the agency endeavors to publish high-value information for each of those audiences in the most accessible forms and formats?

which contributes to the category “Transparency Strategic Action Plan”.

I checked this for two different agencies: Department of Agriculture (USDA) and Department of Transportation (DOT). Let me state very clarify that I picked these two organizations for no particular reason (actually, both have pretty decent open government plans). They only happened to have respectively a “red” and a “green” status for that particular category in their self assessment.

The USDA plan says:

A calendar of new datasets and tools through December 2010 is under development. The calendar of complete datasets and tools will be available by the end of June 2010. USDA will also continue to solicit input from the public, federal agencies, and stakeholders on identification of high-value data and information that can be posted as high-value datasets.

So yes, that’s true, they do not really “identify key audiences etc”, so they gave themselves a “red”.

The DOT rates itself as “green”, but – although there are several pages on the open data process – the only sentences that really addresses the criteria above are the following:

Some examples of high-value data sets that are publicly available now, but not in open formats, include data on safety defects, car recalls, transit ridership, selected air carrier data, and selected transportation fatality data. These data sets will be considered in the data inventory prioritization process that is currently underway.

The DOT will complete a comprehensive Department-wide data inventory, to support the data set selection and release process, by September 30, 2010. After completing this inventory, the DOT will establish timelines for publication of appropriate information not yet available for download in open formats and set specific target dates for release. Once those target dates are formalized, they will be included in the next iteration of the DOT Open Government Plan.

In addition to creating a process for releasing data from the Department and developing a data inventory, the Department will also develop a method to prioritize data sets for release. We are considering enhancing usability by also indicating whether a high-value data set was previously unavailable, available only with a FOIA request, available only for purchase, or available but in a less user-friendly format.

In other terms, the DOT plan (dated April 7) recognizes that a process needs to be established, but says nothing about how that process will look like. More, the first sentence says that there is a prioritization process in place, while the third sentence says that it will be developed. Don’t get me wrong: the DOT plan is very specific about how to deal with open data (see the entire section 3.1), but when it comes to identify in the key audience and prioritize, it falls as short as the USDA plan. yet, it is self-rated as “green”.

[NOTE: The DOT published a revised plan dated April 30 - same as this post and later than the White House assessment - which addresses this particular criteria].

As the US CIO and CTO stated in a recent blog post, “more work remains to be done”. They based their judgment on the self assessments and in fact pointed to only three  agencies – including DOT – as exemplary for having a green status in each criteria. However, as my little example just showed, one can read results and assessments in very different ways.

My personal assessment of open government plans was looking more at the potential for a sustainable implementation rather than completeness. in an area as new as open government, sometimes  less is better than more, and a high-level plan with very little process may have better chances than one that looks fully fledged, and may not prove flexible enough to adapt to the many uncertainties ahead (such as continuous management support, actual use of misuse of open data, and so forth).

There is no doubt that the directive has spurred a lot of very good activity. It is time to share practices but also recognize that agencies are different and pursue different missions and goals: openness must have different flavors and emphasis to add rather than detract from their ability to deliver on their mission.

I am pretty sure that the US CIO and CTO have this very clear in their mind. In fact the best words in that post are those that are nor written. While the URL for the blog post is http://www.whitehouse.gov/blog/2010/04/26/honest-assessment-open-government-initiatives, the title is “An Initial Assessment of Open Government Plans”. The difference between “honest” and “initial” is a very important one, and looks like a Freudian slip. URLs usual carry the initial title with which the post was created. Therefore somebody at the White House realized that this is just an initial assessment, and honesty will come later as all agencies as well as the White House gain more experience with all this.

4 Comments »

Category: open government data     Tags:

4 responses so far ↓

  • 1 Todd Solomon   April 30, 2010 at 2:49 pm

    Andrea, Though I have an obvious bias, I think that “green” was warranted. DOT is exercising particular care to discern its various audiences and target data to those audiences. The challenge is, of course, in the variety. DOT communities consist of people whose interests are in roads and rails, ports and pipelines, transit and trucks, aviation and automobiles, as well as waterways, non-motorized transportation, and a ton of research.

    That said, if you were to cite more of section 3.1.2 in version 1.1 of DOT’s plan, readers would find DOT going out of its way to find high-value information:
    “DOT’s data inventory could contain many types of structured and unstructured information, including, but not limited to, XML data sets and comprehensive reports to external stakeholders (i.e., Congress). The DOT releases numerous comprehensive reports on its websites at varying timeframes. These reports assemble internal DOT data as well as data from state and local departments of transportation or other public entities. Consistent with completing our data inventory, DOT will identify cases in which we provide public information in electronic format, where the underlying data is not exposed. DOT will work with its Operating Administrations, where the data, subject matter expertise and analytical capability reside within the DOT, to develop a process whereby the release of the raw data contained in these reports is concurrently released to data.gov.”

    Further, if you cited section 3.1.4, readers would find a DOT explicitly identifying key audience groups:
    ” The format of the data plays a role in its usability. The Department will work to strike an appropriate balance between formats (such as XML) that serve particular user groups and Web based presentations that can be used and understood by the general public.
    To ensure that the DOT properly accounts for the wide variety of audiences that may consume or review its data, the DOT will work to break down its external stakeholders into like groups. Such groups may include, but are not limited to:
    • Transportation-related Professional Organizations (e.g., American Association of State Highway and Transportation Officials (AASHTO), American Society of Civil Engineers (ASCE)); • Academia, through the University Transportation Centers (UTCs); • The general public; • Application developers and entrepreneurs; and • Engaged public organizations, typically availing themselves of the Freedom of Information Act (FOIA) request process.
    The DOT will continue to identify these unique external stakeholder groups and key audiences and will leverage the collaboration tools outlined elsewhere in this plan to gain a better understanding of their information needs. The DOT will ensure that the process for identifying and releasing high-value information explicitly considers format and usability with respect to user groups. The DOT will complete this activity after the completion of the comprehensive Department-wide data inventory, to support the data set selection and release process. One way to get such feedback about format and usability will be through the 5-star rating system on Data.gov.”

    Whether it deserves its ‘green’ or not, your readers need to know that DOT is clearly looking to publish high-end data in a usable form for as many of its core audiences as it can discern.

  • 2 Andrea Di Maio   April 30, 2010 at 5:09 pm

    Todd, thanks for your thoughtful comment and for pointing out this inconsistency. Actually when I wrote the post I assumed that the assessment referred to the earlier version of the plan (dated April 7), since the White House blog mentioning the scorecard is dated April 27, and Version 1.1 of the DOT plan is dated April 30.
    I guess that version 1.1 was in the making during your assessment, which is why you considered that criteria well addressed. From my perspective, I just felt it was a bit fairer to have the same version assessed across different agencies.
    What really matters is that – as this shows – your open government plan is a living document and evolves as you receive stimuli from outside the department. This is definitely a good symptom for your plan’s sustainability. As I said in my post, also version 1.0 was a good basis, and I believe this is the case for most agencies, in spite of the different length, depth and approach of different plans.
    This time the stimulus has come from within government and in particular from those who are behind the directive itself. As you may know if you have read previous posts on this topic, one of the concerns I have is that a great idea (open government) turns into a compliance exercise: while common assessment criteria serve the purpose of giving everybody a basis for improvement, I am also convinced that the nature, pace and role of open government will and must be different for different agencies.
    The DOT plan, like many others, is built for evolution and for incorporating multiple inputs: I am very much looking forward to when a new version will be triggered by compelling internal department requirements or by the “wisdom of the crowd”, rather than by a set of criteria from the White House.

  • 3 Australia Beats the US Again on Government 2.0   May 3, 2010 at 2:14 pm

    [...] greater promise, now the Australian government’s response appears less than one month after the issuance of the many open government plans by US federal agencies. US initiatives are clearly skewed toward government being in control of [...]

  • 4 Ranking Open Government Plans: What’s the Point?   May 12, 2010 at 9:31 am

    [...] probably less concerned with openness as a critical attribute for their mission. This is something I already observed when looking at the White House own [...]