Blog post

Open Government Data May Be Wrong. So What?

By Andrea Di Maio | March 15, 2011 | 6 Comments

open government data

I can’t resist the temptation of saying “I told you so” to the many supporters of open government that I pissed off repeatedly by underlying open government’s challenges and limitations (see here and here for the latest controversial posts in a long series), after reading an article on NextGov about widespread errors on USASpending.gov, the web site that provides transparency about budgets and spending for the US government. The article goes on highlighting problems with Recovery.gov and Data.gov (for the latter questioning that data is not sufficiently high value for potential users).

Everybody knows that data quality is a key issue with open government, and something that is seen as a considerable cost and risk factor. Of course people do expect government to provide perfectly accurate and up-to-date data, and we all know that when that’s not the case, government becomes and easy target for criticism.

But what if we take a different spin to all this and try to use open government to solve data accuracy problems rather than making them bigger. In the same article, the executive director of the SunLight Foundation, points to a web site where they track those mistakes. This constitutes an essential resource to provide feedback to government. Also, with the same data being available in different ways (e.g. through aggregated information stores like USASpending.gov, individual agency web sites, private enterprises, advocacy groups and so forth), there is no reason why one of them should be officially tagged as the single source of truth, as opposed to using them all to build a more accurate (or less inaccurate) version of the truth.

As I have said so many times, open government should focus more on helping government do its job better – including improving data accuracy – than on creating a platform for as-yet-unspecified players to create new services and value.

We should not consider these mistakes as unforgivable sins, but as a natural part of the process where the “crowd” (professionals, citizens, other agencies) collaborate on making data collectively better.

Open government will succeed if we will accept mistakes as part of the process and their correction as a joint and not just a government responsibility.

Comments are closed

6 Comments

  • Maurizio Morselli says:

    I wasn’t aware that there is such a thing as an “open government”…anywhere?

    The real data will continue to be manipulated and sanitized for the masses and we will never know the truth.

  • Alex Howard says:

    “open government should focus more on helping government do its job better – including improving data accuracy – than on creating a platform for as-yet-unspecified players to create new services and value.”

    These aren’t binary choices. By presenting them this way, you ignore the existence of quite specified players that do exist in healthcare and transit. You also neatly avoid acknowledging that part of the reason you’ve “pissed people off” lies in your rhetorical approach of incendiary headlines that are drawn from the playbook of a blogger at Gawker, not Gartner, rather than the body of analysis itself.

    Accurate data can and does drive accountability, civic utility and economic value. Look at how HHS is moving forward with HeathData.gov.

    I wonder: decades ago, would you have posted analysis that GPS data shouldn’t be released by government for use by as yet unspecified players? How about weather data? How much value would have been lost — how much of the vibrant mobile sector would work — if those decisions hadn’t been made?

    All that being said, your final point is one I’ve heard from many, many people in the open government world: these efforts will only work if trying, failing and rapidly iterating is part of the expectations for new initiatives and services.

  • @Alex – Thanks for your thoughtful comments.
    I do not think I said it is an either-or (you should know me by now). As usual it is a matter of balance. The main thrust of open government (or gov 2.0) is still centered around the government-as-a-platform, and I do understand you support the corporate messaging (i.e. government as a platform is king), but I appreciate that you also have a nuanced view of things.
    Accurate data is desirable but is not necessarily achieved. Where it is not, let’s make sure that interested parties chime in and help improve. I do acknowledge what is happening in some domains, but how examples are there?
    What my post says is NOT that data should not be released to the public. On the contrary, it should even earlier, engaging people in improving that data. And I am so happy I am stating the obvious in my last paragraph: as Latins said “repetita iuvant” and – incidentally – do not forget that people in the open government world are still a tiny minority of all people in the government world.
    Finally, I can’t resist to note that I have – at least partially – “pissed you off” again. As I think I explained, my blogging is personal, fully-fledged analysis is for our clients only. However my headlines are intentional. It seems that articles like the one that triggered this post recognized things that I have been saying for quite some time, already causing a few eyebrows in the “open government world” to raise.

  • Alex Howard says:

    You wrote that open government data efforts should focus on [x] rather than [y]. That’s binary, despite your comment.

    You have not “pissed me off.” Implying as much cheapens your reply in the same way that “I told you so” did your lede. If you had, I would have cited the reasons why and it would show up in my commentary.

    The issues with government data are important, relevant and are beging highlighted during Sunshine Week. (You might blog about what’s happening there, too, if you are interested in what’s happening in that space). Healthy debate and useful analysis of those issues is both relevant and useful to those who are entrusted with improving the situation or holding those doing to account.

    Focusing on improving service delivery is important. It’s unfortunate that you framed the discussion this way, as it muddles the real issues for readers looking for insight. It also means that those people in federal, state and local governments that are working to open data will be faced with managers armed with a Gartner blog post that implying that there are “as-yet-unspecified players to create new services and value” from such data. As you know well, given that you say you’ve been following my reporting, that is not the case.

    On an unrelated note, I find it somewhat ironic that someone whose analysis repeatedly emphasizes the lack of barrier between work/personal use of blogs or social media for government continues to claim that a blog hosted at blogs.gartner.com only reflects your own “personal opinion.” If it’s personal, why is it hosted here? Do you believe that no one won’t quote you from here as “Andrew Di Maio, Gartner government analyst?”

    If I post to my personal blog or a half a dozen media outlets, it comes back to me. That’s the Web we live in now, despite this evidence of your desire to have it be otherwise.

  • Alex, I am glad you were not irritated by my earlier post. Different people have different ways to express their dissatisfaction with a position, and I thought that your references to my “sensationalistic” titles were a sign of you being displeased.
    Let me clear the ground from the personal vs professional ambiguity. As you point out, it would really be weird if I missed the inextricable link between the two, with all my predicament on how to work “at the boundary”. What I meant, in response to your observation that my analysis was incomplete or “sloppy”, was to highlight that fully fledged analysis in the form of research notes is something that I make available to our clients, and can’t be found on a blog. However my positions on this blog, despite being personal as our disclaimer recites, are fully in line with those in my Gartner research. At times I take a more extreme view to stimulate the dialogue, or may challenge some established Gartner positions (an example could be how I see the blend of personal vs corporate collaboration tools), but when I do, this is clearly stated and is anyhow covered by the disclaimer.
    Now, back to this conversation. I concede that I should have said “open government data efforts should focus MORE on [x] than [y]”: I was probably carried away by my passion about open gov being imbalanced toward [y].
    I do know very well that open gov is progressing in certain areas, but maintain that it will remain a niche program unless it is rebalanced toward the government’s own benefit.
    Last month I published a Gartner research note listing our clients’ priorities on technology, IT management and business issues for 2011 as well as for the next 3 years. Gov 2.0 and open gov were almost at the bottom of the top ten, clearly subsumed by technologies and approaches that more directly contribute to cost containment and financial sustainability.
    Open government can definitely help in those areas, but needs to be seen as a means to achieve outcomes rather than just as an outcome in itself.
    Once more, it is not an either-or, but a matter of balance

  • Alorza says:

    I agree. Open Gov is not about a Government that is in possesion of the whole truth but rather a Government that is open for citizens to collaborate in the creation of public value.