The main obstacle to big data adoption is not technology but understanding how to seize opportunities from big data. There is a lot to a movie other than the movie itself: cut episodes, its creators, stories around the film and many more. Here are the “cut episodes” from my recent research on big data opportunities.
Fascinating to me is the powerful shift toward openness: open source, open data and open minds make society as a whole more open. Apache Hadoop has emerged as the leading big data solution; more than 1,200 people across 80 companies have contributed since its beginning in 2005. My colleague Lyn Robison will give a talk about open minds “Smart Swarms and Brain Grids: Mobility for Real-Time Collective Intelligence” at the upcoming Catalyst conference in San Diego. The news on open data sprout one bigger than another: the presidential order Making Open and Machine Readable the New Default for Government Information is out. Bill Gates invests $35M in ‘open source’ science.
I am awed daily at the shift toward openness. Just today I spoke with a healthcare provider intended to share its anonymized data with other clinics and hospitals to get better insights at scale (40 million patients combined).
I spoke last week with the executive of a data-driven company, one of the first (if not the very first) in its industry to realize that information is the company’s business advantage. He told me that they made a decision to switch as much as they can to open source, which pushed engineers who were pretty unexcited about their jobs to the open source community. The intellectual exchange inside the community, meeting likeminded people and the healthy challenges of implementing open source completely rejuvenated these people!
On 1st June, 10,000 people in 95 places participated in the first National Day of Civic Hacking to help local and federal governments cope with the data-driven economy. Governments open up — citizens figure out how to make open data and open interfaces work. I went to the City Camp of Palo Alto — that’s what I saw (and a lot more too):
Oahu bus routes, Austin bike share locations, unlocking prison phone data in Western Massachusetts, bringing in Oakland online answers to questions like How do I apply for a Business License? are just random examples of opening the government data for civic coding. This also illustrates the shift toward borderless organizations.
Companies choose open source as their business strategy. One of the most impressive recent vendor briefings about a truly big idea was with OpenCoin, a company building an open currency system with the Ripple network, an open source, peer-to-peer payment network.
Follow Svetlana on Twitter @Sve_Sic