Prediction Is Hard – Especially About the Future

OK, I admit it – I stole the title from a much smarter man. I thought that man was Yogi Berra, but maybe not – more about that at the end of this post.

Every year, Gartner issues a series of Predicts documents. This year I had the pleasure of doing one for my team on Information Infrastructure Technology. Now, I’m a software guy, and the team I’m on is all software people, so a document assigned to our team would typically be about – well, information software technology. But that would have missed the point rather dramatically, so I connected with a few colleagues and got their OK to use some of their predictions in the small set any document can include. (After that I had to argue with our methodology people about having them appear in two places, but that’s a story you don’t want to hear.)

It was an enjoyable exercise, all things considered, and I was able to collect predictions in several domains I think will have dramatic effects on the information management space. Here are the Planning Assumptions I featured:

• By 2017, IMDBMS as a differentiated DBMS market category will disappear. In-memory will be a “permission to play” capability for DBMS. (Teri Palanca) 
• By 2018, HDDs will still account for 75% to 85% of all petabytes shipped to the server and ECB storage markets, but solid-state solutions will expand to account for 15% to 25% of all mission-critical, near-line and archived data. (John Monroe and Joseph Unsworth.)
• Through 2020, there will be no dominant IoT ecosystem platform; IT leaders will still need to compose solutions from multiple providers. (Al Velosa)
• By 2018, 30% of streaming, near-real-time data integration and data management use cases will be supported by stacks that include Apache Spark. (Lakshmi Randall, Nick Heudecker and me.) 
• Through 2018, 90% of deployed data lakes will be useless as they are overwhelmed with information assets captured for uncertain use cases. (Regina Casonato, Nick Heudecker, Mark Beyer, and me again.)

The list, confined as it was to 5 predictions, leaves out many important topics: cloud, security, Hadoop (my favorite, if you follow my research), emerging categories now collected under NoSQL, the economic impact of open source, the expiration of Oracle’s consent decree on MySQL, and more. But it’s not a bad list, and it touches on memory, storage, the IoT, software and process and governance issues. So I’m pretty happy with it. Gartner clients can find the full document here.

What about Yogi? Most Valuable Player of the American League three times (only 3 other people ever did that) and he got to the World Series as a manager in both leagues. As player, coach, or manager, he appeared in 21 World Series, winning 13 of them (thanks, Wikipedia, for the fast facts). He was a hero of mine as a kid, and learning of his wit and wisdom over the years has only enhanced that. But it appears he did not originate the quote. A little work with Bing found me a whole raft of investigations that ascribe the quote to Danish folk wisdom, even to Neils Bohr. But that’s OK – I’ll always link it to Yogi. After all, even he said (and I’m sure of this one) “I never said most of the things I said.”