Last week, we held our third annual BI Bake Off at the Gartner Data and Analytics Summit in Dallas, Texas. I’ve been doing bake offs for 10 years now, and this one was by far the biggest with almost 600 people attending. Selection to the live panel is based on attendee interest, with Microsoft, Qlik, SAP, and Tableau participating live, and several vendors participating virtually at their booths and via follow up videos.
Doing Good with Data
We ask the vendors to use a consistent data set for the bake off to facilitate a side-by-side comparison. We try to pick data sets that also allow Gartner and vendors to showcase how the data and analytics community can do good with data. We’ve analyzed homelessness, college debt, and this year, the rise in traffic fatalities using Department of Transportation data (full data here). Some experts blame the increase in fatalities on texting and driving, others on low gas prices. Frankly, all the vendor demos were so good that at points during the bake off, I would have rather paused to study the data than assess the product functionality. Here are some actionable insights from this year’s bake off:
- Distracted driving as a percentage of drivers on the road is actually flat. There are more young drivers (ages 25-30) on the road than older; particularly with the economic
recovery, fewer baby boomers have returned to the roads. There is also a difference in risk of fatality based on type of vehicle and where the passenger is sitting – put those toddlers rear middle! (Findings from Microsoft with link to live Power BI app). My take away: experience matters and perhaps we should focus more on ongoing driving skills for younger drivers.
- The most accidents occur on the weekends in the early mornings (2 a.m.) after people leave the bars. (Do we really need to say, don’t drink and drive? Or certainly be extra alert then, take a cab, an Uber, a Lyft, and let’s find a way to reward those designated drivers!) The evening work commute (4-7 p.m.) is the next worst period for accidents. Wyoming is the state with most accidents by population density; New Mexico the state with most distracted drivers by population density. Traffic fatalities have been on the decline for the last 30 years with a peak in 1979-1980. (Findings from Tableau, with Tableau Public dashboard here).
- The most fatalities are on Saturdays, and in terms of holidays, on Thanksgiving. But we seem to have more related to drunk driving on Fourth of July, at least for certain years (I suspect also weather related). (Qlik)
- SAP found the most traffic fatalities are Labor Day, and that male drivers age 19-29 die the most (did I mention my son just turned 19? The difference in worst holiday for fatalities may be in number of years analyzed). They also showed an animation of how drunk driving has changed over time.
- Teenagers get into accidents more frequently between 4-8 PM (which is right after school hours), whereas drivers who are in their 20s get into accidents more frequently in the very early morning or very late at night. Females aged 50+ don’t get into accidents as much for using cellphones compared to males for the same age range. There are way more accidents as the road and bridge conditions get better, and less accidents when they are bad. People pay more attention and aren’t as distracted as much, when they can visibly comprehend the need to focus on driving as a task. (from MicroStrategy)
- All the vendors identified California, Texas, and Florida as having the highest number of traffic fatalities. By harmonizing with U.S. Census data, ClearStory Data identified counties with high volume of fatal accidents per population including Loving, TX, Slope, ND, and King, TX. The national average ratio of Drunk Driving related accidents to Distracted Driving is 11.3. (so bottom line, texting and driving may be grabbing headlines but we still have a bigger drinking and driving problem, albeit improving); West Virginia is an outlier with a ratio of 127.9, while Mississippi has the lowest drunk driving to distracted driving ratio of 3.8. In 2015, accidents with previous offenses rose with previous Speeding up 43%, DWI up 63%, previous accidents up 54% year over year (so let’s focus our efforts on repeat offenders here). (from ClearStory Data)
- It was interesting to see that in Montana the number of accidents when the driver is sober and when he is drunk is almost equal, whereas in most other states the ratio is around 1:5. Also, distracted driving may be a cause across most of the states, but specifically for the south west states, it is actually “looked but did not see”. (from Dundas)
- Young people have almost as much of an elevated accident risk when they are driving in dark lighting conditions (early mornings when they are driving to school for example) as when they are driving drunk or on drugs (this is one more reason to change high school start times). (Salesforce)
- Although California had the highest fatalities of any state in 1990, it has substantially decreased in 2015. Fatalities in Texas, however, have increased. What changes did California make that other states can learn from? (Envision)
Key Findings on BI and Analytic Capabilities
I intentionally structure these bake offs to reveal each vendor’s strengths and weaknesses, so there is not necessarily a clear winner. It is indeed a matter of the best fit, but as we saw in the bake off, there are varying degrees that prospects are duped, how forthright a vendor is, and if a customer wants out-of-the-box capabilities in a single product or interoperability and extensibility across several products. For example, customers seemed perfectly fine with Microsoft’s approach for storytelling via PowerPoint or alerting via Flow, both separate products from the base Power BI. In the Critical Capabilities note, we mainly score vendors on out of the box capabilities, but do consider functionality to embed and extend as a separate set of capabilities. So here was the other surprise from the bakeoff: although this note– which is a deep dive on product functionality– has existed for three years, the majority (90%) of attendees were not familiar with it (dagger to my heart!). The MQ is still the big kahuna.
A number of customers were surprised how much all of the products have changed in just a year. Indeed, the pace of the industry is frenetic! This also isn’t just the bread and butter capabilities, it’s the cool stuff too. Tableau showed its NLP capabilities and Alexa integration, Qlik conversational analytics with bots and Siri SAP the digital board room and smart data discovery within its cloud product, Microsoft integration of survey data and emotion recognition. In this regard, it was inspiring to see so much innovation from big vendors and start ups alike.
Let me know what you find in the data, or better yet, take some of this advice up in your local community. If you have an idea for other data sets to use in future bake offs, I’d love to hear from you.
Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.