This blog post also appears on LinkedIn.
For years – clients have been calling us about social analytics and for years their excitement had been rooted in a particular type of analysis: sentiment analysis.
The promise was that social sentiment analysis could give organizations real time insight into how people felt about a particular marketing campaign, a brand in general, or the customer service they had received. The standard sentiment analysis tools allowed for organizations to see whether social posts were considered to be positive, negative or neutral and more complex tools broke up sentiment into passion and emotion. Additional complexities were addressed – like parsing out phrases within a single post to determine whether one subject of the post was positive while another was negative, without the post being tagged as a resultant neutral.
But some of the largest organizations in the world, working off of some of the most renowned social analytics tools, seem to have soured on social sentiment analysis.
According to these organizations, largely represented by their market insights teams, they discourage their employees and their executives use of sentiment analysis as a definitive measure of success or failure. They say that despite having used multiple tools over the years – and I’m talking the tools which are largely considered to be the leaders in the social analytics space – they have never found sentiment analysis to be particularly accurate.
Perhaps it’s the fault of these vendor’s NLP algorithms, perhaps it’s the simple truth that even people won’t agree on the sentiment of a social media post 100% of the time. It could be western society’s penchant for sarcasm, it could be that there are too many industry specific terms that no vendor could have so many taxonomies for. Maybe our sampling of social data is too biased! Is it our problem for interpreting it wrong or expecting it to work on it’s own without any context? The fact of the matter is that this area has been a massive disappointment to clients and reference customers alike.
If you’re reading this thinking, “well no, it worked for me…” I would love to hear from you in the comments or at my email address jenny.sussin@gartner.com if you need to keep your comments off the record.
Similarly, let me know if you haven’t seen success here and what you think the drivers behind that problem are.
Looking forward to hearing from you!
The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.
Comments are closed
2 Comments
Its very important that people understand the difference between correlation and causation. Social media analysis and sentiment analysis is a test of word of mouth about your brand. In the world of data deluge, consumers resort to heuristics. Social media has also led to peer pressure. Both heuristics and peer pressure push ‘most’ of the people to be influenced by sentiments. But there are cynics, rationalists and people with a brain on their shoulders who will not be deterred.
But, please also note that sentiments influence even stock prices; read animal spirits for that topic.
So, while it is wonderful Jenny that you raise this interesting question, I believe that a professor of NLP from Stanford, you and an Advertising Guru/ PR person can discuss and argue till the cows come home and yet not come to a ‘statistically significant conclusion’ with a ‘reasonable’ sample set.
We at Lexalytics actually agree with the sentiment (hah, I amuse myself) of this article.
Of course social sentiment by itself is a disappointment. On its own, sentiment isn’t actionable. It is simply a clue. It is like looking at one gauge and saying “gee, that looks high” and stopping there. You need context. You need the “why.” It is our customers (and our customers’ customers) that are using sentiment in a comparative way (change over time, comparing with competitors,etc) combined with other signals from the unstructured text content (and structured data) that are successful in making business decisions using social content.
Think about this differently – if you’re looking at hotel reviews, are you just going to look at the star ratings and stop, or are you going to actually look to see if the features (like the beds, or the bathroom) are up to your standards, or are you more concerned about the location or the food? How about the star rating vs. the price? I might want filet mignon on a burger budget, but… That’s the difference between just looking at social sentiment and looking at sentiment + business-relevant context.