Gartner Blog Network


The Disappointment of Social Sentiment Analysis

by Jenny Sussin  |  February 26, 2016  |  2 Comments

This blog post also appears on LinkedIn.

For years – clients have been calling us about social analytics and for years their excitement had been rooted in a particular type of analysis: sentiment analysis.

The promise was that social sentiment analysis could give organizations real time insight into how people felt about a particular marketing campaign, a brand in general, or the customer service they had received. The standard sentiment analysis tools allowed for organizations to see whether social posts were considered to be positive, negative or neutral and more complex tools broke up sentiment into passion and emotion. Additional complexities were addressed – like parsing out phrases within a single post to determine whether one subject of the post was positive while another was negative, without the post being tagged as a resultant neutral.

But some of the largest organizations in the world, working off of some of the most renowned social analytics tools, seem to have soured on social sentiment analysis.

Image by Ryan McGuire

According to these organizations, largely represented by their market insights teams, they discourage their employees and their executives use of sentiment analysis as a definitive measure of success or failure. They say that despite having used multiple tools over the years – and I’m talking the tools which are largely considered to be the leaders in the social analytics space – they have never found sentiment analysis to be particularly accurate.

Image by Ryan McGuire

Perhaps it’s the fault of these vendor’s NLP algorithms, perhaps it’s the simple truth that even people won’t agree on the sentiment of a social media post 100% of the time. It could be western society’s penchant for sarcasm, it could be that there are too many industry specific terms that no vendor could have so many taxonomies for. Maybe our sampling of social data is too biased! Is it our problem for interpreting it wrong or expecting it to work on it’s own without any context? The fact of the matter is that this area has been a massive disappointment to clients and reference customers alike.

Image by Ryan McGuire

If you’re reading this thinking, “well no, it worked for me…” I would love to hear from you in the comments or at my email address jenny.sussin@gartner.com if you need to keep your comments off the record.

Similarly, let me know if you haven’t seen success here and what you think the drivers behind that problem are.

Looking forward to hearing from you!

 

Category: analysis  analytics  sentiment  social-crm  social-media  social-networks  

Tags: analysis  analytics  detection  emotion  facebook  media  negative  networks  neutral  positive  sentiment  social  twitter  

Jenny Sussin
Research Director
6 years with Gartner
7 years IT industry

Jenny Sussin is a Research Director in the ITL Enterprise Software group of Gartner Research, with primary focus on social for CRM. Read Full Bio


Thoughts on The Disappointment of Social Sentiment Analysis


  1. N Singh says:

    Its very important that people understand the difference between correlation and causation. Social media analysis and sentiment analysis is a test of word of mouth about your brand. In the world of data deluge, consumers resort to heuristics. Social media has also led to peer pressure. Both heuristics and peer pressure push ‘most’ of the people to be influenced by sentiments. But there are cynics, rationalists and people with a brain on their shoulders who will not be deterred.
    But, please also note that sentiments influence even stock prices; read animal spirits for that topic.
    So, while it is wonderful Jenny that you raise this interesting question, I believe that a professor of NLP from Stanford, you and an Advertising Guru/ PR person can discuss and argue till the cows come home and yet not come to a ‘statistically significant conclusion’ with a ‘reasonable’ sample set.

  2. Seth Redmore says:

    We at Lexalytics actually agree with the sentiment (hah, I amuse myself) of this article.

    Of course social sentiment by itself is a disappointment. On its own, sentiment isn’t actionable. It is simply a clue. It is like looking at one gauge and saying “gee, that looks high” and stopping there. You need context. You need the “why.” It is our customers (and our customers’ customers) that are using sentiment in a comparative way (change over time, comparing with competitors,etc) combined with other signals from the unstructured text content (and structured data) that are successful in making business decisions using social content.

    Think about this differently – if you’re looking at hotel reviews, are you just going to look at the star ratings and stop, or are you going to actually look to see if the features (like the beds, or the bathroom) are up to your standards, or are you more concerned about the location or the food? How about the star rating vs. the price? I might want filet mignon on a burger budget, but… That’s the difference between just looking at social sentiment and looking at sentiment + business-relevant context.



Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.