Back to Gartner for Marketers Blog

Five Reasons Your Social Analytics Are (Probably) All Wrong

By Martin Kihn | October 21, 2014 | 9 Comments

It hurts me more than it hurts you for me to say this, amigos:

Your social marketing measurement may be downright antisocial.

First, let me ask you a question: Does this look familiar?


Picture1

Hold that thought.

Now, social marketing measurement here means information pulled from common social media listening tools and not firewalled data from your owned channels, e.g., your brand’s own Facebook page. Ninjas among you are aware of flaws, biases and — ahem — issues inherent herein, but a majority of digital marketing analytics consumers may not be.

(While we’re on the topic, let me mention that my colleague Jennifer Polk and I will be hosting a free webinar this Thursday 10/23 at 11am and 1pm Eastern. Details at the Gartner for Marketing Leaders homepage (scroll to the bottom).)

So here are five non-obvious reasons to interrogate the truthiness of your current social marketing analytics dashboards, reports, white papers, assumptions, content marketing efforts, and periodic self-congratulations:

  1. They Include Only Public Posts — While we all feel we live in a world where “nothing is private,” in fact most of the internet is sitting behind some firewall or other and is not (legally) available for us to see. Social networks adhere to privacy policies and social listening tools are limited to interrogating public posts. They are like the ultimate wallflower, a guy going to Facebook or LinkedIn without a single Friend or Contact. By some estimates up to 90% of the conversation about brands on some channels is not public. Forums and wikis, where the most intelligent discussion can happen, are often walled. Many blogs are not public by default.
  2. So They Way Overemphasize Twitter — On the other hand, there is one very popular social network that is almost entirely, fabulously, gloriously public — that is, Twitter. Almost any tweet on anything is captured in all its poorly spelled nuance by your social monitoring tool. Which explains the pie chart above and has given Twitter what is in my humble opinion a vastly overinflated sense of its own place in the world.
  3. Which May Be the Least Unbiased Channel — As I’ve said elsewhere, there are biases inherent in Twitter to bear in mind. To be reductive about it, much of the brand-related discourse on this social network is irrelevant (for example, quoted rap lyrics containing a brand name), and much of it is whimsical. It is a bimodal network, attracting the high and low end of the conversation spectrum. It’s a fast-twitch environment, given to rants reflecting states that rapidly pass. In short, it’s no focus group, people.
  4. They Treat All Channels and Actions Equally — Twitter benefits here as well, as most social dashboards I’ve seen commit the crime of counting each tweet as “1,” each post as “1,” each photo share on Instagram as “1,” each pin on Pinterest as “1” — you get the picture. But a tweet can be (and is, usually) dashed off between elevator rides, while a pin requires some thought, and even a Facebook post inspires us to pause and reflect. Social objects are not created equal. I’ve seen comments in blog posts that are like little essays, complete with footnotes. I’ve seen one-character tweets that make no sense at all. So have you.
  5. Which Ignores Reality, Really — Getting us to the point, at last: Social measurement should actually try to measure something. It should strive to be a representation, an analogy, of actual human behavior and conversation related to your project. If a tweet is — on average — one-third as long as a Facebook post for your brand, then treat it accordingly: weight it 0.33X. If hitting “Like” takes about one-tenth as long as writing a comment, treat it accordingly: weight it 0.1X. (I favor the time-based principle of engagement here, with weights reflecting how long actions take, but there are other approaches.)

So think about adjusting for the above. How? You can weight, weight and weight again. Weight for public vs private posts in channels (to the extent you can estimate these, which I’ll admit isn’t easy). Weight for channels and for engagements within channels. A principle I use here is mentioned in #5. Use your own methods, including a hunch.

As usual, almost anything may be an improvement on business as usual.

Comments are closed

9 Comments

  • You said “Social measurement should actually try to measure something.”

    Agreed, preferably something that’s truly meaningful.

    I crave meaningful insights on “buyer sentiment” — has their opinion changed up/down for the product or service in question? However, primary market research seems to be the only valid way to gain that type of data. Analytics from most online tools doesn’t capture that buyer intent, based upon positive or negative shifts in sentiment.

  • Brian Poe says:

    Great post, Marty! There’s so much chatter around ‘social listening’ so I’m glad to see some research to backup just how little information we have access to as marketers. Marketers like to be the infamous fly on a wall, listening to conversations. It just so happens that we’re flies on one wall of the 7,000 room MGM Grand hotel.

    • Martin Kihn says:

      Thanks Brian – I like your MGM Grand allusion (although I prefer The Cosmopolitan – better restaurants). The thing that makes social analytics so difficult is the social part of it — people are just so darn casual in their online styles!

  • revolooshon says:

    nice story martin – good to see someone hosing down the hype for a change.

  • Sergio says:

    Correlating time spent with length of message seems dangerous. Pascal once wrote: “I would have written a shorter letter, but I did not have the time.”

    • Martin Kihn says:

      Good quote Sergio – certainly a well-written piece of prose is worth savoring for a while. Sadly it’s rare in social networks.

  • Ronny Max says:

    Great post! Successful analytics starts with understanding what the assumptions behind the data. The concept of Data Discovery is just about avoiding false and duplicate data, but also refining what exactly the data suppose to represent.

  • John Walsh says:

    Marty,

    Nice post. The problem that you are speaking to is that social is just one channel or source of information and it can be a low-value or dirty source. So to get to the heart of what is being said, or to your #5 point, you must combine data sources with social. For example let’s say you are a consumer products company and want to gain insight into how your products are being reviewed. First keep in mind social is dirty and its not very effective for individual products and requires a lot of cleaning. So this CPG must combine all customer experience sources; surveys, call center (chat, email and calls) along with social. The other sources I mentioned hold higher value data and must be included.

    By doing this you will get a true sense of the voice of the customer. Which , at the end of the day, is why you are listening in the first place!!!!

  • Avinash Joshi says:

    Great post Martin. Resoinnates perfectly well with me. I have been working with weights but it always is complex to add/allocate weights across a grid of platforms, content, engagements/conversations, sentiment, influence and the works. Would be great if you could share more on the same.