Andrew Frank

A member of the Gartner Blog Network

Andrew Frank
Research VP
5 years at Gartner
30 years IT industry

Andrew Frank specializes in best practices for data-driven marketing, including how organizations can use data to drive sales, loyalty, innovation and other business goals. Andrew also specializes in marketing and advertising technology and business trends …Read Full Bio

Coverage Areas:

This is Not a Product Test

by Andrew Frank  |  October 16, 2008  |  10 Comments

A couple of weeks ago I posted a challenge to the social media monitor community to see if they were listening. They were.

The validity of this test has been challenged. In fact, there was a problem, and Marcel from Radian6 diagnosed it correctly in a comment on David’s blog:

Part of the problem is that they published the same post on two different blogs. Andrew published it on his own blog and then it also was published on the Gartner Media Industry blog about 30 minutes later (if I recall correctly). Some companies posted comments on one blog and not the other, some on both. The Media Industry blog comments appeared to have been moderated, but no indication was given to the commenter that they were moderated – that is why people re-posted their comments (myself included) as it looked like the comment didn’t work. Andrew’s blog, however, worked fine. I suspect that Andrew’s post might have even be automatically cross posted as Gartner does seem to do this, and so perhaps no one was paying close attention to the Gartner blog for incoming comments – given the nature of the post.

I don’t think any harm came from it and it seems to me that Andrew was just making it a general test (and not part of his research) as per his question, "do they eat their own dog food". He later posted on the subject saying that it appears they do.

This was a case of bad timing. Gartner was in the midst of transitioning from our old blog (where the post received 44 responses) and the new blog (where the post received 19 comments, some of which were duplicates as Marcel notes). It was our practice to cross-post to the two blogs at the time, and I apologize to anyone for whom this caused confusion.

So here are some key takeaways:

  1. Social media monitors do work, which was the main point of the test.
  2. Beware of technical issues and timing when conducting blog experiments (even informal ones)

I’m in transit right now but will post the full list of all who responded on either blog next – I want to make sure to get this right – and if I miss anyone, I’m sure I’ll hear about it.

UPDATE: This list is now available here.

Thanks again to everyone who responded, and again sorry to any who felt their voice was not heard.


Category: Uncategorized     Tags:

10 responses so far ↓

  • 1 Marcel LeBrun   October 16, 2008 at 1:32 pm

    Great response, Andrew. It shows that Gartner is listening too!

  • 2 Valerie Combs   October 16, 2008 at 1:48 pm

    Looks like my comment on David’s blog just missed this – well put Andrew!


  • 3 Blake Cahill   October 16, 2008 at 2:16 pm

    VT is glad to see Gartner becoming more active in this space.

    Visible Technologies

  • 4 Alecia O'Brien   October 16, 2008 at 4:35 pm

    Good for you Andrew!


  • 5 David Meerman Scott   October 16, 2008 at 4:37 pm


    Many thanks for your clarification to my blog post. I understand the situation and the clarification is important for the companies affected.

    Your follow-up post will be very valuable and I look forward to seeing it.

    Best, David

  • 6 Scott Hepburn   October 16, 2008 at 4:48 pm

    Well done on the response post, Andrew. While David’s post and the ensuing comments probably weren’t the feedback you were hoping for, I think your fair and measured response speaks volumes about Gartner.

    Unintended “snafus” happen to all of us — thanks for handling it with poise.

  • 7 Diane Thieke   October 16, 2008 at 4:52 pm

    Hi Andrew,

    Thanks for the explanation. I, too, am happy to see Gartner becoming more active. At Dow Jones Insight (formerly Factiva Insight), we’re looking forward to joining this conversation.

    Dow Jones

  • 8 What’s Working in Marketing » Social Media Test Brings Gartner Mea Culpa   October 16, 2008 at 6:00 pm

    [...] Well, to make a long story short, Gartner has stepped up to the plate and acknowledged the problem and explained what happened.  Apparently there was a technology problem.  Here’s Gartner’s repsonse in David’s blog: Here I am, and here’s my response: [...]

  • 9 Adam Zand @NoOneYouKnow   October 17, 2008 at 1:30 pm

    An interesting experiment and I’m glad you acknowledged how Gartner and you erred on David Meerman Scott’s blog: Side note: why did it take Gartner six hours to find your detractors?

    Speed and recognition is important, but the more telling point for business is how they respond. It is great that all the vendors listed (and some you missed) were able to play the hide, seek and comment exercise. However, the truly progressive companies have the technology, but more importantly the intelligence to respond, engage, solve problems and add value.

    Maybe you can set up a new challenge for the vendors to provide a free strategy or tool that benefits the social web and wider community? I can think of plenty of home owners, nonprofits or even impoverished plumbers who could benefit from the brainpower collected in your comments section.

  • 10 Andrew Frank   October 17, 2008 at 1:42 pm


    You wrote
    > Side note: why did it take Gartner six hours to find your detractors?

    It didn’t. It took that amount of time to respond and that’s because this happened while I was at Symposium where I was either presenting or interacting with clients in 1on1 sessions from 7am until 12:30pm when I had to head to the airport. I did the post from there. This seems to be a general hazard for busy people with personal blogs.

    I like your “free tool” idea, although I suspect that there’s currently more pressure among vendors of all types to secure revenue.