This is Not a Product Test

By Andrew Frank | October 16, 2008 | 9 Comments

A couple of weeks ago I posted a challenge to the social media monitor community to see if they were listening. They were.

The validity of this test has been challenged. In fact, there was a problem, and Marcel from Radian6 diagnosed it correctly in a comment on David’s blog:

Part of the problem is that they published the same post on two different blogs. Andrew published it on his own blog and then it also was published on the Gartner Media Industry blog about 30 minutes later (if I recall correctly). Some companies posted comments on one blog and not the other, some on both. The Media Industry blog comments appeared to have been moderated, but no indication was given to the commenter that they were moderated – that is why people re-posted their comments (myself included) as it looked like the comment didn’t work. Andrew’s blog, however, worked fine. I suspect that Andrew’s post might have even be automatically cross posted as Gartner does seem to do this, and so perhaps no one was paying close attention to the Gartner blog for incoming comments – given the nature of the post.

I don’t think any harm came from it and it seems to me that Andrew was just making it a general test (and not part of his research) as per his question, "do they eat their own dog food". He later posted on the subject saying that it appears they do.

This was a case of bad timing. Gartner was in the midst of transitioning from our old blog (where the post received 44 responses) and the new blog (where the post received 19 comments, some of which were duplicates as Marcel notes). It was our practice to cross-post to the two blogs at the time, and I apologize to anyone for whom this caused confusion.

So here are some key takeaways:

  1. Social media monitors do work, which was the main point of the test.
  2. Beware of technical issues and timing when conducting blog experiments (even informal ones)

I’m in transit right now but will post the full list of all who responded on either blog next – I want to make sure to get this right – and if I miss anyone, I’m sure I’ll hear about it.

UPDATE: This list is now available here.

Thanks again to everyone who responded, and again sorry to any who felt their voice was not heard.

9 Comments
  1. 16 October 2008 at 1:32 pm
    Marcel LeBrun says:

    Great response, Andrew. It shows that Gartner is listening too!
    Marcel
    Radian6

  2. 16 October 2008 at 1:48 pm
    Valerie Combs says:

    Looks like my comment on David’s blog just missed this – well put Andrew!

    valerie

  3. 16 October 2008 at 2:16 pm
    Blake Cahill says:

    VT is glad to see Gartner becoming more active in this space.

    Blake
    Visible Technologies

  4. 16 October 2008 at 4:35 pm
    Alecia O'Brien says:

    Good for you Andrew!

    Alecia
    dna13

  5. 16 October 2008 at 4:37 pm
    David Meerman Scott says:

    Andrew,

    Many thanks for your clarification to my blog post. I understand the situation and the clarification is important for the companies affected.

    Your follow-up post will be very valuable and I look forward to seeing it.

    Best, David

  6. 16 October 2008 at 4:48 pm
    Scott Hepburn says:

    Well done on the response post, Andrew. While David’s post and the ensuing comments probably weren’t the feedback you were hoping for, I think your fair and measured response speaks volumes about Gartner.

    Unintended “snafus” happen to all of us — thanks for handling it with poise.

  7. 16 October 2008 at 4:52 pm
    Diane Thieke says:

    Hi Andrew,

    Thanks for the explanation. I, too, am happy to see Gartner becoming more active. At Dow Jones Insight (formerly Factiva Insight), we’re looking forward to joining this conversation.

    Best,
    Diane
    Dow Jones

  8. 17 October 2008 at 1:30 pm
    Adam Zand @NoOneYouKnow says:

    Andrew,
    An interesting experiment and I’m glad you acknowledged how Gartner and you erred on David Meerman Scott’s blog: http://www.webinknow.com/2008/10/gartner-fails-m.html#comment-135047097. Side note: why did it take Gartner six hours to find your detractors?

    Speed and recognition is important, but the more telling point for business is how they respond. It is great that all the vendors listed (and some you missed) were able to play the hide, seek and comment exercise. However, the truly progressive companies have the technology, but more importantly the intelligence to respond, engage, solve problems and add value.

    Maybe you can set up a new challenge for the vendors to provide a free strategy or tool that benefits the social web and wider community? I can think of plenty of home owners, nonprofits or even impoverished plumbers who could benefit from the brainpower collected in your comments section.

  9. 17 October 2008 at 1:42 pm
    Andrew Frank says:

    Adam,

    You wrote
    > Side note: why did it take Gartner six hours to find your detractors?

    It didn’t. It took that amount of time to respond and that’s because this happened while I was at Symposium where I was either presenting or interacting with clients in 1on1 sessions from 7am until 12:30pm when I had to head to the airport. I did the post from there. This seems to be a general hazard for busy people with personal blogs.

    I like your “free tool” idea, although I suspect that there’s currently more pressure among vendors of all types to secure revenue.

Comments are closed.