This is Not a Product Test
By Andrew Frank | October 16, 2008 | 9 Comments
A couple of weeks ago I posted a challenge to the social media monitor community to see if they were listening. They were.
The validity of this test has been challenged. In fact, there was a problem, and Marcel from Radian6 diagnosed it correctly in a comment on David’s blog:
Part of the problem is that they published the same post on two different blogs. Andrew published it on his own blog and then it also was published on the Gartner Media Industry blog about 30 minutes later (if I recall correctly). Some companies posted comments on one blog and not the other, some on both. The Media Industry blog comments appeared to have been moderated, but no indication was given to the commenter that they were moderated – that is why people re-posted their comments (myself included) as it looked like the comment didn’t work. Andrew’s blog, however, worked fine. I suspect that Andrew’s post might have even be automatically cross posted as Gartner does seem to do this, and so perhaps no one was paying close attention to the Gartner blog for incoming comments – given the nature of the post.
I don’t think any harm came from it and it seems to me that Andrew was just making it a general test (and not part of his research) as per his question, "do they eat their own dog food". He later posted on the subject saying that it appears they do.
This was a case of bad timing. Gartner was in the midst of transitioning from our old blog (where the post received 44 responses) and the new blog (where the post received 19 comments, some of which were duplicates as Marcel notes). It was our practice to cross-post to the two blogs at the time, and I apologize to anyone for whom this caused confusion.
So here are some key takeaways:
- Social media monitors do work, which was the main point of the test.
- Beware of technical issues and timing when conducting blog experiments (even informal ones)
I’m in transit right now but will post the full list of all who responded on either blog next – I want to make sure to get this right – and if I miss anyone, I’m sure I’ll hear about it.
UPDATE: This list is now available here.
Thanks again to everyone who responded, and again sorry to any who felt their voice was not heard.