Comments on: Tag Cloud http://blogs.gartner.com/whit_andrews/2008/10/29/tag-cloud/ A member of the Gartner Blog Network Mon, 08 Feb 2010 22:13:19 +0000 hourly 1 http://wordpress.org/?v=3.0.4 By: Whit Andrews http://blogs.gartner.com/whit_andrews/2008/10/29/tag-cloud/comment-page-1/#comment-62 Whit Andrews Wed, 05 Nov 2008 16:22:13 +0000 http://blogs.gartner.com/whit_andrews/?p=100#comment-62 This absolutely is a problem that will not go away; I could not agree more. I think that Dan Tunkelang's point is well-taken, and I am also a believer in transparency as a critical element of future relevancy calculations, especially those in which the intention is to exploit the apparently naive but possibly malicious or soimply calculated behaviors of passionate users. However, as we know, most people decline to invest the efoort in understanding a system such the transparency may be allowed to work effectively. So, as a result, the default settings are allowed to dominate, no matter what. Now, what we have here is the ability to alter the default settings at a very high level, if one can make this work. Dan Sholler, on the other hand, is of course also right. Tagging, however, is even more vulnerable than other kinds of link manipulation. Google bombing requires technical sophistication (you don't think so? explain it to your friend who is not interested in how it works). Piggyback attacks require some sophistication, but overload attacks would be trivially easy for someone with enough interested, or compensated, minions. My intention here is to point out that any roomful of perfect new solutions for relevancy are always vulnerable insofar as they benefit from publicly available tweaking. This absolutely is a problem that will not go away; I could not agree more. I think that Dan Tunkelang’s point is well-taken, and I am also a believer in transparency as a critical element of future relevancy calculations, especially those in which the intention is to exploit the apparently naive but possibly malicious or soimply calculated behaviors of passionate users. However, as we know, most people decline to invest the efoort in understanding a system such the transparency may be allowed to work effectively. So, as a result, the default settings are allowed to dominate, no matter what. Now, what we have here is the ability to alter the default settings at a very high level, if one can make this work.

Dan Sholler, on the other hand, is of course also right. Tagging, however, is even more vulnerable than other kinds of link manipulation. Google bombing requires technical sophistication (you don’t think so? explain it to your friend who is not interested in how it works). Piggyback attacks require some sophistication, but overload attacks would be trivially easy for someone with enough interested, or compensated, minions.

My intention here is to point out that any roomful of perfect new solutions for relevancy are always vulnerable insofar as they benefit from publicly available tweaking.

]]> By: Dan Sholler http://blogs.gartner.com/whit_andrews/2008/10/29/tag-cloud/comment-page-1/#comment-61 Dan Sholler Thu, 30 Oct 2008 15:33:48 +0000 http://blogs.gartner.com/whit_andrews/?p=100#comment-61 Whit, I do need to point out that this problem is not just with tags, but with the entire notion of the web and hypermedia. After all, a tag is just an abstraction for a list of links (all the things that have that tag). this is even how the resources are represented if you get them from del.ici.ous or whoever. The real issue is that links have no machine readable distinctions. In fact, much of the business of the internet (search optimization in particular ) are heuristic attempts to create those kinds of distinctions. Semantic web supposedly gives us a mechanism for attaching metadata to those links, but that only gives the mechanism, and the models still need to be created (possibly by using the same techniques we do today). In the end, any of these techniques are fundamentally statistical, and we all know that the statistics lie when the data set is too small. This is what happens at the long tail, and nothing anyone does can change that. Mr. Tunkelang above suggests that we abandon the statistical techniques for what is essentially a deterministic one, in which each individual controls the level of influence of the various inputs that could be used. this has some appeal as a means of allowing user input to improve the results, but it still relies on the underlying statistical techniques (unless the user is planning to spend a lot of time choosing which of the elements on the list for the del.ici.ous tag "programming" are relevant to his or her current search...) Since the information volumes are so great, these kinds of manual approaches usually cannot work, or they restrict the results to only those items that have been manually vetted. So basically, we end up with two choices: either we use a statistical technique that is pretty much assured of screwing up in the long tail, or we use a manual technique which is assured of being very limited in coverage. Sorry Whit, this one is a problem that IMHO will not go away. Whit,
I do need to point out that this problem is not just with tags, but with the entire notion of the web and hypermedia. After all, a tag is just an abstraction for a list of links (all the things that have that tag). this is even how the resources are represented if you get them from del.ici.ous or whoever. The real issue is that links have no machine readable distinctions. In fact, much of the business of the internet (search optimization in particular ) are heuristic attempts to create those kinds of distinctions.

Semantic web supposedly gives us a mechanism for attaching metadata to those links, but that only gives the mechanism, and the models still need to be created (possibly by using the same techniques we do today). In the end, any of these techniques are fundamentally statistical, and we all know that the statistics lie when the data set is too small. This is what happens at the long tail, and nothing anyone does can change that.

Mr. Tunkelang above suggests that we abandon the statistical techniques for what is essentially a deterministic one, in which each individual controls the level of influence of the various inputs that could be used. this has some appeal as a means of allowing user input to improve the results, but it still relies on the underlying statistical techniques (unless the user is planning to spend a lot of time choosing which of the elements on the list for the del.ici.ous tag “programming” are relevant to his or her current search…) Since the information volumes are so great, these kinds of manual approaches usually cannot work, or they restrict the results to only those items that have been manually vetted.

So basically, we end up with two choices: either we use a statistical technique that is pretty much assured of screwing up in the long tail, or we use a manual technique which is assured of being very limited in coverage.

Sorry Whit, this one is a problem that IMHO will not go away.

]]> By: Daniel Tunkelang http://blogs.gartner.com/whit_andrews/2008/10/29/tag-cloud/comment-page-1/#comment-60 Daniel Tunkelang Thu, 30 Oct 2008 02:41:36 +0000 http://blogs.gartner.com/whit_andrews/?p=100#comment-60 Whit, as you know, I'm a big fan of pointing out the damage done by the adversarial nature of "objective" relevance functions, which is just the academic way of talking about the hostile ecosystem. What we really need--on the web as much as in the enterprise--is transparency and user control. These principles also apply to tagging. Take away the anonymity or quasi-anonymity of tagging, and give users control over which tags affect the user experience. Given that transparency and control, I'll only consider the tags of users I trust to have non-spammy motives and ideally good judgment / taste. We'll get there, even if it's only when the current model collapses under its own weight. Whit, as you know, I’m a big fan of pointing out the damage done by the adversarial nature of “objective” relevance functions, which is just the academic way of talking about the hostile ecosystem. What we really need–on the web as much as in the enterprise–is transparency and user control.

These principles also apply to tagging. Take away the anonymity or quasi-anonymity of tagging, and give users control over which tags affect the user experience. Given that transparency and control, I’ll only consider the tags of users I trust to have non-spammy motives and ideally good judgment / taste. We’ll get there, even if it’s only when the current model collapses under its own weight.

]]>