Google has announced a new web crawler especially for Google News which easily allows news publishers to block their content from Google’s News index, general index or both. Know as Robots Exclusion Protocol (REP), Google is going a step further than the typical block or no block by allowing publishers to block or allow individual news components such as photos or snippets from one or either index (news or general search). It’s doubtful that Google’s move to offer an enhanced REP is in reaction to the recent “news story” in which Microsoft was working with News Corp to block its content from Google, making it exclusively available via Bing. Google’s new effort basically is a means to that end with the subtle message, “good luck with that.”
In homage to the late Paul Harvey, here’s the rest of the story: Google’s forthcoming cloud/content distribution plans dubbed Google Edition suggests a portable, secure storage locker for an individual’s e-books that is accessible across a number of devices surely can extend to news publishers. That virtual storage locker also could contain newspaper and magazine content. As Google develops rich content applications, built in HTML5, for cloud content consumption, a few innovative newspaper readers must be in the works. Google is primed to support any sort of content application it builds or distributes with a variety of business models (ads, ala carte, and subscription) so the company that does not evil hopes to show publishers it is their friend.
The message is clear: if you want to block you content from the most powerful search engine on earth, be our guest. Want to build your own platform for distribution or work with any number of nascent partners (ala Journalism Online) to build paywalls, go right ahead. Google is betting these efforts will fail and that via Google Edition it will play a major role in the content distribution business. At that point, Google no doubt feels newspapers (along with magazines) will line up behind book publishers to ride the cloud into future profitability.