"What it does is it erodes confidence in one of the primary tools for planning," said Sherrill Mane, senior VP of products for the Interactive Advertising Bureau. "Those who are making plans and allocating dollars were using a fundamentally reduced number of users of audience to do their plans. That does ultimately translate to lost money for publishers and websites."
Nielsen says methodology flaws appear to account for a 22% drop in reported time-spent year-over-year online. This revelation has implications beyond the tarnish it puts on Nielsen’s reputation.
First, in the long-standing debate between advocates of panel-based versus census-based (i.e., derived from server data) metrics, panel advocates have now been dealt a serious setback. Beyond long URLs (which are typical of the methods used by social media sites like Facebook to encode session data), other issues have come to light, such as excluding users of Google’s Chrome browser due to its use of long URLs for secure purchases and crashing “people meter” software (which panelists install on their computers which monitors Internet use). These are now being piled on to standard criticisms of panel data which charge that these panels are hardly “random samples” to begin with, and thus all of their data should be regarded suspiciously. Both Nielsen and its main competitor, ComScore, have been moving to “hybrid” methods that combine panel data with census data, and this incident only underscores the urgency of this transition.
Second, the incident also highlights the fact that neither Nielsen nor ComScore has received accreditation from the Media Rating Council (MRC), the group charged by the U.S. Congress with overseeing media ratings systems, although audits have been in progress for several years. Given the billions of dollars now being spent on online media advertising, the use of uncertified currency is clearly unsustainable, and may be costing online publishers millions. This, too, must be urgently addressed.
It’s hard to overstate the significance of Nielsen’s role as currency provider to the media industry (and hence, indirectly, in all industries that depend on media for marketing). Nielsen has borne the brunt of much criticism through its history, and many of its recent studies, such as its Three Screen Report and its Video Consumer Mapping Study, have generated some predictable skepticism, as is to be expected whenever a study billed as “independent” lends support for the core business case of its sponsors (in these cases, the conclusion being that Internet video consumption is yet utterly dwarfed by television consumption in the U.S.).
It’s ultimately a positive sign for online publishers and advertisers that ComScore has mounted a significant competitive challenge to Nielsen, and has plans to take that challenge to Nielsen’s home turf of television. It’s clearly in the interest of these publishers and advertisers, through associations such as IAB, OPA, and 4As, to turn up the heat on MRC and the measurement companies to raise standards of transparency in methodologies and the MRC audit and accreditation process.