How poor metrics undermine digital marketing – The McKinsey Quarterly – How poor metrics undermine digital marketing – Marketing – Digital Marketing October 15, 2008
Posted by Mark Blei in : Uncategorized , add a comment- The digital world has developed faster than the tools needed to measure it. This lag has made it difficult for marketers to fully exploit the Web’s promise as the most targetable and measurable medium in the history of marketing.
- Hobbled by nascent technologies, inconsistent metrics, and a reliance on outdated media models, marketers are failing to tap the Web’s full power. Unless this problem can be addressed, the inability to make accurate measurements of digital advertising’s effectiveness across channels and consumer touch points will continue to promote the misallocation of media budgets and to impede the industry’s growth.
Harsh Times Demand (Even) More Concrete Evidence of Effectiveness September 2, 2008
Posted by Mark Blei in : Uncategorized , add a commentBATAVIA, Ohio (AdAge.com) — Want to know if advertising works? Or how it works? Or what consumers think? Chances are, you’re about to find out as much as or maybe even more than you ever wanted.
A slew of media companies, agencies, research firms and even marketers themselves have ramped up efforts to churn out elaborate studies or research results as selling tools. Reasons vary from the obvious need of media to prove themselves, particularly in tough times, to the fact that data-based pitches have gained currency with the public at large.
By the numbers
+25%
Direct-response TV may be accountable, but it doesn’t always work as planned.
+160%
Average increase in unaided awareness for brands in seven package-goods and fast-food categories in a Yahoo study based on ComScore data
+31%
One-week lift in sales of Campbell’s cream of mushroom soup in April 2006 from a Parade ad in an analysis released late last year from Information Resources Inc.
Of course, it also helps to have the numbers handy because media, agencies and marketers alike are facing a much tougher audience: finance executives.
Only a decade ago, the biggest marketers were run by executives who started their careers in marketing. Today, CEOs of such marketers as Unilever and Kimberly-Clark Corp. come from finance backgrounds. And their joint rival, Procter & Gamble Co., recently has named executives who spent most or all of their careers in finance either as global marketing officers or senior managers over more than a third of the business.
For anyone who wants to crunch numbers, plenty are available.
Parade of data
Parade, after last year commissioning Information Resources Inc. to use marketing-mix models to show sales lift and payback from advertising in the Sunday supplement, is now preparing its third wave of IRI analyses of various brands’ ads and the second wave of using health-care-research firm IMS to do similar analyses of prescription-drug ads, said Mike DeBartolo, exec VP-advertising.
Parade keeps paying for expensive analytics it never used years ago because they help sell ads. Mr. DeBartolo said Parade’s page volume from consumer package-goods, food and over-the-counter drug advertisers is up 36% through July compared to last year after a similar gain in 2007.
Likewise, digital media are finding they have to produce new data to tap a huge pool of media spending that up to now has been largely beyond their grasp, particularly from CPG advertisers.
Direct-response advertisers who’ve been the bread and butter of online media, particularly search, didn’t need studies. They had their sales results.
Not so with CPG advertisers, who generate relatively few sales online and whose abysmally low click-through rates don’t do much to prove effectiveness. So Yahoo, Google and MSN all have ponied up for third-party research this year to prove they’re relevant to the brand marketers.
Searching for results
Google earlier this year released research from Nielsen/Net Ratings and ComScore of an online display and search campaign for Unilever’s Dove deodorant estimating a $530,000 sales lift and 13-percentage-point increase in favorability ratings.
Yahoo recently set out to prove that search had a brand-building impact beyond the clicks, joining MediaVest in a study of 6,000 consumers showing that search ads generated an average 160% increase in unaided brand awareness.
Of course, media have always used data to prove impact. The difference now is that media are having to come up with new types of studies to prove they do more than just reach people, said Joel Rubinson, chief research officer of the Advertising Research Federation.
“Yahoo knows that just focusing on clicks and having an advertising model based on that may be shooting themselves in the foot,” he said, “because there are these other kinds of [branding] benefits.”
He pointed to Microsoft’s “Engagement Mapping” study launched earlier this spring, that works with marketers and agencies including Citi Cards, Sprint, GSD&M Idea City, MindShare and Initiative to demonstrate the cumulative effect of online display or search ads in driving sales rather than just assuming the “last ad clicked” is responsible.
Publishers’ data
For media where marketing-mix models and clicks don’t apply or aren’t practical, proving effectiveness can be harder.
Still, Brenda White, VP-director of print investment at Starcom, is seeing a growing number of publishers come to her with studies from third-party research firms, something the agency has been demanding based on client wishes.
“With the economy the way it is,” she said, “publishers are going to have to do this to hold onto budgets. They’re under increasing pressure from other media, too.”
Since most individual titles aren’t big enough to practically use marketing-mix models, Ms. White sees more publishers using studies from Dynamic Logic or Marketing Evolution that base effectiveness analysis on other metrics, such as changes in brand perceptions.
Similarly, MTV Networks has been looking to build deals around Marketing Evolution’s “return on marketing objectives” system, offering guarantees based on how well campaigns deliver against marketer-chosen objectives such as increasing brand favorability.
Mind candy
Yet it’s not just media getting into studies. So are agencies — such as Ogilvy Action, which recently completed a 14,000-shopper study globally to buttress its standing as a global player in shopper marketing and build interest in the discipline.
Facebook Adopts New Metric for Widgets August 14, 2008
Posted by Mark Blei in : Uncategorized , add a commentCHICAGO Facebook took a step toward standardizing the wild world of Web metrics by switching its engagement rating for applications from reporting daily usage to measuring monthly active users.
The social networking site disclosed the change last week via a company blog aimed at an audience of developers. The goal is to help developers focus on “longer-term engagement” and show them “the number of users who visit [their] application over a longer time frame than just one day,” according to Facebook.
“The new metric will force widget developers to work differently,” said Sonya Chawla, managing director of advertising at Slide, a San Francisco firm that has developed top applications for Facebook like SuperPoke and TopFriends. “A daily count measures a flashy application that does one thing like show a movie preview, for example. A longer time frame [metric] will force widget developers to seek longer term engagement with their widget applications.”
READ THE REST OF THIS ADWEEK ARTICLE BY CLICKING HERE
TO GET MORE GREAT ADWEEK CONTENT INCLUDING NEWSLETTERS CLICK HERE
From the Point Oh! Blogsite November 12, 2007
Posted by Mark Blei in : Uncategorized , add a commentInteresting article at interesting new blog Located at http://point-oh.com/
Point-Oh is Peter Imbres’ blog on marketing and PR
Net Promoter: Buckshot Marketing?
I’ve been meaning to write about net promoter score for a while but it’s one of those big topics that probably needs to be tackled in sections. As I’ve mentioned before, simplicity can often be the key to widespread adoption of a new metric and you can certainly see why NPS appeals to marketers on that level. The only problem is that I really don’t think that every WOM program boils down to “Would you recommend our product/service to a friend?” since WOM interactions are slightly more complex than simple recommendations.
First of all, from a purely statistical standpoint, I highly recommend reading the FREEDyourMind piece on NPS. Among the very valid points that Larry Freed makes are that the margin of error is far too high (he claims an NPS of 24 could be anywhere between 14-34), the scale isn’t clearly defined (he claims 6’s are rarely actual detractors) and that there is no connection between NPS and growth. For even more detail along these lines, I also recommend checking out Bob Thompson’s very detailed piece at Customer Think.
While I am tempted to give Bob and Larry the benefit of the doubt on the statistical relevance, the point I’m most interested in exploring is the connection between NPS and growth. There are certain categories, like automotive and travel/hospitality, that are driven by moving consumers from “passive” to “promoter.” I believe that it’s less a measure of success to get that snapshot of NPS in a random sampling of the target demographic than it is to measure a specific set over time.
It can probably be done in similar manner to brand awareness. Since we basically share a floor with Dynamic Logic, who I consider to be the hands-down best at measuring brand awareness online, I often debate the methodology for this kind of measurement with my friends over there. From a marketing perspective, I would only be interested in seeing an increase in NPS in the same set of surveyed individuals over the period of a marketing interaction. For banner impressions, I believe that the Dynamic Logic method of measuring people who have seen an ad against those who have not makes plenty of sense but I don’t think the same is true for someone who has submitted a video for Doritos Super Bowl promotion and waited around for a month to see who won or someone who has visited H&R Block’s tax advice outpost in Second Life and been incentivized to walk into a retail outlet.
Read the rest of this Point Oh! Post by Clicking Here
R-E-S-P-E-C-T: Why Market Research Just Doesn't Get It September 26, 2007
Posted by Mark Blei in : Uncategorized , add a commentR-E-S-P-E-C-T: Why Market Research Just Doesn’t Get It
Marketers Continue to Ignore an Industry They Need Now More Than Ever
Published: September 24, 2007
It’s a weird dissonance some believe is at the heart of marketing’s malaise. Marketers need good research more than ever but either can’t get it, can’t understand it or can’t accept it. Market research, in other words, has never been so important. Yet market researchers are as unimportant as ever — at least within their organizations.
 |
Like moguls with trophy wives, marketers keep collecting them and keep ignoring them.
The research industry is struggling for respect even as it races to fill vacancies. Fragmentation, complexity and demand for accountability have been a boon for researchers — to the point that their pay and bonuses are rising 10% or more annually, according to recruiting firm Stephen-Bradford Search.
On the other hand, few researchers ever move beyond their staff functions into top marketing or management roles at marketers, agencies or even leading research companies. When the private-equity owners of Nielsen Co., the biggest research firm of all, needed someone to turn the company around, they turned to David Calhoun, one of the hottest products from General Electric Co.’s general-management mill, who spent most of his career in finance.
“Marketers generally distrust research and data,” said Greg Stuart, former CEO of the Interactive Advertising Bureau and co-author of the book “What Sticks,” who’s in the process of launching an undisclosed new business.
Punch in the gut
That attitude helps keep the industry driven largely by gut instinct and what Mr. Stuart calls “tribal custom.” But tradition and gut are increasingly impractical given the complexities of modern marketing, he said.
He cites a relatively simple marketing plan with five elements — positioning, segmentation, TV, print and online media plans — and five choices within each element. It amounts to more than 3,000 permutations. “We’d have to be out of our minds to think we could know what to do in our gut,” he said.
Researchers’ relatively low status within companies and agencies, he said, leads to marketers ignoring their advice or never seeking it in the first place. At the same time, many marketers’ inability to understand the methodology and calculations behind data they get, Mr. Stuart said, “allows research to be abused, sometimes by researchers but also by people trying to sell things.”
Just as marketers often don’t understand research, the researchers often don’t understand why they’re doing it.
Utility
“There is a general belief [among researchers] that over 50% of the research done at companies is wasted,” said Bob Barocci, CEO of the Advertising Research Federation. “They’re asked to do things that, even if the research project is perfect, won’t be useful.”
He attributes much of that to research done purely for defensive purposes to support decisions already made. “It’s covering-your-butt kind of thinking,” he said.
But he also blames research departments for much of the disconnect with marketers. “Often all we do is present numbers,” Mr. Barocci said. “We don’t present insights.”
To be sure, researchers have sought to fix that image by no longer calling research by its old, academic-sounding name.
“I can’t stand the term ‘market research,’” said Alison Zelen, director of consumer and market insights for Unilever’s deodorant business. “I don’t consider myself a market researcher at all. I really consider myself a marketer, which is why I like the [insights] title.”
She termed market research “this huge industry of billions of dollars that anyone basically can do.”
Self-sustaining problem
That outlook has found its way into industry economics, encouraging procurement departments to drive down costs and, in the process, research quality, which in turn undermines marketer confidence in research. It’s a classic vicious circle.
Years before marketers set their procurement officers loose on ad agencies in the 1990s, they did so on research, and with much less hand-wringing.
“The perception is that the only thing important here is price, because any one of these companies will deliver the same results,” Mr. Barocci said. “That’s the cause of procurement getting involved early and quickly [in research buying]. I don’t think anybody ever really believed agencies, on the other hand, were a commodity.”
Research companies are to blame too, he said, for not pushing back, or at least not disclosing what the quality trade-offs will be from low bids.
Quality control
Online research has been a key factor driving down research costs and increasing speed, but some believe it’s also driving down quality. Procter & Gamble Co., the biggest buyer of research in the world, last year disclosed that some online surveys from the same vendors weeks apart delivered significantly different results, as well as online surveys whose results differed significantly from those obtained by mail surveys.
Partly in response, the ARF launched an effort to develop new online-research quality standards by next year. At the same time, Mr. Barocci said the ARF hopes to “start a dialogue on procurement” practices in 2008.
But he said the single biggest issue facing the industry is: “How do we get better data?” He added that continued concerns about the quality of research data in many areas may play the biggest role in undermining the industry’s image.
Research produced as part of the ARF’s online-quality initiative isn’t necessarily a confidence builder. A study recently presented by Knowledge Networks found in some cases substantial divergence from known benchmarks established by the U.S. Census or elsewhere both in online and phone surveys. The sample populations in seven online surveys and one phone survey ranged 10 percentage points or more from the norm in areas such as race, education, income range, movie viewership and travel habits.
A separate study by Burke found an average of 14% of survey takers across 20 online panels were fraudulent or “mentally cheating.” Depending on the panel, between 11% and 20% of respondents contradicted themselves by — at different points in the surveys — saying the same brand both was and was not worth paying more for.
Innovation needed
Kimberly-Clark Chief Marketing Officer Tony Palmer said survey research itself may be part of the industry’s image problem.
“It’s becoming harder and harder to get people’s attention to do research,” Mr. Palmer said. “It’s becoming clearer and clearer that what people say and what they do is different. So there’s a real need to drive research to newer techniques, toward research that deploys anthropology and observation.”
Of course, if more senior managers at marketers or agencies came from research backgrounds, they might be better at sniffing out research-quality issues, understanding quality trade-offs for cheaper research and recognizing how research could be improved. But for many reasons, that isn’t happening and may never.
“When you get to a senior level, it’s about leadership, inspiring people, direction, decisiveness, handling ambiguity,” Mr. Palmer said. “My guess is that the skill set required to make you a really good researcher may be a little different. You tend to
be a little more introverted. … But the industry also needs to ask itself the question: ‘Am I developing people the way I need to be?’”
Susan Gianinno, chairman-CEO of Publicis USA, is a rare senior agency executive from a research background — a former University of Chicago research psychologist who spent several years in or running agency research departments before moving into account management. But she, too, thinks most researchers are ill-suited to run businesses.
Detached researchers
“To move a business forward, you have to come to a judgment and move on pretty quickly, and a really rigorous researcher knows too much about what he doesn’t know,” she said. “Some of the best researchers are better served by being the objective spokesperson for truth and not having to worry about some of the pragmatic aspects of our business.”
But because researchers have become increasingly detached from the creative process, she said, many “don’t know how to take their insights and convert them into actionable, inspiring briefs.”
Unbundling of media from creative agencies over recent decades has widened the divide of research from creativity, she said, because the media agencies took most of the best researchers with them. And research lost favor within its remaining stronghold at creative shops — account planning.
“The whole era of British-imported planning,” she said, “did a huge disservice to strategy and planning because it became too qualitative in the sense that if somebody could just think up an idea or put on a good presentation, they were deemed a good planner.”
While it’s rare for researchers to move into senior marketing and management positions at marketers, too, it’s not unheard of. One example is Irene Rosenfeld, chairman-CEO of Kraft Foods, who after getting a Ph.D. in marketing and statistics from Cornell University started in the General Foods research department in 1981.
Success story?
Research executives believe the relative clout people from research backgrounds had at Kraft and General Foods in decades past probably contributed to Kraft becoming the leading adopter of now widely used marketing-mix modeling to evaluate advertising effectiveness.
Gregg Ambach, VP-analytic services at ImmediateFX and a veteran both of Kraft and Campbell Soup Co., said Kraft was well ahead of Campbell in developing marketing analytics. “Certainly there was no one with a career path similar to Irene’s in the Campbell organization,” he said. “Some of that is probably recognizing the value of the staff function.”
Of course, the Kraft experience hasn’t always been a glowing endorsement of the power of analytics. With sales growth stuck, around 3%, Kraft has been lagging behind some of its big peers, such as Kellogg and Nestlé. And in one of the more extreme forms of procurement hardball in 2003 (before Ms. Rosenfeld took over), Kraft asked some market-research firms to rebate money they’d already been paid.
Even the biggest proponents of research don’t necessarily believe putting more researchers in charge is the answer. But putting more researchers into C-level roles, perhaps as chief strategy officers, may help.
Bringing data to the table
“I like to have a quant jock or a researcher by my side in running a business,” Mr. Stuart said. “I like to have direct access to that expertise.”
The other solution, he said, is to make research a bigger part of general marketers’ training.
He said he sees hope for more “fact-based decision-making” taking hold in marketing through such parallels as Major League Baseball, where the numbers-driven “Moneyball” talent development of Oakland Athletics General Manager Billy Beane has gained a growing foothold in recent years.
“That kind of change is generational,” Mr. Stuart said. “It’s going to be a long, slow slog.”
If baseball’s any indication, he’s right: The A’s are playing under .500 this year.
The Unspoken Truth July 31, 2007
Posted by Mark Blei in : Uncategorized , add a commentThe Unspoken Truth
Posted July 31st, 2007 by Gregory Wilson
There is a lot of talk these days about measuring results and whether advertising agencies should be more proactive when it comes to finding ways to be accountable for their work.
To date, agencies have been somewhat hesitant. And, while the reasons are many, there is one universal truth that goes unspoken. For most agencies, it is far more lucrative to be paid for the possibility of success, than it is for the actuality of results.
The good news is that a new engagement study, recently released by Omnicom Group’s OMD, may encourage a few agencies to be more open to the idea of accountability.
What the results from the study indicate is that one engaged viewer is worth eight regular viewers. Not to mention that for the brands involved, factoring engagement into the equation increased measurable return on investment 15% to 20% over models that only factored in GRPs.
Coincidently, Nielsen/NetRatings recently announced that they would start measuring time spent rather than page views.
Why is this important?
Because if “time spent” is measurable on a page, which it is, then it is also measurable when someone clicks into and out of a commercial on a digital platform. When do they click out? Usually when the commercial becomes less engaging for them.
I know. Many will argue that time spent does not equal engagement. So for now, let’s just say that it does appear to be a fairly good indicator of engagement if the viewers are allowed to activate the commercial and leave when they want to.
Now what would happen if an advertising agency said that instead of receiving a fee for creating a commercial based on hours worked, they would like to be paid based on how long their commercial involved the viewer for?
In other words, the more engaging their work, the more the agency would make. The less engaging the work, the less they’d make.
Would advertisers be willing to work this way?
Considering that an engaged viewer is worth eight regular viewers, and that ROI increases 15% to 20% with engagement, you would think that advertisers would be motivated to take their agencies up on this.
Except for that unspoken truth thing: for most agencies, it is far more lucrative to be paid for the possibility of success, than it is for the actuality of results.
Fortunately, most is not all.
And there are a handful of agencies — Crispin, Goodby, Wieden, BBDO — that are, for good reason, confident in the work they create. You’d think that they would like nothing more than to be paid based on how well their work engages the viewer.
The fact is, under the current labor-based compensation model, engaging work and non-engaging work are compensated equally. As most agencies are better at the latter than the former, the outcry has been minimal. Failure, has in fact, proven to be quite lucrative for most agencies.
It’s only the truly brilliant shops that are leaving money on the table.
Should things remain the same now that we can determine engaging commercials from non-engaging commercials on digital platforms? And especially now that it seems as if engaging commercials are worth more to the advertiser?
The digital marketplace and its new measurement capabilities offer the opportunity to liberate advertising from the clutches of mediocrity. And to let those agencies that are better than the rest rise even further to the top — both in the work that they do, and in the way that they’re paid for it.
Nielsen Launches Online Video Measurement Service May 8, 2007
Posted by Mark Blei in : Uncategorized , add a comment| Nielsen Launches Online Video Measurement Service | |
| by Gavin O’Malley, Tuesday, May 8, 2007 6:00 AM ET | |
| IN AN INCREASINGLY VIDEO-DRIVEN ONLINE universe, Nielsen//NetRatings on Monday released a new syndicated measurement service, VideoCensus, which combines panel and census research methodologies to provide an end-to-end accounting of audience size, demographic composition, engagement and competitive activity. “By harnessing the unique strengths of both panel and Web analytics measurement tools, we have a ‘best of breed’ service that can be used for planning and post analysis on the Internet,” said Manish Bhatia, executive vice president, NetRatings. The VideoCensus methodology employs two patented assets–NetRatings’ desktop meter and SiteCensus content-tagging technology. To facilitate reporting, online video publishers, technology providers and networks attach a piece of NetRatings code to their video delivery platforms. Once enabled, this code allows NetRatings to collect a census count of the viewing activity. The NetRatings desktop meter further dissects and analyzes the video data to provide granular insight into viewer engagement with specific video channels, programs, and clips. Using a technology-agnostic collection system, VideoCensus measures all methods of streaming media delivery regardless of application, protocol or viewing platform. Through its proprietary combination of technologies, VideoCensus reports on viewing of cached content, peer-to-peer programs, and digital rights-managed video streams. Troy Young, chief marketing officer of VideoEgg, said VideoCensus has already “given us the ability to get accurate counts of activity across our network, along with the panel-based demographic data we need to position our audience.” Young noted that VideoEgg is now delivering more than 425 million video streams a month, across more than 70 social networking sites.
|
