Dynamic Logic's Ken Mallon comments on recent OPA/comScore research June 30, 2009
Posted by Mark Blei in : Ken Mallon, Staff posts, industry news , trackback Ken Mallon Dynamic Logic’s SVP Custom Solutions & Ad Effectiveness Consulting speaks out about recent research released by the Online Publishers Association in conjunction with comScore.
Ken is a 25-year research veteran having applied research methods, statistics and data mining expertise to a variety of fields including health, pharmaceuticals, marketing, and internet behavior. At Dynamic Logic, Ken heads the Custom Solutions team, providing specialized research and consulting services to top clients. Before joining Dynamic Logic, Ken was Director of Product Development, Director of Marketing Solutions, and Director of Data Mining at Yahoo!, where he helped to create Yahoo!’s data-mining group, as well as their behavioral targeting system, unique consumer insights, and passively collecting data and advertising products.
Ken holds a Master of Health Science in biostatistics from Johns Hopkins University, a Master of Science in statistics from Stanford University and a Bachelor of Science in secondary math education from University of Connecticut. Kens comments appear below.
Last week, OPA and comScore released some research titled “The Silent Click: Building Brands Online”. Dynamic Logic is a long-time friend of the OPA (we love you, Stu!), but I just had to comment on this. While we agree with one of the key messages of the research – to stop putting emphasis on CTR – we disagree with the overarching framework and believe some of the research methods are misleading.
FRAMEWORK
I won’t start with a methodological discussion (methods geeks, skip to the bottom of the article where we can have a nice statistical dialogue). Instead, let’s start higher with the overall framework. In the key findings (slide 52) this research states:
Search + Site Visitation + e-Commerce spending = a smart formula for measuring display ad effectiveness.
We fundamentally disagree with this concept. On page 5 of the deck, Carrie Frolich, Managing Director Digital, Mediaedge: cia, is quoted as saying:
“Remember why you’re advertising … You are not advertising for clicks … What you’re advertising for is to sell me stuff or change perception, and that’s what we need to be measuring against.”
Frolich is right-on. This is very similar to what I say on just about every panel and conference speaking engagement in which I participate:
“At the end of the day, you care about two things – did my advertising help sell something or did it change someone’s opinion. Everything else is a surrogate or noise.”
We believe that the right formula is:
Perception changes + sales (both online and offline) = the best formula for measuring display ad effectiveness.
We believe that post-view behaviors play a key role diagnostically and add color to the above.
The brand perceptions part can be handled via surveys (passively collected comments via blogs and otherwise can play a role as well) and the sales impact part must utilize one of several methods for capturing both online and offline sales which I’ll go into later.
I want to digress for a moment into a pharmaceutical industry parallel. During my years as a statistical scientist at UCSF, Genentech and Amgen, I was taught that at the end of the day, when talking about drug interventions, a medicine must do one of two things. It must either save lives or improve quality of life. All other things are surrogates or noise. Sound familiar? If a drug improved your cholesterol, but didn’t improve your quality of life or reduce the risk of death, would you take it? Probably not. If an ad format increased interaction but didn’t make people like your brand better and didn’t increase sales, was it effective?
I know a lot of this sounds like I’m discrediting post-view behavior. I don’t mean to do that. Post-view behaviors such as driving search, visits to the advertiser site, brochure downloads and trailer views can be important diagnostic tools for understanding lifts or lack thereof in brand perception and sales. But, by themselves, they are not ultimate measures of ad effectiveness. There is one post-view behavior worth clarifying – e-commerce. In the Dynamic Logic framework – it’s all about perceptions and sales – e-commerce is included in the latter. It’s part of the sales impact measurement. Online sales, although they are technically a post-view behavior, are not a surrogate. They are part of the real deal.
Now, let’s get back to Carrie Frolich. If she’s right and I believe she is, then why doesn’t the research presented include any brand perception results? And, why is the sales impact information only limited to online and not offline sales?
Dynamic Logic is known for measuring the branding impact of digital advertising and we do this via live web-intercept survey. But, what some of our clients are increasingly discovering is that we can do much more. We are currently advocating a broad-based way of measuring the impact of display ads – brand impact + offline sales estimation + e-commerce (where appropriate) + other post-view behaviors to add color and for diagnostic use.
Some nine months ago, we discussed internally the impact of the economy and wondered what we might be able to do for our clients who were feeling increasing pressure. What we heard again and again from our clients is “ROI, ROI – we want more ROI and … it can’t be expensive to measure”. Publishers, agencies, advertisers and technology providers are under increased pressure from management and clients to deliver results. This type of pressure has always existed in digital but now it’s heavier than ever.
As a result of this need, we have begun thinking very creatively about what we can do. We worked out a simple, straight-forward and relatively inexpensive way of estimating the offline and online sales impact of display advertising (and website exposure) applicable to just about any advertised product. This, combined with our patented approach to measuring changes in brand perception, completed the sales + perception = display ad impact formula we advocate and has been reinforced by Carrie Frolich. Also, since our tracking technology could be used to capture post-view behaviors such as e-commerce, we had that element as well.
What was missing were other post-view behaviors such as impact on search and impact on website visitation other than the advertiser site. An example of this would be measuring the traffic to kbb.com following display ad exposure to an automotive campaign.
Meanwhile, TNS became one of our cousin companies, opening the door to closer collaboration and data integration with Cymfony and Compete. Sometimes, when you ask for something, you get it! By collaborating with Compete, we can now estimate the impact of display ads on search volume and website traffic to any prominent site. And, what of Cymfony? What role can they play? Longer-term, we hope to use their technology to enhance the measure of perceptions (among other things). This is currently assessed largely via surveys, but could be enhanced with passively-collected data across the Web.
So, to sum up regarding the framework,
INCOMPLETE: Search + Site Visitation + e-Commerce spending = a smart formula for measuring display ad effectiveness.
CORRECT: Perception changes + sales (both online and offline) = the best formula for measuring display ad effectiveness.
And, by the way, although some of our capabilities are not fully productized in this regard, we can deliver today on this way of measuring digital advertising effectiveness and can do so in a relatively cost-effective way.
Thank-you, Carrie!
MISLEADING METHODS
And, now on the more tedious discussion of the methods used in this research. First, I give OPA and comScore credit for being responsible researchers and using the term correlation. Also, they didn’t do statistical testing to compare the control and exposed group. Since the groups were not scientifically equivalent, not testing is the right thing to do since statistical significance would have no meaning anyway.
That being said, I think more could have been done to make the comparison groups more comparable. This is important because, although they are careful to use the term correlation so that people should not conclude causality, we all know that the untrained reader will immediately conclude causality.
As an example, let’s look at slide 16. It shows that visitors to a brand’s site who had been exposed to that brand’s advertising ended up spending more time on the site and consumed more pages. Firstly, let me say this. I believe in display advertising. I believe online ad exposure is likely to cause someone to end up spending more time on a website. I’ve just seen too much controlled data to support it. So, I have to believe it. However,
1) This research doesn’t prove that ad exposure leads to greater site consumption
2) The authors don’t directly claim it to be true (they use the careful and correct term, correlation), but
3) Most people will arrive at the conclusion that online ad exposure drives increased brand consumption in terms of website time spent and pages visited (I’ve read dozens of tweets with that interpretation)
Although, again, I believe (3) is likely to be true, I’m just saying that it doesn’t follow from the data presented. Although the authors cover themselves statistically by not testing and by using the term correlation, I feel it’s a bit irresponsible given that much of the audience isn’t well-trained in statistical interpretation.
I have the same complaint for nearly all the comparisons that are made in this presentation. It’s a bit worse in cases where there is an unexposed versus exposed comparison. Let’s look at slide 17 for example. It shows that those who were exposed to advertising had 7% more e-commerce than unexposed (similar results on slide 18).
First, look at the x-axis. The range of x-axis is from 200 to 250. When I was a statistics student I learned that shortening the axis to make the difference seem larger, is a reporting no-no. But, that’s largely a pet-peeve – I can see why you’d want to do that. My main criticism is the selection of the unexposed as the control group. To avoid misleading results, one must select the comparison group very carefully and since it wasn’t done scientifically, great care should be taken to make the groups as comparable as possible either by weighting, multivariate regression, stratification or other techniques.
Let me explain just one possible reason for having a 7% difference that has nothing to do with ad exposure. Take the example of re-targeting. That’s when you show an ad to someone because they previously visited your site (and you smartly cookied them) or they previously performed a search in your category or for your brand. If you do a retargeting campaign, you are selecting a subset of the population who has a far greater chance of being interested in and therefore greater chance of purchasing your product. Same goes for good behavioral targeting (there’s good and there’s bad – but, I’ll save that topic for another blog). So, it very well could be that the exposure itself played no role in later e-commerce. It could just be a function of good targeting. And, if the unexposed group possibly came from other websites than the exposed, then in-context targeting could have been the reason for increased e-commerce.
Thus, again, although I believe in online ads and believe they increase e-commerce, the research presented doesn’t show it. The same methodological criticism applies to the rest of the results.
Again, we are supporters of the OPA. But, we think comScore did them somewhat of a disservice with this research. I believe that online advertising drives e-commerce, increases search and has many other positive brand benefits, I don’t think this research proves it.
If you want to measure the complete impact of display advertising – brand impact and sales, talk to us at Dynamic Logic. We can provide valuable and robust scientific data within our framework for measuring display ad effectiveness:
Perception changes + sales (both online and offline) = the best formula for measuring display ad effectiveness.
Best,
Ken Mallon
Senior Vice President
Custom Solutions & Ad Effectiveness Consulting
Follow us on twitter: Dynamiclogic Join our Facebook Page Join our group on LinkedIn
Comments
Ken, our research with the OPA was never intended to measure the lift caused by online display advertising. It simply described the characterisitics of the audience reached by display advertising across different types of online content. For a definitive evaluation of the lift caused by display ads please see our research white paper “Whither the Click?” published in the current issue of the Journal of Advertising Research or accessible here:
http://www.comscore.com/Press_Events/Presentations_Whitepapers/2008/How_Online_Advertising_Works_Whither_The_Click
Gian- Thanks for taking the time to give us your thoughts.
I’ve passed your comment onto Ken.
Respectfully,
Mark Blei
Mark Blei | Global Business Development & Marketing …Blog guy?| Dynamic Logic – A Millward Brown Company
Gian,
The main point of what I wrote is about the framework. We agree with Carrie that it’s all about perceptions and sales. Post-view behavior is an important diagnostic tool but shouldn’t be part fo the effectiveness formula. That’s a fundamental difference.
Regarding methods, I understand that you didn’t intend to prove causality. I pointed that out in my article and gave you credit for using the right terminology. However, that doesn’t mean it didn’t mislead. Based on what I have seen tweeted and based on comments we have received directly, many people have mis-interpreted the findings. Balancing with more controlled findings or including more caveats may have helped.
[...] This post was Twitted by arthurbarbato [...]
Ken:
In the presentation of results from “The Silent Click” study (which I personally handled in SF and NY) I made it crystal clear that this study did not use a test / control design coupled with a pretest / test period and as such was never intended to measure or establish a causal relatioship between ad exposure and sales. Rather, it was intended to describe the characterisitcs of the audience for display ads. This is no different from what has been done for decades in traditional media to show the value of the audience for TV, Radio, Print ads. So, to criticize the design of the OPA study seems to me to be taking jabs at a generally accepted research method. Nonetheless, I’ll be sure to pass your comments on to Pam Horan, President of the OPA, so that she can respond to you directly.
While we’re discussing the subject of “misleading comments”, I’d like to address the claims you make in your article that DL is the first company to be able to measure the impact of display campaigns on sales (Yes, Ken, that’s what a reader would conclude from reading your article. Did you intend to create this perception?) In fact the 200+ studies included in comScore’s “Whither the Click” paper (which prove the positive impact of display ads on site visitation, trademark search queries, online and offline sales — all in the face of minimal clicks) were conducted over a period of several years — long before your parent company acquired Compete and you developed any capability of measuring sales and not just attitudes.
Sounds to me like your criticism of the OPA study is a lot like the pot calling the kettle black.
Gian,
Thanks for your comments. Have a look at my reply from 1July. I have addressed the methodological issue. I understand you didn’t intent to mislead. However, based on the tweets I have read and questions I’ve received directly, people did indeed walk away with some misperceptions. Also, just because a methodology has been used for a long time, doesn’t make it good. Digital media has an advantage over non-digital in that one can determine exposure with a much higher degree of certaintly. Thus, the digital community tends to have somewhat higher research expectations in terms of controlled experiments.
As far as sales impact measurement, that’s been possible for a very long time in one form or another. And, neither I, nor Dynamic Logic make any claims to have been the first.
But, all this is a distraction from the main point of what I wrote. The main point was around our differences in terms of the framework for measurement. We agree with Carrie that perceptions + sales = the right way to measure display ad effectiveness. Whereas you focused on post-view online behaviors such as search, site visitation and e-commerce. E-commerce fits into the sales part of our formula but, we believe site visitation and search, althought very useful diagnostically, are typically not ad effectiveness metrics by themselves.
Finally, I am not criticizing the OPA. They support lots of great industry research. I disagree on the framework which I assume was developed by Comscore and not OPA. And, I think OPA would have been better served had more rigorous controls been used in the research. The results likely would have been positive either way, but the research community as a whole would have been more accepting of the findings had the research been better controlled.
Gian, we agree on many things and I think debate is very healthy for our industry. Perhaps we should arrange a public live debate. I’d welcome it.
Best,
Ken
Hi Ken:
Unfortunately, you seem unable or unwilling to grasp the fact that the research we recently conducted was not, as I have said in my previous posts on your blog — and as I made crystal clear in my presentation of the results — intended to establish a cause and effect relationship between display ads and sales. It simply described the characteristics of Internet users who are exposed to display ad campaigns. Many advertisers, publishers and media planners have told me that this type of analysis is of significant value to them and I reject your assertion that the approach is somehow flawed.
Based on the feedback that we have received from people who either attended the recent presentation or viewed the deck, we see no evidence that they have misunderstood the objectives and nature of the research.
In addition, as I explained in my previous posts and further elaborate below, comScore’s recent study was conducted as a supplement to previous comScore research (published in this month’s issue of the Journal of Advertising Research) that did firmly establish a causal relationship between ad exposure and lift in search, site visits, and sales. So, your claims about “misinterpretation” are really moot.
That said, your logic in claiming that trademark search and site visitation driven by display ads are irrelevant metrics escapes me. They are, after all, elements of many brands’ marketing plans and, as such, critical activities behind which marketers put substantial money in their efforts to persuade a consumer to purchase the advertised brand. In fact, research conducted by comScore (which I was invited to present at Professor Jerry Wind’s “Empirical Generalizations in Advertising” conference at the Wharton Business School and which was just published in the June issue of the Journal of Advertising Research) shows that display ads lift search queries and site visitation, both of which, in turn, help drive sales. It would be folly to ignore an understanding of the important role of these variables in helping a display campaign close a sale.
Let me be clear, I have no problem with your suggestion of including attitudinal metrics as diagnostics. They can certainly add value. But, I disagree with your assertion that search and site visitation not be used. It makes me wonder whether your understanding of how online advertising works isn’t based on the fact that Dynamic Logic has traditionally focused only on attitudinal metrics in its research and so you’ve never really had the opportunity to fully understand the synergy that exists between display and search ads and eventual sales? If so, you really need to get caught up. comScore’s Brand Metrix service provides not only attitudinal metrics but also a measurement of behavioral response, including lifts in search queries, site visitation, online and offline sales. Clients find this to be very valuable. Academics also, and I’m happy to forward you a copy of the Journal of Advertising Research so that you can grasp the significance of comScore’s research. You can also see a summary here:
http://www.comscore.com/Press_Events/Press_Releases/2008/11/Value_of_Online_Advertising
You should also read eMarketer’s current article “How Online Brand Advertising Can Influence Every Step Along the Purchase Funnel”:
http://www.emarketer.com/brandmeasurement/index.php/online-brand-advertising-influence-purchase/ This article does an excellent job of showing how display advertising can change consumer behavior and the benefits of measuring that behavior in all its manifestations. For example, because the conversion rates following a search query are so high, it’s valuable for an advertiser to understand if their display campaign has been effective in lifting the number of trademark search queries. You can’t gauge this by just measuring attitudes.
Over and out
Gian
p.s. Thanks for allowing me to promote comScore’s ad effectiveness measurement capabilities on the Dynamic Logic blog. It’s much appreciated.
Gian,
Thanks for your comments. I still think the best way to continue this is via a live debate. I notice you did not respond to my inquiry along those lines. And, again, I’d like to emphasize that we agree on many things including a common desire to decrease or even eliminate the focus many have on CTR.
First, I’ll address and try to put to bed this issue of being misleading. As I’ve repeatedly written, I know that you did not intentionally mislead. But, I’ve seen evidence that people arrived at false conclusions (”lift” in headlines, tweets, direct inquiries from clients). I understand you have not experienced this. So, at this point, I think we should just agree to disagree and drop this issue.
But, on to the issue of ad effectiveness measurement framework, I think this is something of strong interest to the community and well worth continued debate. Your report with the OPA stated that the best way to measure ad effectiveness is by measuring impact on search, site visitation and e-commerce. We contend that the best approach is to measure changes in perceptions and sales (online and offline) supported diagnostically by post-view behavioral information.
I do not believe impact on search and site visitation is “irrelevant” as you have mis-quoted me. I wrote that these behaviors are “an important diagnostic tool” — pretty far from saying they’re irrelevant.
Furthermore, I believe advertisers should routinely measure all three: 1) impact on perceptions, 2) impact on sales (both online and offline) and 3) diagnostic behavioral information including changes in search, site visitation to the client site as well as related sites. And, I’m glad Dynamic Logic can offer this breadth of measurement.
Let me explain why perceptions and sales must take priority over post-view non e-commerce behaviors. Which of these scenarios do you think an advertiser would prefer?
The display ad campaign was shown to
A) Increase sales both offline and online but did not increase search volume or traffic to the site
B) Increased awareness, brand favorability and purchase intent, but did not increase search or site traffic
C) Increased search volume and traffic but did not change brand perceptions and did not increase sales.
I think 90% of advertisers would rank order the above, as written — A is first, then scenario B then C.
Here’s the bottom line. A campaign that increases sales is successful (with the caveat that perceptions weren’t harmed in the process). End of conversation. The buck, literally, stops there.
By contrast, a campaign that increases search and traffic may or may not be considered a success depending on what happens with brand perceptions and sales.
At the end of the day, if an ad campaign didn’t change anyone’s opinions and didn’t impact sales, it didn’t work. This is the same sentiment shared by Carrie Frolich, a person you quoted in your research.
I hope this helped clarify things for you and I look forward to a live debate.
Ken
Ken:
Let me summarize where we are.
In one of my previous posts, I have already made it clear that I agree that attitudes are important diagnostics. We have always included them in comScore’s Brand Metrix product, which also takes into account the deleterious impact of cookie deletion (which can lead to a serious understatement of ad impact). Clients tell me that Dynamic Logic doesn’t include any adjustments for cookie deletion in its products and so understates the impact of online ad campaigns.
I’m pleased to see that you now agree that sales, search and site visitation are also important diagnostics. DL has historically only provided attitudes, so it’s a step forward to hear you now say that actual behavior is also important. I do want to point out that these variables have always been included in comScore’s ad effectiveness measurement services.
However, there is one other extremely important dimension that comScore includes in its evaluation of the effectiveness of a media plan, and which you fail to mention. That is, what is the reach / frequency that was actually delivered in the media plan and what are the characteristics of the people who received the ad impressions? Cookie deletion is again the culprit here because it can cause the actual delivered media plan to be very different from what was planned to be delivered. comScore routinely provides clients with post-buy measurements and clients are telling me that DL doesn’t provide this capability. So in your example, I think you’re dead wrong. The buck doesn’t stop at A. Both B and C could have failed to cause a change in behavior because the correct media plan wasn’t delivered. You fail to even mention this frequent occurrence, so it looks to me like Dynamic Logic’s approach again misses the mark.
Gian
p.s. Happy to join you in a public debate. As you probably know, I am very active on the speaking circuit and I would be delighted to see you participate on one of my upcoming panels.
Gian,
Let me also summarize where we are. Firstly, we both want to move the industry away from CTR. We both agree that it can be a terrible measure of ad effectiveness. And, if we focus on moving people away from that, I’m sure we’ll make an impact together on the industry.
Assuming someone has bought into other measures of ad effectiveness, you’ve stated publicly that the best way to measure display = search + site visitation + e-commerce. I’ve said it’s about changes in perceptions + sales impact (online and offline). You eventually said that perceptions play a role diagnostically. I’ve said post-view behaviors are important diagnostically. I’ve made the point that if a campaign impacts sales but doesn’t affect online post-view behaviors, then it’s a success. Whereas the opposite is not true — campaigns can impact behaviors without being successful.
You are now saying we’ve failed to bring reach and frequency into the equation. If so, then we have both failed in that respect. It wasn’t part of your formula either. That being said, what you are talking about is an extension of ad effectiveness called impacted reach which is defined as reach times impact. Impact, as I’ve said before is changes in perceptions + sales. This is typically represented as a percentage — percent of people buying per million impressions, percent of people with changed perceptions per million impressions (etc.). So, impacted reach (reach times impact) is the number of people impacted by a campaign. We can calculate this for clients, not just in terms of perception changes, but also estimates for numbers of items sold (online and offline) due to the advertising.
I agree that on a campaign basis, one should examine impacted reach. We’ve had a tool built into our MarketNorms database (of over 5,000 campaigns) for the last several years, that allows one to calculate impacted reach and perform “what if” scenarios. Some times clients just take the ad effectiveness results and calculate the impacted reach themselves using their own reach and frequency data.
You are right that historically we’ve been the brand impact company. And, it’s true that historically comScore has been a traffic reporting company. You’ve moved into ad effectiveness where we’ve been for 10 years. We’ve come at this from different directions, but we now have competing solutions that can incorporate measures of perceptions, post-view behaviors and sales.
So, in the end, we are doing the same things — but we feel we are doing it just a bit better. And, we think your strong emphasis on online behavior (your entire formula for measurement included *only* online behavioral metrics), missed the mark.
* Perceptions. We track the entire campaign and survey people within the media footprint of the campaign. You take a needle-in-a-haystack approach, searching through the panel to find people exposed to the campaign, then seeing how many of those are willing to take a survey. The size of your panel limits the depth of insights you can provide and, in some cases, makes this measurement infeasible.
You’ve made a lot of the subject of cookie deletion, not just on this blog but elsewhere. What percentage of people do you think delete cookies between the time they see an ad and the time they take our survey? The percentage is small. We’ve done conservative simulations that show that the impact on our metrics is tiny and often smaller than the rounding error in the metrics (eg, an increase of 6.1 is measured to be 6.0 or 6.05). Since we are a media neutral company, it’s important for us to be conservative in these regards. We can provide clients with estimates that account for the very small bias that cookie deletion introduces but so far, after they are convinced how small it is, no one asks for it. They like the neutrality and are glad we don’t report misleading results.
* Post-view behaviors. Yes, you have more experience but we have partnered with Compete.com who is quite experienced and has a larger panel than yours. This means we can get more granular on certain more rare behaviors such as e-commerce. And, our sample sizes will be larger which measure greater statistical reliability.
* Sales. Your formula didn’t even include offline sales. You restricted things to online. We’ve developed a reliable and relatively inexpensive system for tracking through to offline sales and are about to announce extensions and enhancements to this.
A final word about media neutrality. Notice that our formula ad fx = changes in perceptions + sales, works for all media. People believed it this formula the internet became popular and it continues to be the best general way to assess advertising effectiveness.
Regarding a debate, I think a neutral environment rather than one of us being part of the others’ panel, would work best.
Best,
Ken
Ken:
Sorry it’s taken me a while to post a reply. I’ve been engrossed with matters in the real world
Let’s stop pretending that a campaign can be reduced to a simple equation for every situation. You keep coming back to a single slide in our OPA presentation that said “search + site visitation + e-Commerce = a smart formula for measuring display ad effectiveness.” If one is measuring online impact, I contend that that’s a perfectly valid approach. I was not suggesting that one ignore offline sales and, in fact, we’ve been pioneers in measuring it.
It is interesting to me that you take issue with the use of trademark search and site visitation in measuring ad effectiveness. I suggest that your criticisms apply equally to brand perception, which you keep including as a critical evaluation metric. There is no justification for your claim that brand perception is the right thing to evaluate, but trademark search and site visitation are not. In fact, your own management team disagrees with you. Why, just the other day your company announced the following:
“On July 13 Compete announced the availability of its new online advertising effectiveness measurement capability which analyzes the impact of online advertising on consumers’ online behaviors such as engagement and conversion on brand sites, post-exposure search behavior and rival and third-party site visitation.”
http://www.dynamiclogic.com/na/pressroom/releases/?id=711
This is the same thing of which I’ve been trying to convince you for the better part of three weeks now, but still you insisted that : “post-view behavior shouldn’t be part of an effectiveness formula….search and visits to the advertiser site are not ultimate measures of ad effectiveness. In the Dynamic Logic framework – it’s all about perceptions and sales”.
Note especially the use of the term “ad effectiveness” in your own company’s release. Your management must have forgotten to let you know that they agree with me (and disagree with you) regarding what constitutes valid measures of ad effectiveness and that search and site visitation are indeed acceptable measures. I do notice that they didn’t bother to include a quote from you in their release. That should tell you something.
Your comments on cookie deletion are also rather uninformed. For example, it’s clear that you don’t understand comScore’s Brand Metrix methodology. We don’t – as you claim — survey our own panel in our methodology. And as far as cookie deletion is concerned, your methodology has no way of knowing that the person you define as a control respondent (who is supposed to have not seen a campaign ad at any time) isn’t just a computer whose ad cookies were deleted at some point in the past. That results in a contaminated control group and an understatement of ROI. We adjust for it. Dynamic Logic doesn’t.
I’m going to leave this discussion with a final suggestion. Before you publicly criticize your competitors (remember, you started this) and waste everyone’s time with your personal opinions, you should check-in with your superiors at DL and your compatriots at Compete to make sure you really understand your company’s official product strategy, positioning and philosophy. As Steve Martin once said to John Candy: “Next time you start a story, have a point.”
Gian
Gian,
I’m not pretending. I really do believe that, except for some rare cases, one truly does have as one’s ultimate goal either selling more product or changing opinions. I don’t take issue and never have taken issue with search and site visitation as metrics to help understand ad effectiveness. I simply believe they belong in the diagnostic/supportive bucket.
As far as your wrong perception that I’ve somehow not alligned with my management, I’ll be writing on this blog in the near future about our fantastic new relationship with Compete.com and how our new offering fits in with our overall framework for ad effectiveness measurement. If you wish to reply to that blog posting, feel free. This one has run it’s course.
I believe I’ve stated my opinions very clearly. Your framework falls apart and, in fact, you’ve moved away from it in this blog. The framework I’ve stated stands up to the test of time and, in fact, works for any medium.
Ken