Dynamic Logic's Ken Mallon comments on recent OPA/comScore research June 30, 2009
Posted by Mark Blei in : Ken Mallon, Staff posts, industry news , comments closed Ken Mallon Dynamic Logic’s SVP Custom Solutions & Ad Effectiveness Consulting speaks out about recent research released by the Online Publishers Association in conjunction with comScore.
Ken is a 25-year research veteran having applied research methods, statistics and data mining expertise to a variety of fields including health, pharmaceuticals, marketing, and internet behavior. At Dynamic Logic, Ken heads the Custom Solutions team, providing specialized research and consulting services to top clients. Before joining Dynamic Logic, Ken was Director of Product Development, Director of Marketing Solutions, and Director of Data Mining at Yahoo!, where he helped to create Yahoo!’s data-mining group, as well as their behavioral targeting system, unique consumer insights, and passively collecting data and advertising products.
Ken holds a Master of Health Science in biostatistics from Johns Hopkins University, a Master of Science in statistics from Stanford University and a Bachelor of Science in secondary math education from University of Connecticut. Kens comments appear below.
Last week, OPA and comScore released some research titled “The Silent Click: Building Brands Online”. Dynamic Logic is a long-time friend of the OPA (we love you, Stu!), but I just had to comment on this. While we agree with one of the key messages of the research – to stop putting emphasis on CTR – we disagree with the overarching framework and believe some of the research methods are misleading.
FRAMEWORK
I won’t start with a methodological discussion (methods geeks, skip to the bottom of the article where we can have a nice statistical dialogue). Instead, let’s start higher with the overall framework. In the key findings (slide 52) this research states:
Search + Site Visitation + e-Commerce spending = a smart formula for measuring display ad effectiveness.
We fundamentally disagree with this concept. On page 5 of the deck, Carrie Frolich, Managing Director Digital, Mediaedge: cia, is quoted as saying:
“Remember why you’re advertising … You are not advertising for clicks … What you’re advertising for is to sell me stuff or change perception, and that’s what we need to be measuring against.”
Frolich is right-on. This is very similar to what I say on just about every panel and conference speaking engagement in which I participate:
“At the end of the day, you care about two things – did my advertising help sell something or did it change someone’s opinion. Everything else is a surrogate or noise.”
We believe that the right formula is:
Perception changes + sales (both online and offline) = the best formula for measuring display ad effectiveness.
We believe that post-view behaviors play a key role diagnostically and add color to the above.
The brand perceptions part can be handled via surveys (passively collected comments via blogs and otherwise can play a role as well) and the sales impact part must utilize one of several methods for capturing both online and offline sales which I’ll go into later.
I want to digress for a moment into a pharmaceutical industry parallel. During my years as a statistical scientist at UCSF, Genentech and Amgen, I was taught that at the end of the day, when talking about drug interventions, a medicine must do one of two things. It must either save lives or improve quality of life. All other things are surrogates or noise. Sound familiar? If a drug improved your cholesterol, but didn’t improve your quality of life or reduce the risk of death, would you take it? Probably not. If an ad format increased interaction but didn’t make people like your brand better and didn’t increase sales, was it effective?
I know a lot of this sounds like I’m discrediting post-view behavior. I don’t mean to do that. Post-view behaviors such as driving search, visits to the advertiser site, brochure downloads and trailer views can be important diagnostic tools for understanding lifts or lack thereof in brand perception and sales. But, by themselves, they are not ultimate measures of ad effectiveness. There is one post-view behavior worth clarifying – e-commerce. In the Dynamic Logic framework – it’s all about perceptions and sales – e-commerce is included in the latter. It’s part of the sales impact measurement. Online sales, although they are technically a post-view behavior, are not a surrogate. They are part of the real deal.
Now, let’s get back to Carrie Frolich. If she’s right and I believe she is, then why doesn’t the research presented include any brand perception results? And, why is the sales impact information only limited to online and not offline sales?
Dynamic Logic is known for measuring the branding impact of digital advertising and we do this via live web-intercept survey. But, what some of our clients are increasingly discovering is that we can do much more. We are currently advocating a broad-based way of measuring the impact of display ads – brand impact + offline sales estimation + e-commerce (where appropriate) + other post-view behaviors to add color and for diagnostic use.
Some nine months ago, we discussed internally the impact of the economy and wondered what we might be able to do for our clients who were feeling increasing pressure. What we heard again and again from our clients is “ROI, ROI – we want more ROI and … it can’t be expensive to measure”. Publishers, agencies, advertisers and technology providers are under increased pressure from management and clients to deliver results. This type of pressure has always existed in digital but now it’s heavier than ever.
As a result of this need, we have begun thinking very creatively about what we can do. We worked out a simple, straight-forward and relatively inexpensive way of estimating the offline and online sales impact of display advertising (and website exposure) applicable to just about any advertised product. This, combined with our patented approach to measuring changes in brand perception, completed the sales + perception = display ad impact formula we advocate and has been reinforced by Carrie Frolich. Also, since our tracking technology could be used to capture post-view behaviors such as e-commerce, we had that element as well.
What was missing were other post-view behaviors such as impact on search and impact on website visitation other than the advertiser site. An example of this would be measuring the traffic to kbb.com following display ad exposure to an automotive campaign.
Meanwhile, TNS became one of our cousin companies, opening the door to closer collaboration and data integration with Cymfony and Compete. Sometimes, when you ask for something, you get it! By collaborating with Compete, we can now estimate the impact of display ads on search volume and website traffic to any prominent site. And, what of Cymfony? What role can they play? Longer-term, we hope to use their technology to enhance the measure of perceptions (among other things). This is currently assessed largely via surveys, but could be enhanced with passively-collected data across the Web.
So, to sum up regarding the framework,
INCOMPLETE: Search + Site Visitation + e-Commerce spending = a smart formula for measuring display ad effectiveness.
CORRECT: Perception changes + sales (both online and offline) = the best formula for measuring display ad effectiveness.
And, by the way, although some of our capabilities are not fully productized in this regard, we can deliver today on this way of measuring digital advertising effectiveness and can do so in a relatively cost-effective way.
Thank-you, Carrie!
MISLEADING METHODS
And, now on the more tedious discussion of the methods used in this research. First, I give OPA and comScore credit for being responsible researchers and using the term correlation. Also, they didn’t do statistical testing to compare the control and exposed group. Since the groups were not scientifically equivalent, not testing is the right thing to do since statistical significance would have no meaning anyway.
That being said, I think more could have been done to make the comparison groups more comparable. This is important because, although they are careful to use the term correlation so that people should not conclude causality, we all know that the untrained reader will immediately conclude causality.
As an example, let’s look at slide 16. It shows that visitors to a brand’s site who had been exposed to that brand’s advertising ended up spending more time on the site and consumed more pages. Firstly, let me say this. I believe in display advertising. I believe online ad exposure is likely to cause someone to end up spending more time on a website. I’ve just seen too much controlled data to support it. So, I have to believe it. However,
1) This research doesn’t prove that ad exposure leads to greater site consumption
2) The authors don’t directly claim it to be true (they use the careful and correct term, correlation), but
3) Most people will arrive at the conclusion that online ad exposure drives increased brand consumption in terms of website time spent and pages visited (I’ve read dozens of tweets with that interpretation)
Although, again, I believe (3) is likely to be true, I’m just saying that it doesn’t follow from the data presented. Although the authors cover themselves statistically by not testing and by using the term correlation, I feel it’s a bit irresponsible given that much of the audience isn’t well-trained in statistical interpretation.
I have the same complaint for nearly all the comparisons that are made in this presentation. It’s a bit worse in cases where there is an unexposed versus exposed comparison. Let’s look at slide 17 for example. It shows that those who were exposed to advertising had 7% more e-commerce than unexposed (similar results on slide 18).
First, look at the x-axis. The range of x-axis is from 200 to 250. When I was a statistics student I learned that shortening the axis to make the difference seem larger, is a reporting no-no. But, that’s largely a pet-peeve – I can see why you’d want to do that. My main criticism is the selection of the unexposed as the control group. To avoid misleading results, one must select the comparison group very carefully and since it wasn’t done scientifically, great care should be taken to make the groups as comparable as possible either by weighting, multivariate regression, stratification or other techniques.
Let me explain just one possible reason for having a 7% difference that has nothing to do with ad exposure. Take the example of re-targeting. That’s when you show an ad to someone because they previously visited your site (and you smartly cookied them) or they previously performed a search in your category or for your brand. If you do a retargeting campaign, you are selecting a subset of the population who has a far greater chance of being interested in and therefore greater chance of purchasing your product. Same goes for good behavioral targeting (there’s good and there’s bad – but, I’ll save that topic for another blog). So, it very well could be that the exposure itself played no role in later e-commerce. It could just be a function of good targeting. And, if the unexposed group possibly came from other websites than the exposed, then in-context targeting could have been the reason for increased e-commerce.
Thus, again, although I believe in online ads and believe they increase e-commerce, the research presented doesn’t show it. The same methodological criticism applies to the rest of the results.
Again, we are supporters of the OPA. But, we think comScore did them somewhat of a disservice with this research. I believe that online advertising drives e-commerce, increases search and has many other positive brand benefits, I don’t think this research proves it.
If you want to measure the complete impact of display advertising – brand impact and sales, talk to us at Dynamic Logic. We can provide valuable and robust scientific data within our framework for measuring display ad effectiveness:
Perception changes + sales (both online and offline) = the best formula for measuring display ad effectiveness.
Best,
Ken Mallon
Senior Vice President
Custom Solutions & Ad Effectiveness Consulting
Follow us on twitter: Dynamiclogic Join our Facebook Page Join our group on LinkedIn
New Free White Paper ! BRAND VALUE OF RICH MEDIA AND VIDEO ADS June 30, 2009
Posted by Mark Blei in : white paper , add a commentA new report, co-authored by DoubleClick and Dynamic Logic, is available free on our website at at:
http://www.dynamiclogic.com/na/research/whitepapers/docs/DL_DoubleClick_June09.pdf
For every online display campaign, advertisers must decide what ad formats to use for best results. To help, DoubleClick teamed up with Dynamic Logic to study the impact of ad format selection on branding goals. In the joint report titled “The Brand Value of Rich Media and Video Ads,” we compare the branding strengths of four common display advertising formats:
- Image – GIF or JPG ads
- Simple Flash – Ads with a short animation and single click-through link
- Rich Media with Video - Ads capable of responding to user interaction, such as mouseovers, keyboard inputs, or clicks without a click-through, that also contain video
- Rich Media without Video – Ads capable of responding to user interaction, such as mouseovers, keyboard inputs, or clicks without a click-through, that do not contain video
The report provides detailed information on how each of these formats impacts aided brand awareness, online ad awareness, message association, brand favorability and purchase intent. For example, study findings show that, on average:
- Rich Media formats are the most successful at driving purchase intent
- Rich Media without Video is unique among the formats in its ability to positively impact all five brand metrics
- Simple Flash is the least effective of the ad formats studied
- For brand favorability, aided brand awareness and purchase intent metrics, Rich Media with Video provides a statistically significant improvement over Simple Flash at a 90% confidence level
The report concludes with a cheat sheet to help guide your ad format decisions and suggests best practices for achieving branding goals. We hope you enjoy this and it helps you. Please feel free to contact us directly with any questions at 212-844-3700 and tell the person answering the phone that you read about it on our Blog!
Meet Stephanie Teng June 30, 2009
Posted by Mark Blei in : Staff posts , add a commentRecently we began a new feature where we introduce you to a few new staff members at Dynamic Logic every week . Here they will discuss their specialties, new challenges they may have faced and how the challenges were overcome, interesting projects they have worked on and new things that they have learned.
We hope that it will serve as a vehicle to allow you to get to know us better and through this to have a better understanding of our capabilities and the types of unique challenges we meet head on everyday. 
My name is Stephanie Teng, and I’m the graphic designer within Dynamic Logic Marketing in the New York office. Like most people, I’m able to stay in touch with friends via social media tools. During my sophomore year in college, my Kinesiology class partner sent an instant message, ordering me to “Join The Facebook pls” so that we could “be friends.” Today, “social media” are still buzzwords, having capabilities far beyond counting one’s number of friends. The marketing team has worked with many DL departments to explore the social media niche. I recently worked closely with our VP of Digital Strategy, helping to develop materials to further examine DL’s holistic approach to social media measurement. During this process, I was surprised to learn exactly how broad the social media space is, with light bulb moments of realizing “THAT’S social media? …Well, that does make sense…” I thought I was already a pro at updating my Twitter status, posting photos to Facebook, and reading my favorite blogs (like this one right here). The social media landscape is quite beyond my usual fares, including but certainly not limited to syndication, P2P, wikis and collaborative projects, consumer reviews, and endless more. I just needed a reminder to step outside the personal daily routine to realize the even more massive scope of this technology. Out of the office, I enjoy being an avid Penn State alumna, watching loud/obnoxious movies, finding the best eats in NYC, learning French (again and again since high school), and learning how to breakdance. Headspins are in the queue. I’ll also admit that I follow a few celebrity Twitter accounts.
If you’re interested in how Dynamic Logic can help your next campaign perform optimally for you. We invite you to call us at 212-844-3700 or email us at answers (at) dynamic logic (dot) com. Tell them that you read about us here!
Sorry took a vacation day June 30, 2009
Posted by markblei in : Uncategorized , add a commentMore Posts tomorrow!
Meet Vivian Takach June 24, 2009
Posted by Mark Blei in : Staff posts , add a comment
Recently we began a new feature where we introduce you to a few new staff members at Dynamic Logic every week . Here they will discuss their specialties, new challenges they may have faced and how the challenges were overcome, interesting projects they have worked on and new things that they have learned.
We hope that it will serve as a vehicle to allow you to get to know us better and through this to have a better understanding of our capabilities and the types of unique challenges we meet head on everyday.
My name is Vivian Takach and I am a Senior Research Executive in Dynamic Logic’s San Francisco office.
Recently while I was watching TV I saw an ad for a consumer packaged good targeted at women that offended me so much my initial reaction was to never buy the product again. Taking a step back from my reaction as a consumer, I started to think that the brand may not have had anything to do with the campaign beyond hiring an agency, and that the sentiments may not be their own.
I began thinking about how advertising can strongly connect a consumer with a brand, and that this connection could be a very negative one. I did some research on the CPG campaign that offended me, to find that I was not alone. Many women’s groups had felt the same way, and intended to boycott the brand. I visited the product’s website to find pages upon pages of negative comments. If only measuring site visitation, the campaign may actually be measured as a success. Through my research I also saw that although misguided, the campaign had been carefully thought out and researched prior to launching.
What I found has led me to start doing further research within Dynamic Logic’s extensive database on the lasting impact of bad advertising campaigns. Even with the best of intentions and carefully planned creatives, consumers can respond in a deeply negative way. What does this mean for brand perceptions in the long run? How long will it take for the brand to return to previous baselines in regards to consumers’ impressions? Does the brand need to find a new target entirely, having alienated the original target audience? I am hoping my research will help in some way guide our clients who are in similar situations back to a healthier place for their products.
When not studying ad impact at work or boring my husband with it at home, I spend my time with my two adorable cats and am a big scooter fan, and have been in an all girls’ scooter club for nearly 5 years now.
If you’re interested in how Dynamic Logic can help your next campaign perform optimally for you. We invite you to call us at 212-844-3700 or email us at answers (at) dynamic logic (dot) com. Tell them that you read about us here!
Google and Dynamic Logic Co-Present Research on Rich Media Ad Effectiveness at ARF June 24, 2009
Posted by Mark Blei in : Uncategorized , add a commentAt the ARF’s Audience Measurement 4.0 yesterday, Amy Fayer from Dynamic Logic co-presented results with Google from a custom MarketNorms study measuring the effectiveness of different ad formats at achieving different branding goals. We have released this white paper for you to download (pdf),
There are already a few articles in which the research has been covered - Video In Rich Media Ads More Likely To Lead Customers To Purchase by MediaPost and WebProNews Google Analyzes Rich Media Ad Effectiveness .
If you have any questions related to this presentation or our capabilities please feel free to call us at 212-844-3728 and tell them you read about it in the Dynamic Logic Blog!
For new readers or readers coming over from our old Blogger site June 23, 2009
Posted by Mark Blei in : Uncategorized , add a commentWe have added a link to allow you to subscribe to our blog daily in newsletter format by going here and filling out the requested information.
Additionally the comment spam is a little thik around these parts . I might miss your comment or it might take a very long time for me to approve it since each needs moderation. If you want to make a comment or draw my attention to a legitimate one please use the comment form in the about us section over here.
Thanks for stopping by. we are working to make this new site a bit better and more functional every day.
Mark Blei
Meet Stefan Sellberg June 23, 2009
Posted by markblei in : Staff posts, Uncategorized , add a commentLast week we began a new feature where we introduce you to a few new staff members at Dynamic Logic every week . Here they will discuss their specialties, new challenges they may have faced and how the challenges were overcome, interesting projects they have worked on and new things that they have learned.
In invite you to meet Stefan Sellberg
“My name is Stefan Sellberg and I’m the Vice President of Client and Market Development at Dynamic Logic. I have been with Dynamic Logic for over 8 years and I’ve really enjoyed watching the industry mature as an Advertising and content delivery mechanism as well as a platform for conducting Research.
A client came to me earlier in the year asking if we could help them measure a campaign leveraging TV, Print, Online and OOH. They were deploying a two pronged creative approach using both brand and topical advertising. The goals and objectives were to understand the following:
- How is each medium working in isolation and what are the synergistic effects of exposure to multiple mediums
- What creative theme (brand vs. topical) is working most efficiently in driving Brand and Persuasion metrics
- What is the ideal media mix and how can I can reallocate media to optimize campaign performance
I recommended using our CrossMedia solution that incorporates a path model and media simulator/optimization tool. This approach would provide the client a good understanding of campaign performance and allow them to adjust GRP, Print insertion dates and online delivery to optimize performance. While I cannot get into specific results due to client confidentiality, I can say the campaign measurement approach was a huge success. The client found the media simulator extremely helpful and has since made recommendations to adjust GPR weigh to optimize performance.
In addition to media and advertising effectiveness measurement, I like spending my spare time golfing, camping, and enjoying life with friends and family in the thousands of restaurant and bars in Manhattan”
If you’re interested in how Dynamic Logic can help your next campaign perform optimally for you. We invite you to call us at 212-844-3700 or email us at answers (at) dynamic logic (dot) com. Tell them that you read about us here!
Meet Jennifer Schulze June 22, 2009
Posted by Mark Blei in : Staff posts , add a commentLast week we began a new feature where we introduce you to a few new staff members at Dynamic Logic every week . Here they will discuss their specialties, new challenges they may have faced and how the challenges were overcome, interesting projects they have worked on and new things that they have learned.
We hope that it will serve as a vehicle to allow you to get to know us better and through this to have a better understanding of our capabilities and the types of unique challenges we meet head on everyday.
I invite you to meet Jennifer Schulze.
My name is Jennifer Schulze and I am on the Client Service team in the Chicago office. Something I noticed that has been coming up again lately is what a powerful online segment moms are, particularly within the social media landscape. According to a recent eMarketer article I came across, an estimated 34 million mothers go online at least once a month and given that moms tend to be the primary purchasers for the household, it is no surprise that marketers continue to be eager to tap into such a segment.
I see an increasing number of brands opting for highly customized ad opportunities in hopes of gaining favor within this group. One of my clients within the CPG consumables industry has been experimenting more and more with completely customized ad spends within popular social media and community spaces geared towards moms. They have seen positive results from such ventures thus far. For their study that recently closed, we have observed that customized placements heavily integrated throughout mom communities provided increases in purchase intent and other key metrics. I am predicting that this client as well as others will continue to expand their efforts to better market to moms.
The creative executions utilized in these campaigns will no doubt play a vital role in efforts to build relationships with and persuade moms. For any campaign online ads must stand out, clearly convey the message, and have relevance. Hopefully those with moms in mind will take steps to ensure that they are using the most effective and efficient creative to achieve their advertising objectives
If you’re interested in how Dynamic Logic can help your next campaign perform optimally for you. invite you to call us at 212-844-3700 or email us at answers (at) dynamic logic (dot) com. Tell them that you read about us here!
.
Meet Vincent Blaney June 17, 2009
Posted by Mark Blei in : Uncategorized, white paper , add a commentThis week we began a new feature where we introduce you to a few new staff members at Dynamic Logic every week . Here they will discuss their specialties, new challenges they may have faced and how the challenges were overcome, interesting projects they have worked on and new things that they have learned.
I invite you to meet Vincent Blaney
We hope that it will serve as a vehicle to allow you to get to know us better and through this to have a better understanding of our capabilities and the types of unique challenges we meet head on every day.
Hi, my name is Vincent Blaney and I am a Associate Research Director based in the London office, I live in Maida Vale which for those who do not know is just round the corner from Abbey Road and being a big Beatles fan I kinda like that.
Currently I am very interested in the use of online video and approaches to it. It seems to me from being involv
ed in several projects (using video on sites rather than pre-roll) that there is currently a tendency to pick up a TV ad and drop it on the net. As of yet I have not seen this work for the brand and I think there needs to be a development in terms of adapting these ads for the online environment. Video does work but you need to put yourself into the consumers shoes and approach it in the correct way, I am sitting at my PC now listening to Arcade Fire and my exposure to a video unit will be purely visual. Lets make some small but significant changes to improve them e.g. subtle brand logo presence and subtitle text to re-enforce what may otherwise be lost.
I try to enjoy my life outside of work by playing Golf quite badly and often ill-tempered, but also love to travel and see new countries and cities. Music and reading are both passions of mine which I love to relax with and I am currently struggling to learn Spanish which I hope I succeed at because otherwise my Spanish girlfriend may kill me!
If you’re interested in how Dynamic Logic can help your next video campaign perform optimally for you. Or, if you have some golf tips to keep Vincents temper down. I invite you to call us at 212-844-3700 or email us at answers (at) dynamic logic (dot) com. Tell them that you read about us here!
