Every advertiser wishes they knew exactly how their online ad campaigns are performing relative to their competitors. The truth is that the very qualities that have accelerated growth of expenditure towards the digital marketplace are the same ones that make it difficult for marketers to benchmark their performance.
Having the ability to serve creative in a live auction environment is the stuff dreams were made of pre-internet days. Everyone agrees that the power to micro-manage bids against highly desirable and custom targeted audience impressions in real-time is nothing short of awesome. While ad tech continues to perfect the media selling and buying process, the dynamics of an open market keep the ability to accurately compare campaign performances out of reach.
At IAB Canada’s MIXX event last September, David Weinberger delivered an insightful talk where he challenged the validity of A/B testing in a digital world. Weinberger pointed out that the vast amount of variables from one moment to the next in online activity makes it impossible to understand whether one piece of communication is truly more effective than another. This is because each impression is served with its own unique set of circumstances. The variances in these circumstances make gauging performance impossible.
Following are some of the factors that have a profound effect on the performance of a modern online advertising campaign:
Quality of Inventory Purchased
Advertisers that demand higher quality inventory have tight filters set up in their buying program. These filters require varying levels of investment in ad tech and may include, but not be limited to, things like:
- 100% viewable inventory
- Assurances of valid (human) traffic
- White lists to assure brand safety
- Selective exchanges – private exchanges or custom markets
Competitive Environment
As with any auction-based marketplace, the pricing is heavily influenced by the competitive activity. Competitive bidding fluctuates heavily online and can be extremely unpredictable in every category. Advertisers can enter and exit the arena rapidly. Factors like price elasticity and inventory levels present additional complexity to the evaluation of relative performance.
Campaign Strategy
Strategy remains the secret sauce for advertisers. This includes everything from segmentation planning and re-targeting application to careful day parting and the use of frequency capping. Each campaign has its own footprint.
It may seem useful to study year over year campaign performance however, maintaining that same campaign footprint in the interests of benchmarking as opposed to optimizing in real time in the pursuit of better results seems like an absurdity.
Creative
Creative is one of the most obvious obstacles to performance benchmarking. There are infinite renditions of creative pieces that can run online. As discussed earlier, comparing the effectiveness of online ad creative in a largely uncontrolled environment makes it impossible to draw campaign level conclusions. To be clear, a consumer is never presented with two or three ad concepts at the same time. Consumers will not “vote” on your creative for advertisers to understand which piece resonates best. If a single advertiser’s own ad cannot be evaluated in an open-market environment, how can we expect to benchmark against hundreds of competing ads using that same logic?
The Unattainable Promise of Benchmarking
While some efforts are being made to collect and aggregate mass amounts of data to provide advertisers with a notion of benchmarking, this movement presents a slippery slope towards gross misrepresentation of digital media’s effectiveness. This type of data collection is unable to provide what it claims to and in an environment where advertisers are demanding transparency in their digital media investment; the irony of this ambulance-chase is not lost on the supply chain.
Ad Tech has recently come under fire by many skeptics who feel there is an over expenditure in the delivery costs of digital inventory. It seems irrational to propose that advertisers spend even more of their media dollars (in the name of transparency or clarity on value) on a subscription to benchmarking that is based on flawed logic.
The Bottom Line
The pursuit of the holy grail of (unreliable) financial benchmarking of online advertising effectiveness is worth far less than the chase for better strategy, tight controls over inventory being purchased and accurate reporting. Wise marketers know to invest in smart digital strategists who are acutely aware that the devil is indeed in the dynamics.
IAB Canada will continue to work with the industry and its various associations to help define parameters for appropriate evaluation of digital media effectiveness. In the meantime, hold on to your log files.