An overview of industry viewpoints on measurement techniques
As the digital ecosystem rapidly evolves, Multi-Touch Attribution (MTA) is gaining noticeable traction among forward-thinking digital players in our industry. Many advertisers, however, have been using Marketing Mix Modeling (MMM) for much longer and are hesitant to move beyond this tried and true methodology. For a detailed definition of MTA and MMM, please see Appendix I.
The conversation is ongoing over which of the two is better, with arguments ranging from turnaround time to granularity, cost, and actionability. Despite the debate, we believe that these two approaches are not mutually exclusive and could be complementary when used correctly. In fact, several companies have applied their own unified approach to combine the benefits of both MMM and MTA. However, confusion persists among many marketers and other stakeholders as vendors do not have a common definition of a unified approach.
To address this topic, the IAB Measurement & Attribution Committee assembled a working group, conducted an industry-wide survey, and convened an expert panel to understand the adaptation and the applications of these two approaches to bring clarity to the marketplace confusion. This paper focuses on answering the following:
- Why and how to use MMM and MTA together
- What to expect from MMM and MTA including restrictions, and tradeoffs
- How to choose a solutions provider
Rather than endorse any specific approach, we aim to empower our members with knowledge and transparency so each marketer can make their own informed decisions.
Outcome-Based Measurement: Focusing on the Business Bottom Line
A panel of experts convened by IAB highlight an important aspect of outcome-based measurement: It provides a simple number that allows CMOs to prove to their boards the value of marketing efforts, and it allows marketers to speak the same language with stakeholders across their business.
Media effectiveness has historically been measured in terms of delivery metrics like reach, frequency, and Gross Rating Point (GRP) — with results often sitting outside of a comprehensive and consistent measurement framework. This has made it difficult for marketers to measure the impact of their marketing investment in a cohesive way. However, within the digital media space we can measure both media delivery (reach, frequency) and performance metrics (conversion rate, engagement) together so that we can link performance, reach, and revenue into a single view of media investment and how it drives growth. This is the ideal scenario. However, since some of these metrics do not directly translate to the business bottom line, it’s not always possible — and this keeps executive teams up at night.
The industry has been shifting its focus from vanity metrics such as click-through rate (CTR) to outcome-based measurement as the latter ties media performance to the metrics that are crucial to a business’ growth. This enables marketing leaders to assess the performance of their media investments with clearly defined financial-related metrics such as sales or store visits. In other words, this empowers marketers to better understand where their media investments are driving growth and plan for future investments in a more data-driven way. Furthermore, outcome-based measurement and reporting build consistency and familiarity over time for marketing leaders, strengthens a business’ ability to plan and evaluate all their marketing investment options by making results accessible and easy to interpret.
Industry Performance Benchmarks
While MMM has been used for decades in marketing measurement, MTA was developed when digital media became popular. As with every relatively new measurement method, there have not been many benchmarks available to the industry.
Industry benchmarking can be a tricky subject, especially when applying various techniques to quantify ROI. There are a variety of factors that affect the amount of marketing-driven sales that will be quantified in a measurement solution, including the product category, sales, and media channels, amount of marketing investment (both absolute and as a percent of sales), data availability, data freshness, and many more.
Reconciling the measured ROI performance within a brand’s marketing ecosystem can be further complicated by the number of data and media partners that have their own bespoke methodologies. This proliferation of single channel, single partner, limited data measurements is driving brands to a standardized, cross-channel solution.
During the panel discussion, it was agreed that regardless which technique, brands switching to a new measurement vendor often report double-digit returns in marketing investment performance, reflecting the lack of a true baseline comparison. Our expert panel shared their perspectives: We have seen MTA studies claim that the performance improved from 3% to 30% or higher. 30% seemed to be on the higher side, yet still possible, especially in a new study, with a new vendor where a baseline is lacking. However, the greater the number is, the harder it is to sustain.
Most of our panelists agreed that 3 to 4% is more realistic, especially in an ongoing study.
As for MMM vs. MTA, MMM tends to produce higher marketing contribution measurements. This is primarily because it can include more touchpoints than MTA, especially offline non-addressable tactics such as TV, radio, out-of-home (OOH), point-of-purchase (POP), and sales promotion. When comparing the benefit of MMM and MTA, there are tradeoffs to each technique. MMM tends to provide more measured marketing ROI because it can include more touchpoints and provide an understanding of how to improve ROI. MTA provides more tactical understanding of how to improve ROI, but for fewer touchpoints.
Reasons Marketers Use MMM and MTA
In addition to validating marketing ROI to clients or management, many survey respondents shared that they use MMM for prediction and planning/budget allocation. In their own words:
As for MTA, most respondents use it to measure digital campaigns:
Challenges and Benefits of Using Both MMM and MTA
Most survey respondents and expert panelists mentioned that they use MMM and MTA for different purposes — and that the results often do not match. Marketers saw conflicting information and struggled to marry the learning from both in a meaningful way. Some mentioned that they have different teams owning each measurement and that they don’t integrate data from both studies.
Some of our panelists believe that MTA alone is a flawed approach and that the way to address the issues is with a unified approach. However, while a unified approach solves the challenge of having multiple vendors and reports that are not well aligned, planning and implementation become more complex. For specifics, please see the section called Understanding the Restrictions and Tradeoffs.
When Not to Use MMM or MTA
While MMM and MTA have different setup criteria, use different datasets, and apply different methodologies, they do share some commonalities: Both approaches need careful analytic design, proper implementation, and someone to interpret the results and apply the learnings to future plans. Without proper resources, there is a high likelihood of failure.
Specific constraints for MMM and MTA include the following:
For MMM to obtain results that are meaningful and actionable, the marketing investment needs to reach a certain scale. While most people agree with this concept, there’s no agreement upon a minimum threshold.
Some long-time veterans suggest 50-100 GRPs if the advertisers are using network TV, depending on the length of the flight. However, it does not apply to advertisers who do not use national TV since iGRP does not translate the same.
For MTA the threshold is lower since it measures fewer touchpoints and it can be set up on a campaign level instead of a multi-year modeling dataset. Nevertheless, there are other trade-offs of scale. In MTA, the complexity increases with the scope, from the number of media channels and platforms to different KPIs, to the overall size and the duration of the campaigns. For more information, see the sections below for Media Mix and Offline Metrics.
While the cost of implementation and support vary, no one has ever said running MMM or MTA is cheap. Our survey results revealed that cost was one of the common pain points for both MMM and MTA for advertisers.
However, when compared to the overall marketing budget, the cost of measurement can be relatively low. One general reference suggests that MMM isn’t worthwhile if the annual budget is less than $5 million for a minimum of two consecutive years, whereas MTA may be supported on a campaign basis only as part of ROI planning. Of course, this is a rough estimate and there are other conditions things to consider, such as media mix.
MMM works when there is variation in media mix, but not such drastic variation that it’s outside of historic bounds, especially when the results are to be used for future planning. For example, if the media mix is a continuous flight with the same weight for each platform, MMM is not the right approach. Nor will it work if there’s a drastic change to the media mix, say from 70% TV and 30% digital to 90% digital and 10% TV, as there will be no benchmark for the model to function.
On the other hand, MTA works well for many digital campaigns due to its trackability. One of the most challenging aspects, however, is working with walled gardens such as Google, Facebook, and Twitter since third-party measurement providers won’t have visibility into user-level impression data inside the walls. If the media plan has a sizable portion in these walled gardens, the solution for walled garden measurement will be well-executed A/B testing and/or MMM.
What’s the best way to manage walled gardens?
MTA was originally designed to track digital exposures to digital conversions. However, few brands are allocating 100% of their budget to digital. There are several challenges to accurately tracking online and offline media and conversions to the same individual (i.e., identity resolution) from insufficient data, shared devices, and cookie rejections, to flawed matching methodologies. While it is possible to apply MTA to an online/offline campaign with both online/offline KPIs, it adds more complexity and potentially increases the error rate from misattribution.
Understanding the Restrictions and Tradeoffs: Data Collection, Data Governance, Identity, Walled Gardens, and Privacy
A robust data collection plan is imperative for any measurement solutions (“garbage in, garbage out”). The biggest hurdle to timely delivery of insights across MTA and MMM is often delivery of accurate data. Not only do data restatements delay results and disturb model accuracy, they also add costs by way of rerunning or restating results. Delivery issues are particularly pronounced for MTA since results are delivered and used daily or weekly, while MMM results are typically delivered monthly or quarterly.
Strong incoming data delivery practices are required for both MTA and MMM to ensure data accuracy. Frequently occurring data issues like spend actualization for TV and radio media types can be dealt with systemically. Where delays occur due to spend actualization, planned spend can be used and adjusted appropriately until the actualized data is ready to replace it.
When setting up MMM and MTA solutions, consider all possible data sources. Typically, there is a tradeoff between completeness, time to deployment, and solution complexity. For MTA it may be straightforward to gain access to traditional browser-based ad exposures, but mobile/in-app inventory, over-the-top (OTT) exposures, walled gardens, and non-digital addressable data each present their own challenges.
A user of MTA must consider their entire marketing budget and understand that the fastest-to-deployment solution may only cover a fraction of media spend. A solution built in phases can find a balance between completeness of data sources and time to deployment, where the lowest-hanging fruit is built first, and harder to access data sources are added in the future. What becomes critical with this approach is a modeling methodology that accounts for the to-be-added marketing channels.
On the MMM side, the same tradeoff between completeness and time to deployment exists, however the pain points tend to lie in capturing adequate historical data. Generally speaking, MMM requires roughly three years of historical data across media types and model outcomes (KPIs). In many cases, however, history is lost or was never recorded. As a result, media which is just being tested in market is hard or impossible to read. MMM often focuses on large time-period data deliverables with measurement at the media channel level. Running the analysis with an overly simplistic data delivery format can obscure underlying data issues, like a missing media type, campaign name, or other variables. Applying the right data filters during a data pull (and checking this data) becomes critical to an accurate model.
As MTA solutions become more complex and data sources proliferate, a rich suite of data validation and audit reporting tools are critical for success. This suite must automatically flag data issues and raise them to relevant parties, as well as provide easy-to-digest human-readable reports so any user can perform a high-level health check on the data collection plan, similar to solutions in place for MMM today.
Due to the omnichannel nature of modern marketing plans, identity resolution is a critical piece to any MTA project. Cookie-based measurement, while still useful, tells an incomplete story. The most successful MTA projects are built on an identity graph that allows marketers to tie their easily trackable digital addressable channels, walled garden platforms, offline addressable channels, and first-party data to a single identity graph through personally identifiable information (PII) onboarding and direct integrations with platforms. While dozens of companies claim that they have developed a great solution, there are gaps yet to be addressed.
Walled gardens have been difficult to measure, but times are changing. Platforms like Google, Facebook, Twitter, and others have historically been protective of their user-level data in the name of privacy and allow marketers to granularly measure performance only within the context of their platform.
A/B testing and macro-level measurement (e.g., MMM) has always been possible for walled garden advertising. However, recent developments in data clean rooms and advertiser demand for transparency have slightly opened the door for granular measurement. A select few measurement vendors have established relationships with these platforms to enable transparent cross-channel measurement.
We live in a rapidly evolving privacy landscape. General Data Protection Regulation (GDPR), the California Consumer Privacy Act (CCPA), the NAI 2020 Code of Conduct, and browser makers’ war on third-party cookies is making granular measurement more difficult than ever. The measurement industry must respect consumers’ privacy preferences while at the same time work within the new regulations to create useful and actionable measurement solutions. MMM deals primarily with de-identified and aggregated data sets, so the effects of the new privacy landscape are mostly limited to MTA projects.
Trackability of users across digital properties is significantly degraded in this new privacy landscape, especially when technologies rely on traditional browser-based third-party cookies. Innovative tracking technologies like browser fingerprinting can help marketers temporarily mitigate the limitations but at the expense of their users’ privacy. There are other methods like first-party PII onboarding and probabilistic clustering that can help as well. Marketers must weigh the tradeoffs when making decisions about their measurement tech stack.
In the larger view of a consumer or customer’s identity, we believe that an industry-wide standard is necessary to successfully continue to measure and act on identity. That is why the IAB is currently working with all the identity consortiums, as well as the browser makers, to come up with a democratized solution that benefits both advertisers and consumers.
What is needed is a holistic solution that is privacy compliant: A solution that provides relevancy and enables meaningful connections between advertisers and consumers without creating an identity arms race across the ecosystem; a solution that enables robust privacy controls that the consumer can manage on their end.
Choosing the Right Vendor for Measurement
Choosing the right vendor(s) is critical to driving campaign, marketing, and business success. Given the evolving landscape and breadth of providers across MTA and MMM, it’s important to ask the right questions before diving in. Key areas to investigate include:
Vertical Knowledge and Experience
Vendors should work with advertisers to understand their industry. While some measurement basics are the same across industries, developing hypotheses about what influences sales requires specific knowledge. This experience also allows for more accurate inclusion of the most important drivers of sales in the model. The more comprehensive a model is, the more likely it will accurately attribute influence on the right touchpoints.
Vertical experience is often seen in a vendor’s benchmarks. Robust benchmarks come from deep experience.
Transparency and Clarity of Modeling Approach
Understanding how a vendor developed their model and its inner workings makes it easier to understand its distinct point of view, limitations, and the results. Vendors should be willing to explain in detail their approach and even open their black box to show how their metrics work. Many vendors publish all or parts of their approach to allow peers to scrutinize and improve on it.
Granularity, Accuracy, and Access to Data
How deep and wide is the data pool, how correct is it, and how automatically does the vendor collect it? This is the meat and potatoes of measurement, but these elements make up most of the meal. Does the vendor have direct access to data and the right partnerships? Does the vendor use a single, centralized analytics platform? Is the data granular enough to make actionable decisions (e.g., store-level data versus a geographic area)? What regular testing does the vendor do on the data to ensure errors (there are always errors) are caught, corrected, and eliminated before reporting?
How Actionable Are the Insights
Gone are the days when reporting on what happened is enough. Today, all measurement needs to point to what’s next, and it needs to be specific enough to drive strategy and allocation. Specific insights around campaigns, tactics, brands, distributors, or geographies can drive specific actions and improve results. Reports that show what happened at too macro a level don’t allow teams to tease out what is really driving results or how to optimize performance.
Conclusion: The Evolution Continues
The digital revolution has unleashed enormous new measurement and attribution capabilities, with some predicting that the industry would quickly achieve the holy grail of understanding precisely which ads and media spur web and store visits, purchases, customer engagement, and more. After all, the advertising and marketing industry had never seen such an impressive application of data, technology, and investment in such a short period of time.
The reality is more complicated. That same revolution has also brought enormous complexity. And, as with all revolutions, it hasn’t slowed down long enough for the current challenges to be addressed before new ones arrive.
Success in this world should be measured in how far we have come, not necessarily how far we have to go, and the industry’s capabilities in measurement and attribution have come a long way indeed. This can be seen across the ecosystem, but perhaps most acutely in how the relationship between MMM and MTA has evolved. Industry leaders continue to vigorously seek the attribution holy grail, now more than ever with a more unified understanding of how both MMM and MTA can contribute.
Appendix I: MMM, MTA, and Unified Measurement Models Defined
Marketing Mix Modeling (MMM)
Established in the 1960s, Marketing Mix Modeling or MMM is a statistical analysis of aggregate sales, marketing, and business drivers data that quantifies the impact of different marketing channels and tactics (the marketing mix) on financial outcomes over time. The result is insights and recommendations that can be used to optimize marketing investment allocations and predict future outcomes.
MMM attempts to answer questions such as: “What was the return on ad spend on mobile last year?” and “What would sales be if we shift 10% of the budget allocation to addressable TV?” Since MMM measures results on an aggregate level, it can include marketing tactics that are not addressable such as radio or broadcast TV.
Multi-Touch Attribution (MTA)
The ultimate goal of Multi-Touch Attribution or MTA is to properly measure the impact of marketing activities on the metric associated with a conversion event at a granular level and to use these insights to guide decisions about future marketing spend. The fundamental question that MTA seeks to answer is “What is the expected change in propensity to convert that was the result of an impression (or any form of interaction with the customer)?” This measurement must also consider the innate propensity that different customers will convert (e.g., purchase) without any exposure to marketing.
Many attribution methods are based on predetermined weights that are used to proportionately assign credit for converting events (e.g., purchases) to the marketing treatments preceding it. Simple weight-based allocations like first- or last-click, equal attribution, or time-dependent weights do not get to true incrementality.
Unified Measurement Model
A new marketing measurement approach that integrates multiple statistical techniques, such as, but not limited to MMM and MTA, to assign business value to each strategic and tactical factor affecting marketing performance across all customer touchpoints. It resolves the differences commonly seen when MTA and MMA are used separately.
There are several ways to build a unified measurement model; one approach is a three-step implementation process involving MMM models that inform MTA and are recalibrated to ensure consistency.
In the first step, marketing mix models are created, quantifying the impact of addressable and non-addressable business drivers. In the second step, an MTA model is built using inputs from the MMM. The non-addressable impacts are fed into the MTA model to control for non-addressable drivers. The addressable coefficients from the MMM model are used to calibrate the estimation of the MTA coefficients. The third step in the process is the alignment of coefficients of the addressable media across the MMM and MTA models to ensure consistent measurement.
Appendix II: The Current MMM and MTA Landscape (Survey Results)
Discovery: What Users Say About MMM and MTA
To better understand how MMM and MTA are perceived and used, the IAB Measurement & Attribution Committee conducted a fact-finding survey to gauge usage, familiarity, benefits, challenges, pain points, and future plans for MMM and MTA. The study collected responses from 116 advertisers, agencies, media/publishers, and marketing companies from November 2018 to January 2019. Based on the survey results, we learned that:
- MMM is more broadly adopted than MTA, especially among advertisers.
- Cost, knowledge, data collection, data quality, and transparency are common pain points for both approaches.
Marketers are more familiar with MMM. Survey results indicated that our respondents are more familiar with MMM than MTA, especially among marketers (advertisers and agencies). While the confidence among marketers seemed high, we did notice a repeated theme of knowledge and transparency cited as common pain points regardless of respondent backgrounds. It supports our approach of education, clarification, and simplification as the base for reconciliation.
MMM is also more adopted than MTA. Overall, MMM is more adopted among our respondents than MTA. And marketers are more likely to use MMM than other industry players.
This finding, along with the level of familiarity and common pain points, established a solid base for us to collaborate and reconcile MMM with MTA.
- Overall, MMM is more adopted among our respondents than MTA. And marketers are more likely to use MMM than the other industry players. This finding, along with the level of familiarity and common pain points, established a solid base for us to collaborate and reconcile MMM with MTA.
The high cost of adoption and lack of knowledge, data collection, data quality, and transparency are the common pain points for both MMM and MTA.
- While marketers like the holistic approach for MMM, especially when it comes to budget allocation, there are some pain points too including cost, timeliness, and not enough granularity.
- When it comes to MTA, understanding the consumer journey was mentioned repeatedly as a benefit. However, the lack of transparency, lack of understanding, and questionable methodology were called out as a disadvantage, in addition to its focus on digital media only and its lack of offline attribution.
- The lack of transparency in the methodology was mentioned under both MMM and MTA as a reason for not using either approach.
- While lack of understanding could be the overall barrier, for respondents who are familiar with both, the hurdle of reconciling MTA learnings with historical learnings and MMM insights is the key to adopting both.
The Interactive Advertising Bureau (IAB) empowers the media and marketing industries to thrive in the digital economy. Its membership is comprised of more than 650 leading media companies, brands, and the technology firms responsible for selling, delivering, and optimizing digital ad marketing campaigns. The trade group fields critical research on interactive advertising, while also educating brands, agencies, and the wider business community on the importance of digital marketing. In affiliation with the IAB Tech Lab, IAB develops technical standards and solutions. IAB is committed to professional development and elevating the knowledge, skills, expertise, and diversity of the workforce across the industry. Through the work of its public policy office in Washington, D.C., the trade association advocates for its members and promotes the value of the interactive advertising industry to legislators and policymakers. Founded in 1996, IAB is headquartered in New York City.
For more information, please visit iab.com.
About the IAB Data Center of Excellence
The IAB Data Center of Excellence is an independently funded and staffed unit within IAB, founded to enhance existing IAB resources and to drive the data agenda for the digital media, marketing, and advertising industry. The Data Center of Excellence’s mission is to define boundaries, reduce friction, and increase value along the data chain, for consumers, marketers, and the ecosystem that supports them.
For more information on how to get involved, please contact [email protected].
This report would not be possible without the guidance and direction of the IAB Reconciliation of Marketing Mix Modeling and Multi-Touch Attribution Working Group led by Joe Pilla, IAB Data Center of Excellence. We extend our thanks and deepest appreciation.
We would also like to acknowledge the contributions of the following individuals and organizations:
Sable Mi, Chief Research Officer, NinthDecimal (Co-chair)
Peter Minnium, President, Ipsos (Co-chair)
Karl Brautigam, Disney Interactive
Ferdinand David, Dunn & Bradstreet
Margit Kittridge, Dynata
Mike Menkes, Analytic Partners
Matt Zambelli, Neustar
IAB Reconciliation of MMM and MTA Working Group
Ahava Digital Group
Allen Media, LLC
Dun & Bradstreet
IAB would like to extend a special thanks to the following individuals for sharing their professional perspective.
Lindsay Blanch, Hill Holiday /Trilia Media
Jessie Dawes, Shiseido
Ross-boy Link, Marketing Attribution
Maggie Merklin, Analytic Partners
Shannon Versaggi, Lowe’s Companies