How do I measure Direct Mail?
Direct Mail DoE, using holdouts, is the classical approach for measuring Catalog and Direct mail. When expertly designed, has the ability to deliver on the promise of incrementality measurement at the drop and keycode level.
Typically, holdouts are selected by picking the nth name within each keycode (or granular RFM-type segment) within each campaign. By carefully selecting test and control groups from both house CRM and prospecting files DoEs can control for addressable and non-addressable media as well as non-media factors at the group level, focusing on the true incrementality provided by Catalog tactics at the campaign and audience level once statistical significance has been achieved, usually in a matter of months for most catalog focused retailers. These reads can then be meaningfully updated afterwards for each new catalog campaign, typically in as little as a week after any new mailing when Measured is providing the match back analysis.
MMM measures Catalog impact on aggregated business outcomes at the channel or sub-channel level, providing insights into channel contribution relative to other addressable and non-addressable media and informs strategic planning and forecasting at the annual or quarterly level.
MTA attributes the impact of catalog impressions on conversion events at the user level. For CRM related catalog activity based on an internal file of existing customers, catalogs are typically mailed to known users then connected to online or in-store purchase events based on an internal CRM file and known user ID. However, in order to accurately attribute credit to catalog at the impression level, even for known customer IDs, it is required to connect CRM based user IDs typically based on place ID (name/address) or email to digital impressions and user IDs served across multiple devices (desktop, mobile, browsers, etc.) typically based on cookie IDs. In order to do this effectively a third party identity and cross-device provider is typically required at potentially significant additional cost to the marketer. Without this additional data service MTA typically over attributes credit to catalog as it inaccurately assumes that little or no digital impressions have been served to or impacted the decision making process. Even when this service is included in MTA at an additional cost, match rates rarely exceed 50%. Prospecting catalog marketing activities executed via third party vendors are subject to all of the same aforementioned limitations with the added complexity that mailings sent at the place ID level must also be connected to conversions by unknown users either online at the cookie ID level or in-store via credit card or other PII collected at the time of purchase. All of these challenges for both CRM and prospecting at the addressable level are in addition to MTAs previously mentioned and well known limitations in accounting for non-addressable media and non-media factors.