Understanding Geo Lift Testing in Marketing: A Comprehensive Guide

This article explains how Geo Lift Testing helps to understand the effectiveness of marketing strategies and make data-driven decisions.

What is geo lift testing?

Geo lift testing, also known as geographic lift testing, is a method used in marketing research to measure the impact of advertising campaigns or other marketing measures. It involves selecting a specific geographical area in which a particular advertising measure is carried out, while this measure is not used in a comparable control area. The idea is to compare sales figures or other relevant performance indicators in both areas in order to determine the actual effect of the advertising on sales or customer behavior.

Important aspects of Geo Lift Testing are:

1. selection of regions: The test and control regions must be carefully selected so that they are comparable in terms of demographic, economic and other relevant factors.

2. measurement of the results: Results are analyzed by comparing sales data or other KPIs (Key Performance Indicators) between the test regions and the control regions. This may involve simple comparisons or more complex statistical analysis.

3. adjustment for confounding factors: It is important to control for external confounding factors that could influence the results. For example, this could be seasonal fluctuations or other marketing activities.

Once the "lift" or impact has been established in the test region, this effect can often be used to extrapolate the potential impact of the campaign to larger or other markets.

Geo Lift Testing is often used in industries where regional marketing campaigns play a role, such as retail, fast-moving consumer goods or the telecommunications industry. This method has the advantage of providing relatively realistic insights into the effectiveness of marketing strategies as it takes place in a natural environment, as opposed to simulated tests or surveys.

Definition of Geo Lift Testing

Geo Lift Testing is an experimental research method in the field of marketing that is used to evaluate the effectiveness of advertising measures in specific geographical regions. In this approach, at least two geographically defined areas are selected: a test region in which a specific advertising or marketing campaign is carried out and a control region in which this measure does not take place. By comparing performance indicators such as sales figures, customer visits or other relevant metrics between these two regions, the direct influence of the marketing measure on consumer behavior or sales can be viewed in isolation. This approach helps companies understand the extent to which their promotional efforts actually lead to an increase in business performance by measuring the "lift" or increase generated by the marketing actions.

Difference between geolift, multi-touch attribution (MTA) and marketing mix modeling

Geo Lift Testing, Multi-Touch Attribution (MTA) and Marketing Mix Modeling (MMM) are three methods used to analyze and evaluate the effectiveness of marketing strategies, with each method having its specific strengths and limitations.

Geo Lift Testing focuses on the effect of marketing measures in geographically defined areas. Test regions in which campaigns are carried out are compared with control regions that do not experience such advertising measures. The main advantage of this method lies in its ability to provide clear and causal evidence of the effectiveness of a marketing action, particularly useful for offline and localized campaigns. However, Geo Lift Testing is limited to geographic analysis and requires comparable regions to achieve valid results, making it less suitable for digital or globally oriented campaigns.

Multi-Touch Attribution (MTA), on the other hand, offers a detailed analysis of all marketing contacts along the customer journey. By evaluating each touchpoint, MTA can show how individual interactions across different channels contribute to the end goal. This is particularly valuable for companies with complex digital marketing strategies that want to enable precise optimization of budget allocation to the most effective channels. The challenges with MTA lie in data privacy regulations and the complexity of fully capturing all customer interactions in times of diminishing signals, as well as the possible neglect of synergies between channels.

Marketing Mix Modeling (MMM) uses historical data to analyze the effectiveness of various marketing elements as well as external factors such as the economic climate or competitor actions. MMM is excellent for strategic planning and budgeting as it provides a broad overview of the entire marketing mix and its impact. However, the method can be less effective in dynamic markets or when evaluating new marketing channels, as it offers less granular insights at the level of individual customer interactions and does not provide real-time results.

In summary, each method offers valuable insights depending on the specific needs and circumstances of the business. Geo Lift Testing is ideal for causal analysis in physical regions, MTA provides deep insights into digital customer journeys, and MMM provides a comprehensive perspective on the influence of the entire marketing mix, including external factors.

Marketing mix models (MMM) and geo-experiments can be used together. Use the results of a geo-lift test to adjust your MMM results and ensure that they are as close to experimental causality as possible. You can also use GeoLift to determine the impact of the MMM budget recommendation per channel and compare the impact with the usual business campaigns.

As there are fewer and fewer mechanisms such as cookies and mobile IDs, techniques based on aggregation and modeling are becoming increasingly necessary. MTA in particular is becoming increasingly inaccurate. However, these changes do not affect MMM or geo-experiments that use aggregated data.

While triangulation is the buzzword here, Google seems to be focusing on calibration. In their Modern Measurement Playbook, they propose a solution to the question that constantly comes from marketers: "How do I use my different measurement methods together?"

Here's how Google's calibration method works:
1️⃣ Take your attribution ROAS (say for Q1)
2️⃣ Take your geo-experimental iROAS (say from a test you ran in Q1)
3️⃣ Divide the iROAS by the ROAS, to get your calibration multiplier
4️⃣ Multiply your future attribution ROAS (for Q2) by the calibration multiplier to get an estimated iROAS for Q2.

This example of calibration is good because it shows how to combine geo-testing with attribution ROAS (for Google, attribution ROAS means reports within the platform, but it's probably meant to mean Google Analytics or MTA reports as well).

It also shows how you use experiments at specific points in time to determine the incrementality of future periods. This is helpful because most marketers don't do ongoing incrementality testing (there are both operational and financial costs to ongoing testing).

Technologies and tools for geo lift testing

Tools such as Haus and Twigeo, as well as the open source options from Meta and Google, offer simple approaches to performing geo lift testing. The underlying methods are usually the following.

Differences-in-differences: This is the most popular and simplest approach to perform an inference for a geo test. Usually not recommended unless the predicted effect size is really large. DID has fallen out of use mainly due to its reliance on the assumption of parallel trends, which rarely occurs in actual experiments.

CausalImpact (Google): Uses a Bayesian time series modeling technique to predict the counterfactual scenario. This is particularly useful when strong prior knowledge is available or when the KPI metric is not available and modeling must be done using a proxy.

GeoX (Google):: Uses a linear model to predict the counterfactual time series. It is positioned as a more robust solution for paired market studies and also has superior small sample properties (small number of test/control markets).

Synthetic Control Method (Synth): Uses a weighted combination of conversions of the control markets to create a counterfactual scenario for test markets. This method has gained a lot of popularity recently, but can produce biased results if a perfect match cannot be found from the pool of controls.

Augmented Synthetic Control Method (ASCM): The Augmented Synthetic Control Method (ASCM) is an extension of the Synth method. Modifications to the original method include covariates to improve the test setup and reduce bias, as well as negative weights in the weighting system. Although these additions provide more flexibility and adaptability to meet marketing needs, the use of covariates without careful consideration can have the opposite effect and introduce new biases.

Each method has its own characteristic features, and one might suit your testing needs better than another. You also have the option of combining methods to take advantage of certain features that are present in one method and not in the other to optimize a test design for your individual case. For example, you can start with ASCM to create a synthetic control group and then use techniques such as stratified matching to refine the selection of the set-up. A case like this is very advanced and is usually beyond the requirements of how most tests are conducted.

In most cases, the focus should not be on selecting the "right" method, but on defining the target, understanding the underlying data and applying the right transformations. It is an iterative process.

Structural problems of geo lift testing

• The inability to synchronize targeting between different platforms such as Google and Meta means that inaccuracies are a given.
• Location data is inaccurate and unreliable.
• Cost and resource intensive: Running GeoLift experiments requires additional resources, including access to location data, technology infrastructure and expertise in data analysis.
• Sampling bias: Geo Lift tests rely on partial data and can lead to unclear or simply incorrect conclusions.
• Conversion loss: If 50% of the target area is not to receive ads, there will be 50% fewer conversions - and that for several weeks. Possibly too high a price to pay to measure effectiveness.
• Seasonality: A 4-5 week Geo Lift test will not be able to factor out the weather - quite the opposite. The weather can also behave very differently in different geographical regions.
• Measurement delay: It can take weeks to analyze the effects. During this time, no far-reaching changes should be made to the campaigns, e.g. budgets, creation.

Why is geo lift testing still relevant?

Geo-testing has become more accessible and the tools have become cheaper, making it a common analysis practice. While geo-testing will never be the most accurate measurement method, it may be the best option available.

The ongoing stream of privacy policies now limits even 1st party data. Many hope that one of the many online measurement solutions being worked on will create a working standard - but waiting for that to happen would be naïve. Advertisers are just beginning to process the impact of Apple's AppTrackingTransparency framework and are getting to grips with GA4 and less data, while more and more data is disappearing.

Data-driven decision making with the help of Geo Lift Testing

By using Geo Lift Testing, marketers can make data-driven decisions based on objective measurements of the actual impact of marketing efforts. This enables companies to allocate their resources more efficiently and continuously optimize their marketing strategies.

Geo-testing: an underestimated tool for growth marketing

Geo-testing is fundamentally a flexible method of measuring incrementality in a privacy-friendly way. It is an indispensable tool for app marketers on iOS (ATT, SKAD) and for all marketers who advertise on hard-to-measure channels such as TV, OTT or ATL. Geo Lift tests require clean data, knowledge of marketing science and an understanding of common methodologies.

As traditional digital tracking becomes less reliable and privacy regulations increase, you as a growth marketer have the opportunity to gain ground with geo-testing.