Incrementality testing for multi-location businesses: a field guide

Incrementality testing for multi-location businesses is a simple idea with a fussy execution. You change advertising in some markets, leave it alone in others, and measure how many more customers came through the doors where the ads changed than the comparison says would have come anyway. That gap is the number your CFO actually wants.

This guide is for the people who run the budget: a marketing lead at a 30-club gym brand, an operator with 60 quick-service units, the agency that buys their local media. It covers which test to run, how big and how long, what data you need, a worked example with the arithmetic shown, and the mistakes that turn eight weeks of work into a shrug.

What incrementality testing means when you have locations

An ecommerce brand can split users into exposed and unexposed groups and watch checkouts. You can’t. Most of your customers pay at a counter, a front desk or a chair, and no pixel follows them there. So the unit of the experiment has to be the market, not the person.

That changes three things. The outcome is a count you already keep by location: guests, new members, new patients, booked jobs, car counts. The treatment is a change in spend by geography, which every major platform supports. And the comparison is other markets, weighted so they behaved like the test markets before anything changed.

In the US the natural unit is usually the Nielsen Designated Market Area, because that is how TV, radio and much local digital is bought. There are 210 of them (Source: Nielsen, 2025), and a brand with 40 locations typically touches 12 to 25. Smaller units such as ZIP codes or radius targets work for search and social, though they leak more.

Why multi-location brands are built for incrementality tests

Here is the quiet advantage. You run the same concept, at roughly the same prices, in many places at once. Weather, holidays, payday weeks and the national economy hit most of those places together. When spend changes in a handful of markets and not the others, the steady markets show what would have happened without the change.

You don’t even need to plan it every time. Spend already moves: a regional manager pushes for more in Phoenix, a new store gets launch money, a budget cut lands unevenly. Each of those is a natural experiment hiding in your data. A planned test is just the same comparison with the messiness removed in advance.

The flip side: one market is not a test. A single holdout city can drift for reasons nobody will ever discover (a highway closure, a rival’s grand opening, a local news story). You need several test markets and a pool of comparison markets large enough to build a good blend.

Which incrementality test design fits your question?

Write the decision down first. “Should we keep spending $40k a month on paid social?” is a test. “Is marketing working?” is a mood. The question then picks the design.

Go-dark tests give the cleanest answer to “is this working?”, and they make finance happy because they save money during the test. They also make regional operators nervous. Heavy-up tests are the easier sell internally, but they answer a slightly different question: the value of the next dollar, not the average dollar. Both are legitimate. Just don’t run a heavy-up and then claim it proved the whole channel pays.

If you want the full mechanics of holdouts, buffers and spillover, our guide to geo holdout tests for multi-location brands goes deeper. Comparing the two main designs head to head is covered in matched market testing vs geo holdout.

The data you need (you probably have it)

• Weekly outcomes by location. Covers from Toast or Square, member sign-ups from your club software, new patients from Dentrix or Open Dental, booked jobs from ServiceTitan. A year of history is ideal; 26 weeks is the practical floor.
• Weekly spend by market and channel. Meta and Google Ads both report spend by region or DMA. For radio, out-of-home and mail, a spreadsheet from your buyer is fine.
• A location list. Location ID, market, open date and close date. That’s it.
• An events calendar. Openings, remodels, price changes, LTOs, local promotions, outages. Anything that moves a market for reasons other than ads.

Notice what isn’t on the list: customer IDs, pixels, call tracking, loyalty matching. Market-level testing works on totals, which is why it survives cookie loss and iOS privacy changes untouched.

How big and how long should the test be?

This is where most tests die, quietly, before launch. If the lift you can realistically expect is smaller than the normal week-to-week wobble in your numbers, the test will come back “inconclusive” and somebody will read that as “ads don’t work.”

A planning rule that holds up: measure the standard deviation of the weekly gap between your test markets and their comparison blend over the pre-period. The minimum detectable effect (MDE) for a standard test is roughly 2.8 times that number divided by the square root of the test length in weeks. With 6% weekly noise and 8 weeks, that is 2.8 × 6% ÷ 2.83 ≈ 5.9%. Weeks aren’t fully independent, so treat it as a floor and pad it.

Then check the budget can produce that lift. If the test markets see 300 new members a week, a 6% lift is 18 members a week. At a guessed $150 per extra member, that takes about $2,700 a week of added spend. Spend less and you are buying an expensive “we don’t know.”

A worked example: heavy-up test for a fitness brand

Illustrative numbers throughout. A 36-club fitness brand wants to know whether more paid social in January buys more members, or just more leads. It picks 8 test markets containing 14 clubs, randomly chosen from matched pairs, and raises Meta spend by $6,000 a week across them for 8 weeks. The other 19 markets keep spend flat. Two markets with club openings are excluded.

Two things jump out. First, a before-and-after comparison would have claimed a lift of 20% (2,960 against 2,464, which is 308 × 8). Half of that was January. Second, Meta’s 1,140 leads at roughly $42 each looked like a bargain. The members the spend actually added cost $167.

Is $167 good? That’s a business question. If a member is worth $600 in first-year contribution, yes, even at the top of the range. If your churn means a member is worth $250, the top of the range ($320) loses money and you’d want a second, longer test before scaling.

Reading the result location by location

The pooled answer is the one to make the channel decision on. The per-location answers are where operators get interested, and where you need some discipline.

A single club’s range will be wide. If the pooled lift is 10.8% and one club shows 25% while another shows −3%, both may be noise. Look for patterns that repeat across several locations, like suburban clubs outperforming downtown ones, rather than ranking individual stores. Then design the next test to check the pattern.

Report in the language of the room. “288 extra members, $167 each, range $114 to $320” survives a budget meeting. “10.8% lift, p = 0.02” does not.

Common incrementality testing mistakes

• Running a two-week test. It will lie to you. Weekly numbers are noisy, ads take time to build, and customers who saw the ad in week two may show up in week five.
• Picking test markets by hand. The markets a regional VP volunteers are the ones he’s most optimistic about. Pair similar markets, then flip a coin within each pair.
• Letting other things change. A price increase, a menu launch or an extra mailer in only the test markets makes the result unreadable. Freeze everything you can.
• Using platform conversions as the outcome. The point is to measure customers at your locations. Measuring leads or modeled visits brings back the problem you were trying to escape.
• Counting new stores. A club that opened during the test will look like a 400% lift. Exclude markets with openings, closures or remodels.
• Peeking and stopping early. Checking weekly and stopping when it looks good guarantees a flattering, wrong answer. Fix the end date before launch.
• Treating “not significant” as “zero.” An inconclusive test says the effect is smaller than the MDE, not that it doesn’t exist. Say what size of effect it could have missed.

Planned tests vs always-on reads

Planned tests are the gold standard for a big decision, but you can’t run them for every channel every quarter. Most brands get more value from an always-on read of the natural experiments already in their data, with a planned test two or three times a year on the channels carrying the most money.

The always-on read uses the same comparison logic. Whenever spend in a market moves, compare that market with a weighted blend of markets where it held steady. It is less clean than a randomized test, so it should come with a range and an honest flag when the comparison is weak.

Questions people ask about incrementality testing

Start with one question, one channel and the data you already export. The first clean answer usually pays for the next five tests.

Written by The Countlift team at First Byte in The Woodlands, Texas. Questions or corrections? Email hello@countlift.com.