Confounding Variable

Analytics

Also: Confounder · Lurking Variable

What it isA hidden factor skewing your result

The riskCorrelation mistaken for causation

FixControl for it or randomise it away

Shows up inA-B tests, seasonality, campaign timing

Quick definition

A confounding variable is a factor outside your test that influences both the thing you changed and the result you're measuring, making it look like your change caused an effect it didn't. It's the hidden third variable that quietly explains a correlation.

What it actually means

Imagine you launch a new ad creative on the same day a competitor goes out of stock. Sales jump. You credit the creative. The real driver was the competitor's stockout, sending their customers looking for alternatives. That stockout is a confounding variable. It moved alongside your test and dragged the result with it, without you ever measuring it.

Confounding variables are the reason correlation and causation stay separate concepts. They hide behind seasonality, pricing changes, PR mentions, weather, algorithm updates, even payday cycles. Any factor that shifts at the same time as your test and also affects the outcome is a candidate.

This matters most in A/B testing and attribution work. A poorly randomised experiment lets confounders creep in through the back door: one variant gets shown more to mobile users, or runs a day longer, or launches during a sale. Suddenly your conversion rate difference isn't about the headline you changed. It's about who saw it and when.

Good experiment design doesn't eliminate confounders through cleverness. It eliminates them through randomisation, forcing the confounding factors to distribute evenly across your test groups so they cancel out.

If you can't name what else changed at the same time as your test, you can't trust what your test told you.

How it shows up

Confounding shows up as results that are too good, too fast, or that don't replicate. A campaign that seemingly doubles CPA overnight when nothing about the campaign itself changed. An A/B test where the winning variant happened to run over a weekend. A churn drop that coincides with a pricing change nobody flagged in the readout. The tell is usually a second event nobody controlled for sitting on the same timeline as the metric you're celebrating.

The Australian context

Australia's smaller market makes confounders easier to spot but also easier to ignore, because sample sizes are already thin and teams are reluctant to slice the data further. A retail campaign launched the same week as a public holiday, or during EOFY sales, is a common local trap. Segmentation by state also matters here. A lockdown or weather event in one state can quietly confound a national test that looks clean in aggregate.

Where people get this wrong

Assuming a clean A-B testing setup means no confounders exist.Randomisation reduces confounding risk, it doesn't guarantee zero. Uneven traffic splits, timing drift between variants, and external events can still slip through even in a well-built test.

Spotting a strong correlation and stopping there.A strong correlation between two moving lines feels like proof. It's only ever a prompt to look for a shared cause, not a conclusion in itself.

Blaming or crediting the one variable everyone's watching.Teams tend to attribute results to whatever they intentionally changed, because that's the story they're prepared to tell. The variables nobody was tracking are exactly the ones most likely to be doing the work.

Common questions

What's a simple example of a confounding variable?

Ice cream sales and drowning deaths both rise in summer. Neither causes the other. Warm weather is the confounding variable driving both. In marketing, a seasonal spike hitting your test at the same time as a creative change plays the same role.

How do I stop confounding variables from ruining my A-B tests?

Randomise properly, run variants for the same time period, and avoid launching tests near known events like sales, holidays or pricing changes. If you can't avoid the timing, at least log it so you can explain unusual results later.

Is a confounding variable the same as bias?

They're related but not identical. Bias is a systematic error in how you collect or measure data. A confounding variable is a real external factor influencing your outcome. Bias can create the appearance of confounding, and confounding left unmanaged can introduce bias into your conclusions.

Can you ever fully remove confounding variables?

Not fully, but you can manage the risk. Random assignment across a large enough sample size spreads confounders evenly across groups so their effect cancels out statistically, even though the confounder itself is still technically present.

Debrief

Get the next one

No spam. No fluff. Just the next article, straight to your inbox.

Keep exploring

About New Rebellion

New Rebellion is a marketing intelligence consultancy. We build tools, score Australian businesses on how their marketing actually performs, and publish Debrief every day. This dictionary is part of how we work in the open.

How we think →