Control vs Treatment
AnalyticsAlso: Control Group · Treatment Group · Test Group vs Control Group
Quick definition
Control and treatment are the two groups compared in an experiment, most commonly A/B testing. The control group experiences the current, unchanged version of something. The treatment group experiences the new version being tested. Comparing outcomes between the two groups shows whether the change caused a real difference.
The two groups, explained
The baseline. Sees the current page, email or offer with nothing changed.
The variant. Sees the new headline, layout or offer you're actually testing.
What it actually means
Think of a clinical drug trial. One group gets the real medicine, the other gets a placebo, and nobody tells the patients which is which. Marketing experiments borrow the same logic. The control group keeps seeing what already exists. The treatment group sees the change you're testing, whether that's a new call-to-action, a different price or a redesigned landing page.
The entire point of a control is comparison. Without one, you can't tell whether a lift in conversion rate came from your change or from something else entirely, a seasonal spike, a paid media push, a competitor going down for maintenance. The control absorbs all the noise that has nothing to do with your test.
This is the backbone of A/B testing, but it applies wherever you're isolating cause from coincidence: email subject lines, ad creative, pricing pages, onboarding flows. The names change (test versus control, variant versus baseline) but the mechanism is identical. One group is your reality check.
A treatment group without a genuine control is just a story with no comparison point.
How it shows up
This shows up any time traffic is split for a test. Your A/B testing tool randomly assigns visitors to control or treatment, usually at a fifty-fifty split, sometimes weighted if you're de-risking a bigger change. It shows up in email platforms as 'Group A / Group B' sends, in ad platforms as experiment arms, and in product as feature flags where one cohort sees the old flow and another sees the new one. The moment someone says 'we tested this and it worked', the first question worth asking is what the control group looked like.
The Australian context
Australian sites generally carry less traffic than their US or UK equivalents, which matters more here than most teams realise. Smaller sample sizes mean tests need to run longer to reach statistical significance, and splitting an already modest audience into control and treatment can stretch a test out for months rather than weeks. The fix isn't giving up on controls. It's testing bigger, more obvious changes first, since subtle tweaks need traffic volumes many Australian businesses simply don't have.
Where people get this wrong
Related terms
Common questions
What's the difference between control and treatment groups?
The control group sees the existing, unchanged version of whatever you're testing. The treatment group sees the new variant. Comparing outcomes between the two isolates whether your change actually caused a difference, rather than something else happening at the same time.
How big should the control group be?
Usually matched to the treatment group, often a fifty-fifty split, so both sides have enough data to reach statistical significance around the same time. Skewed splits are fine if you're de-risking a bigger change, but they extend how long the test needs to run.
Can you run a test without a control group?
You can, but you're no longer running an experiment. Without a control, you're comparing your result to a guess about what would have happened anyway, which means seasonality, external events and normal variance all get credited to your change.
Why did my treatment group win but the result didn't hold after launch?
This usually means the test ended before reaching statistical significance, or the control and treatment groups weren't comparable to begin with. A result that looks real during the test but fades post-launch is the classic sign of an underpowered sample size.
Debrief
Get the next one
No spam. No fluff. Just the next article, straight to your inbox.
Keep exploring
About New Rebellion
New Rebellion is a marketing intelligence consultancy. We build tools, score Australian businesses on how their marketing actually performs, and publish Debrief every day. This dictionary is part of how we work in the open.
How we think →