Deduplication
Data & TrackingAlso: Dedupe · Record Deduplication
Quick definition
Deduplication is the process of finding and merging duplicate records in a database, most commonly customer or lead records in a customer relationship management (CRM) system or email platform. Without it, one person can exist as two or three separate contacts, splitting their history and skewing reports.
What it actually means
Duplicate records build up quietly. Someone fills out a form with a work email, then again later with a personal one. A sales rep manually adds a lead that already exists under a slightly different spelling. Two systems sync and each creates its own copy of the same contact. None of it looks urgent in the moment.
The damage shows up later. Segmentation splits one customer's activity across two profiles, so their behaviour looks smaller than it is. Attribution gets muddled because a purchase links to one record while the ad clicks live on another. Email marketing sends the same person two versions of a campaign, which looks sloppy and can hurt deliverability.
Deduplication is the fix, but it's rarely a single clean pass. Most platforms merge on exact matches (same email, same phone number) which catches the obvious cases. The harder duplicates, same person with a typo'd surname or a different email entirely, need fuzzy matching or manual review. Left unmanaged, a CRM's duplicate rate creeps upward every month new leads flow in.
A customer who exists three times in your CRM doesn't feel like three customers to your team. They feel like a mess.
How it shows up
Deduplication shows up as a maintenance task inside your CRM or email platform, usually a built-in tool that flags likely duplicates for merging. It also shows up indirectly wherever counts look inflated: total contact numbers that don't match expected customer volumes, segmentation that undercounts engaged users, or the same person receiving duplicate emails from an automated sequence.
Where people get this wrong
Related terms
Common questions
How often should I run deduplication?
Set an automated check monthly at minimum, more often if you import lists regularly or run frequent lead-gen campaigns. Waiting longer lets duplicates compound and makes the eventual cleanup more time-consuming.
Can deduplication be fully automated?
Exact-match duplicates (identical email or phone number) can be merged automatically. Fuzzy matches, where names or details are similar but not identical, need a human check before merging to avoid combining two different people by mistake.
What happens to a contact's history when records merge?
Most platforms let you choose which record is the primary and pull activity history, email engagement and purchase data from both into the surviving record. Always check the merge preview before confirming, since some fields can be overwritten rather than combined.
Does deduplication affect email deliverability?
Yes. Sending the same campaign twice to one person because they exist as two records looks like spam behaviour to inbox providers and increases unsubscribe and complaint rates. Clean lists protect your sender reputation.
Debrief
Get the next one
No spam. No fluff. Just the next article, straight to your inbox.
Keep exploring
About New Rebellion
New Rebellion is a marketing intelligence consultancy. We build tools, score Australian businesses on how their marketing actually performs, and publish Debrief every day. This dictionary is part of how we work in the open.
How we think →