Two people built the same report — and got numbers 18% apart

TL;DR — The dataset behind the weekly report is assembled by hand: four exports, one spreadsheet, one person. Two people building “the same” report got numbers 18% apart, and neither did anything wrong. What a scheduled pipeline that cleans, de-duplicates and flags its exceptions changes.

Monday, 08:40

The ops lead at a 26-person B2B firm starts the week the same way: export from the CRM, export from billing, export from the ads account, the events sheet someone on marketing maintains. Paste into one spreadsheet, de-duplicate by eye, refresh the pivot, send it to leadership by noon.

Three and a half hours. About 2,300 rows. Accurate in the way a hand-built thing is accurate — exactly as accurate as the person who built it, that Monday.

In August she took two weeks off. Someone else built the report from the same sources, following the same steps as best he could reconstruct them. The pipeline number came out 1,576. Hers was 1,860.

Nobody did anything wrong. He de-duplicated by company name; she uses the domain. He skipped the events sheet’s 200 rows marked “unqualified”; she pulls them in and filters later. His CRM export ran a Thursday. Each choice was defensible. Not one of them was written down.

That report isn’t a process. It’s roughly forty judgment calls that live in one person’s fingers.

Where the Monday goes

Count it for this firm:

The pull. Four exports into one sheet: 3.5 hours when nothing misbehaves. Five when a source quietly renames a column.
The de-dupe. About 180 duplicate pairs a month — “Acme Ltd” against “Acme Limited” against “ACME” — merged by eye, with no record of which row won or why.
The gaps. The report splits pipeline by industry and headcount, and about 40% of rows are missing one or both. Someone looks up ~30 companies a month on LinkedIn, four minutes each — and the enrichment is stale within weeks.
The as-of problem. Each export is current as of the moment it was pulled, which is a different moment for each source. The dashboard refreshes when someone remembers. Decisions get made on numbers nobody can date.

That’s ~15 hours a month, plus a dataset that depends on one person not being on holiday.

That is the shape of a job a pipeline should own.

What a pipeline changes

It runs on a schedule, not on a Monday. Every morning at 06:00 it pulls all four sources. A failed pull is retried, then flagged — a source going down can’t silently shrink the dataset.
One shape. Domains lowercased and stripped, dates in one format, one row per company, keyed on the domain — not on how someone happened to type the name.
De-duplication by rule, with a confidence line. Obvious merges happen automatically; the ~15 pairs a week the rules won’t touch are listed for a human. The same decision gets made every run, and there’s a record of which row won.
Enrichment where terms allow it. Missing industry, headcount, location filled against external records — only where the source’s terms permit — and every filled field is stamped with its source and date. “Where did this number come from” finally has an answer.
It lands somewhere you own. A warehouse table, a database, a sheet. The version everyone reads is the version the pipeline wrote — not the copy someone pasted into a deck.
Every run leaves a record. Row counts per source, what merged, what was enriched, which rows need a human. Nothing silently dropped, nothing silently guessed.

The numbers, before and after

Assembly: 3.5 hours every Monday → ~20 minutes reviewing the exception list.
Duplicates: ~180 a month merged by eye → auto-merged by rule, ~15 a week flagged for review.
Firmographics: ~40% of rows missing → filled where allowed, source- and date-stamped.
Freshness: “as of whenever the export was pulled” → as of 06:00 today.
Two builders, one report: 1,860 vs 1,576 → one dataset, one definition.

The arithmetic

About 15 hours a month becomes about two — exception review and a spot check. At a conservative blended rate, roughly $8,000 a year. Real, but not the point.

The point is the decisions made on top of the dataset: the campaign judged on numbers missing their newest 200 rows, the pitch that quoted a headcount a year stale, the pricing call made on the 18%-off figure because that was the one that made it onto the slide.

How it starts

One messy source and one clean output you actually want — built end to end first, including the de-duplication rules and a first pass at normalisation, in one to two weeks. What you see is the dataset and the run log: what it pulled, what it merged, what it couldn’t resolve. More sources get added once the shape is proven, and each one costs less than the last.

If your week starts with someone pasting exports into a spreadsheet, that’s the first pipeline worth building. We pull data from your sources on a schedule, clean and de-duplicate it, enrich it where the terms allow, and land one clean dataset your team and your dashboards can rely on: Data pipelines & enrichment.