The channel filter on my dashboard silently broke ROAS: how to attribute ad spend across sales channels

Someone I work with clicked a channel filter on a performance dashboard I'd built with Claude Code — a small React app fed by a nightly data build, deployed to Cloudflare Pages — chose one marketplace, and told me the numbers were wrong. They were.

Not in the way I expected. The revenue filtered correctly. Orders filtered correctly. Average basket filtered correctly. The return on ad spend sat there, unmoved, showing exactly the same figure it showed with every channel selected.

I had built a filter that quietly refused to touch the one number everybody actually looks at.

You cannot filter a ratio you never attributed

The reason is embarrassing once you say it out loud. Revenue is tracked per sales channel — the marketplace, the shop, each storefront. Ad spend is tracked per advertising channel and country — this platform in that market, that platform in this one. Those are two different taxonomies wearing the same word.

So filtering the numerator was trivial and filtering the denominator was undefined. Rather than fail, the code did the polite thing: it left the denominator alone. Total spend, filtered revenue, a ratio that meant nothing and looked authoritative.

I have written about how a ratio is two numbers and both of them can move. This is the uglier cousin of that problem: a ratio where one number moves and the other cannot, and nothing in the interface tells you which.

A filter that silently does nothing is worse than a filter that isn't there. The missing one you work around. The broken one you believe.

The fix was a set of assumptions, written down where people can see them

Attribution here is not a modelling exercise, it is bookkeeping with a few honest judgement calls. Marketplace advertising maps to that marketplace. Search and social advertising maps to the owned shops, because that is where those clicks land — and where there is more than one owned shop, it splits by their share of revenue in the window you are looking at.

That last one is an assumption, not a measurement. So it is printed on the page. Anyone reading a filtered figure can see the rule that produced it, disagree with it, and argue about it. That is the whole point. As the measurement literature keeps pointing out, platform-reported returns routinely overstate results by counting the same conversion more than once, and attribution, mix modelling and incrementality answer three different questions rather than competing to answer one. If your allocation rule is a guess, the useful move is to make it a visible guess.

Then the invariant. With every channel selected, the dashboard must reproduce the old numbers exactly — same revenue, same spend, same ratio, to the last decimal. If the attribution logic leaks a franc anywhere, that check catches it before anyone else does. It found two leaks. The reconciliation against the total spend snapshot balanced to the cent, which is the only reason I trust the per-channel splits at all.

Then the same bug, one layer down

Filter to the marketplace that only sells in one country, and a heatmap kept showing every country in the estate.

Same root cause. That heatmap was built from a country-by-product data cube that had no channel dimension in it. The frontend could not filter what the pipeline had never split.

The tempting fix is an approximation in the interface — scale the country rows by that channel's share and move on. It would have looked right, and it would have been a second silently-wrong number sitting under the first one.

The actual fix was upstream in the Python build step, and it turned out to be free. The four sources feeding that cube were already one channel each — the shop's own analytics, the marketplace feed, the storefront markets, the retail marketplaces. The channel dimension had been there the whole time and was being flattened away at build. Exporting it unflattened took an afternoon and made the filter exact instead of approximate. Sum of channels equals the old country totals; that invariant is tested.

This is a pattern I run into constantly with data plumbing: the structure you need already exists in the source and gets destroyed on the way in. Look upstream before you build a model of something you are currently throwing away.

Two things I would now do on day one

Label what a control cannot do. Some views genuinely have no channel dimension and some have no country dimension. Those are now marked as exempt from the relevant filter, in the interface, in words. A user who can see that a control does not apply here is informed. A user watching a control apparently apply and change nothing is being misled.

Never let a rate be the headline without its parts. The daily advertising chart used to plot return on spend alone, which meant a tiny order against tiny spend produced a spectacular spike. It now plots revenue as bars and the ratio as a line over them. Same data, and suddenly nobody mistakes noise for a good day. That is the same lesson as insisting you know what went into a single blended number before you steer by it.

The general version, and the reason I keep coming back to this: as Kaplan and Norton put it thirty years ago in the Harvard Business Review piece that opened with it, what you measure is what you get. The corollary nobody quotes is that what you measure badly is also what you get. A filter that appears to work is a measurement system making a promise on your behalf.

Two hours of work. The bug was never in the filter. It was in believing that two things called "channel" were the same thing.

Sources & further reading

External
Kaplan & Norton — The Balanced Scorecard: Measures That Drive Performance (HBR)
Media Measurement and the Assisted Own Goal: Attribution, Marketing-Mix Models, and Individual-Level Incrementality
From platform ROAS to true incrementality

Related posts
Our ROAS doubled. We hadn't sold more.
Four sales channels into one number
2,390 ways to say 29 things