Your analytics dashboard is lying to you by leaving things out

A brand I work with had a story they told about themselves. "We mostly sell bundles." Everyone in the room believed it. It shaped how they thought about pricing, about the catalogue, about which ads to run. The trouble is, nobody had actually looked.

So I looked. And the looking turned out to be the hard part — not because the data didn't exist, but because the tool we'd been trusting couldn't see it.

The connector showed me everything except the thing I needed

We pipe this brand's web analytics through an off-the-shelf connector into a warehouse, the way most teams do now. It's clean, it's automatic, and for revenue totals and conversion rates it's perfectly good. I asked it a simple question: what's the actual product split? Which items are in which orders?

It couldn't answer. The connector exposes order-level and visit-level aggregates — total revenue, total items, total conversions — but no product-level granularity. No SKUs. From thirty thousand feet, the numbers looked complete. There was just no way to land the plane.

This is the quiet failure mode of modern data stacks. The dashboard doesn't lie by being wrong. It lies by being silent about what it leaves out. You get a confident-looking number and no signal that a whole dimension is missing underneath it.

A tool that answers nine of your ten questions will happily let you forget you ever asked the tenth.

Knowing the data exists is half the job

Here's where twenty years of doing this actually pays off. I knew the underlying analytics platform tracks ecommerce at the line-item level. The data was sitting right there in the source system; the connector just wasn't carrying it downstream. So I wrote a thin query straight against the platform's Reporting API, pulling the product reports the connector never surfaced.

That's the whole move. Not a new platform, not a big migration, not a six-week BI project. A small, sharp tool pointed at a source I already understood. It's the same instinct I wrote about when I pointed an AI agent at a live ad account — the value isn't the tool, it's knowing exactly which question to ask it and where the real answer lives.

What the data actually said

Over the course of a few days around 320 orders. And the number that ended the debate: 1.14 items (sku) per order.

That's not a bundle business. On most days there wasn't a single multi-item order at all. The "we mostly sell bundles" story was, at best, describing a single product that happens to ship as a kit — which is a completely different thing, and changes nothing about how multi-product the basket really is. Either way, the belief that had been steering decisions didn't survive contact with the line items.

This is exactly the reality check McKinsey keeps pointing at: data should be a complement to and occasionally a correction of seasoned gut feel, not a replacement for it. The gut had a hypothesis. The line items had the answer. You need both, but you have to actually go and get the second one.

The part nobody automates

I could have accepted the dashboard. Most people do — that's what it's there for, and pushing past it costs time. But accepting a tool's blind spot as the edge of reality is how confident organisations make slow, expensive mistakes. The judgement is in not trusting the clean number, in knowing the missing dimension exists, and in caring enough to go pull it.

That restraint — refusing to act on the comfortable summary — is the same muscle I lean on in almost every mandate. I've written before about picking the one question worth chasing instead of boiling the ocean. And it's the through-line of why I think deep domain expertise is the only real leverage left: tools are abundant now, judgement isn't. Anyone can stand up a dashboard. Knowing when it's quietly lying to you is the rare bit.

The bundle question took an afternoon to settle once I stopped trusting the summary. The story had survived for months only because nobody had looked underneath it.

Your dashboard isn't lying. It's just not telling you what it doesn't know. That's your job.

Sources & further reading

External
Matomo Reporting API reference (product/ecommerce item reports)
Matomo Ecommerce reporting guide
McKinsey — Making data-driven marketing decisions

Related posts
I pointed the Meta MCP at a real ad account — here's what it found
The Art of Restraint
The Death of Generic AI