Behavioral · medium
A number on the dashboard suddenly spikes. Walk me through what you check.
Asked in BI Developer interviews.
Short answer
Decide whether it is real or an artifact: did upstream data change, did a join fan out, did a filter break, or did one real event drive it. Segment to localize it, compare against the source of truth, and only then explain it to stakeholders. Reporting a broken number as real is worse than catching it late: verify before you raise it.
How to answer it
Work outside in
Check the cheap explanations before the expensive ones. Most spikes are not the business changing.
1. Is it real? does the raw source agree with the dashboard?
a spike in one and not the other is a pipeline
problem, not a business event
2. Did the pipeline
change? a deploy, a model change, a schema change
check what shipped in the last 24 hours FIRST
3. Did the definition
change? a filter removed, a join added, a metric edited
4. Did collection
change? new tracking, an SDK upgrade, duplicate events
from a double-fired tag
5. Is it a duplicate? a join fanning out, or a rerun that inserted twice
row count against distinct key count answers this
6. Is it one segment? slice by source, country, platform, account
a single account or bot ring is a common cause
7. Is it seasonal? compare with the same day last week and last year
before calling anything anomalous
8. Is it genuinely
the business? a campaign, a launch, press, a competitor outageThe two-minute triage
Compare row counts and distinct key counts for the spike day against the previous week. If rows grew and distinct keys did not, it is duplication and you can stop looking at the business entirely. That single check resolves a large share of these.
Then decide what to say
A spike that is real and explainable gets a note on the dashboard. A spike caused by the pipeline gets fixed and backfilled, and the people who saw the wrong number get told. The communication is part of the job: a number that moved and was never explained costs trust even after it is corrected.
Key points
- Suspect the pipeline before the business. Deploys are more common than miracles.
- Row count against distinct count separates duplication from real growth in one query.
- Slice by segment early. One bad account or a bot ring is a frequent cause.
- Compare with the same day last week and last year before calling it an anomaly.
- Whatever the cause, close the loop with whoever saw the wrong number.
FAQ
- What if nothing explains it? Widen the window and check upstream sources. An unexplained spike is not resolved just because it went away.
- Should there be alerting? Yes: freshness, row count and metric anomaly detection catch most of these before a human notices.
- What if it is real and good? Still verify it. A real spike and a double-counted one look identical on the chart.
The 30-second answer
I would work outside in and suspect the pipeline before the business, because a deploy is far more likely than a miracle.
First, does the raw source agree with the dashboard? If the spike is in one and not the other it is a pipeline problem. Then what shipped in the last day: a model change, a schema change, a definition edit, or new tracking that is double firing.
The single most useful check is row count against distinct key count for that day versus the previous week. If rows grew and distinct keys did not, it is duplication from a fan-out or a rerun, and I can stop looking at the business entirely. After that I would slice by segment, since one account or a bot ring is a common cause, and compare with the same day last week and last year before calling anything anomalous.
In the interview
- What they are checking: that you verify before you escalate, and that you know the fan-out signature.
- Common mistake: forwarding the dashboard screenshot with "did we do something on Tuesday?" before checking whether Tuesday loaded twice.