Generate your anomaly test with Elementary AI
Let our Slack chatbot create the anomaly test you need.
dimension_anomalies watches how your rows split across the values of a field, rather than how big the table is. It catches problems a volume test misses: one country stops sending data, a payment provider drops to zero, an unexpected new value shows up — all while the total row count barely moves.
For the list of settings, see dimension_anomalies. This page covers the choices that only come up with this test.
Every value is checked on its own
The test counts rows for each value of yourdimensions, in each time bucket. Each value is then compared against its own past, with its own expected range.
JP falling to zero is compared only to JP’s own history, so it gets caught even though the table total hardly changed. That is what this test is for.
The trade-off is that more values means more noise. With 500 values, every run is 500 chances for a false alarm. Small values are the worst offenders: something that averages 5 rows a day swings a lot in percentage terms, so it drifts in and out of range on its own. Stick to fields with a few stable values, and use where_expression to leave out the rest.
Listing two columns watches combinations
Listing two columns does not test each column separately. Elementary joins them into one value with; in between, then watches every combination it finds. With dimensions: [country, device_os], what gets watched is US; ios, US; android, DE; ios, and so on. Empty values appear as the word NULL, so a jump in US; NULL is a real signal that device_os has gone missing.
Elementary saves the history under the exact list you configured, in the exact order. If you add a column, remove one, or just reorder them, the test can no longer find its old history and starts collecting again from scratch. Changing this list is really creating a new test.
New and missing values
- A value goes missing. Once Elementary has seen a value, it records a
0for buckets where that value has no rows. This is what lets it spot a value that disappears. - A new value shows up. A value Elementary has never seen has no past to compare against, so it cannot be flagged on the run where it first appears. To catch unexpected new values, add an
accepted_valuestest. - A value stays gone. Once a value has been at zero for the whole period, Elementary stops watching it, so retired values do not keep alerting.
A long detection period can hide problems
The training period and the detection period always overlap. This is not a mistake you can avoid by picking different period lengths: the expected range for a bucket is worked out from that value’s numbers up to and including that bucket, so the days being tested are also part of what they are tested against. That is harmless whendetection_period covers a single bucket. It works against you when it covers several — which, with the default 2-day detection period and daily buckets, it already does.
Say something breaks on Monday and stays broken through Wednesday, with detection_period: 3 days. Monday gets flagged. Monday’s bad number then becomes part of what Tuesday is compared against, pulling the average up and widening the range. By Wednesday the test is comparing against a range the problem itself stretched. A long problem covers its own tracks — you get one alert, then silence that looks like it was fixed.
Two related limits. A value with only one past data point cannot be checked at all, so the default 14-day training_period means 14 buckets for each value — not much for values that only show up now and then. And seasonality splits each value again by day of week, so every value needs far more history.
How to think about detection_delay
detection_delay moves the whole tested window further into the past. It is tempting to always set a one-day delay so the test never looks at a half-finished day — but Elementary only ever tests finished buckets, so you don’t need a delay for that.
Your job does not need to run at midnight for this to hold. With daily buckets, the newest day tested is always yesterday, whether dbt starts at 00:30, 02:00 or 23:00. Today’s partial data is never tested, without setting anything.
What a delay is really for is data that shows up late. If yesterday’s load does not finish until 04:00 but your tests run at 02:00, yesterday looks finished on the calendar but is still missing rows in the warehouse, and every value looks like it dropped.
A delay also only moves in whole buckets, so a small delay is not a small change. Daily buckets, dbt running at 02:00:
A four-hour delay pushes the cutoff back past midnight, so it costs you a whole day — the same as asking for one. So think in whole days: no delay when your data reliably lands before the tests run (the usual case, and the default), one day when late data is normal and you are fine hearing about yesterday tomorrow.
Time buckets are built from calendar dates, but the cutoff for “how recent” comes from when dbt started running, and dbt uses UTC. If your timestamp column holds local time rather than UTC, the newest day can get tested before that day has actually ended where your data lives. Check which day your test is really looking at, and add a delay if it is testing too early.
With no timestamp_column
The test counts rows per value across the whole table and saves one number per run. Its history is built from past runs, so a 14-day training period needs roughly 14 runs on different days. time_bucket, detection_delay and the period settings have no buckets to work with. Use this for dimension tables and reference data. If rows are only ever added, set a timestamp_column.
Reading a failure
- The failure count counts buckets, not values. One value that looks wrong across three buckets counts as three failures.
- The results include each affected value’s whole history, not just the bad buckets, so the report and the alert graph can show the problem in context.