Skip to main content

Generate your anomaly test with Elementary AI

Let our Slack chatbot create the anomaly test you need.
The duration for retracting the detection period. That’s useful in cases which the latest data should be excluded from the test. For example, this can happen because of scheduling issues- if the test is running before the table is populated for some reason. The detection delay is the period of time to ignore, after the detection period.
  • Default: 0
  • Relevant tests: Anomaly detection tests with timestamp_column
Detection Delay

​
How it works?

The detection_delay param only works for tests that have timestamp_column configuration. It does not affect the other duration parameters, like detection_period or training_period — it moves the whole tested window into the past instead of making it shorter.

​
You don’t need a delay to avoid testing a half-finished day

Elementary only ever tests finished time buckets. The newest one it looks at is the last bucket that had already ended when dbt started, minus any delay. Your job does not need to run at midnight for this to hold. With daily buckets, the newest day tested is always yesterday, whether dbt starts at 00:30, 02:00 or 23:00 — today’s partial data is never tested, without setting anything. What a delay is really for is data that shows up late. If yesterday’s load does not finish until 04:00 but your tests run at 02:00, yesterday looks finished on the calendar but is still missing rows in the warehouse, and the test sees a drop. A delay also only moves in whole buckets, so a small delay is not a small change. Daily buckets, dbt running at 02:00: A four-hour delay pushes the cutoff back past midnight, so it costs you a whole day — the same as asking for one.
Before adding a delay, see whether a different column solves it. If timestamp_column is when something happened (occurred_at) but the rows only land hours later, switch to when it was loaded (loaded_at, _synced_at). Grouping by arrival time avoids the problem instead of waiting it out.
Time buckets are built from calendar dates, but the cutoff for “how recent” comes from when dbt started running, and dbt uses UTC. If your timestamp column holds local time rather than UTC, the newest day can get tested before that day has actually ended where your data lives. Check which day your test is really looking at, and add a delay if it is testing too early.
For the full trade-off, see Understanding dimension anomalies.