Generate your anomaly test with Elementary AI
Let our Slack chatbot create the anomaly test you need.
elementary.metric_stability
Detects changes to a time bucket’s own previously measured aggregates after the bucket has settled.
Use it for historical revenue, costs, or other measures that should stop changing once late data has arrived. Anomaly detection compares different periods against each other; it may notice some effects of a restatement, but it does not directly enforce the expectation that settled history stays put.
Use Case
“Alert me if last month’s revenue changes after the books are closed.”Test Logic
- Measure the selected column metrics per time bucket, reusing Elementary’s metric caching
- Ignore any bucket younger than
min_bucket_age, and any measurement taken before the bucket reached that age - For each remaining bucket, compare the newest measurement against its own earlier measurements: the previous one (
last_check), the earliest retained one (first_check), or both - If the change exceeds
max_change_percent→ FAIL (change_type: value_changed) - If a previously measured bucket or dimension has no current metric in a window the test actually rescanned → FAIL (
change_type: missing_bucket) - Otherwise → PASS
Test configuration
Required configuration:columns, metrics, timestamp_column, min_bucket_age
data_tests:
— elementary.metric_stability:
arguments:
columns: [column name] # Required
metrics: [metric name] # Required
timestamp_column: column name # Required
min_bucket_age: # Required
period: [second | minute | hour | day | week]
count: int
change_since: [last_check | first_check] # Optional - default: [last_check]
max_change_percent: int # Optional - default: 0
time_bucket: # Optional
period: [hour | day | week | month]
count: 1
dimensions: [sql expression] # Optional
where_expression: sql expression # Optional
days_back: int # Optional - derived from min_bucket_age
backfill_days: int # Optional - default: days_backParameters
Choosing a timestamp column
Choose a business or event timestamp whose historical periods you want to protect. A row’s ingestion or last-modified timestamp can move it between buckets when the row is updated, which answers a different question.Coverage and cost
min_bucket_age measures time since the bucket ended. Measurements taken before that age are excluded from both baselines, because a bucket’s own settling would otherwise become its baseline. The first eligible measurement only establishes a baseline; a pass at that point does not yet verify historical stability.
days_back bounds the observation window. The drift example above protects daily buckets roughly 28 to 90 days old, not all historical data, and corrections outside that window are not detected. Incremental models and sources also use backfill_days to control re-measurement; it defaults to days_back, and an explicitly shorter backfill window reduces coverage to the buckets actually scanned on that run.
Without an explicit window, the test derives one from the settling age (roughly twice the age, with room for whole buckets). That is a convenience default, not a business retention policy. Set the window to cover the corrections you care about, and run often enough to measure each eligible bucket more than once. Longer windows increase rescanning and metric-history storage costs.
Baselines and legitimate corrections
last_checkcompares against the previous eligible measurement. With zero tolerance, 100 → 120 fails, and a subsequent 120 passes. New measurements become the baseline automatically, including measurements from failing runs.first_checkcompares against the earliest retained eligible measurement. It catches cumulative drift: 100 → 110 → 120 exceeds a 15% threshold overall even though each step is smaller. A corrected 120 keeps failing against 100 until it returns within tolerance or the bucket leaves coverage.- Selecting both fails if either comparison exceeds the threshold.
last_check when a change should be reported once and then accepted automatically. Choose first_check when continued deviation should remain a failure.
Dimensions
Each bucket and dimension combination is a separate metric with its own history and baseline. A restatement confined to one dimension value is still reported, and a dimension value that stops appearing is reported as a missing bucket. Every extra combination is another measurement to store and compare.Failure details
With test result samples enabled, stored samples include the bucket, column, metric, dimensions, current and baseline values, measurement timestamps, and absolute and percentage deltas. Normal Elementary sample limits and privacy controls apply.change_type: value_changed reports numeric movement above the configured percentage threshold. max_change_percent: 1 means 1%, not 100%. The default is zero with a tiny relative floor that suppresses floating-point aggregation noise. Movement away from a zero baseline always fails, because relative change is undefined there.
change_type: missing_bucket means a previously measured bucket or dimension has no current metric in a window the test actually rescanned. It fails regardless of the percentage tolerance, reports a NULL current value alongside the last observed value, and remains a failure while the bucket is missing and within coverage. It does not invent a zero for aggregates such as average or minimum.
Notes
time_bucket.countmust be 1. A multi-step bucket cannot be measured on a stable grid across runs, so the test would never report a changemin_bucket_ageacceptssecond,minute,hour,dayandweek. Express an age over longer calendar periods in days, for example{count: 60, period: day}- A
days_backtoo small to leave room for whole buckets pastmin_bucket_ageraises an error rather than passing without ever comparing anything - Stable aggregates do not guarantee unchanged source rows. Offsetting changes can cancel in a sum, so pair this test with row-level checks when record immutability is the requirement