- Data Anomaly Detection Tests
- Anomaly detection tests configuration
Data Anomaly Detection Tests
Anomaly detection tests configuration
The anomaly detection tests configuration is defined in .yml
files in your dbt project, just like in native dbt tests.
Elementary tests have three levels of configurations:
- Test arguments - Test specific arguments.
- Table configuration - Configure the timestamp column and details of a monitored table.
- Global vars - Optional configuration parameters of the operation.
Test arguments
Test specific configurations, configured in the yml for each test (like dbt severity
, limit
, etc).
Configuration for all anomaly detection tests:
Configuration for specific anomaly detection tests:
Table configuration
Elementary tests timestamp_column
can be configured for a model / source.
This will apply to any test defined on the table that has no timestamp_column
param.
Data monitoring global vars
Elementary has several global vars used for tests configurations.
You can change their defaults by adding them with a new value in your dbt_project.yml
under the vars key.
dbt_project.yml
# optional #
# Global vars for data monitoring#
vars:
days_back: 14 # maximum timeframe for collecting metrics and analyzing anomalies
anomaly_sensitivity: 3 # sensitivity of anomaly detection
backfill_days: 2 # days to backfill on each run, adjust to your data delays