1. Data Anomaly Detection Tests
  2. Anomaly detection tests configuration

The anomaly detection tests configuration is defined in .yml files in your dbt project, just like in native dbt tests.

Elementary tests have three levels of configurations:

  1. Test arguments - Test specific arguments.
  2. Table configuration - Configure the timestamp column and details of a monitored table.
  3. Global vars - Optional configuration parameters of the operation.

Test arguments

Test specific configurations, configured in the yml for each test (like dbt severity, limit, etc).

Configuration for all anomaly detection tests:

Configuration for specific anomaly detection tests:

Table configuration

Elementary tests timestamp_column can be configured for a model / source. This will apply to any test defined on the table that has no timestamp_column param.

Data monitoring global vars

Elementary has several global vars used for tests configurations. You can change their defaults by adding them with a new value in your dbt_project.yml under the vars key.

# optional #
# Global vars for data monitoring#
  days_back: 14 # maximum timeframe for collecting metrics and analyzing anomalies
  anomaly_sensitivity: 3 # sensitivity of anomaly detection
  backfill_days: 2 # days to backfill on each run, adjust to your data delays