elementary.event_freshness_anomalies

Monitors the freshness of event data over time, as the expected time it takes each event to load - that is, the time between when the event actually occurs (the event timestamp), and when it is loaded to the database (the update timestamp).

This test compliments the freshness_anomalies test and is primarily intended for data that is updated in a continuous / streaming fashion.

The test can work in a couple of modes:

  • If only an event_timestamp_column is supplied, the test measures over time the difference between the current timestamp (“now”) and the most recent event timestamp.
  • If both an event_timestamp_column and an update_timestamp_column are provided, the test will measure over time the difference between these two columns.

Test configuration

Required configuration: event_timestamp_column Default configuration: anomaly_direction: spike to alert only on delays.

tests:   — elementary.event_freshness_anomalies:     event_timestamp_column: column name     update_timestamp_column: column name     where_expression: sql expression     anomaly_sensitivity: int     detection_period:       period: [hour | day | week | month]       count: int     training_period:       period: [hour | day | week | month]       count: int     time_bucket:       period: [hour | day | week | month]       count: int     seasonality: day_of_week     detection_delay:       period: [hour | day | week | month]       count: int     ignore_small_changes:       spike_failure_percent_threshold: int       drop_failure_percent_threshold: int     anomaly_exclude_metrics: [SQL expression]