Skip to main content

Generate your anomaly test with Elementary AI

Let our Slack chatbot create the anomaly test you need.
 elementary.event_freshness_anomalies Monitors the freshness of event data over time, as the expected time it takes each event to load - that is, the time between when the event actually occurs (the event timestamp), and when it is loaded to the database (the update timestamp). This test compliments the freshness_anomalies test and is primarily intended for data that is updated in a continuous / streaming fashion. The test can work in a couple of modes:
  • If only an event_timestamp_column is supplied, the test measures over time the difference between the current timestamp (“now”) and the most recent event timestamp.
  • If both an event_timestamp_column and an update_timestamp_column are provided, the test will measure over time the difference between these two columns.

​
Test configuration

Required configuration: event_timestamp_column Default configuration: anomaly_direction: spike to alert only on delays. 
data_tests:
  — elementary.event_freshness_anomalies:
    arguments:
      event_timestamp_column: column name
      update_timestamp_column: column name
      where_expression: sql expression
      anomaly_sensitivity: int
      detection_period:
        period: [hour | day | week | month]
        count: int
      training_period:
        period: [hour | day | week | month]
        count: int
      time_bucket:
        period: [hour | day | week | month]
        count: int
      seasonality: day_of_week
      detection_delay:
        period: [hour | day | week | month]
        count: int
      ignore_small_changes:
        spike_failure_percent_threshold: int
        drop_failure_percent_threshold: int
      anomaly_exclude_metrics: [SQL expression]
models:
  - name: < model name >
    data_tests:
      - elementary.event_freshness_anomalies:
          arguments:
            event_timestamp_column: < timestamp column > # Mandatory
            update_timestamp_column: < timestamp column > # Optional
            where_expression: < sql expression >
            time_bucket: # Daily by default
              period: < time period >
              count: < number of periods >