Metrics

In Elementary, you can monitor any metric you want on the content of your data, view the metric and set up anomaly detection tests on it!

​

How does it work?

Elementary uses a type of dbt test to collect metrics on your data and sync them up to Elementary Cloud. The metrics will be collected each time you run dbt test, in a way that is similar to how training data for anomaly tests is collected. A metric can be collected with or without an anomaly detection test configured on it.

​

How to set up a data content metric?

The monitored metrics are set up in the code, in a way similar to dbt tests.

No mandatory configuration, however it is highly recommended to configure a timestamp_column.

tests:
  — elementary.collect_metrics:
    timestamp_column: column name
    time_bucket:
      period: [hour | day]
      count: int
    dimensions: sql expression
    metrics: monitors list
     name: string
     type: monitor type
     columns: list
    where_expression: sql expression

models:
  - name: < model name >
    tests:
      - elementary.collect_metrics:
        timestamp_column: < timestamp column >
        time_bucket: # Daily by default
          period: < time period >
          count: < number of periods >
        dimensions: < list of dimensions to group by >
        metrics: < list of metrics >
          - name: < user defined name for metric >
            type: < which metric to calculate >
            columns: < which columns to calculate the metric on- for column metrics >
        where_expression: < sql expression >
        cloud_monitored: < boolean, should Elementary automatically create anomaly tests for the collected metrics? >

Upon running dbt test, your data is split into buckets:

• The size of each bucket is configured by the time bucket field
• Each row in the table is assigned to a bucket based on the timestamp of the timestamp_column selected
• If dimensions were selected, each combination of dimensions will create a separate metric and then we compute the metric (or metrics) of choice for each bucket. We save the metrics in the Elementary schema and sync it to Elementary Cloud whenever you sync your environment.

The metric chart will be visible in the Metrics screen, along with all the metrics Elementary has collected. If a test was created for the metric, it will be visible in Elementary just like any other test.

To include alerts on Metrics Tests in your alert rules, simply edit your alert rule and check the box “Cloud content tests” under “Test Categories”.