Send Slack alerts

Elementary enriches alerts with table owners).

If you want to tag a model owner in a slack alert:

• Use ’@’ and the email prefix of the slack user.
• For example, if we want to tag a user named Jessica with an email jessica.jones@marvel.com in our Slack workspace, simply add the email prefix (with lower case) jessica.jones as follows to your model schema.yml / properties.yml:

models:
  - name: my_model_name
    meta:
      owner: "@jessica.jones"

It is possible to tag multiple owners as well:

models:
  - name: my_model_name
    meta:
      owner: ["@jessica.jones", "@joe.joseph"]

Elementary supports configuring description to tests alerts.

To set it up, simply add the description to your test in the properties.yml

tests:
  - test_name:
    meta:
      description: "This is the test description"

You can use tags to provide context to your alerts.

You can also use it to tag a group or a channel in a slack alert:

• Add it as model tag and use ’#’ as the prefix of the channel name.
• For example, to tag the marketing team’s data ops channel add the following to your model schema.yml / properties.yml.

tests:
  - test_name:
    meta:
      tags: ["#marketing", "#support"]

If you want to tag users on an alert:

• Use ’@’ and the email prefix of your slack user, and to ‘subscribers’ under a meta field to your properties.yml file.
• For example, if we want to tag a user named Jessica with an email jessica.jones@marvel.com in our Slack workspace, use “@jessica.jones”.

models:
  - name: my_model_name
    meta:
      alerts_config:
        subscribers: "@jessica.jones"
    columns:
      - name: column_name
        tests:
          - unique:
              meta:
                alerts_config:
                  subscribers: "@luke.cage"

It is possible to tag multiple subscribers as well:

models:
  - name: my_model_name
    meta:
      alerts_config:
        subscribers: ["@jessica.jones", "@luke.cage"]

By default Elementary uses the Slack channel that was configured in the Slack integration. Elementary supports configuring custom slack channels that are configured on your models / sources / tests and snapshots.

• If you configure a custom slack channel for a model, all the test alerts that belong to this model will be sent to this custom slack channel.
• If you configure a custom slack channel for both a model and a test, the test channel will override the model channel.
• If you configure a custom slack channel and you decide to group your alerts by table into a single message, it will be sent to the model channel (even if a differnt channel was configured on the test level)

To set it up, simply add the relevant channel to your models in the properties.yml:

models:
  - name: marketing_leads
    meta:
      alerts_config:
        channel: marketing_data_ops

If your models / tests are in folders by department / team, another useful option is to configure the channel in your dbt_project.yml file:

models:
  marketing_bi:
    +meta:
      alerts_config:
        channel: marketing_data_ops

tests:
  marketing_bi:
    +meta:
      alerts_config:
        channel: marketing_data_ops

You can also configure a custom slack channel for a specific test:

models:
  - name: marketing_leads
    columns:
      - name: column_name
        tests:
          - unique:
              meta:
                alerts_config:
                  channel: marketing_data_ops

Elementary supports the following alert fields:

• table: Displays the table name of the test
• column: Displays the column name of the test
• description: Displays the description of the test
• owners: Displays the owners of the model on which the test is running
• tags: Displays the dbt tags of the test/model
• subscribers: Displays the subscribers of the test/model
• result_message: Displays the returned message from the test result
• test_parameters: Displays the parameters that were provided to the test
• test_query: Displays the query of the test
• test_results_sample: Displays a sample of the test results

By default, all of the fields are shown in the alerts. Elementary supports configuring alert fields on your dbt project / models and tests.

• If you configure alert fields on your dbt project, all the test alerts of all of your tests will display only the configured alert fields.
• If you configure alert fields for a model, all the test alerts that belong to this model will display only the configured alert fields.
• If you configure alert fields for both a model and a test, the test configured alert fields will override the model configured alert fields (same as for the dbt project configured alert fields).

To set it up globaly for your project, add the desired alert fields to your models and tests in the dbt_project.yml file:

models:
  marketing_leads:
    +meta:
      alerts_config:
        alert_fields: ["description", "owners", "tags", "subscribers"]

tests:
  marketing_leads:
    +meta:
      alerts_config:
        alert_fields: ["description", "owners", "tags", "subscribers"]

To set it up for a model, add the desired alert fields to your model in the properties.yml:

models:
  - name: marketing_leads
    meta:
      alerts_config:
        alert_fields: ["description", "owners", "tags", "subscribers"]

You can also configure alert fields for a specific test:

models:
  - name: marketing_leads
    columns:
      - name: column_name
        tests:
          - unique:
              meta:
                alerts_config:
                  alert_fields: ["description", "owners", "tags", "subscribers"]

Elementary supports filtering alerts using a selector. Elementry edr monitor command will notify only on the selector’s matched alerts.

There are 3 selectors supported by elementary:

• tag - Notify on models/sources/tests that are tagged with the provided tag selector (notice that tests can be matched on their model’s/source’s tag).
• owner - Notify on models/sources/tests that their owner is provided owner selector (notice that tests can be matched on their model’s/source’s owner).
• model - Notify on the model/source and its tests.

To filter alerts by tag:

edr monitor --select tag:critical
edr monitor --select tag:finance

To filter alerts by owner:

edr monitor --select config.meta.owner:@jeff
edr monitor --select config.meta.owner:@jessy

To filter alerts by model:

edr monitor --select model:customers
edr monitor --select model:orders

edr monitor --select customers
edr monitor --select orders

Elementary support configuring suppression interval for alerts. By default, the suppression interval for all of the alerts is set to 0. Elementary won’t send any alert that is generated within suppression interval.

alert_suppression_interval can accept values of 0-24 (including unrounded numbers) - this number represents the number of hours for which alerts will be skipped.

To set it up globaly for your project, add the alert suppression interval to your models and tests in the dbt_project.yml file:

models:
  marketing_leads:
    +meta:
      alerts_config:
        alert_suppression_interval: 24


tests:
  marketing_leads:
    +meta:
      alerts_config:
        alert_suppression_interval: 24

To set it up for a model, add the desired alert suppression interval to your model in the properties.yml:

models:
  - name: marketing_leads
    meta:
      alerts_config:
        alert_suppression_interval: 24

You can also configure alert suppression interval for a specific test:

models:
  - name: marketing_leads
    columns:
      - name: column_name
        tests:
          - unique:
              meta:
                alerts_config:
                  alert_suppression_interval: 24

By default, Elementary sends a single alert to notify on each failure. When using single alerts, the alert will include extensive information for fast triage.

Elementary also supports grouping alerts by table. In this case, a single Slack notification will be generated containing all test warnings/failures/errors as well as the errors associated with the model. The created notification will contain a union of the relevant owners, tags and subscribers. Due to their nature, grouped alerts will contain less information on each issue. As always, you can use our (detailed report) for easy triage.

To group alerts by table:

edr monitor --group-by table

Grouping can also be configured through the yml files. To set it up globaly for your project, add the configuration to your models in the dbt_project.yml file:

models:
 marketing_bi:
   +meta:
     alerts_config:
     # alerts on models in marekting_bi should be grouped by table:
       slack_group_alerts_by: table

To set it up for a model, add the configuration to your model in the properties.yml:

models:
 - name: marketing_leads
   meta:
     alerts_config:
       # all alerts on marketing_leads should group together to one slack message:
       slack_group_alerts_by: table

Grouping by table can be configured globally (in the dbt_project.yml) but if you wish to override it for a specific model where you want a single alert for each failure, you can add the configuration to your model in the properties.yml:

models:
 - name: marketing_leads
   meta:
     alerts_config:
       # alerts on marketing_leads will not be grouped:
       slack_group_alerts_by: alert