Generate observability report

Elementary data observability report can be used for visualization and exploration of data from the dbt-package tables. That includes dbt test results, Elementary anomaly detection results, dbt artifacts, tests runs, etc.

Before you start

Before you can start using the report, make sure to install the dbt package, configure a profile and install the CLI. This is required for the report to work.

dbt package installation

Configure connection profile

Configuring the Elementary Profile

In order to connect, Elementary needs a connection profile in a file named profiles.yml. This profile will be used by the CLI, to connect to the DWH and find the dbt package tables.

The easiest way to generate the profile is to run the following command within the dbt project where you deployed the elementary dbt package (works in dbt cloud as well):

dbt run-operation elementary.generate_elementary_cli_profile

Ensure that you fill in your password and any other missing fields after you paste the profile in your local profiles.yml file.

Copy the output, fill in the missing fields and add the profile to your profiles.yml. Here is a demonstration:

Elementary profile details and format

Path: HOME_DIR/.dbt/profiles.yml
Profile name: elementary
Schema name: The schema of elementary models, default is <your_dbt_project_schema>_elementary

## SNOWFLAKE ##
## By default, edr expects the profile name 'elementary'.      ##
## Configure the database and schema of elementary models.     ##
## Check where 'elementary_test_results' is to find it.        ##

elementary:
  outputs:
    default:
      type: snowflake
      account: [account id]

      ## User/password auth ##
      user: [username]
      password: [password]

      role: [user role]
      database: [database name]
      warehouse: [warehouse name]
      schema: [schema name]_elementary
      threads: 4

Install CLI

Install Elementary CLI

To install the monitor module run:

pip install elementary-data

Run one of the following commands based on your platform (no need to run all):

pip install 'elementary-data[snowflake]'
pip install 'elementary-data[bigquery]'
pip install 'elementary-data[redshift]'
pip install 'elementary-data[databricks]'
pip install 'elementary-data[athena]'
pip install 'elementary-data[trino]'
pip install 'elementary-data[clickhouse]'
## Postgres doesn't require this step

Run edr --help in order to ensure the installation was successful.

If you're receiving command not found: edr please check our troubleshooting guide.

Generate Tests Report UI

Execute `edr report` in your terminal

After installing and configuring the CLI, execute the command:

edr report

The command will use the provided connection profile to access the data warehouse, read from the Elementary tables, and generate the report as an HTML file.

Generating a report for single invocation

Elementary support filtering the report by invocation on generation. The filtered report will only include data for the selected invocation (This applies only on the Test Results and Lineage screens).

There are 3 ways to filter the report by invocation:

Last invocation - Filter by the last invocation (dbt test / dbt build) that ran.
Invocation time - Filter by the closest invocation (dbt test / dbt build) to the provided time (the provided time should be in ISO format local time).
Invocation id - Filter by the provided invocation (dbt test / dbt build) id.

Filters usage example:

edr report invocation filters

edr report --select last_invocation
edr report --select "invocation_time:2022-12-25 10:10:35"
edr report --select invocation_id:XXXXXXXXXXXXX

Elementary offers 3 methods that make it easy to share the report with others:

Send via Slack
Host on Amazon S3
Host on Google Cloud Storage (GCS)

Refer to this guide for detailed instructions.

On this page

Before you start
Generate Tests Report UI
Execute edr report in your terminal
Generating a report for single invocation
Sharing the report

Before you start

Before you can start using the report, make sure to install the dbt package, configure a profile and install the CLI. This is required for the report to work.

dbt package installation

Configure connection profile

Configuring the Elementary Profile

In order to connect, Elementary needs a connection profile in a file named profiles.yml. This profile will be used by the CLI, to connect to the DWH and find the dbt package tables.

The easiest way to generate the profile is to run the following command within the dbt project where you deployed the elementary dbt package (works in dbt cloud as well):

dbt run-operation elementary.generate_elementary_cli_profile

Ensure that you fill in your password and any other missing fields after you paste the profile in your local profiles.yml file.

Copy the output, fill in the missing fields and add the profile to your profiles.yml. Here is a demonstration:

Elementary profile details and format

Path: HOME_DIR/.dbt/profiles.yml
Profile name: elementary
Schema name: The schema of elementary models, default is <your_dbt_project_schema>_elementary

## SNOWFLAKE ##
## By default, edr expects the profile name 'elementary'.      ##
## Configure the database and schema of elementary models.     ##
## Check where 'elementary_test_results' is to find it.        ##

elementary:
  outputs:
    default:
      type: snowflake
      account: [account id]

      ## User/password auth ##
      user: [username]
      password: [password]

      role: [user role]
      database: [database name]
      warehouse: [warehouse name]
      schema: [schema name]_elementary
      threads: 4

Install CLI

Install Elementary CLI

To install the monitor module run:

pip install elementary-data

Run one of the following commands based on your platform (no need to run all):

pip install 'elementary-data[snowflake]'
pip install 'elementary-data[bigquery]'
pip install 'elementary-data[redshift]'
pip install 'elementary-data[databricks]'
pip install 'elementary-data[athena]'
pip install 'elementary-data[trino]'
pip install 'elementary-data[clickhouse]'
## Postgres doesn't require this step

Run edr --help in order to ensure the installation was successful.

If you're receiving command not found: edr please check our troubleshooting guide.

Generate Tests Report UI

Execute `edr report` in your terminal

After installing and configuring the CLI, execute the command:

edr report

The command will use the provided connection profile to access the data warehouse, read from the Elementary tables, and generate the report as an HTML file.

Generating a report for single invocation

There are 3 ways to filter the report by invocation:

Last invocation - Filter by the last invocation (dbt test / dbt build) that ran.
Invocation time - Filter by the closest invocation (dbt test / dbt build) to the provided time (the provided time should be in ISO format local time).
Invocation id - Filter by the provided invocation (dbt test / dbt build) id.

Filters usage example:

edr report invocation filters

edr report --select last_invocation
edr report --select "invocation_time:2022-12-25 10:10:35"
edr report --select invocation_id:XXXXXXXXXXXXX

Elementary offers 3 methods that make it easy to share the report with others:

Send via Slack
Host on Amazon S3
Host on Google Cloud Storage (GCS)

Refer to this guide for detailed instructions.

On this page

Before you start
Generate Tests Report UI
Execute edr report in your terminal
Generating a report for single invocation
Sharing the report

Generate observability report

Before you start

How to install Elementary dbt package?

Configuring the Elementary Profile

Elementary profile details and format

Install Elementary CLI

Generate Tests Report UI

Execute `edr report` in your terminal

Generating a report for single invocation

Elementary OSS

Quickstart

Guides

Configuration & usage

Deployment

Integrations

Community & Support

Release Notes

Generate observability report

Before you start

How to install Elementary dbt package?

Configuring the Elementary Profile

Elementary profile details and format

Install Elementary CLI

Generate Tests Report UI

Execute `edr report` in your terminal

Generating a report for single invocation

​Before you start

​Generate Tests Report UI

​Execute edr report in your terminal

​Generating a report for single invocation

​Sharing the report

Elementary OSS

Quickstart

Guides

Configuration & usage

Deployment

Integrations

Community & Support

Release Notes

​Before you start

​Generate Tests Report UI

​Execute edr report in your terminal

​Generating a report for single invocation

​Sharing the report

Before you start

Generate Tests Report UI

Execute `edr report` in your terminal

Generating a report for single invocation

Sharing the report

Before you start

Generate Tests Report UI

Execute `edr report` in your terminal

Generating a report for single invocation

Sharing the report