Elementary dbt package includes exposure validation tests, implemented as dbt tests.
These tests can detect changes in your models’ columns that break downstream exposures, such as BI dashboards.
Within your models
directory, add a file called exposures.yml
version: 2
exposures:
- name: customers
label: Customer Dashboard
type: dashboard
maturity: high
url: https://your.bi.tool/dashboards/1
description: >
Shows customer growth
depends_on:
- ref('customers')
owner:
name: Callum McData
email: [email protected]
For each exposure you wish to verify, add a new property called meta
:
version: 2
exposures:
- name: customers
label: Customer Dashboard
type: dashboard
maturity: high
url: https://your.bi.tool/dashboards/1
description: >
Shows customer growth
depends_on:
- ref('customers')
owner:
name: Callum McData
email: [email protected]
meta:
referenced_columns:
- column_name: "customer_id"
In Elementary Cloud, the exposures and exposure validation tests are added
automatically, leveraging an integration with your BI tool.
You can optionally also specify the column data type (data_type
)
version: 2
- name: returned_orders
label: Returned Orders
type: dashboard
maturity: high
url: https://your.bi.tool/dashboards/2
description: >
Returned orders over time
depends_on:
- ref('returned_orders')
owner:
name: Callum McData
email: [email protected]
meta:
referenced_columns:
- column_name: "order_id"
data_type: "numeric"
In addition, if your exposure depends on several nodes, you can depend explicitly per column which source should be tested:
version: 2
- name: returned_orders
label: Returned Orders
type: dashboard
maturity: high
url: https://your.bi.tool/dashboards/2
description: >
Returned orders over time
depends_on:
- ref('returned_orders')
- ref('orders')
owner:
name: Callum McData
email: [email protected]
meta:
referenced_columns:
- column_name: "order_id"
data_type: "numeric"
node: ref('returned_orders')
- column_name: "customer_id"
data_type: "numeric"
node: ref('orders')
Adding the Elementary exposure tests for your module
For each module schema you wish to verify the exposure dependencies, add the elementary exposure tests:
...
- name: returned_orders
description: This table contains all of the returned orders
config:
tags: ["finance"]
tests:
- elementary.volume_anomalies:
tags: ["table_anomalies"]
timestamp_column: "order_date"
- elementary.exposure_schema_validity:
tags: [elementary]
We recommend adding a tag to the tests so you can execute these in a dedicated run using the selection parameter --select tag:elementary
.
Upon running the tests, if breaking changes are detected in the model, the test will fail and in the test result query you’ll be able to see the reasons:
SELECT 'customers' as exposure, 'https://your.bi.tool/dashboards/1' as url, 'different data type for the column customer_id numeric vs numeric' as error
UNION ALL SELECT 'customers' as exposure, 'https://your.bi.tool/dashboards/1' as url, 'order_id column missing in the model' as error
What does it mean when a test fails?
When a test fails, it means that an exposure is potentially broken due to a missing or wrongly-typed column. Open your BI tool to validate, and if you make any changes to your dashboards be sure to update the exposures.yml
and your model schema accordingly.