Data Tests Overview

Data tests are useful for validating and enforcing explicit expectations on your data.

Elementary enables data validation and result tracking by leveraging dbt tests and dbt packages such as dbt-utils, dbt-expectations, and Elementary. This rich ecosystem of tests covers various use cases, and is widely adopted as a standard for data validations. Any custom dbt generic or singular test you develop will also be included.
Additionally, users can create custom SQL tests in Elementary, and add tests in bulk from the UI.

The combination of dbt tests, Elementary monitors, custom SQL tests and the rich dbt testing ecosystem provides the ability to achieve wide and comprehensive coverage.

​

Supported data tests

​

Test configuration

One of the design principals in Elementary is that users should manage configuration in their code. This enables to maintain the same workflow for building the pipeline and configuring coverage, it makes observability and governance part of the development cycle, and provides control in review process and version management.

However, adding many tests in code is tedious, and configuration in code isn’t usable for everyone.

In Elementary, we designed a flow to incorporate the good of both worlds:

• Configuration in code or in UI - The UI test configuration flow opens pull requests to the code base through the code repository integration.
• The code is the single source of truth - As configuration from UI goes to code, the code remains the place where configuration is managed and maintained.

​

Benefits of leveraging dbt tests

dbt tests are very powerful. The ease of use, simplicity, and usefulness in the dev process is unmatched. When you adopt any observability tool, you will still use dbt tests. This is why in Elementary, dbt tests are first class citizens.

There are several benefits to this approach:

• Single interface for all observability - Prevent the distribution of monitoring between different tools. All configuration is in code, all the results are in one interface.
• Avoid duplicate work and vendor lock in - The tests you implemented already are effective in Elementary, as well as additional configuration. The future tests you add will remain in your code if you decide to offboard.
• Control of schedule and cost - You have control of configuration and scheduling, tests can be executed when data is actually loaded and validation is needed.
• Prevent bad data from propagating - As tests are in pipeline, you can leverage dbt build and fail the pipeline on critical test failures.
• Rich ecosystem - The community of dbt users developes and supports various testing use cases.

​

dbt Test Hub

To help you find the test that is right for your use case, we created the dbt Test Hub. It's a searchable catalog of all the tests supported in Elementary, with their descriptions and example use cases.

The tests are also segmented to use cases, so you can easily find the different options for addressing your detection use case.