Install Elementary dbt package
To start using Elementary, you need to add our dbt package to your dbt project.
What are dbt packages and packages.yml?
What are dbt packages and packages.yml?
A dbt package is additional Jinja and SQL code that is added to your project, for additional functionality. In fact, each package is a dbt project. By adding a package to your project, you are adding the package code to be part of your project, you can reference its macros, execute its models, and so on.
Add packages to your project by creating a packages.yml file under the main project directory (where your dbt_project.yml is), and adding the relevant package. After you add a new package, run dbt deps to actually pull its code to your project. This is also how you update packages.
Some packages we recommend you check out: dbt_utils, dbt_date, codegen.
Video: How to install Elementary dbt package?
Step-by-step: Install Elementary dbt package
Add elementary to `packages.yml`
Add the following to your packages.yml file (if missing, create it where dbt_project.yml is):
Add to your `dbt_project.yml`
This means Elementary models will have their own schema.
Depending on your project custom schema macro, the schema will be named elementary or <target_schema>_elementary.
Make sure your user has permissions to create schemas.
Important: Materialization config
Important: Materialization config
For elementary to work, it needs to create some of the models as incremental tables. Make sure that there are no global materialization configurations that affect elementary, such as:
Make sure to place the 'elementary' configuration under the models key, and other configs under your project name.
Example:
If you change materialization settings, make sure to run dbt run -s elementary --full-refresh.
Important: Allowing Elementary to override dbt's default materializations (relevant from dbt 1.8)
Important: Allowing Elementary to override dbt's default materializations (relevant from dbt 1.8)
Starting from dbt 1.8 and above, 3rd party dbt packages cannot override dbt materializations without an explicit configuration from users. Elementary utilizes this dbt mechanism to add reporting functionality to non-Elementary tests (such as native dbt tests and expectations):
- Collection of the total number of failed rows.
- Collection of failed row samples.
In order for these features to work, it is required to add the following flag to your dbt_project.yml file:
Please note that after setting this flag you may see a deprecation warning from dbt. This is a temporary measure and we are working with the dbt team on a longer term solution.
Import the package and build Elementary models
This will mostly create empty tables, that will be updated with artifacts, metrics and test results in your future dbt executions.
Run tests
Validate the installation by running some tests:
After you ran your tests, we recommend that you ensure that the results were loaded to elementary_test_results table.
What happens now?
Once the elementary dbt package has been installed and configured, your test results, run results and dbt artifacts will be loaded to elementary schema tables.
If you see data in these models you completed the package deployment (Congrats! 🎉).
What’s next?
Install Elementary dbt package
To start using Elementary, you need to add our dbt package to your dbt project.
What are dbt packages and packages.yml?
What are dbt packages and packages.yml?
A dbt package is additional Jinja and SQL code that is added to your project, for additional functionality. In fact, each package is a dbt project. By adding a package to your project, you are adding the package code to be part of your project, you can reference its macros, execute its models, and so on.
Add packages to your project by creating a packages.yml file under the main project directory (where your dbt_project.yml is), and adding the relevant package. After you add a new package, run dbt deps to actually pull its code to your project. This is also how you update packages.
Some packages we recommend you check out: dbt_utils, dbt_date, codegen.
Video: How to install Elementary dbt package?
Step-by-step: Install Elementary dbt package
Add elementary to `packages.yml`
Add the following to your packages.yml file (if missing, create it where dbt_project.yml is):
Add to your `dbt_project.yml`
This means Elementary models will have their own schema.
Depending on your project custom schema macro, the schema will be named elementary or <target_schema>_elementary.
Make sure your user has permissions to create schemas.
Important: Materialization config
Important: Materialization config
For elementary to work, it needs to create some of the models as incremental tables. Make sure that there are no global materialization configurations that affect elementary, such as:
Make sure to place the 'elementary' configuration under the models key, and other configs under your project name.
Example:
If you change materialization settings, make sure to run dbt run -s elementary --full-refresh.
Important: Allowing Elementary to override dbt's default materializations (relevant from dbt 1.8)
Important: Allowing Elementary to override dbt's default materializations (relevant from dbt 1.8)
Starting from dbt 1.8 and above, 3rd party dbt packages cannot override dbt materializations without an explicit configuration from users. Elementary utilizes this dbt mechanism to add reporting functionality to non-Elementary tests (such as native dbt tests and expectations):
- Collection of the total number of failed rows.
- Collection of failed row samples.
In order for these features to work, it is required to add the following flag to your dbt_project.yml file:
Please note that after setting this flag you may see a deprecation warning from dbt. This is a temporary measure and we are working with the dbt team on a longer term solution.
Import the package and build Elementary models
This will mostly create empty tables, that will be updated with artifacts, metrics and test results in your future dbt executions.
Run tests
Validate the installation by running some tests:
After you ran your tests, we recommend that you ensure that the results were loaded to elementary_test_results table.
What happens now?
Once the elementary dbt package has been installed and configured, your test results, run results and dbt artifacts will be loaded to elementary schema tables.
If you see data in these models you completed the package deployment (Congrats! 🎉).