Skip to main content
Elementary’s on-run-end hooks collect test results and dbt artifacts to provide comprehensive observability. However, for large projects or when running only a subset of models, these hooks can add significant time to your dbt runs. This guide explains how to control the timing of on-run-end hooks to make them more efficient and avoid unnecessary runs while maintaining the observability you need.

​
Overview

The Elementary on-run-end hooks perform several operations that can impact run time:
  • Metadata artifacts upload: Uploads all project metadata (models, tests, sources, etc.)
  • Run results upload: Uploads test results and model execution results
  • dbt invocation upload: Uploads dbt invocation metadata
For large projects, these operations can take several minutes. The following strategies help you control when these hooks run to optimize efficiency and reduce unnecessary uploads.

​
Control Metadata Artifacts Upload Timing

The most effective way to reduce on-run-end time is to control when metadata artifacts are uploaded. Instead of uploading all project metadata (models, tests, etc.) on every run, you can upload artifacts on a schedule that matches when your project actually changes.

​
Configuration

Add the following to your dbt_project.yml:
dbt_project.yml
vars:
  disable_dbt_artifacts_autoupload: true

​
Upload Artifacts When Needed

When you need to update metadata (e.g., after schema changes or adding new models), upload artifacts either manually or via a recurring job (e.g., daily): 
dbt run --select elementary.edr.dbt_artifacts
Metadata artifacts only need to be uploaded when your project structure changes (new models, tests, sources, etc.). For most runs, you only need run results, which are much faster to upload.

​
Control Other Hooks Timing

You can further control when different parts of the on-run-end logic run by setting additional variables. Adjust these based on what data you need Elementary to collect and when you need it.

​
Available Configuration Variables

dbt_project.yml
vars:
  disable_run_results: true
  disable_tests_results: true
  disable_dbt_invocation_autoupload: true
  • disable_run_results: Controls when model execution results are uploaded
  • disable_tests_results: Controls when test results are uploaded
  • disable_dbt_invocation_autoupload: Controls when dbt invocation metadata is uploaded
Controlling these hooks will reduce the amount of data processed and uploaded at the end of each run, but may impact the completeness of reports and monitoring. Only control hooks for data you don’t need on every run.

​
Limit Hooks to Production or Specific Targets

If you only need full observability in production, configure Elementary to run hooks only for specific targets. This is especially useful when you want to:
  • Speed up local development
  • Reduce costs in non-production environments
  • Focus monitoring on production workloads

​
Configuration by Target

Configure hooks to run only for specific targets:
dbt_project.yml
vars:
  disable_dbt_artifacts_autoupload: "{{ target.name != 'prod' }}"
  disable_run_results: "{{ target.name != 'prod' }}"
  disable_tests_results: "{{ target.name != 'prod' }}"
  disable_dbt_invocation_autoupload: "{{ target.name != 'prod' }}"

​
Disable Entire Package for Non-Production

Alternatively, you can disable the entire Elementary package for non-production targets:
dbt_project.yml
models:
  elementary:
    +enabled: "{{ target.name == 'prod' }}"
Disabling the entire package will prevent Elementary tests from running. We recommend using the hook disablement vars instead, which allows Elementary tests to run while skipping the artifact collection.
For more details on configuring Elementary for different environments, see the Dev/Prod Configuration guide.

​
Next Steps

After configuring your on-run-end hooks timing:
  1. Set up metadata sync schedule: If you disabled autoupload, create a process (manual or recurring job, e.g., daily) to sync metadata when your project changes
  2. Configure sync scheduling: Set up webhook-triggered syncs for real-time updates. See the Sync Scheduling guide for details
  3. Monitor data completeness: Verify that Elementary Cloud still receives the data needed for your monitoring use cases
For more information about configuring Elementary for different environments, see the Dev/Prod Configuration guide.