Skip to main content
Elementary’s on-run-end hooks collect test results and dbt artifacts to provide comprehensive observability. However, for large projects or when running only a subset of models, these hooks can add significant time to your dbt runs. This guide explains how to control the timing of on-run-end hooks to make them more efficient and avoid unnecessary runs while maintaining the observability you need.

​
Overview

The Elementary on-run-end hooks perform several operations that can impact run time:
  • Metadata artifacts upload: Uploads all project metadata (models, tests, sources, etc.)
  • Run results upload: Uploads test results and model execution results
  • dbt invocation upload: Uploads dbt invocation metadata
For large projects, these operations can take several minutes. The following strategies help you control when these hooks run to optimize efficiency and reduce unnecessary uploads.

​
Control Metadata Artifacts Upload Timing

The most effective way to reduce on-run-end time is to control when metadata artifacts are uploaded. Instead of uploading all project metadata (models, tests, etc.) on every run, you can upload artifacts on a schedule that matches when your project actually changes.

​
Configuration

Add the following to your dbt_project.yml:
dbt_project.yml
vars:
  disable_dbt_artifacts_autoupload: true

​
Upload Artifacts When Needed

When you need to update metadata (e.g., after schema changes or adding new models), upload artifacts either manually or via a recurring job (e.g., daily): 
dbt run --select elementary --vars '{"enable_elementary_models": true}'
For the enable_elementary_models pattern in dbt_project.yml, see Can I disable / exclude the Elementary models?.
Metadata artifacts only need to be uploaded when your project structure changes (new models, tests, sources, etc.). For most runs, you only need run results, which are much faster to upload.

​
Control Other Hooks Timing

Important: Disabling hooks (other than metadata artifacts) means you will lose this data for those runs. Only disable hooks for runs you don’t want to monitor (e.g., local development).
You can further control when different parts of the on-run-end logic run by setting additional variables.

​
Available Configuration Variables

dbt_project.yml
vars:
  disable_run_results: true
  disable_tests_results: true
  disable_dbt_invocation_autoupload: true
  • disable_run_results: Controls when model execution results are uploaded
  • disable_tests_results: Controls when test results are uploaded
  • disable_dbt_invocation_autoupload: Controls when dbt invocation metadata is uploaded
Disabling these hooks means you will lose this data for those runs. Only disable hooks for runs you don’t want to monitor (e.g., local development). This will reduce the amount of data processed and uploaded at the end of each run, but will impact the completeness of reports and monitoring for those runs.

​
Disable test table cleanup during on-run-end

Elementary creates temporary tables during test execution and cleans them up at the end of each run. For large projects with many tests, this cleanup phase can become a significant portion of the on_run_end time. If you observe in the logs that the slow phase is the test table cleanup, you can disable it during on_run_end and offload it to a dedicated scheduled job instead.

​
Disable automatic cleanup

Add the following to your dbt_project.yml:
dbt_project.yml
vars:
  clean_elementary_temp_tables: false

​
Run cleanup in a dedicated job

After disabling automatic cleanup, schedule a recurring dbt operation to clean up stale test tables: 
dbt run-operation elementary.cleanup_stale_test_tables
You can pass optional arguments to control the cleanup behavior: 
dbt run-operation elementary.cleanup_stale_test_tables --args '{"hours": 48, "limit": 1000}'
Parameters:
  • hours (default: 24): Only delete test tables that are older than this many hours. Increase this value if you want to retain temporary tables for longer before cleanup.
  • limit (default: 2000): Maximum number of tables to delete per run. Use this to cap the cleanup operation and avoid long-running jobs when many stale tables have accumulated.
We recommend scheduling this job to run daily. If a large number of stale tables have built up, the limit parameter prevents the first cleanup from running indefinitely — you can run the job multiple times until all stale tables are removed.

​
Limit Hooks to Production or Specific Targets

If you only need full observability in production, configure Elementary to run hooks only for specific targets. This is especially useful when you want to:
  • Speed up local development
  • Reduce costs in non-production environments
  • Focus monitoring on production workloads

​
Configuration by Target

Configure hooks to run only for specific targets:
dbt_project.yml
vars:
  disable_dbt_artifacts_autoupload: "{{ target.name != 'prod' }}"
  disable_run_results: "{{ target.name != 'prod' }}"
  disable_tests_results: "{{ target.name != 'prod' }}"
  disable_dbt_invocation_autoupload: "{{ target.name != 'prod' }}"

​
Disable Entire Package for Non-Production

Alternatively, you can disable the entire Elementary package for non-production targets:
dbt_project.yml
models:
  elementary:
    +enabled: "{{ target.name == 'prod' }}"
Disabling the entire package will prevent Elementary tests from running. We recommend using the hook disablement vars instead, which allows Elementary tests to run while skipping the artifact collection.
For more details on configuring Elementary for different environments, see the Dev/Prod Configuration guide.

​
Next Steps

After configuring your on-run-end hooks timing:
  1. Set up metadata sync schedule: If you disabled autoupload, create a process (manual or recurring job, e.g., daily) to sync metadata when your project changes
  2. Configure sync scheduling: Set up webhook-triggered syncs for real-time updates. See the Sync Scheduling guide for details
  3. Monitor data completeness: Verify that Elementary Cloud still receives the data needed for your monitoring use cases
For more information about configuring Elementary for different environments, see the Dev/Prod Configuration guide.