Monitoring the performance of your data pipeline is critical for maintaining data quality, reliability, and operational efficiency. Proactively monitoring performance issues enables to detect bottlenecks and opportunities for optimization, prevent data delays, and avoid unnecessary costs.

Elementary monitors and logs the execution times of:

  • dbt models
  • dbt tests

​
Models performance

Navigate to the Model Duration tab.

The table displays the latest execution time, median execution time, and execution time trend for each model. You can sort the table by these metrics and explore the execution times over time for the models with the longest durations

It is also useful to use the navigation bar to filter the results, and see run times per tag/owner/folder.

​
Tests performance

Navigate to the Test Execution History tab.

On the table you can see the median execution time and fail rate per test. You can sort the table by this time column, and detect tests that are compute heavy.

It is also useful to use the navigation bar to filter the results, and see run times per tag/owner/folder.

Incident managementOverview and best practices