April 12, 2023: v0.7.7 Python, v0.7.5 dbt package

🔥 What’s new?

  • Seasonality and Anomaly direction 📈
    • On this release we are introducing 2 new anomaly detection tests configurations. The goal is to improve the accuracy and unlock more detection use cases. 🎯💪
    • We prepared a short video to explain 📽👇🏻
    • seasonality: day_of_week 🗓
      • If enabled, daily values will be compared only to the same week day previous values.
      • This configuration would improve accuracy and detection of datasets that have different volume and metrics on weekends and different weekdays.
    • anomaly_direction: [ both | spike | drop ] ↗️↘️
      • By default, elementary alerts on both spikes and drops of a metric. Using this configuration, you can limit anomaly tests to alert only on the wanted direction.
      • For example - If you want volume anomalies to fail only on drop in row count, and not on a spike, just add anomaly_direction: drop to the test config.
  • on-run-end performance improvements
    • We fixed problems in artifacts caching, and implemented an incremental upload. This means that upon a change to models / tests / sources - elementary will now only update the changed resources.
    • This should improve the performance of the hooks, and we have additional ideas and planned tasks around further improvements.
    • Special thanks to @panapost for helping with this effort 🤩

🐞 Bug fixes

  • Added support to unsupported time buckets on Bigquery (week, month), and made buckets full and consistent to calendar times - Thank you @Kenneth Roos and @Julian Schulte for reporting!
  • Added log rotation to edr.log to prevent large log files that won’t open - Thank you @Stéphane Burwash and @Grant Mehrer for reporting!
  • Fixed elementary tests custom names issue - Thank you @Fabien Traventhal for reporting!
  • Fixed subscribers parsing issue - Thank you @Muthu for reporting!
  • Added target-path to upload_source_freshness - Requested by @Ange Melhuish!