April 12, 2023: v0.7.7 Python, v0.7.5 dbt package
🔥 What’s new?
- Seasonality and Anomaly direction 📈
- On this release we are introducing 2 new anomaly detection tests configurations. The goal is to improve the accuracy and unlock more detection use cases. 🎯💪
- We prepared a short video to explain 📽👇🏻
seasonality: day_of_week 🗓
- If enabled, daily values will be compared only to the same week day previous values.
- This configuration would improve accuracy and detection of datasets that have different volume and metrics on weekends and different weekdays.
anomaly_direction: [ both | spike | drop ] ↗️↘️
- By default, elementary alerts on both spikes and drops of a metric. Using this configuration, you can limit anomaly tests to alert only on the wanted direction.
- For example - If you want volume anomalies to fail only on drop in row count, and not on a spike, just add
anomaly_direction: drop to the test config.
- on-run-end performance improvements ⏳
- We fixed problems in artifacts caching, and implemented an incremental upload. This means that upon a change to models / tests / sources - elementary will now only update the changed resources.
- This should improve the performance of the hooks, and we have additional ideas and planned tasks around further improvements.
- Special thanks to @panapost for helping with this effort 🤩
🐞 Bug fixes
- Added support to unsupported time buckets on Bigquery (week, month), and made buckets full and consistent to calendar times - Thank you @Kenneth Roos and @Julian Schulte for reporting!
- Added log rotation to
edr.log to prevent large log files that won’t open - Thank you @Stéphane Burwash and @Grant Mehrer for reporting!
- Fixed elementary tests custom names issue - Thank you @Fabien Traventhal for reporting!
- Fixed subscribers parsing issue - Thank you @Muthu for reporting!
target-path to upload_source_freshness - Requested by @Ange Melhuish!