July 20, 2023: v0.9.1 Python, v0.9.0 dbt package
🔥 What’s new?
- New Seasonality Options: hour_of_day and hour_of_week 🌞🌙
- Thanks to the brilliant contribution from Moss Pauly, you now have two new seasonality options: hour_of_day and hour_of_week.
- These options will help in making the anomaly detection tests more accurate for datasets with strong seasonality impact.
- Check out the docs
- Data for calculating Quality Score 📊
- We made the following changes to help calculating data sets quality score (refer to this great article for more info):
- We’ve added two new columns, elementary_test_results.failed_row_count and dbt_tests.quality_dimension. This data enables you to gain deeper insights into data quality and better track your data pipeline’s health.
- We automatically fill these fields for common test (dbt, dbt_utils, and dbt_expectations), if you would like them to work for other test, or overwrite the default ones, you will need to fill the test’s meta with:
- quality_dimension - Name of the quality dimension this test measures.
- failed_row_count_calc - An SQL expression to calculate the amount of failed rows from the test (count(*), sum(n_records), etc).
- Thank you Saurabh Jha for working with us on this!
- This feature is still experimental, and is not reflected in the report nor the alerts.
- Alert Suppression in CLI 🚦
- Now, you can suppress alerts in the Command Line Interface (CLI) using the —suppression-interval flag.
- This feature gives you more control over your alerts and helps you manage notifications effectively.
- Check out the docs
💫 More changes
- Order Slack messages by test run time ⏰
- We’ve added a new option to order Slack messages by test run time. This makes it easier to track test results and prioritize action items - Thanks Willi Mueller for suggesting!
- Set SQL expression as timestamp_column
- Support for dbt where Parameter
🐛 Bugfix 🐛
- We’ve resolved a pesky bug that caused dbt_columns to malfunction in specific access configurations when using BigQuery.
- Table anomalies tests fix - Reported by Scott Anderson !
- New dimension values will no linger cause dimension anomalies test to fail - Thanks Jason Deng for reporting!