Skip to main content

Generate your anomaly test with Elementary AI

Let our Slack chatbot create the anomaly test you need.
dimensions: [list of SQL expressions] The test will group the results by a given column / columns / valid select sql expression. Under dimensions you can configure the group by expression. Using this param segments the tested data per dimension, and each dimension is monitored separately. For example - A column_anomalies test monitoring for null_rate with dimensions configured will monitor the null_rate of values in the column, grouped by dimension, and will fail if in a specific dimension there is an anomaly in null_rate. It is best to configure low-cardinality fields as dimensions.
  • Default: None
  • Relevant tests: dimension_anomalies, column_anomalies, all_columns_anomalies
  • Configuration level: test

​
Listing more than one column

Listing two columns does not test each column separately. Elementary joins them into one value with ; in between, then watches every combination it finds. With dimensions: [country, device_os], what gets watched is US; ios, US; android, DE; ios, and so on. Empty values appear as the word NULL.
Combinations multiply fast. 20 countries and 10 operating systems is up to 200 things to watch, each needing its own history, and each combination has fewer rows than either column on its own. To watch two fields separately, write two tests.
Elementary saves the history under the exact list you configured, in the exact order. If you add a column, remove one, or just reorder them, the test can no longer find its old history and starts collecting again from scratch.
For how to choose dimensions and read the results, see Understanding dimension anomalies.