Adding Elementary Tests
This tutorial will add a few common tests to our data set.
To read more about Elementary’s suite of tests click here.
There are 3 models in our tutorial:
Copy the text below into your
You have now added Elementary tests to your project. Now let’s see how they are run and visualized.
There are 3 models in our tutorial:
- Customers
- Orders
- Returned Orders
schema.yml file. We will now navigate there and add tests for each of our models.
A
schema.yml file that includes all the below tests can be found at the bottom of this page, so there is no need to manually add each test.Test 1: Customer Signups
Test 1: Customer Signups
First, we will use the tables_anomalies test to perform a row_count. This tests counts the number of rows created in a given period to determine if there have been any anomalies in the number of signups in a given time period.We will add the volume_anomalies test as follows:Now that we have configured a test, we should define a column to use for our timestamp. This will be used to create time buckets for anomaly detection. We select the signup_date column as seen below:This test will fail if there are any days (as defined by signup_date) where the number of rows exceeds 3 standard deviations above/below the mean.
Test 2: Returned Orders
Test 2: Returned Orders
Similar to Test 1, we will use the volume_anomalies test to detect an anomalous number of returned orders in a given time period. In this test, however, we will define the timestamp column at the test level - instead of at the model level.
Test 3: Dimension Anomalies in the Orders Model
Test 3: Dimension Anomalies in the Orders Model
The elementary.dimension_anomalies tests can be used to check for anomalies in the distribution of data in low cardinality fields. First, we will select the status column in the Orders model to run this test on:Next, we will define a timestamp column for determining time buckets:This test will gather row count metrics for each value in the status column and will fail if the distribution deviates substantially from the mean.
Test 4: Column Anomalies in the Customers Model
Test 4: Column Anomalies in the Customers Model
We will use the elementary.column_anomalies test to monitor the count of customers with 0 orders. To do so, we will define a zero_count test under elementary.column_anomalies for the number_of_orders column:
Notice that we already defined the timestamp_column at the model level, so we don’t have to define it in the test. This page has in-depth details on test priorities.
Copy the text below into your
schema.yml file to add all of the above tests to our project:
Final schema.yml file
Final schema.yml file
You have now added Elementary tests to your project. Now let’s see how they are run and visualized.