You can use GitLab’s CI in order to run
edr commands in an automated manner.
For example, you can use it to send a data observability report on changes in your dbt project, monitor for data issues periodically and send alerts to your Slack, and much more.
edr requires a dbt connection profile called
elementary that points to your Elementary schema in the warehouse.
More information on that can be found here.
Accordingly, we’ll need to provide the pipeline with that profile.
If you already have a
profiles.yml in your repository, simply add the
elementary profile to it, and make sure the credentials are loaded as variables.
edr will automatically pick up on the
Alternatively, you can write your
profiles.yml as a whole to a GitLab variable.
In order to setup the Elementary pipeline in your GitLab repository, you’ll need to create a file at the root of the project called
.gitlab-ci.yml with the following content.
image: name: ghcr.io/elementary-data/elementary:<TAG> # Replace <TAG> with edr's version. entrypoint: [""] elementary: script: - echo "$PROFILES_YML" > profiles.yml # Only required if `profiles.yml` doesn't exist in repository and using `PROFILES_YML` secret. - edr monitor --slack-webhook "$SLACK_WEBHOOK" - edr send-report --aws-access-key-id "$AWS_ACCESS_KEY_ID" --aws-secret-access-key "$AWS_SECRET_ACCESS_KEY" --s3-bucket-name "$S3_BUCKET_NAME" artifacts: paths: - edr.log
image property defines the Docker image to be used within the pipeline.
In this case, we’ll be using Elementary’s official Docker image.
elementary property defines the job and the
edr commands that will be ran.
It also uploads the
edr.log as a job artifact which can be useful for debugging purposes.
Feel free to edit the
.gitlab-ci.yml to your liking and make sure to add the required variables when doing so.
For more information about GitLab’s CI/CD capabilities.