GitLab CI

You can use GitLab’s CI in order to run edr commands in an automated manner.

For example, you can use it to send a data observability report on changes in your dbt project, monitor for data issues periodically and send alerts to your Slack, and much more.

​

Setup

​

Profile

edr requires a dbt connection profile called elementary that points to your Elementary schema in the warehouse. More information on that can be found here. Accordingly, we’ll need to provide the pipeline with that profile.

If you already have a profiles.yml in your repository, simply add the elementary profile to it, and make sure the credentials are loaded as variables. edr will automatically pick up on the profiles.yml file.

Alternatively, you can write your profiles.yml as a whole to a GitLab variable.

​

Pipeline

In order to setup the Elementary pipeline in your GitLab repository, you’ll need to create a file at the root of the project called .gitlab-ci.yml with the following content.

image:
  name: ghcr.io/elementary-data/elementary:<TAG> # Replace <TAG> with edr's version.
  entrypoint: [""]

elementary:
  script:
    - echo "$PROFILES_YML" > profiles.yml # Only required if `profiles.yml` doesn't exist in repository and using `PROFILES_YML` secret.
    - edr monitor --slack-webhook "$SLACK_WEBHOOK"
    - edr send-report --aws-access-key-id "$AWS_ACCESS_KEY_ID" --aws-secret-access-key "$AWS_SECRET_ACCESS_KEY" --s3-bucket-name "$S3_BUCKET_NAME"
  artifacts:
    paths:
      - edr.log

The image property defines the Docker image to be used within the pipeline. In this case, we’ll be using Elementary’s official Docker image.

The elementary property defines the job and the edr commands that will be ran. It also uploads the edr.log as a job artifact which can be useful for debugging purposes.

Feel free to edit the .gitlab-ci.yml to your liking and make sure to add the required variables when doing so.

For more information about GitLab’s CI/CD capabilities.