orchestrator
job_name
job_id
job_url
job_run_id
job_run_url
env_vars
.
Orchestration tools usually have default environment variables, so this might happen automatically. The list of supported orchestrators and default env vars is in the following section.
These are the env vars that are collected:
ORCHESTRATOR
, JOB_NAME
, JOB_ID
, JOB_URL
, JOB_RUN_ID
, JOB_RUN_URL
To configure env_var
for your orchestrator, refer to your orchestrator's docs.
env_var
and var
exist, the var
will be prioritized.
To pass job data to Elementary using var
, use the --vars
flag in your invocations:
var / env_var | Format |
---|---|
orchestrator | One of: airflow , dbt_cloud , github_actions , prefect , dagster |
job_name, job_id, job_run_id | String |
job_url, job_run_url | Valid HTTP URL |
env_vars
/ vars
.
The following default environment variables are supported out of the box:
Orchestrator | Env vars |
---|---|
dbt cloud | orchestrator job_id: DBT_CLOUD_JOB_ID job_run_id: DBT_CLOUD_RUN_ID job_url: generated from DBT_ACCOUNT_ID , DBT_CLOUD_PROJECT_ID , DBT_CLOUD_JOB_ID job_run_url: generated from ACCOUNT_ID , DBT_CLOUD_PROJECT_ID , DBT_CLOUD_RUN_ID |
Github actions | orchestrator job_run_id: GITHUB_RUN_ID job_url: generated from GITHUB_SERVER_URL , GITHUB_REPOSITORY , GITHUB_RUN_ID |
Airflow | orchestrator |
--vars
:
dbt_invocations
. You could also use the view job_run_results
which groups invocation by job.