Transformation & Orchestration
Airflow
Jobs information
Elementary can collect jobs data from Airflow, including job run url
, job name
, job id
.
These are presented in the lineage node info tab.
To collect jobs info for Airflow, check out this guide.
Syncs schedule
For near-real-time results, it is recommended to change the sync configuration from the default hourly syncs to webhook triggered syncs and invoke using Airflow.