DBT Airflow Factory

Python Version PyPI Version Downloads Maintainability Test Coverage

Introduction

The factory is a library for parsing DBT manifest files and building Airflow DAG.

The library is expected to be used inside an Airflow environment with a Kubernetes image referencing dbt.

dbt-airflow-factory’s main task is to parse manifest.json and create Airflow DAG out of it. It also reads config YAML files from config directory and therefore is highly customizable (e.g., user can set path to manifest.json). DAG building is an on-the-fly process without materialization. Also, the process may use Airflow Variables as a way of configuration.

Community

Although the tools was created by GetInData and used in their project it is open-sourced and every one is welcome to use and contribute to make it better and even more usefull.