ELT (Extract Load Transform) DAG file for Cloud Composer (Airflow)
How to:
- Install airflow
pip install "airflow[gcp,mysql]"
- Run airflow webserver & scheduler
- Create a mysql connection on webserver under Admin > Connections
- Change values on the
devfest2020.py
file as commented - Move the python file to dag folder