Tag: Airflow
How to run a Spark Job in Airflow hosted on a different server using BashOperator ?
user April 2, 2022 0 Comments on How to run a Spark Job in Airflow hosted on a different server using BashOperator ?
In this article we will discuss on how we can trigger a PySpark Job running on a AWS EMR from…
How to create a Airflow DAG(Scheduler) to execute a redshift query ?
user February 22, 2022 0 Comments on How to create a Airflow DAG(Scheduler) to execute a redshift query ?
Use case : We have a redshift query (an insert sql ) to load data from another table on daily…