Tag: Airflow
Airflow : Using Boto3 in Airflow
Boto3 is the Amazon Web Services (AWS) SDK for Python, which allows Python developers to write software that makes use…
Advantages of using AWS MWAA (AWS Managed Workflows for Apache Airflow)
AWS Managed Workflows for Apache Airflow (MWAA) is a fully managed service that makes it easy to run Apache Airflow…
How to pass snowflake connection parameters in snowflake operator in airflow
In order to pass connection parameters to the SnowflakeOperator in Airflow, you need to first create a connection object in…
Airflow dags not getting refreshed/updating. How to do it manually?
Once we created DAG in Airflow, we expect it to get refreshed the same time. Once we refresh the Airflow…
How to run a Spark Job in Airflow hosted on a different server using BashOperator ?
In this article we will discuss on how we can trigger a PySpark Job running on a AWS EMR from…
How to create a Airflow DAG(Scheduler) to execute a redshift query ?
Use case : We have a redshift query (an insert sql ) to load data from another table on daily…