While the web-based UI is a common interface for interacting with Airflow, many tasks, including triggering DAGs, can be performed directly from the console. In this guide, we’ll walk you through how to run an Airflow DAG, specifically named freshers_daily_viewership, directly from the console.
Prerequisites:
An installed and configured Apache Airflow instance.
A defined DAG with the name freshers_daily_viewership in your Airflow DAGs folder.
Access to the console or terminal where Airflow commands can be run.
Step-by-Step Procedure to Run the DAG:
Navigate to the Airflow Home Directory (Optional):
Before running any Airflow commands, it’s often a good practice to navigate to the Airflow home directory. This step might be optional depending on your setup.
cd $AIRFLOW_HOME
Check if the DAG is Present:
Before triggering the DAG, ensure that it exists and is recognized by Airflow.
airflow list_dags
In the output, look for freshers_daily_viewership. If it’s present, you can proceed.
Inspect the DAG (Optional):
If you want to inspect the tasks and structure of the DAG before running it, use:
airflow list_tasks freshers_daily_viewership --tree
This command provides a tree view of all the tasks in the freshers_daily_viewership DAG.
Run the DAG:
To trigger the DAG run from the console, use the following command:
airflow trigger_dag freshers_daily_viewership
This will initiate a run of the freshers_daily_viewership DAG. By default, the run’s execution date will be set to the current date and time. If you wish to specify a different execution date, you can use the –exec-date option followed by the desired date in the YYYY-MM-DD format.
Monitor the DAG Run (Optional):
If you wish to monitor the DAG’s progress directly from the console, you can use the airflow task_state command to get the status of individual tasks. For example, to check the state of a task named task_1 in the freshers_daily_viewership DAG for today’s date, you’d use:
airflow task_state freshers_daily_viewership task_1 $(date +"%Y-%m-%d")
The ability to trigger and monitor Airflow DAGs directly from the console provides flexibility, especially for automation, testing, or debugging purposes. The above step-by-step guide ensures you can confidently run the freshers_daily_viewership DAG or any other DAG in your Airflow environment from the console. Always remember to monitor your DAGs, especially when manually triggered, to ensure they complete successfully and as expected.
Read more on Airflow here : Airflow
