Airflow dags not getting refreshed/updating. How to do it manually?

Apache Airflow

Once we created DAG in Airflow, we expect it to get refreshed the same time. Once we refresh the Airflow UI also the latest changes or dag(if new dag ) Cant be seen in the Airflow UI. Normally airflow refreshes dags every 30 seconds, so we expect to see the latest changes in 30 seconds . The common scenario where the dags are not getting refreshed may be due to some error in the dag python code. Another scenario , is if there are duplicate id ( same dag name for two files, can occour when you copy a code to create a new one ) .  To get the dags refreshed manually you can do the following. Log on to the server where your airflow is installed( If its a virtual environment , go to that path )  and perform the following steps. 

source workspace/virtualenv/bin/activate
export AIRFLOW_HOME=~/workspace/airflow
airflow dags list

The above will list all the dags in that server path. 

No if you want to get the error list as well then you have to give -v  in the last command above. To also retrieve a list of errors, you need to append -v to the final command mentioned above

airflow dags list -v


Author: user

Leave a Reply