rsync : Stands for ‘remote sync’. This tool is used for synchronizing files and directories between two locations.
Example
"rsync -avh -e "$ssh_command" "$CI__DIR"/myscripts/dags/ "$server":work/airflow/dags/user/ --delete --exclude '__pycache__' --omit-dir-times"rsync -avh -e “$ssh_command”: The rsync command is initiating the synchronization process. The -a flag tells rsync to archive, which means it will preserve symbolic links, file permissions, user & group ownerships, and timestamps during the sync. The -v flag stands for verbose, which means it will show the progress of the files being synced. The -h flag is for human-readable, meaning it will display file sizes in a format easier for humans to read (like KB, MB, GB, etc.). The -e flag is used to specify the remote shell to use. In this case, it’s using whatever shell is specified in the “$ssh_command” variable.
“$CI_DIR”/myscripts/dags/ “$server”:work/airflow/dags/user/: These are the source and destination of the synchronization. “$CI_DIR”/myscripts/dags/ is the source directory on the local machine, where “$CI_DIR” is a variable that holds the path to the project directory. “$server”:work/airflow/dags/user/ is the destination on the remote server, where “$server” is a variable that holds the address of the remote server.
–delete: This flag means that any files or directories at the destination that are not present at the source will be deleted.
–exclude ‘__pycache__’: This flag tells rsync to exclude any files or directories named __pycache__ during the synchronization process. This is generally used to exclude Python cache files that are not necessary to be synced.
–omit-dir-times: This flag tells rsync to omit directories when it is preserving modification times (-t). The modification times are not particularly useful for directories since they can change even when the files within the directory have not been modified, and syncing them could unnecessarily consume resources.
Please note that the variables (“$ssh_command”, “$CI_DIR”, “$airflow_server”) enclosed in the $ and quotation marks are placeholders for actual values. These variables would have been defined earlier in the script or in the environment.
