In this article, we'll dive into the various settings you can configure in this file,…
Category: article
DBT : How to restrict your project to only work with a range of dbt versions.
dbt allows you to specify a required version of dbt in your dbt_project.yml file using the require-dbt-version key. This feature…
DBT : How to specify a custom list of directories to be removed by the dbt clean
One of the key features of DBT is its ability to create “clean” target tables, which are optimized for querying…
Ways to get the distribution of a column in Python, depending on the type of data
There are several ways to get the distribution of a column in Python, depending on the type of data and…
PySpark : How to create a map from a column of structs : map_from_entries
pyspark.sql.functions.map_from_entries map_from_entries(col) is a function in PySpark that creates a map from a column of structs, where the structs have…
PySpark : Converting Unix timestamp to a string representing the timestamp in a specific format
pyspark.sql.functions.from_unixtime The “from_unixtime()” function is a PySpark function that allows you to convert a Unix timestamp (a long integer representing…
PySpark : Check if two or more arrays in a DataFrame column have any common elements
pyspark.sql.functions.arrays_overlap The arrays_overlap function is a PySpark function that allows you to check if two or more arrays in a…
PySpark : Combine the elements of two or more arrays in a DataFrame column
pyspark.sql.functions.array_union The array_union function is a PySpark function that allows you to combine the elements of two or more arrays…
PySpark : Sort an array of elements in a DataFrame column
pyspark.sql.functions.array_sort The array_sort function is a PySpark function that allows you to sort an array of elements in a DataFrame…
Apple’s iOS 16.3 software update . All about Security Fixes, Advanced Data Protection and Hardware Keys
iOS 16.3 is a software update for Apple’s iOS operating system. This update includes security fixes, advanced data protection features,…
PySpark : How to sort a dataframe column in ascending order while putting the null values first ?
pyspark.sql.Column.asc_nulls_first In PySpark, the asc_nulls_first() function is used to sort a column in ascending order while putting the null values…