software_engineering Archives

Artificial Intelligence interview questions

1. What is Artificial Intelligence ? Artificial intelligence is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans…

PySpark : Inserting row in Apache Spark Dataframe.
In PySpark, you can insert a row into a DataFrame by first converting the DataFrame…
PySpark : Connecting and updating postgres table in spark SQL
Apache Spark is an open-source, distributed computing system that can process large amounts of data…
PySpark : Explain in detail whether Apache Spark SQL lazy or not ?
Yes, Apache Spark SQL is lazy. In Spark, the concept of "laziness" refers to the…
In pyspark what is the difference between Spark spark.table() and spark.read.table()
In PySpark, spark.table() is used to read a table from the Spark catalog, whereas spark.read.table()…
How can you convert PySpark Dataframe to JSON ?
pyspark.sql.DataFrame.toJSON There may be some situation that you need to send your dataframe to a…
PySpark: How to add months to a date column in Spark DataFrame (add_months)
I have a use case where I want to add months to a date column…
PySpark : LongType and ShortType data types in PySpark
pyspark.sql.types.LongType pyspark.sql.types.ShortType In this article, we will explore PySpark's LongType and ShortType data types, their…
PySpark : How to decode in PySpark ?
pyspark.sql.functions.decode The pyspark.sql.functions.decode Function in PySpark PySpark is a popular library for processing big data…
PySpark : Explanation of MapType in PySpark with Example
MapType in PySpark is a data type used to represent a value that maps keys…
PySpark : How decode works in PySpark ?
One of the important concepts in PySpark is data encoding and decoding, which refers to…

Tag: software_engineering

How to run dataframe as Spark SQL – PySpark

Apache Storm interview questions

Apache PIG interview questions

Apache Spark interview questions

Data communication interview questions

Artificial Intelligence interview questions

Data Structure interview questions

Trending

Recent Posts

Featured Posts – Slider Widget

Setting up Minikube using docker driver on ubuntu

Optimizing PySpark queries with adaptive query execution – (AQE) – Example included

Transferring elastic IP between AWS accounts – Step by step process

Handling NULL values in dynamic SQL insert statements using Python

PySpark : Calculate the Euclidean distance or the square root of the sum of the squares of its arguments using PySpark.

PySpark : How to perform compute covariance using covar_pop and covar_samp with PySpark

Automated email responses using Gmail and google sheets with Google apps script

Navigating job dependencies in AWS glue – Managing ETL workflows

Airflow scheduler does not appear to be running. Last heartbeat was received 20 minutes ago. The DAGs list may not update : Resolved

Spark repartition() vs coalesce() – A complete information

Most Viewed Posts