There are situation you may need to programmatically transfer file from SFTP server to you…
Category: article
Python throwing as NameError: name ‘__file__’ is not defined – Solution
On Executing os.path.dirname(os.path.realpath(__file__)) in python interactive shell, you will get the error NameError: name ‘__file__’ is not defined. This is…
Amazon API Gateway quick reference and cheat sheet
1. Amazon API Gateway is an AWS service for creating, publishing, maintaining, monitoring, and securing REST, HTTP, and WebSocket APIs…
How to drop multiple partition in Hive by giving condition.
Hive Partitions is a good and easy way to organizes Hive tables into partitions by dividing tables into different parts…
How to delete a partition data as well from Hive external table on DROP command?
As you know external tables are tables where Hive does not manage the data of the External table. So when…
How to convert a hive managed table to external table without recreating it ?
In Hive, Managed tables / Internal table are Hive owned tables and the tables data are managed and controlled by…
How to copy data from Redshift to Hive ? How to unload from Redshift in Parquet format ? Explained with Sample code .
There may be some business scenario to copy daily data from Redshift to Hive. For the compression , it is…
How to do Force serialization in AWS Redshift table by locking all tables?
You can force serialization by locking all tables in each session. The LOCK command blocks operations that would result in…
Serializable isolation violation on table in Redshift : ERROR: 1023 : Resolved solution
When you run concurrent Amazon Redshift operations in different sessions, there is a probability of getting “ERROR: 1023 DETAIL: Serializable…
How to concatenate multiple columns in a Spark dataframe
concat_ws : With concat_ws () function you can concatenates multiple input string columns together into a single string column, using…
PySpark-How to create and RDD from a List and from AWS S3
In this article you will learn , what an RDD is ? How can we create an RDD from a…