Pandas, a linchpin in Python’s data analysis toolkit, is equipped with an array of statistical functions. These functions are indispensable for exploring, understanding, and deriving insights from datasets. This article introduces some of the most crucial statistical functions available in Pandas.
Core Statistical Functions in Pandas
1. Descriptive Statistics
a. .describe()
Offers a quick overview of the central tendencies, dispersion, and shape of a dataset’s distribution.
b. .mean()
Calculates the mean of the values for the requested axis.
c. .median()
Finds the median, which is the value separating the higher half from the lower half of a data sample.
d. .mode()
Determines the mode or the value that appears most frequently in a dataset.
2. Measures of Spread
a. .std()
Computes the standard deviation, a measure of the amount of variation or dispersion in a set of values.
b. .var()
Calculates the variance, quantifying the degree of spread in a set of data points.
c. .quantile()
Finds the quantile, a value below which a certain percent of observations fall.
3. Correlation and Covariance
a. .corr()
Evaluates the correlation between columns in a DataFrame, offering insights into the relationship between variables.
b. .cov()
Computes the covariance, indicating the direction of the linear relationship between variables.
Practical Application with Sample Data
To illustrate these functions, let’s use a simple dataset:
import pandas as pd
# Learning @ Freshers.in
data = {
'Age': [25, 30, 35, 40, 45],
'Salary': [50000, 55000, 60000, 65000, 70000]
}
df = pd.DataFrame(data)
# Applying Statistical Functions
print("Describe:\n", df.describe())
print("Mean:\n", df.mean())
print("Standard Deviation:\n", df.std())
print("Correlation:\n", df.corr())When to Use Statistical Functions
- Exploratory Data Analysis (EDA): To get a quick overview and understand the basic properties of the dataset.
- Data Cleaning: Identifying outliers or errors in the data.
- Data Modeling: Understanding relationships between variables before building predictive models.
