Pandas

On This Page

• 14.1  Why Pandas? — The Data Analyst's #1 Tool
• 14.2  Series
• 14.3  DataFrame
• 14.4  Reading CSV & Excel Files
• 14.5  Cleaning Data & Missing Values
• 14.6  Filtering & Sorting
• 14.7  Merge & Join
• 14.8  GroupBy
• 14.9  Pivot Table
• 14.10  apply(), map(), replace()
• 14.11  Mini Project — Sales Data Analysis
• 14.12  Practical Exercises
• 14.13  Module Quiz (MCQs)
• 14.14  Module Assignment
• 14.15  Interview Questions — Module 14
• 14.16  Student Notes Page

Module 14: Pandas

14.1  Why Pandas? — The Data Analyst's #1 Tool

Pandas is the single most important library for Data Analytics in Python. It gives you the DataFrame — a table-like structure identical to an Excel sheet — along with powerful, fast tools to clean, filter, transform, and analyze data of any size.

pip install pandas

import pandas as pd

14.2  Series

A Series is a single column of data with labeled positions (an index) — think of it as one column from an Excel sheet.

marks = pd.Series([78, 85, 92, 66], index=["Priya","Rahul","Anita","Vikram"])
print(marks)
print(marks["Anita"])

14.3  DataFrame

A DataFrame is a full table made of multiple Series (columns) — the core structure you will use for almost all real data analytics work.

data = {
    "Name": ["Priya", "Rahul", "Anita", "Vikram"],
    "Course": ["SEO", "Google Ads", "SEO", "Meta Ads"],
    "Fee": [12000, 15000, 12000, 13000]
}
df = pd.DataFrame(data)
print(df)
print(df.shape)      # (rows, columns)
print(df.columns)    # column names

14.4  Reading CSV & Excel Files

df = pd.read_csv("students.csv")
df_excel = pd.read_excel("students.xlsx", sheet_name="Sheet1")

print(df.head())      # first 5 rows
print(df.tail(3))     # last 3 rows
print(df.info())      # column types, non-null counts
print(df.describe())  # statistical summary of numeric columns

14.5  Cleaning Data & Missing Values

Real-world data is messy — missing values, duplicates, and wrong data types are extremely common. Cleaning this data is often 70-80% of a data analyst's actual job.

print(df.isnull().sum())          # count missing values per column

df_cleaned = df.dropna()          # remove rows with any missing value
df_filled = df.fillna(0)          # replace missing values with 0
df["Fee"] = df["Fee"].fillna(df["Fee"].mean())   # fill with average

df = df.drop_duplicates()         # remove duplicate rows

14.6  Filtering & Sorting

seo_students = df[df["Course"] == "SEO"]
high_fee = df[df["Fee"] > 12000]
both = df[(df["Course"] == "SEO") & (df["Fee"] >= 12000)]

print(seo_students)

df_sorted = df.sort_values("Fee", ascending=False)
print(df_sorted)

14.7  Merge & Join

merge() combines two DataFrames based on a common column — exactly like Excel's VLOOKUP, but far more powerful.

students = pd.DataFrame({"ID":[1,2,3], "Name":["Priya","Rahul","Anita"]})
fees = pd.DataFrame({"ID":[1,2,3], "Fee":[12000,15000,12000]})

merged = pd.merge(students, fees, on="ID")
print(merged)

14.8  GroupBy

groupby() splits data into groups based on a column, then applies a calculation to each group — similar to Excel's SUBTOTAL or a Pivot Table's grouping logic.

revenue_by_course = df.groupby("Course")["Fee"].sum()
print(revenue_by_course)

avg_by_course = df.groupby("Course")["Fee"].mean()
print(avg_by_course)

14.9  Pivot Table

pivot_table() reshapes and summarizes data — Pandas' equivalent of Excel's Pivot Table feature.

pivot = pd.pivot_table(df, values="Fee", index="Course", aggfunc="sum")
print(pivot)

14.10  apply(), map(), replace()

df["Fee_with_GST"] = df["Fee"].apply(lambda x: round(x * 1.18, 2))
print(df[["Name", "Fee", "Fee_with_GST"]])

14.11  Mini Project — Sales Data Analysis

import pandas as pd

df = pd.read_csv("sales_data.csv")   # columns: Date, Product, Region, Units, Revenue

print(df.isnull().sum())
df = df.dropna()

revenue_by_region = df.groupby("Region")["Revenue"].sum().sort_values(ascending=False)
print("Revenue by Region:\n", revenue_by_region)

top_product = df.groupby("Product")["Units"].sum().idxmax()
print("Best-Selling Product:", top_product)

14.12  Practical Exercises

Basic (5 Questions)

1. Create a DataFrame of 5 students with Name, Course, and Fee columns.

2. Print the first 3 rows and the column names of your DataFrame.

3. Filter your DataFrame to show only students paying more than Rs.12000.

4. Sort your DataFrame by Fee in descending order.

5. Check for missing values in a sample DataFrame using isnull().sum().

Intermediate (5 Questions)

1. Load a CSV file and print df.info() and df.describe().

2. Group your student DataFrame by Course and calculate total and average fee per course.

3. Merge two DataFrames (students and fees) on a common ID column.

4. Use apply() to add an 18% GST column to your fee data.

5. Use replace() to rename a course value in your DataFrame.

Advanced (5 Questions)

1. Build a complete sales analysis: read a CSV, clean missing values, group by region, and find the top-performing region.

2. Create a pivot table showing total revenue by Product and Region together.

3. Build a 'Customer Churn Flag' column using apply() that marks customers as 'At Risk' if their purchase count is below a threshold.

4. Merge 3 different DataFrames (students, fees, attendance) into a single combined DataFrame.

5. Build a complete 'Monthly Revenue Report' that reads a CSV, cleans it, groups by month, and exports the summary to a new CSV.

14.13  Module Quiz (MCQs)

14.14  Module Assignment

Assignment: Institute Revenue Dashboard (Pandas)

• Create a CSV with 15 rows: Name, Course, Fee, Enrollment_Month.
• Load it into Pandas and clean any missing values.
• Calculate total and average revenue per Course using groupby().
• Create a pivot table showing revenue by Course and Enrollment_Month.
• Use apply() to add a Fee_with_GST column, and export the final cleaned DataFrame to a new CSV.

Expected Output: A complete, working revenue analysis pipeline built entirely using Pandas — from raw CSV to a clean, summarized report.

14.15  Interview Questions — Module 14

• Q: What is the difference between a Series and a DataFrame?
  A: A Series is a single labeled column of data, while a DataFrame is a full table made up of multiple Series (columns) sharing a common index.
• Q: How do you handle missing values in Pandas?
  A: Using dropna() to remove rows with missing data, or fillna() to replace missing values with a default, mean, or other computed value.
• Q: What is the difference between merge() and concat()?
  A: merge() combines DataFrames based on a common key column (like a SQL join), while concat() simply stacks DataFrames on top of or alongside each other without matching on a key.
• Q: What does groupby() actually do internally?
  A: It follows a 'split-apply-combine' pattern — splitting data into groups by a key, applying an aggregation function to each group, then combining the results back into a single structure.

14.16  Student Notes Page

Use this space to write down key points, doubts, and your own examples from today's session.