The Mathematics Behind Principal Component Analysis

Mathematics Behind PCA

Principal Component Analysis (PCA) is a dimensionality reduction technique used in machine learning and statistics to transform a dataset with many correlated variables into a smaller set of uncorrelated variables called principal components.

Steps behind Principal Component Analysis.

Step 1: Take the whole dataset consisting of d+1 dimensions and ignore the labels such that our new dataset becomes d dimensional.

Step 2:  Compute the mean of every dimension of the whole dataset.

Step 3: Mean Center the Data

Step 4: Compute the sample covariance matrix

Step 5: Compute Eigenvalues

Step 6: Compute Eigenvectors

Step 7: Principal Components

Step 8: Projection

Practice Problems

Matrix 1: The "E-Commerce Customer" Data

This matrix represents customer behavior on a shopping website. It’s perfect for teaching Customer Segmentation.

• Features: Number of Visits, Items in Cart, Total Spend (normalized).

        

 Identify which "row" (customer) is likely your most valuable user.

Matrix 2: The "Sensor Reading" Data

This matrix represents technical readings from a manufacturing machine. This is great for teaching Anomaly Detection.

• Features: Temperature (°C), Vibration Level, Power Consumption.

Matrix 3: The "Social Media Engagement" Data

This matrix tracks the performance of different posts. This is a classic use case for Feature Weighting.

• Features: Likes, Shares, Comments.

Thank you :)