Dimensionality Reduction: PCA in Code¶
Why this matters¶
The earlier bridge lessons explained the idea:
This page makes that idea visible with one small PCA run.
The goal is not to build the best dimensionality-reduction result. The goal is to read simple code and output, then connect the output back to the theory:
Mental model¶
PCA creates new coordinate columns.
Those new columns are not original columns like size_m2, rooms, or price_k. They are summary axes learned from patterns in the original columns.
So after PCA, one apartment is still one row, but it is described with fewer numbers.
Core ideas¶
- PCA takes a table of numeric features as input.
- Standardization puts the features on comparable scales before PCA judges spread.
n_components=2asks PCA to create two new summary columns.- The reduced data has the same number of rows as the original data.
- The reduced data has fewer columns than the original data.
- PCA values can be negative because they are coordinates on new axes.
- Explained variance ratio says how much spread each PCA component kept.
One tiny PCA run¶
This invented apartment dataset has six rows and three numeric columns:
import numpy as np
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
X = np.array([
[35, 1, 210],
[45, 2, 260],
[55, 2, 330],
[70, 3, 430],
[85, 3, 520],
[100, 4, 650],
])
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X_scaled)
print("Original shape:", X.shape)
print("Reduced shape:", X_pca.shape)
print("\nFirst three apartments after PCA:")
print(np.round(X_pca[:3], 2))
print("\nExplained variance ratio:")
print(np.round(pca.explained_variance_ratio_, 2))
Output:
Original shape: (6, 3)
Reduced shape: (6, 2)
First three apartments after PCA:
[[-2.39 -0.24]
[-1.35 0.31]
[-0.82 -0.06]]
Explained variance ratio:
[0.98 0.02]
How to read the output¶
The shape changed¶
Before PCA:
That means:
After PCA:
That means:
The number of rows did not change. PCA did not remove apartments.
The number of columns changed because PCA replaced three original features with two new summary features.
The PCA columns are new coordinates¶
The first three reduced rows are:
Each row still describes one apartment.
But these two numbers are not:
They are coordinates on PCA's new axes:
Negative values are normal. They just mean the apartment lies on one side of a learned PCA axis.
Explained variance tells you what the components kept¶
The explained variance ratio is:
Read this as:
In this tiny dataset, size_m2, rooms, and price_k move together strongly. Bigger apartments tend to have more rooms and higher prices.
That is why one PCA component can keep most of the spread.
This does not mean PCA discovered the best possible prediction features. It only means these two PCA components kept almost all of the numeric spread in this tiny input table.
When the PCA signal is weaker¶
The output above is a strong output signal for compression because two components kept nearly all of the spread:
Not every PCA result looks like that.
Suppose a 2D PCA result printed:
Together, the first two components keep:
That means the 2D PCA view kept 75% of the spread and left 25% out.
That is a weaker output signal.
It might still be useful for a rough visualization, but it is not strong evidence that two PCA components are enough for a faithful compression.
Use this careful wording:
PCA is eligible here, but this output gives weaker evidence that a 2D PCA view is enough.
I would check the plot or compare model performance before relying on it.
How this connects to the full lesson¶
When the full lesson uses the Wine dataset, the table is larger:
The same reading habit applies:
shape before PCA -> how many rows and original columns went in
shape after PCA -> how many rows and new component columns came out
explained variance -> how much spread the components kept
For deciding whether PCA is appropriate before running code, see Dimensionality Reduction: Why PCA and LDA Work Differently.
Common traps¶
The PCA numbers are not original columns
After PCA, the reduced columns are new component coordinates. They are not size_m2, rooms, or price_k.
Negative PCA values are not errors
PCA values are positions on learned axes. Being negative just means the row lies on one side of that axis.
High explained variance is not the same as best prediction
PCA preserves spread. It does not know target labels unless a later supervised model uses the reduced features.
Eligible does not mean useful
PCA may be logically allowed for numeric data, but you still need output or metrics to decide whether it helped.
Check yourself¶
What shape did the data have before PCA?
(6, 3): six rows and three original feature columns.
What shape did the data have after PCA?
(6, 2): six rows and two new PCA component columns.
Are the two PCA columns original apartment columns?
No. They are new summary coordinates created from the original columns.
What does the explained variance ratio tell you?
How much of the original spread each PCA component kept.
If two PCA components keep 75% of the spread, what should you say?
PCA may still be useful for a rough 2D view, but the output gives weaker evidence that two components are enough for faithful compression.
Next¶
Next, read Dimensionality Reduction: LDA in Code.
That page shows the supervised version: LDA uses labels to create class-separating component columns.
Source anchors¶
- Supports:
study-guide/docs/lessons/05-dimensionality-reduction.md - Source file:
notebooks/Module2/05-Dimensionality Reduction.ipynb - Key source concepts prepared here: PCA, standardization, component columns, transformed shape, explained variance ratio