PRINCIPAL COMPONENT ANALYSIS

PRINCIPAL COMPONENT ANALYSIS

4-Feature Numerical Example

Mathematics • Physics • Chemistry • Biology

STEP 01 Get the Data

Consider the marks obtained by five students in four subjects: Mathematics, Physics, Chemistry and Biology.

Student	Mathematics	Physics	Chemistry	Biology
A	2	1	5	7
B	4	3	4	6
C	6	5	8	9
D	8	7	6	8
E	10	9	10	11

Original Data Matrix

X = [ 2  1  5  7 ]
[ 4  3  4  6 ]
[ 6  5  8  9 ]
[ 8  7  6  8 ]
[10  9  10  11]

STEP 02 Compute the Mean Vector (μ)

μ = [ Mean(Math), Mean(Physics), Mean(Chemistry), Mean(Biology) ]

Mean(Math) = (2 + 4 + 6 + 8 + 10) / 5 = 6

Mean(Physics) = (1 + 3 + 5 + 7 + 9) / 5 = 5

Mean(Chemistry) = (5 + 4 + 8 + 6 + 10) / 5 = 6.6

Mean(Biology) = (7 + 6 + 9 + 8 + 11) / 5 = 8.2

Mean Vector:
μ = [ 6, 5, 6.6, 8.2 ]

STEP 03 Subtract Mean from the Given Data

X_centered = X − μ

Student	Math − 6	Physics − 5	Chemistry − 6.6	Biology − 8.2
A	−4	−4	−1.6	−1.2
B	−2	−2	−2.6	−2.2
C	0	0	1.4	0.8
D	2	2	−0.6	−0.2
E	4	4	3.4	2.8

X_centered = [ −4  −4  −1.6  −1.2 ]
[ −2  −2  −2.6  −2.2 ]
[ 0   0    1.4    0.8 ]
[ 2   2   −0.6  −0.2 ]
[ 4   4    3.4    2.8 ]

STEP 04 Calculate the Covariance Matrix

Following the Gate Vidyalay calculation style, covariance is calculated using 1/n.

C = (1/n) X_centered^TX_centered

C = [ 8.00  8.00  4.80  4.00 ]
[ 8.00  8.00  4.80  4.00 ]
[ 4.80  4.80  4.64  3.68 ]
[ 4.00  4.00  3.68  2.96 ]

Covariance Matrix obtained successfully.

STEP 05 Calculate Eigenvectors and Eigenvalues

Eigenvalues

Component	Eigenvalue	Variance Explained
PC1	21.5758	91.42%
PC2	2.0053	8.50%
PC3	0.0189	0.08%
PC4	0.0000	0.00%

Total Variance = 21.5758 + 2.0053 + 0.0189 + 0 = 23.6000

Principal Eigenvector — PC1

PC1 = [ −0.59800 ]
[ −0.59800 ]
[ −0.41254 ]
[ −0.33854 ]

The sign of an eigenvector is arbitrary. Therefore, the equivalent positive eigenvector can also be used:

PC1 = [ 0.59800 ]
[ 0.59800 ]
[ 0.41254 ]
[ 0.33854 ]

PC1 explains approximately 91.42% of the total variance. Therefore, PC1 is selected as the principal component.

Second Principal Component — PC2

PC2 ≈ [ −0.37660 ]
[ −0.37660 ]
[ 0.69244 ]
[ 0.48669 ]

STEP 06 Choosing Components and Forming Feature Vector

PC1 contains approximately 91.42% of the total variance. Therefore, the four original features can be reduced primarily to one principal component.

Feature Vector = [ PC1 ]

Feature Vector = [ 0.59800 ]
[ 0.59800 ]
[ 0.41254 ]
[ 0.33854 ]

Dimensionality Reduction: Original dimensions = 4
Reduced dimensions = 1
Variance retained ≈ 91.42%

STEP 07 Deriving the New Dataset

PC1 Score = X_centered × PC1

Student	PC1 Score
A	−5.8503
B	−4.2094
C	0.8484
D	2.0768
E	7.1345

FINAL PCA DATASET

Student	Mathematics	Physics	Chemistry	Biology	PC1
A	2	1	5	7	−5.8503
B	4	3	4	6	−4.2094
C	6	5	8	9	0.8484
D	8	7	6	8	2.0768
E	10	9	10	11	7.1345

PCA Visualization

4-Feature Dataset — PC1 Projection

Interpretation:
The four original features are transformed into principal components. PC1 captures approximately 91.42% of the total variation, making it the most important direction in the dataset.

Final PCA Summary

Item	Result
Original Features	4
Mathematics Mean	6.0
Physics Mean	5.0
Chemistry Mean	6.6
Biology Mean	8.2
PC1 Eigenvalue	21.5758
PC2 Eigenvalue	2.0053
PC3 Eigenvalue	0.0189
PC4 Eigenvalue	0.0000
PC1 Variance Explained	91.42%
PC2 Variance Explained	8.50%
PC3 Variance Explained	0.08%
PC4 Variance Explained	0.00%
Reduced Dimension	4 → 1

4 FEATURES → 1 PRINCIPAL COMPONENT

PC1 retains approximately 91.42% of the total variance.

Created by
BIJAN KRISHNA PAUL
BHAIRAB GANGULY COLLEGE