PRINCIPAL COMPONENT ANALYSIS

PRINCIPAL COMPONENT ANALYSIS

3-Feature Numerical Example

Mathematics • Physics • Chemistry

STEP 01 Get the Data

Consider the marks obtained by five students in three subjects: Mathematics, Physics and Chemistry.

Student	Mathematics	Physics	Chemistry
A	2	1	5
B	4	3	4
C	6	5	8
D	8	7	6
E	10	9	10

Original Data Matrix

X = [ 2  1  5 ]
[ 4  3  4 ]
[ 6  5  8 ]
[ 8  7  6 ]
[10  9  10]

STEP 02 Compute the Mean Vector (μ)

μ = [ Mean(Math), Mean(Physics), Mean(Chemistry) ]

Mean(Math) = (2 + 4 + 6 + 8 + 10) / 5 = 6

Mean(Physics) = (1 + 3 + 5 + 7 + 9) / 5 = 5

Mean(Chemistry) = (5 + 4 + 8 + 6 + 10) / 5 = 6.6

Mean Vector: μ = [ 6, 5, 6.6 ]

STEP 03 Subtract Mean from the Given Data

X_centered = X − μ

Student	Math − 6	Physics − 5	Chemistry − 6.6
A	−4	−4	−1.6
B	−2	−2	−2.6
C	0	0	1.4
D	2	2	−0.6
E	4	4	3.4

X_centered = [ −4  −4  −1.6 ]
[ −2  −2  −2.6 ]
[ 0   0    1.4 ]
[ 2   2   −0.6 ]
[ 4   4    3.4 ]

STEP 04 Calculate the Covariance Matrix

Following the calculation style used in the Gate Vidyalay example, the covariance matrix is calculated using 1/n.

C = (1/n) X_centered^TX_centered

C = 1/5 × [ 40  40  24 ]
       [ 40  40  24 ]
       [ 24  24  23.2 ]

Covariance Matrix:

C = [ 8.00  8.00  4.80 ]
[ 8.00  8.00  4.80 ]
[ 4.80  4.80  4.64 ]

STEP 05 Calculate Eigenvectors and Eigenvalues

Eigenvalues

Component	Eigenvalue	Variance Explained
PC1	19.1711	92.88%
PC2	1.4689	7.12%
PC3	0.0000	0.00%

Total Variance = 19.1711 + 1.4689 + 0 = 20.6400

Principal Eigenvector — PC1

PC1 = [ 0.64065 ]
[ 0.64065 ]
[ 0.42325 ]

PC1 explains approximately 92.88% of the total variance. Therefore, PC1 is selected as the principal component.

Second Principal Component — PC2

PC2 = [ −0.29928 ]
[ −0.29928 ]
[ 0.90602 ]

STEP 06 Choosing Components and Forming Feature Vector

Since PC1 explains 92.88% of the total variance, it can be selected as the main reduced feature.

Feature Vector = [ PC1 ]

Feature Vector = [ 0.64065 ]
[ 0.64065 ]
[ 0.42325 ]

Dimensionality Reduction: The original dataset contains 3 features. After PCA, the major information can be represented using 1 principal component with approximately 92.88% variance retention.

STEP 07 Deriving the New Dataset

PC1 Score = X_centered × PC1

Student	PC1 Calculation	PC1 Value
A	(−4)(0.64065)+(−4)(0.64065)+(−1.6)(0.42325)	−5.8024
B	(−2)(0.64065)+(−2)(0.64065)+(−2.6)(0.42325)	−3.6630
C	(0)(0.64065)+(0)(0.64065)+(1.4)(0.42325)	0.5925
D	(2)(0.64065)+(2)(0.64065)+(−0.6)(0.42325)	2.3087
E	(4)(0.64065)+(4)(0.64065)+(3.4)(0.42325)	6.5642

FINAL PCA DATASET

Student	Mathematics	Physics	Chemistry	PC1
A	2	1	5	−5.8024
B	4	3	4	−3.6630
C	6	5	8	0.5925
D	8	7	6	2.3087
E	10	9	10	6.5642

PCA Visualization

3-Feature Dataset — PC1 Projection

Interpretation:
PC1 combines Mathematics, Physics and Chemistry into a single new feature. Because PC1 explains approximately 92.88% of the variance, most of the information in the original three-dimensional dataset is retained.

Final PCA Summary

Item	Result
Original Features	3
Mathematics Mean	6.0
Physics Mean	5.0
Chemistry Mean	6.6
PC1 Eigenvalue	19.1711
PC2 Eigenvalue	1.4689
PC3 Eigenvalue	0.0000
PC1 Variance Explained	92.88%
PC2 Variance Explained	7.12%
PC3 Variance Explained	0.00%
Reduced Dimension	3 → 1

3 FEATURES → 1 PRINCIPAL COMPONENT

PC1 retains approximately 92.88% of the total variance.

Created by
BIJAN KRISHNA PAUL
BHAIRAB GANGULY COLLEGE