🌸 K-Nearest Neighbors (KNN)

🌸 K-Nearest Neighbors (KNN)

Step-by-step mathematical demonstration of KNN using the Iris dataset.

1. What is KNN?

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm used mainly for classification and regression.

For classification, KNN looks at the nearest training examples and uses their labels to predict the class of a new data point.

New Flower ↓ Calculate Distance ↓ Sort Distances ↓ Select K Neighbors ↓ Majority Voting ↓ Prediction

2. Iris Training Dataset

Point	Sepal Length	Sepal Width	Petal Length	Petal Width	Class
P1	5.1	3.5	1.4	0.2	Setosa
P2	4.9	3.0	1.4	0.2	Setosa
P3	6.4	3.2	4.5	1.5	Versicolor
P4	6.9	3.1	4.9	1.5	Versicolor
P5	6.3	3.3	6.0	2.5	Virginica
P6	5.8	2.7	5.1	1.9	Virginica

1 New Flower

We want to classify a new flower.

X = (5.0, 3.1, 1.5, 0.2)

Sepal Length = 5.0 Sepal Width = 3.1 Petal Length = 1.5 Petal Width = 0.2

2 Choose K

K = 3

We will select the three closest flowers.

3 Euclidean Distance Formula

d(X,P) = √[ (x₁-p₁)² + (x₂-p₂)² + (x₃-p₃)² + (x₄-p₄)² ]

The four features are:

x₁ = Sepal Length x₂ = Sepal Width x₃ = Petal Length x₄ = Petal Width

4 Calculate Distance to P1

X = (5.0, 3.1, 1.5, 0.2) P1 = (5.1, 3.5, 1.4, 0.2)

d(X,P1) = √[ (5.0-5.1)² + (3.1-3.5)² + (1.5-1.4)² + (0.2-0.2)² ]
= √[ 0.01 + 0.16 + 0.01 + 0 ]
= √0.18
= 0.424

P1 belongs to Setosa.

5 Calculate Distance to P2

X = (5.0, 3.1, 1.5, 0.2) P2 = (4.9, 3.0, 1.4, 0.2)

d(X,P2) = √[ (5.0-4.9)² + (3.1-3.0)² + (1.5-1.4)² + (0.2-0.2)² ]
= √[ 0.01 + 0.01 + 0.01 + 0 ]
= √0.03
= 0.173

P2 belongs to Setosa.

6 Calculate Distance to P3

P3 = (6.4, 3.2, 4.5, 1.5)

d(X,P3) = √[ (5.0-6.4)² + (3.1-3.2)² + (1.5-4.5)² + (0.2-1.5)² ]
= √[ 1.96 + 0.01 + 9.00 + 1.69 ]
= √12.66
= 3.558

P3 belongs to Versicolor.

7 Calculate Distance to P4

P4 = (6.9, 3.1, 4.9, 1.5)

d(X,P4) = √[ (5.0-6.9)² + (3.1-3.1)² + (1.5-4.9)² + (0.2-1.5)² ]
= √[ 3.61 + 0 + 11.56 + 1.69 ]
= √16.86
= 4.106

8 Calculate Distance to P5

P5 = (6.3, 3.3, 6.0, 2.5)

d(X,P5) = √[ (5.0-6.3)² + (3.1-3.3)² + (1.5-6.0)² + (0.2-2.5)² ]
= √[ 1.69 + 0.04 + 20.25 + 5.29 ]
= √27.27
= 5.222

9 Calculate Distance to P6

P6 = (5.8, 2.7, 5.1, 1.9)

d(X,P6) = √[ (5.0-5.8)² + (3.1-2.7)² + (1.5-5.1)² + (0.2-1.9)² ]
= √[ 0.64 + 0.16 + 12.96 + 2.89 ]
= √16.65
= 4.080

10 Sort the Distances

Rank	Point	Distance	Class
1	P2	0.173	Setosa
2	P1	0.424	Setosa
3	P3	3.558	Versicolor
4	P6	4.080	Virginica
5	P4	4.106	Versicolor
6	P5	5.222	Virginica

11 Select K = 3 Neighbors

Nearest Neighbor 1: P2 → Setosa → 0.173

Nearest Neighbor 2: P1 → Setosa → 0.424

Nearest Neighbor 3: P3 → Versicolor → 3.558

12 Majority Voting

Setosa: P1 + P2 = 2 Votes

Versicolor: P3 = 1 Vote

Virginica: = 0 Votes

🏆 Prediction = IRIS SETOSA

13. Complete KNN Process

New Flower

→

Choose K

→

Distance

→

Sort

→

K Neighbors

→

Voting

→

Prediction

14. ⭐ Interactive Step-by-Step KNN

① Input

X = (5.0, 3.1, 1.5, 0.2)

This is the new flower that we want to classify.

② Choose K

K = 3

The three nearest training points will participate in voting.

③ Distance Formula

d(X,P) = √[ (x₁-p₁)² + (x₂-p₂)² + (x₃-p₃)² + (x₄-p₄)² ]

④ Calculate Distance

d(X,P2) = √0.03 = 0.173

P2 → Setosa

⑤ Sort Distances

0.173 → P2
0.424 → P1
3.558 → P3
4.080 → P6
4.106 → P4
5.222 → P5

⑥ Select 3 Neighbors

P2 → Setosa
P1 → Setosa
P3 → Versicolor

⑦ Voting

Setosa = 2
Versicolor = 1
Virginica = 0

The class with the highest number of votes wins.

⑧ Final Prediction

🌸 IRIS SETOSA

15. Important KNN Terms

KNN Training Data Test Data K Value Neighbor Distance Euclidean Distance Feature Classification Majority Voting Prediction Feature Scaling

B.K. PAUL

Bhairab Ganguly College

Artificial Intelligence & Machine Learning Learning Module

© 2026 B.K. PAUL • Bhairab Ganguly College