🧠 KNN Using an ANN-Style Learning Model
Step-by-step mathematical explanation using
Age, Salary and Credit Score
⚠️ Important Concept
KNN and ANN are different algorithms.
KNN = K-Nearest Neighbors
ANN = Artificial Neural Network
Here, KNN is represented in an ANN-style input → processing → output structure so that the mathematical process is easier to understand.
KNN = K-Nearest Neighbors
ANN = Artificial Neural Network
Here, KNN is represented in an ANN-style input → processing → output structure so that the mathematical process is easier to understand.
1. ANN-Style Representation of KNN
INPUT LAYER
Age
Salary
Credit Score
Age
Salary
Credit Score
→
DISTANCE LAYER
d₁
d₂
d₃
...
d₁
d₂
d₃
...
→
K NEIGHBORS
Nearest K
Points
Nearest K
Points
→
OUTPUT
Approved
OR
Rejected
Approved
OR
Rejected
1 Training Dataset
Suppose a bank has the following historical customer data.
| Customer | Age | Salary (₹000) | Credit Score | Decision |
|---|---|---|---|---|
| A | 25 | 30 | 650 | Rejected |
| B | 28 | 35 | 670 | Rejected |
| C | 30 | 40 | 700 | Approved |
| D | 35 | 50 | 720 | Approved |
| E | 40 | 60 | 750 | Approved |
| F | 45 | 70 | 780 | Approved |
2 New Customer
Age = 32
Salary = ₹45,000
Credit Score = 710
Salary = ₹45,000
Credit Score = 710
We want to determine whether the new customer's credit card application should be approved.
3 Choose K
K = 3
The three nearest customers will vote on the final decision.
4 Why Feature Scaling is Important
Notice that:
Age ≈ 32
Salary ≈ 45
Credit Score ≈ 710
Credit Score has a much larger numerical value. If we directly calculate Euclidean distance, Credit Score could dominate the distance calculation. Therefore, KNN generally benefits from feature scaling.
Age ≈ 32
Salary ≈ 45
Credit Score ≈ 710
Credit Score has a much larger numerical value. If we directly calculate Euclidean distance, Credit Score could dominate the distance calculation. Therefore, KNN generally benefits from feature scaling.
Standardization:
z = (x − μ) / σ
For teaching simplicity, we will use normalized values below.
5 Normalized Input
| Customer | Age | Salary | Credit Score | Decision |
|---|---|---|---|---|
| A | 0.00 | 0.00 | 0.00 | Rejected |
| B | 0.15 | 0.125 | 0.154 | Rejected |
| C | 0.25 | 0.25 | 0.385 | Approved |
| D | 0.50 | 0.50 | 0.538 | Approved |
| E | 0.75 | 0.75 | 0.769 | Approved |
| F | 1.00 | 1.00 | 1.00 | Approved |
New Customer
X = (0.35, 0.375, 0.462)
6 Euclidean Distance
d(X,P)
=
√[
(x₁-p₁)²
+
(x₂-p₂)²
+
(x₃-p₃)²
]
Here:
x₁ = Age
x₂ = Salary
x₃ = Credit Score
x₂ = Salary
x₃ = Credit Score
7 Distance from Customer A
X = (0.35, 0.375, 0.462)
A = (0, 0, 0)
A = (0, 0, 0)
d(X,A)
=
√[
(0.35−0)²
+
(0.375−0)²
+
(0.462−0)²
]
= √[ 0.1225 + 0.140625 + 0.213444 ]
= √0.476569
≈ 0.690
= √[ 0.1225 + 0.140625 + 0.213444 ]
= √0.476569
≈ 0.690
8 Distance from Customer B
B = (0.15,0.125,0.154)
d(X,B)
=
√[
(0.35−0.15)²
+
(0.375−0.125)²
+
(0.462−0.154)²
]
= √[ 0.04 + 0.0625 + 0.094864 ]
= √0.197364
≈ 0.444
= √[ 0.04 + 0.0625 + 0.094864 ]
= √0.197364
≈ 0.444
9 Distance from Customer C
C = (0.25,0.25,0.385)
d(X,C)
=
√[
(0.35−0.25)²
+
(0.375−0.25)²
+
(0.462−0.385)²
]
= √[ 0.01 + 0.015625 + 0.005929 ]
= √0.031554
≈ 0.178
= √[ 0.01 + 0.015625 + 0.005929 ]
= √0.031554
≈ 0.178
10 Distance from Customer D
D = (0.50,0.50,0.538)
d(X,D)
=
√[
(0.35−0.50)²
+
(0.375−0.50)²
+
(0.462−0.538)²
]
= √[ 0.0225 + 0.015625 + 0.005776 ]
= √0.043901
≈ 0.209
= √[ 0.0225 + 0.015625 + 0.005776 ]
= √0.043901
≈ 0.209
11 Distance from Customer E
d(X,E)
≈
0.530
Customer E is farther away than C and D.
12 Distance from Customer F
d(X,F)
≈
1.005
Customer F is the farthest among the six examples.
13 Sort the Distances
| Rank | Customer | Distance | Decision |
|---|---|---|---|
| 1 | C | 0.178 | Approved |
| 2 | D | 0.209 | Approved |
| 3 | B | 0.444 | Rejected |
| 4 | E | 0.530 | Approved |
| 5 | A | 0.690 | Rejected |
| 6 | F | 1.005 | Approved |
14 Select K = 3 Neighbors
Nearest 1:
C → Approved
Nearest 2: D → Approved
Nearest 3: B → Rejected
Nearest 2: D → Approved
Nearest 3: B → Rejected
15 Majority Voting
Approved
= 2 Votes
Rejected = 1 Vote
Rejected = 1 Vote
Approved has the majority.
16 Final Prediction
💳 CREDIT CARD APPROVAL
✅ APPROVED
✅ APPROVED
17. KNN Process in ANN-Style Form
INPUT
Age
Salary
Credit Score
Age
Salary
Credit Score
→
DISTANCE
d(A)
d(B)
d(C)
d(D)
d(E)
d(F)
d(A)
d(B)
d(C)
d(D)
d(E)
d(F)
→
K = 3
C
D
B
C
D
B
→
OUTPUT
APPROVED
APPROVED
⭐ Interactive Step-by-Step KNN
① Input Layer
Age = 32
Salary = ₹45,000
Credit Score = 710
Salary = ₹45,000
Credit Score = 710
② Choose K
K = 3
Three nearest customers will vote.
③ Distance Layer
d = √[
(Age difference)²
+
(Salary difference)²
+
(Credit Score difference)²
]
The distance is calculated for every training customer.
④ Sort Distances
C = 0.178
D = 0.209
B = 0.444
E = 0.530
A = 0.690
F = 1.005
D = 0.209
B = 0.444
E = 0.530
A = 0.690
F = 1.005
⑤ Select Neighbors
C → ApprovedD → Approved
B → Rejected
⑥ Voting
Approved = 2Rejected = 1
2 > 1
Therefore: Approved wins.
⑦ Final Prediction
Customer's Credit Card✅ APPROVED
18. Complete Mathematical Flow
INPUT
↓
Age + Salary + Credit Score
↓
Feature Scaling
↓
Calculate Distance
↓
Sort Distances
↓
Choose K = 3
↓
Select 3 Nearest Neighbors
↓
Majority Voting
↓
Final Classification
↓
💳 APPROVED
19. KNN vs ANN
| KNN | ANN |
|---|---|
| Instance-based algorithm | Neural network |
| Uses distance | Uses weighted sums |
| Uses K neighbors | Uses neurons |
| Uses majority voting | Uses activation functions |
| No weight training | Weights are learned |
| Usually lazy learning | Training is required |
No comments:
Post a Comment