🌳 Decision Tree Using ANN-Style Visualization
Step-by-step Decision Tree mathematics using Age, Salary and Credit Score
⚠️ Important
Decision Tree is NOT an Artificial Neural Network.
This example uses an ANN-style layer visualization to explain the flow of a Decision Tree.
Decision Tree → Entropy + Information Gain
ANN → Weights + Bias + Activation Function
This example uses an ANN-style layer visualization to explain the flow of a Decision Tree.
Decision Tree → Entropy + Information Gain
ANN → Weights + Bias + Activation Function
ANN-Style Representation
INPUT LAYER
Age
Salary
Credit Score
Age
Salary
Credit Score
→
DECISION NODE
Best Feature
Information Gain
Best Feature
Information Gain
→
SPLIT
YES
NO
YES
NO
→
OUTPUT
APPROVED
REJECTED
APPROVED
REJECTED
1 Training Dataset
| Customer | Age | Salary ₹000 | Credit Score | Decision |
|---|---|---|---|---|
| A | 22 | 25 | 580 | Rejected |
| B | 25 | 30 | 600 | Rejected |
| C | 30 | 40 | 650 | Rejected |
| D | 35 | 50 | 700 | Approved |
| E | 40 | 60 | 730 | Approved |
| F | 45 | 75 | 760 | Approved |
2 Calculate Parent Entropy
There are 6 customers.
Approved = 3
Rejected = 3
Approved = 3
Rejected = 3
P(Approved) = 3 / 6 = 0.5
P(Rejected) = 3 / 6 = 0.5
Entropy(S) = −[0.5 log₂(0.5)] −[0.5 log₂(0.5)]
Entropy(S) = 1
P(Rejected) = 3 / 6 = 0.5
Entropy(S) = −[0.5 log₂(0.5)] −[0.5 log₂(0.5)]
Entropy(S) = 1
Parent Entropy = 1.0
3 Find the Best Feature
The Decision Tree checks which feature produces the best separation.
| Feature | Information Gain |
|---|---|
| Age | 0.459 |
| Salary | 0.459 |
| Credit Score | 1.000 |
Highest Information Gain =
Credit Score
Therefore:
Credit Score becomes the Root Node.
Therefore:
Credit Score becomes the Root Node.
4 Create Root Node
Credit Score ≤ 675?
YES
REJECTED
NO
APPROVED
5 Mathematical Split
Credit Score ≤ 675
YES: 580, 600, 650
→ Rejected
NO: 700, 730, 760
→ Approved
YES: 580, 600, 650
→ Rejected
NO: 700, 730, 760
→ Approved
6 Calculate Child Entropy
YES Branch
Rejected = 3
Approved = 0
Total = 3
P(Rejected)=3/3=1
P(Approved)=0/3=0
Entropy = −(1 log₂1) −(0 log₂0)
Entropy = 0
Approved = 0
Total = 3
P(Rejected)=3/3=1
P(Approved)=0/3=0
Entropy = −(1 log₂1) −(0 log₂0)
Entropy = 0
NO Branch
Approved = 3
Rejected = 0
P(Approved)=1
P(Rejected)=0
Entropy = 0
Rejected = 0
P(Approved)=1
P(Rejected)=0
Entropy = 0
7 Information Gain Calculation
IG
=
Parent Entropy
−
Weighted Child Entropy
IG = 1 − [(3/6 × 0) + (3/6 × 0)]
IG = 1 − 0
IG = 1.0
IG = 1 − [(3/6 × 0) + (3/6 × 0)]
IG = 1 − 0
IG = 1.0
Perfect separation achieved!
Credit Score is an excellent decision feature for this simplified dataset.
Credit Score is an excellent decision feature for this simplified dataset.
8 Classify a New Customer
New Customer:
Age = 32
Salary = ₹45,000
Credit Score = 710
Age = 32
Salary = ₹45,000
Credit Score = 710
9 Follow the Decision Tree
Question:
Is Credit Score ≤ 675?
710 ≤ 675 ?
FALSE
Therefore follow: NO → APPROVED
Is Credit Score ≤ 675?
710 ≤ 675 ?
FALSE
Therefore follow: NO → APPROVED
💳 CREDIT CARD DECISION
✅ APPROVED
✅ APPROVED
10 Decision Tree as ANN-Style Flow
INPUT
x₁ = Age
x₂ = Salary
x₃ = Credit Score
↓
FEATURE SELECTION
Information Gain
↓
DECISION NODE
Credit Score ≤ 675 ?
↓
BRANCH
YES / NO
↓
OUTPUT
Approved / Rejected
x₁ = Age
x₂ = Salary
x₃ = Credit Score
↓
FEATURE SELECTION
Information Gain
↓
DECISION NODE
Credit Score ≤ 675 ?
↓
BRANCH
YES / NO
↓
OUTPUT
Approved / Rejected
Decision Tree vs ANN
| Decision Tree | ANN |
|---|---|
| Entropy | Loss Function |
| Information Gain | Gradient Descent |
| Decision Node | Neuron |
| Feature Split | Weighted Sum |
| Branch | Connection |
| Leaf Node | Output Neuron |
🎯 Interactive Step-by-Step Explanation
Input:
Age = 32
Salary = ₹45,000
Credit Score = 710
Age = 32
Salary = ₹45,000
Credit Score = 710
Parent Entropy = 1
Child Entropy = 0
Child Entropy = 0
Information Gain
= 1 − 0
= 1.0
= 1 − 0
= 1.0
Root Feature:
🌳 Credit Score
🌳 Credit Score
Credit Score ≤ 675?
710 ≤ 675?
FALSE
NO → APPROVED
710 ≤ 675?
FALSE
NO → APPROVED
🎉 FINAL RESULT
CREDIT CARD APPROVED
CREDIT CARD APPROVED
No comments:
Post a Comment