🌸 K-Nearest Neighbors (KNN)
Implementation on Iris Dataset using Python
Classification • Accuracy • Confusion Matrix • Classification Report
📌 Objective
The objective is to build a KNN classification model using the famous Iris dataset. The model learns from flower measurements and predicts whether a flower belongs to Iris Setosa, Iris Versicolor, or Iris Virginica.
```The Iris dataset contains four important numerical features:
- 🌿 Sepal Length
- 🌿 Sepal Width
- 🌸 Petal Length
- 🌸 Petal Width
💻 Complete Python Program
# ---------------------------------------------------------
# KNN IMPLEMENTATION ON IRIS DATASET
# ---------------------------------------------------------
# 1. Import required libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import (
accuracy_score,
confusion_matrix,
classification_report,
ConfusionMatrixDisplay
)
# ---------------------------------------------------------
# 2. Load Iris Dataset
# ---------------------------------------------------------
iris = load_iris()
X = iris.data
y = iris.target
feature_names = iris.feature_names
target_names = iris.target_names
print("Feature Names:")
print(feature_names)
print("\nTarget Classes:")
print(target_names)
# ---------------------------------------------------------
# 3. Create DataFrame
# ---------------------------------------------------------
df = pd.DataFrame(X, columns=feature_names)
df["species"] = [target_names[i] for i in y]
print("\nFirst Five Records:")
print(df.head())
# ---------------------------------------------------------
# 4. Split Dataset into Training and Testing Data
# ---------------------------------------------------------
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.20,
random_state=42,
stratify=y
)
print("\nTraining Samples:", len(X_train))
print("Testing Samples :", len(X_test))
# ---------------------------------------------------------
# 5. Feature Scaling
# ---------------------------------------------------------
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
# ---------------------------------------------------------
# 6. Create KNN Model
# ---------------------------------------------------------
k = 5
knn = KNeighborsClassifier(n_neighbors=k)
# ---------------------------------------------------------
# 7. Train the Model
# ---------------------------------------------------------
knn.fit(X_train_scaled, y_train)
# ---------------------------------------------------------
# 8. Predict Test Data
# ---------------------------------------------------------
y_pred = knn.predict(X_test_scaled)
print("\nActual Values:")
print(y_test)
print("\nPredicted Values:")
print(y_pred)
# ---------------------------------------------------------
# 9. Calculate Accuracy
# ---------------------------------------------------------
accuracy = accuracy_score(y_test, y_pred)
print("\nAccuracy:", round(accuracy * 100, 2), "%")
# ---------------------------------------------------------
# 10. Confusion Matrix
# ---------------------------------------------------------
cm = confusion_matrix(y_test, y_pred)
print("\nConfusion Matrix:")
print(cm)
# ---------------------------------------------------------
# 11. Classification Report
# ---------------------------------------------------------
print("\nClassification Report:")
print(
classification_report(
y_test,
y_pred,
target_names=target_names
)
)
# ---------------------------------------------------------
# 12. Display Confusion Matrix
# ---------------------------------------------------------
disp = ConfusionMatrixDisplay(
confusion_matrix=cm,
display_labels=target_names
)
disp.plot()
plt.title("KNN Confusion Matrix - Iris Dataset")
plt.show()
# ---------------------------------------------------------
# 13. Test a New Flower
# ---------------------------------------------------------
new_flower = np.array([
[5.1, 3.5, 1.4, 0.2]
])
new_flower_scaled = scaler.transform(new_flower)
prediction = knn.predict(new_flower_scaled)
print("\nNew Flower Measurements:")
print(new_flower)
print("\nPredicted Species:")
print(target_names[prediction[0]])
# ---------------------------------------------------------
# END
# ---------------------------------------------------------
🔎 Step-by-Step Explanation
```1️⃣ Load Dataset
load_iris() loads the built-in Iris dataset from
Scikit-learn.
2️⃣ Select Features
Four measurements are used as input features: sepal length, sepal width, petal length and petal width.
3️⃣ Split Dataset
80% of the observations are used for training and 20% are used for testing.
4️⃣ Scaling
StandardScaler puts the numerical features on a comparable scale. This is useful for distance-based algorithms such as KNN.
5️⃣ Choose K
Here K = 5. The model considers the five nearest training observations when making a prediction.
6️⃣ Train Model
knn.fit() prepares the KNN classifier using the
scaled training data.
7️⃣ Prediction
The model finds nearby training examples and assigns the class having the majority vote.
8️⃣ Evaluation
Accuracy, confusion matrix and classification report are used to evaluate the classifier.
📐 How KNN Works Mathematically
```KNN calculates the distance between a new data point and existing training points. A common distance measure is Euclidean distance.
After calculating the distances, KNN selects the K nearest observations and uses majority voting to determine the predicted class.
Example: If K = 5 and the nearest five flowers contain:
- 3 → Setosa
- 1 → Versicolor
- 1 → Virginica
The new flower is classified as Iris Setosa because Setosa receives the majority vote.
```📊 Evaluation Measures
```| Measure | Meaning |
|---|---|
| Accuracy | Percentage of correctly classified observations. |
| Confusion Matrix | Shows actual classes versus predicted classes. |
| Precision | Measures how many predicted members of a class were actually correct. |
| Recall | Measures how many actual members of a class were correctly identified. |
| F1-score | Harmonic mean of precision and recall. |
🎓 Important Points for Students
```- KNN is a supervised machine learning algorithm.
- It can be used for classification and regression.
- KNN is called a lazy learning algorithm because it does not build a conventional model during training.
- The value of K strongly influences the prediction.
- Small K values can make the model sensitive to noise.
- Large K values can make the model less sensitive to local patterns.
- Feature scaling is important because KNN relies on distances.
- For Iris classification, the four flower measurements are used to predict one of three species.