🔬 K-Means Clustering Performance Evaluation
Iris Dataset using Python
Clustering → Validation Metrics → Visualization → Interpretation
📌 What is the Purpose?
K-Means is an unsupervised machine learning algorithm that divides observations into a selected number of groups called clusters. Creating clusters alone is not enough; we also need to determine whether those groups are meaningful and well separated.
This example evaluates the clustering result using three complementary ideas:
Measures how appropriately observations fit their assigned cluster.
Measures the relationship between cluster compactness and separation.
Examines the relationship between generated clusters and known Iris categories.
🌸 Step 1 — Prepare the Iris Dataset
The Iris dataset provides four numerical measurements for each flower. K-Means uses these measurements to discover groups without being given the species names during clustering.
| Feature | Description |
|---|---|
| Sepal Length | Length of the sepal |
| Sepal Width | Width of the sepal |
| Petal Length | Length of the petal |
| Petal Width | Width of the petal |
💻 Complete Python Program
# =========================================================
# K-MEANS CLUSTERING PERFORMANCE EVALUATION
# IRIS DATASET
# =========================================================
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import (
silhouette_score,
davies_bouldin_score
)
# ---------------------------------------------------------
# 1. Load the Dataset
# ---------------------------------------------------------
iris = load_iris()
X = iris.data
actual_labels = iris.target
print("Feature Names:")
print(iris.feature_names)
print("\nClass Names:")
print(iris.target_names)
# ---------------------------------------------------------
# 2. Create a DataFrame
# ---------------------------------------------------------
df = pd.DataFrame(
X,
columns=iris.feature_names
)
df["Actual Species"] = [
iris.target_names[value]
for value in actual_labels
]
print("\nFirst Five Rows:")
print(df.head())
print("\nDataset Size:")
print(df.shape)
# ---------------------------------------------------------
# 3. Standardize the Input Features
# ---------------------------------------------------------
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
# ---------------------------------------------------------
# 4. Build the K-Means Model
# ---------------------------------------------------------
k = 3
kmeans = KMeans(
n_clusters=k,
random_state=42,
n_init=10
)
# ---------------------------------------------------------
# 5. Perform Clustering
# ---------------------------------------------------------
clusters = kmeans.fit_predict(X_scaled)
df["Cluster"] = clusters
print("\nCluster Assignments:")
print(df[["Actual Species", "Cluster"]].head(10))
# ---------------------------------------------------------
# 6. Display Centroids
# ---------------------------------------------------------
print("\nCluster Centers:")
print(kmeans.cluster_centers_)
# ---------------------------------------------------------
# 7. Calculate Silhouette Score
# ---------------------------------------------------------
silhouette = silhouette_score(
X_scaled,
clusters
)
print("\nSilhouette Score:")
print(round(silhouette, 4))
# ---------------------------------------------------------
# 8. Calculate Davies-Bouldin Index
# ---------------------------------------------------------
davies_bouldin = davies_bouldin_score(
X_scaled,
clusters
)
print("\nDavies-Bouldin Index:")
print(round(davies_bouldin, 4))
# ---------------------------------------------------------
# 9. Compare Clusters with Actual Species
# ---------------------------------------------------------
comparison = pd.crosstab(
df["Cluster"],
df["Actual Species"]
)
print("\nCluster vs Actual Species:")
print(comparison)
# ---------------------------------------------------------
# 10. Visualize the Clusters
# ---------------------------------------------------------
plt.figure(figsize=(9, 6))
plt.scatter(
X_scaled[:, 0],
X_scaled[:, 1],
c=clusters,
s=55,
alpha=0.8
)
plt.xlabel("Standardized Sepal Length")
plt.ylabel("Standardized Sepal Width")
plt.title("K-Means Clustering of Iris Dataset")
plt.show()
# ---------------------------------------------------------
# 11. Evaluate Several Values of K
# ---------------------------------------------------------
k_range = range(2, 8)
silhouette_results = []
db_results = []
for current_k in k_range:
model = KMeans(
n_clusters=current_k,
random_state=42,
n_init=10
)
current_clusters = model.fit_predict(X_scaled)
silhouette_results.append(
silhouette_score(
X_scaled,
current_clusters
)
)
db_results.append(
davies_bouldin_score(
X_scaled,
current_clusters
)
)
# ---------------------------------------------------------
# 12. Plot Silhouette Scores
# ---------------------------------------------------------
plt.figure(figsize=(9, 5))
plt.plot(
list(k_range),
silhouette_results,
marker="o"
)
plt.xlabel("Number of Clusters")
plt.ylabel("Silhouette Score")
plt.title("Silhouette Score vs Number of Clusters")
plt.grid(True)
plt.show()
# ---------------------------------------------------------
# 13. Plot Davies-Bouldin Scores
# ---------------------------------------------------------
plt.figure(figsize=(9, 5))
plt.plot(
list(k_range),
db_results,
marker="o"
)
plt.xlabel("Number of Clusters")
plt.ylabel("Davies-Bouldin Index")
plt.title("Davies-Bouldin Index vs Number of Clusters")
plt.grid(True)
plt.show()
# ---------------------------------------------------------
# 14. Print Final Evaluation
# ---------------------------------------------------------
print("\n========================================")
print("K-MEANS EVALUATION")
print("========================================")
print("Selected K :", k)
print("Silhouette Score :", round(silhouette, 4))
print("Davies-Bouldin Index :", round(davies_bouldin, 4))
print("\nGeneral interpretation:")
print("Higher Silhouette Score -> better")
print("Lower Davies-Bouldin -> better")
# =========================================================
# END
# =========================================================
🧩 Step-by-Step Explanation
1️⃣ Load Data
The program imports the Iris dataset and separates its numerical measurements from the species information.
2️⃣ Standardize
The four measurements are standardized so that differences in feature scale do not dominate distance calculations.
3️⃣ Select K
The initial model uses K = 3, producing three groups from the flower measurements.
4️⃣ Create Clusters
K-Means assigns observations to nearby centroids and repeatedly updates the centroids until the grouping becomes stable.
5️⃣ Silhouette Score
This score indicates whether points are closer to their own group than to neighboring groups.
6️⃣ Davies-Bouldin
This index considers cluster spread and separation. Lower values generally indicate better-defined groups.
7️⃣ Compare Results
The cross-tabulation helps inspect how the discovered clusters correspond to the known species labels.
8️⃣ Try Different K
Several cluster counts are tested so that students can observe how K affects the evaluation measures.
📈 Silhouette Score
The Silhouette Score considers both the compactness of a point's own cluster and its separation from another nearby cluster.
Here a represents the average distance from a sample to other samples in its own cluster, while b represents its average distance to the nearest alternative cluster.
A value closer to 1 generally indicates stronger clustering, while values near zero suggest overlapping groups.
📉 Davies-Bouldin Index
The Davies-Bouldin Index evaluates how similar each cluster is to its most similar neighboring cluster by considering within-cluster spread and between-cluster separation.
Therefore, when comparing candidate K values, a smaller Davies-Bouldin value is usually preferable.
📊 Understanding the Cluster Comparison
K-Means creates labels such as 0, 1 and 2. These numbers have no inherent meaning such as "Setosa" or "Virginica". They simply identify the clusters discovered by the algorithm.
The original species labels can therefore be used after clustering to inspect how closely the discovered groups correspond to the known categories.
Species Name → original Iris dataset label
🎯 Why Test Different K Values?
Selecting the number of clusters is an important part of K-Means. Instead of assuming that one value is always correct, we can evaluate multiple values and compare their validation scores.
| Metric | Preferred Direction | Interpretation |
|---|---|---|
| Silhouette | Higher | Better cohesion and separation |
| Davies-Bouldin | Lower | Better cluster separation relative to spread |
🎓 Complete Workflow
↓
Feature Preparation
↓
Standardization
↓
K-Means
↓
Cluster Assignment
↓
Silhouette + Davies-Bouldin
↓
Cluster / Species Comparison
↓
Test Multiple K Values
↓
Interpret the Results
Key idea: K-Means discovers groups without using the species labels. The validation metrics help us judge the quality of those groups, while the known Iris categories provide an additional way to interpret the result.
No comments:
Post a Comment