🌳 Decision Tree Classification Iris Dataset using Python

🌳 Decision Tree Classification

Iris Dataset using Python

Data Loading → Train/Test Split → Decision Tree → Prediction → Evaluation → Visualization

📌 Introduction

A Decision Tree is a supervised machine learning algorithm that makes decisions by repeatedly splitting data according to feature values. It has a tree-like structure consisting of a root node, internal nodes, branches and leaf nodes.

In this example, the Decision Tree is used to classify Iris flowers into three species:

🌸 Setosa 🌺 Versicolor 🌷 Virginica

```

🌼 Iris Dataset

```

The Iris dataset contains measurements of iris flowers using four numerical features:

Feature	Description
Sepal Length	Length of the sepal
Sepal Width	Width of the sepal
Petal Length	Length of the petal
Petal Width	Width of the petal

```

💻 Complete Python Program

# =========================================================
# DECISION TREE CLASSIFICATION ON IRIS DATASET
# =========================================================

# 1. Import required libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.tree import plot_tree

from sklearn.metrics import (
    accuracy_score,
    confusion_matrix,
    classification_report,
    ConfusionMatrixDisplay
)

# =========================================================
# 2. Load the Iris Dataset
# =========================================================

iris = load_iris()

X = iris.data
y = iris.target

print("Feature Names:")
print(iris.feature_names)

print("\nTarget Names:")
print(iris.target_names)

# =========================================================
# 3. Create DataFrame
# =========================================================

df = pd.DataFrame(
    X,
    columns=iris.feature_names
)

df["Species"] = [
    iris.target_names[i]
    for i in y
]

print("\nFirst Five Rows:")
print(df.head())

print("\nDataset Shape:")
print(df.shape)

# =========================================================
# 4. Separate Input and Output
# =========================================================

X = df[iris.feature_names]
y = df["Species"]

# Convert species names into numerical target values
y = iris.target

# =========================================================
# 5. Train-Test Split
# =========================================================

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.20,
    random_state=42,
    stratify=y
)

print("\nTraining Samples:", X_train.shape[0])
print("Testing Samples :", X_test.shape[0])

# =========================================================
# 6. Create Decision Tree Model
# =========================================================

model = DecisionTreeClassifier(
    criterion="gini",
    max_depth=3,
    random_state=42
)

# =========================================================
# 7. Train the Model
# =========================================================

model.fit(X_train, y_train)

print("\nDecision Tree Training Completed.")

# =========================================================
# 8. Make Predictions
# =========================================================

y_pred = model.predict(X_test)

print("\nActual Values:")
print(y_test)

print("\nPredicted Values:")
print(y_pred)

# =========================================================
# 9. Calculate Accuracy
# =========================================================

accuracy = accuracy_score(
    y_test,
    y_pred
)

print("\nAccuracy:")
print(round(accuracy * 100, 2), "%")

# =========================================================
# 10. Confusion Matrix
# =========================================================

cm = confusion_matrix(
    y_test,
    y_pred
)

print("\nConfusion Matrix:")
print(cm)

# =========================================================
# 11. Classification Report
# =========================================================

print("\nClassification Report:")

print(
    classification_report(
        y_test,
        y_pred,
        target_names=iris.target_names
    )
)

# =========================================================
# 12. Display Confusion Matrix
# =========================================================

display = ConfusionMatrixDisplay(
    confusion_matrix=cm,
    display_labels=iris.target_names
)

display.plot()

plt.title(
    "Decision Tree - Iris Dataset Confusion Matrix"
)

plt.show()

# =========================================================
# 13. Visualize Decision Tree
# =========================================================

plt.figure(
    figsize=(18, 10)
)

plot_tree(
    model,
    feature_names=iris.feature_names,
    class_names=iris.target_names,
    filled=True,
    rounded=True,
    fontsize=11
)

plt.title(
    "Decision Tree for Iris Dataset"
)

plt.show()

# =========================================================
# 14. Predict a New Flower
# =========================================================

new_flower = np.array([
    [5.1, 3.5, 1.4, 0.2]
])

prediction = model.predict(
    new_flower
)

print("\nNew Flower:")
print(new_flower)

print("\nPredicted Species:")
print(
    iris.target_names[prediction[0]]
)

# =========================================================
# END OF PROGRAM
# =========================================================

🧠 Step-by-Step Explanation

```

1️⃣ Load Dataset

load_iris() loads the built-in Iris dataset from Scikit-learn.

2️⃣ Select Features

Four measurements are used as input: sepal length, sepal width, petal length and petal width.

3️⃣ Split Data

The dataset is divided into training and testing sets. Here, 80% is used for training and 20% for testing.

4️⃣ Create Tree

DecisionTreeClassifier() creates the classification model. The example uses the Gini criterion and maximum depth 3.

5️⃣ Train Model

model.fit() learns decision rules from the training data.

6️⃣ Prediction

The trained tree applies its learned rules to the test observations and predicts their flower species.

7️⃣ Evaluation

Accuracy, confusion matrix and classification report are calculated to measure model performance.

8️⃣ Visualization

plot_tree() displays the complete decision-making structure of the trained tree.

```

📐 Mathematics Behind Decision Tree

```

Gini Impurity

The Gini criterion measures how impure a node is. A perfectly pure node has Gini impurity equal to zero.

Gini = 1 − Σ pᵢ²

Here, pᵢ represents the proportion of samples belonging to class i. The tree searches for splits that reduce impurity.

Entropy

Entropy = − Σ pᵢ log₂(pᵢ)

Entropy measures the disorder or uncertainty in a node. Decision Trees can use either Gini impurity or entropy depending on the selected criterion.

```

🌳 Understanding the Tree

```

ROOT NODE
Feature Test

↓

Branch 1
Condition satisfied

Branch 2
Another condition

Branch 3
Another condition

↓

LEAF NODE
Final Iris Species

```

🎯 New Flower Prediction

```

The program tests a new flower with the measurements:

Sepal Length = 5.1 cm   |   Sepal Width = 3.5 cm   |   Petal Length = 1.4 cm   |   Petal Width = 0.2 cm

The Decision Tree follows its learned rules and returns the predicted Iris species.

```

✅ Advantages of Decision Tree

```

• Easy to understand and interpret.
• Can be visualized as a tree.
• Works well with numerical features.
• Does not require feature scaling.
• Useful for both classification and regression.
• Produces understandable decision rules.

⚠️ Limitations

• A very deep tree can overfit the training data.
• Small changes in data can sometimes produce a different tree.
• Tree complexity should be controlled using parameters such as max_depth, min_samples_split and min_samples_leaf.

```

🎓 Student Summary

```

The complete machine learning workflow is:

Iris Dataset ➜ Feature Selection ➜ Train/Test Split ➜ Decision Tree ➜ Training ➜ Prediction ➜ Accuracy ➜ Confusion Matrix ➜ Classification Report ➜ Tree Visualization

Key idea: A Decision Tree repeatedly asks feature-based questions until it reaches a leaf node containing the final predicted class.

```

🌐 Machine Learning with Python • Decision Tree • Iris Dataset

🌐 Created by Bijan Krishna Paul

📚 Reference: Kaggle – Decision Tree Iris Dataset