Association Rule Mining (Apriori Algorithm) Using Python

 

Association Rule Mining (Apriori Algorithm)

Note: Association Rule Mining is an Unsupervised Machine Learning technique. It is mainly used for Market Basket Analysis to discover relationships between items frequently purchased together.

---

🟦 Program Aim

Aim:

To implement the Association Rule Mining (Apriori Algorithm) using Python and identify products that are frequently purchased together.

---

🟩 Algorithm Used

Apriori Algorithm

---

🟨 Problem Statement

A supermarket wants to analyze customer shopping patterns. By examining previous transactions, the store aims to identify products that are frequently purchased together. This information helps improve product placement, cross-selling, and promotional strategies.

---

🟪 Step 1: Install Required Library

Install the mlxtend package (only once).

pip install mlxtend

Explanation

• mlxtend stands for Machine Learning Extensions.
• It provides the Apriori algorithm and functions for generating association rules.

---

🟦 Step 2: Import Required Libraries

import pandas as pd
from mlxtend.preprocessing import TransactionEncoder
from mlxtend.frequent_patterns import apriori, association_rules

Explanation

• pandas → Used to create and manipulate data.
• TransactionEncoder → Converts transaction data into a True/False matrix.
• apriori() → Finds frequent itemsets.
• association_rules() → Generates association rules from frequent itemsets.

---

🟩 Step 3: Create the Transaction Dataset

transactions = [
    ["Milk", "Bread", "Butter"],
    ["Milk", "Bread"],
    ["Milk", "Butter"],
    ["Bread", "Butter"],
    ["Milk", "Bread", "Butter", "Eggs"],
    ["Bread", "Eggs"],
    ["Milk", "Eggs"]
]

Explanation

Each inner list represents one customer's shopping basket.

Customer	Purchased Items
1	Milk, Bread, Butter
2	Milk, Bread
3	Milk, Butter
4	Bread, Butter
5	Milk, Bread, Butter, Eggs
6	Bread, Eggs
7	Milk, Eggs

---

🟨 Step 4: Convert Transactions into Binary Format

encoder = TransactionEncoder()

encoded_data = encoder.fit(transactions).transform(transactions)

df = pd.DataFrame(encoded_data, columns=encoder.columns_)

Explanation

The Apriori algorithm requires data in binary (True/False or 1/0) format.

The dataset becomes:

Bread	Butter	Eggs	Milk
True	True	False	True
True	False	False	True
False	True	False	True
True	True	False	False
True	True	True	True
True	False	True	False
False	False	True	True

---

🟦 Step 5: Display the Dataset

print(df)

Explanation

Displays the converted transaction matrix used for mining frequent itemsets.

---

🟩 Step 6: Find Frequent Itemsets

frequent_items = apriori(df, min_support=0.3, use_colnames=True)

print(frequent_items)

Explanation

• min_support = 0.3 means an itemset must appear in at least 30% of all transactions.
• use_colnames=True displays product names instead of column numbers.

Example Output:

Support	Itemsets
0.71	{Milk}
0.71	{Bread}
0.57	{Butter}
0.43	{Eggs}
0.43	{Milk, Bread}
0.43	{Milk, Butter}

---

🟨 Step 7: Generate Association Rules

rules = association_rules(
    frequent_items,
    metric="confidence",
    min_threshold=0.7
)

print(rules)

Explanation

This step generates association rules using:

• Metric = Confidence
• Minimum Confidence = 70%

Example Rule:

Milk  → Bread

Meaning:

Customers buying Milk are likely to buy Bread as well.

---

🟥 Step 8: Display Selected Columns

print(rules[['antecedents',
             'consequents',
             'support',
             'confidence',
             'lift']])

Explanation

This displays the most important measures:

Antecedent	Consequent	Support	Confidence	Lift
Milk	Bread	0.43	0.75	1.05
Bread	Butter	0.43	0.60	1.04

---

🟪 Complete Python Program

import pandas as pd
from mlxtend.preprocessing import TransactionEncoder
from mlxtend.frequent_patterns import apriori, association_rules

transactions = [
    ["Milk", "Bread", "Butter"],
    ["Milk", "Bread"],
    ["Milk", "Butter"],
    ["Bread", "Butter"],
    ["Milk", "Bread", "Butter", "Eggs"],
    ["Bread", "Eggs"],
    ["Milk", "Eggs"]
]

encoder = TransactionEncoder()

encoded_data = encoder.fit(transactions).transform(transactions)

df = pd.DataFrame(encoded_data, columns=encoder.columns_)

print("Transaction Dataset")
print(df)

frequent_items = apriori(df,
                         min_support=0.3,
                         use_colnames=True)

print("\nFrequent Itemsets")
print(frequent_items)

rules = association_rules(frequent_items,
                          metric="confidence",
                          min_threshold=0.7)

print("\nAssociation Rules")
print(rules[['antecedents',
             'consequents',
             'support',
             'confidence',
             'lift']])

---

🟩 Sample Output

Transaction Dataset

   Bread  Butter  Eggs  Milk
0   True    True  False  True
1   True   False  False  True
2  False    True  False  True
3   True    True  False False
4   True    True   True  True
5   True   False   True False
6  False   False   True  True

Frequent Itemsets

support     itemsets

0.71        {Milk}

0.71        {Bread}

0.57        {Butter}

0.43        {Eggs}

0.43        {Milk, Bread}

...

Association Rules

Milk → Bread

Bread → Butter

---

🟦 Step-by-Step Working of the Algorithm

Transaction Data
        │
        ▼
Convert into Binary Matrix
        │
        ▼
Apply Apriori Algorithm
        │
        ▼
Find Frequent Itemsets
        │
        ▼
Generate Association Rules
        │
        ▼
Display Support, Confidence & Lift

---

🟨 Important Terms

Term	Description
Support	Frequency of an itemset appearing in all transactions.
Confidence	Probability that customers who buy item A also buy item B.
Lift	Measures the strength of the relationship between two items. A lift value greater than 1 indicates a positive association.
Frequent Itemset	A group of items that appears frequently in the dataset.
Association Rule	A rule showing the relationship between two or more items (e.g., Milk → Bread).

---

🌍 Real-Life Applications

• 🛒 Market Basket Analysis
• 🛍 Product Recommendation Systems
• 🏪 Store Shelf Arrangement
• 💳 Banking Product Recommendations
• 🎬 Movie Recommendation Systems
• 🌐 E-commerce Websites (Amazon, Flipkart)
• 🍔 Restaurant Combo Offers

---

🎯 Viva Questions

1. What is Association Rule Mining?
2. What is the Apriori Algorithm?
3. Define Support, Confidence, and Lift.
4. What is a Frequent Itemset?
5. Why is TransactionEncoder used?
6. What is the purpose of min_support?
7. What is the purpose of min_threshold in association rules?
8. Give two real-life applications of Association Rule Mining.

---

⭐ One-Line Revision

Association Rule Mining uses the Apriori algorithm to discover frequently occurring item combinations and generate rules such as "If a customer buys Milk, they are also likely to buy Bread."

Naïve Bayes Algorithm in Machine Learning Using Python

 

Naïve Bayes Algorithm in Machine Learning

---

🟦 Program Aim

Aim:

To implement the Gaussian Naïve Bayes Algorithm using Python and predict whether a patient has Diabetes or is Healthy based on their blood sugar level.

---

🟩 Algorithm Used

Gaussian Naïve Bayes (GaussianNB)

---

🟨 Problem Statement

A hospital wants to predict whether a patient is Healthy or has Diabetes based on the patient's Blood Sugar Level.

---

🟪 Step 1: Import Required Library

from sklearn.naive_bayes import GaussianNB

Explanation

• sklearn is the Scikit-learn library.
• naive_bayes is the module that contains Naïve Bayes algorithms.
• GaussianNB is used for continuous numerical data (e.g., blood sugar, age, height, weight).

---

🟦 Step 2: Create the Training Dataset

X = [
    [85],
    [90],
    [95],
    [140],
    [150],
    [160]
]

Explanation

X represents the input feature (Independent Variable).

Each value is the patient's Blood Sugar Level (mg/dL).

Patient	Blood Sugar
Patient 1	85
Patient 2	90
Patient 3	95
Patient 4	140
Patient 5	150
Patient 6	160

The algorithm uses these values for learning.

---

🟩 Step 3: Create the Output Labels

y = [
    "Healthy",
    "Healthy",
    "Healthy",
    "Diabetes",
    "Diabetes",
    "Diabetes"
]

Explanation

y represents the target variable (Dependent Variable).

Blood Sugar	Output
85	Healthy
90	Healthy
95	Healthy
140	Diabetes
150	Diabetes
160	Diabetes

The algorithm learns the relationship between blood sugar levels and health status.

---

🟨 Step 4: Create the Gaussian Naïve Bayes Model

model = GaussianNB()

Explanation

This line creates an object of the Gaussian Naïve Bayes classifier.

The model is now ready to be trained.

---

🟪 Step 5: Train the Model

model.fit(X, y)

Explanation

The fit() function trains the model using the training data.

• X = Input data (Blood Sugar)
• y = Output labels (Healthy / Diabetes)

During training, the model:

• Calculates the prior probability of each class.
• Calculates the likelihood of each blood sugar value for each class.
• Uses Bayes' Theorem to estimate probabilities.

---

🟦 Step 6: Predict for a New Patient

prediction = model.predict([[145]])

Explanation

The patient's blood sugar level is 145 mg/dL.

The model calculates:

• Probability of Healthy
• Probability of Diabetes

It selects the class with the higher probability.

---

🟩 Step 7: Display the Prediction

print("Prediction =", prediction[0])

Explanation

prediction is returned as a list (or array).

Using [0] retrieves the first (and only) predicted result.

Possible Output:

Prediction = Diabetes

---

🟥 Step 8: Complete Python Program

# Import Gaussian Naïve Bayes
from sklearn.naive_bayes import GaussianNB

# Training Data (Blood Sugar Levels)
X = [
    [85],
    [90],
    [95],
    [140],
    [150],
    [160]
]

# Output Labels
y = [
    "Healthy",
    "Healthy",
    "Healthy",
    "Diabetes",
    "Diabetes",
    "Diabetes"
]

# Create Model
model = GaussianNB()

# Train Model
model.fit(X, y)

# Predict New Patient
prediction = model.predict([[145]])

# Display Result
print("Prediction =", prediction[0])

---

🟦 Sample Output

Prediction = Diabetes

---

🟩 Step-by-Step Workflow

Start
   │
   ▼
Import GaussianNB
   │
   ▼
Create Training Dataset (X)
   │
   ▼
Create Output Labels (y)
   │
   ▼
Create GaussianNB Model
   │
   ▼
Train Model using fit()
   │
   ▼
Enter New Blood Sugar Value
   │
   ▼
Predict using predict()
   │
   ▼
Display Prediction
   │
   ▼
End

---

🟨 Line-by-Line Explanation

Line	Code	Description
1	from sklearn.naive_bayes import GaussianNB	Imports the Gaussian Naïve Bayes classifier.
2	X = [...]	Creates the input feature (blood sugar values).
3	y = [...]	Creates the output labels (Healthy/Diabetes).
4	model = GaussianNB()	Creates the Naïve Bayes model.
5	model.fit(X, y)	Trains the model using the training data.
6	prediction = model.predict([[145]])	Predicts the class for a new patient.
7	print(prediction[0])	Displays the predicted class.

---

🟪 Why Gaussian Naïve Bayes?

Gaussian Naïve Bayes is suitable because the feature (blood sugar level) is a continuous numerical value.

Examples of continuous data include:

• Blood Sugar
• Age
• Height
• Weight
• Salary
• Temperature

---

🟦 Advantages

• ✔ Easy to implement
• ✔ Fast training and prediction
• ✔ Works well with small datasets
• ✔ Handles continuous numerical data
• ✔ Effective for classification problems

---

🟥 Limitations

• ❌ Assumes all features are independent.
• ❌ Performance may decrease if features are highly correlated.
• ❌ Sensitive to the quality of training data.

---

🟩 Applications

• 🏥 Disease Diagnosis
• 📧 Spam Email Detection
• 😊 Sentiment Analysis
• 📰 News Classification
• 🌐 Language Detection
• 💳 Fraud Detection

---

📝 Viva Questions

1. What is Naïve Bayes?
2. Why is it called Naïve?
3. What is Gaussian Naïve Bayes?
4. What is the purpose of fit()?
5. What is the purpose of predict()?
6. What is the difference between Gaussian, Multinomial, and Bernoulli Naïve Bayes?
7. Why is prediction[0] used?
8. Which Python library provides the Naïve Bayes algorithm?

---

🎯 Key Points for Exams

• Algorithm: Gaussian Naïve Bayes
• Library: sklearn.naive_bayes
• Model Class: GaussianNB()
• Training Method: fit()
• Prediction Method: predict()
• Input: Continuous numerical values
• Output: Predicted class (e.g., Healthy or Diabetes)

---

⭐ One-Line Revision

Gaussian Naïve Bayes is a supervised machine learning algorithm that uses Bayes' Theorem and probability to classify continuous numerical data by assuming that all input features are independent.

k-Nearest Neighbors (KNN) Algorithm Using Python

 

k-Nearest Neighbors (KNN) Algorithm

---

🟦 Program Aim

Aim:

To implement the K-Nearest Neighbors (KNN) Classification Algorithm using Python and predict whether a person's height is classified as Short or Tall.

---

🟩 Algorithm Used

K-Nearest Neighbors (KNN) Classifier

---

🟨 Problem Statement

A school wants to classify students into two categories:

• Short
• Tall

based on their Height (in cm) using the K-Nearest Neighbors (KNN) algorithm.

---

🟪 Step 1: Import Required Library

First, import the KNeighborsClassifier class from the sklearn.neighbors module.

from sklearn.neighbors import KNeighborsClassifier

Explanation

• sklearn is the Scikit-learn library.
• neighbors contains the KNN algorithm.
• KNeighborsClassifier() is used for classification problems.

---

🟦 Step 2: Create the Training Dataset

X = [
    [150],
    [160],
    [170],
    [180]
]

Explanation

X represents the input feature (Independent Variable).

Here, the input is the Height of students.

Student	Height (cm)
Student 1	150
Student 2	160
Student 3	170
Student 4	180

The KNN algorithm stores these training examples.

---

🟩 Step 3: Create the Output Labels

y = [
    "Short",
    "Short",
    "Tall",
    "Tall"
]

Explanation

y represents the output labels (Dependent Variable).

Height	Category
150	Short
160	Short
170	Tall
180	Tall

These are the correct answers used to train the model.

---

🟨 Step 4: Create the KNN Model

model = KNeighborsClassifier(n_neighbors=3)

Explanation

• KNeighborsClassifier() creates the KNN model.
• n_neighbors=3 means the model will consider the 3 nearest neighbors while making a prediction.

Why choose K = 3?

The algorithm checks the three closest training data points and predicts the class that appears most frequently among them.

---

🟦 Step 5: Train the Model

model.fit(X, y)

Explanation

The fit() method trains the model.

Syntax:

model.fit(input_data, output_labels)

Here,

• X → Heights of students
• y → Categories (Short/Tall)

During training, KNN stores the dataset instead of creating a mathematical model.

---

🟩 Step 6: Predict a New Data Point

prediction = model.predict([[175]])

Explanation

We want to predict the category of a student whose height is 175 cm.

The model calculates the distance between 175 cm and all training data points.

---

🟨 Step 7: Display the Result

print("Prediction =", prediction[0])

Output

Prediction = Tall

Explanation

Since the majority of the nearest neighbors are classified as Tall, the algorithm predicts:

Prediction = Tall

---

🟦 Complete Python Program

from sklearn.neighbors import KNeighborsClassifier

# Training Data (Height in cm)
X = [
    [150],
    [160],
    [170],
    [180]
]

# Output Labels
y = [
    "Short",
    "Short",
    "Tall",
    "Tall"
]

# Create KNN Model
model = KNeighborsClassifier(n_neighbors=3)

# Train the Model
model.fit(X, y)

# Predict for a New Student
prediction = model.predict([[175]])

# Display the Result
print("Prediction =", prediction[0])

---

🟪 Step-by-Step Working of KNN

Step 1️⃣ Import the KNN library

⬇

Step 2️⃣ Create the training dataset

⬇

Step 3️⃣ Create the output labels

⬇

Step 4️⃣ Choose the value of K

⬇

Step 5️⃣ Train the model using fit()

⬇

Step 6️⃣ Enter a new data point

⬇

Step 7️⃣ Calculate the distance from the new point to all training points

⬇

Step 8️⃣ Select the K nearest neighbors

⬇

Step 9️⃣ Count the majority class (Majority Voting)

⬇

Step 🔟 Display the predicted result

---

🟥 Workflow

        Training Data
             │
             ▼
   Choose Value of K (K=3)
             │
             ▼
      Train the Model
             │
             ▼
      New Data (175 cm)
             │
             ▼
   Calculate Distances
             │
             ▼
 Find 3 Nearest Neighbors
             │
             ▼
     Majority Voting
             │
             ▼
      Final Prediction
          (Tall)

---

🟩 Distance Calculation Example

Suppose the new student's height is 175 cm.

Training Height	Distance from 175	Category
150	25	Short
160	15	Short
170	5	Tall
180	5	Tall

The 3 nearest neighbors are:

Height	Category
170	Tall
180	Tall
160	Short

Majority Voting

• Tall = 2 votes
• Short = 1 vote

➡ Final Prediction = Tall

---

🟦 Expected Output

Prediction = Tall

---

🟨 Explanation of Important Functions

Function	Description
KNeighborsClassifier()	Creates the KNN classifier model
n_neighbors=3	Selects the 3 nearest neighbors
fit(X, y)	Stores the training dataset
predict()	Predicts the category for new data

---

🟩 Advantages

• ✔ Simple and easy to understand
• ✔ No complex training process
• ✔ Suitable for classification and regression
• ✔ Works well with small datasets
• ✔ Easy to implement

---

🟥 Limitations

• ❌ Slow for large datasets
• ❌ Sensitive to noisy data
• ❌ Choosing the correct value of K is important
• ❌ Performance decreases with high-dimensional data

---

🟦 Applications

• 🏥 Disease Diagnosis
• 📧 Spam Email Detection
• 😊 Face Recognition
• 🎬 Movie Recommendation
• 🛒 Product Recommendation
• 🌸 Flower Classification
• 👤 Customer Segmentation

---

📝 Viva Questions

Q1. What is KNN?

Answer:
K-Nearest Neighbors (KNN) is a supervised machine learning algorithm that predicts the class of a new data point by analyzing the K nearest training examples.

---

Q2. What does K represent?

Answer:
K represents the number of nearest neighbors considered while making a prediction.

---

Q3. Why is an odd value of K preferred?

Answer:
An odd value (e.g., 3, 5, 7) helps avoid ties during majority voting in binary classification.

---

Q4. Does KNN require a training phase?

Answer:
KNN has no explicit training phase. It simply stores the training data and performs calculations during prediction.

---

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm that classifies a new data point by finding the K nearest neighbors using a distance metric and assigning the class based on majority voting (classification) or average value (regression).