🖼️ Entropy in Image Processing

Measuring the information, randomness and complexity present in an image using probability and information theory.

1️⃣ What is Image Entropy?

Image entropy is a numerical measure of the amount of information or uncertainty contained in an image.

It is calculated from the probability distribution of the image's intensity values.

H = − Σ pᵢ log₂(pᵢ)

🔵 Low Entropy

Most pixels have similar intensity values. The image is relatively uniform.

Example: A plain white wall.

🟢 High Entropy

Many different intensity values occur with significant probabilities.

Example: A detailed natural scene.

🟣 Entropy Unit

When logarithm base 2 is used, entropy is measured in:

bits/pixel

🟠 8-bit Image

An 8-bit grayscale image contains 256 possible intensity values.

H_max = log₂(256) = 8 bits

2️⃣ Mathematical Derivation

Suppose an image contains N pixels and has L possible gray levels. Let nᵢ be the number of pixels having gray level i.

pᵢ = nᵢ / N

The information associated with gray level i is:

I(i) = −log₂(pᵢ)

The average information is obtained by multiplying each information value by its probability:

H = Σ pᵢ I(i)

Therefore:

H = −Σ pᵢ log₂(pᵢ)

3️⃣ Worked Example — Simple Image

Consider a small grayscale image whose pixels contain four intensity levels.

Gray Level	Number of Pixels	Probability pᵢ
0	4	4/16 = 0.25
85	4	4/16 = 0.25
170	4	4/16 = 0.25
255	4	4/16 = 0.25

Therefore:

H = −[0.25log₂(0.25) + 0.25log₂(0.25) + 0.25log₂(0.25) + 0.25log₂(0.25)]

Since:

log₂(0.25) = −2

Therefore:

H = −[4 × 0.25 × (−2)]

H = 2 bits/pixel

4️⃣ Entropy Calculation — Interactive

Enter probabilities for four intensity groups. The probabilities should approximately add up to 1.

Enter probabilities and click Calculate Entropy.

5️⃣ Histogram and Entropy

The histogram represents the distribution of intensity values. Entropy is calculated from the normalized histogram.

Important: A histogram alone does not directly give entropy. First convert histogram frequencies into probabilities:

pᵢ = nᵢ / N

Then calculate:

H = −Σ pᵢ log₂(pᵢ)

6️⃣ Low Entropy vs High Entropy

Property	Low Entropy	High Entropy
Intensity variation	Low	High
Image complexity	Low	High
Uniformity	High	Low
Information content	Low	High
Typical histogram	Concentrated	Spread out
Example	Plain background	Detailed texture

7️⃣ Entropy of a Uniform 8-bit Image

For a completely uniform image, suppose every pixel has intensity 128.

Then:

p(128) = 1

Therefore:

H = −1 × log₂(1)

H = 0 bits/pixel

There is no uncertainty because we already know that every pixel has the same intensity.

8️⃣ Maximum Entropy

Maximum entropy occurs when all possible intensity values are equally likely.

For an 8-bit grayscale image:

pᵢ = 1/256

Therefore:

H = −Σ(1/256)log₂(1/256)

Since there are 256 terms:

H = −256 × (1/256) × log₂(1/256)

H = log₂(256)

H = 8 bits/pixel

9️⃣ Entropy in Image Processing — Applications

🔍 Image Segmentation

Entropy can help identify regions containing different levels of texture and information.

🗜️ Image Compression

Entropy gives an estimate of the theoretical lower bound of average bits needed to represent image information.

🧩 Texture Analysis

Highly textured regions generally contain greater intensity variation and may have higher entropy.

🖥️ Image Quality Analysis

Entropy can be used as one statistical feature when comparing images or image-processing results.

🔐 Image Security

Entropy is commonly considered when evaluating randomness in image-encryption results.

🤖 Computer Vision

Entropy can be used as a feature for classification and region analysis.

🔟 Important Numerical Examples

Example 1 — Two intensity levels

Suppose:

p₁ = 0.5,   p₂ = 0.5

H = −[0.5log₂(0.5)+0.5log₂(0.5)]

H = 1 bit/pixel

Example 2 — Unequal probabilities

p₁ = 0.8,   p₂ = 0.2

H = −[0.8log₂(0.8)+0.2log₂(0.2)]

H ≈ 0.722 bits/pixel

Because one intensity dominates, the entropy is lower than the 50%-50% case.

Example 3 — Four equally probable levels

p₁=p₂=p₃=p₄=0.25

H = 2 bits/pixel

1️⃣1️⃣ Entropy Animation

The following animation shows how image pixels are converted into probabilities and finally into entropy.

🖼️ Image
Pixels

➜

📊 Histogram
nᵢ

➜

📐 Probability
pᵢ = nᵢ/N

➜

🧮 Information
−log₂(pᵢ)

➜

📈 Entropy
H

Click Start Animation.

1️⃣2️⃣ Important Formula Summary

Probability:   pᵢ = nᵢ/N

Information:   I(i) = −log₂(pᵢ)

Image Entropy:   H = −Σ pᵢlog₂(pᵢ)

Maximum entropy:   H_max = log₂(L)

8-bit grayscale:   H_max = log₂(256) = 8 bits/pixel

🎯 Concept in One Line

🖼️ Image → 📊 Histogram → 📐 Probability → 🧮 Information → 📈 Entropy

Higher entropy generally means a more varied and information-rich intensity distribution, while lower entropy indicates a more predictable or uniform distribution.

Created by Bijan Krishna Paul
B.K.PAUL Bhairab Ganguly College