🧠 Three-Layer ANN — Digit Recognition Step-by-step mathematical example for recognizing a handwritten digit using a 16 × 10 pixel image.

🧠 Three-Layer ANN — Digit Recognition

Step-by-step mathematical example for recognizing a handwritten digit using a 16 × 10 pixel image.

1. What Is a Three-Layer ANN?

In this example, the ANN contains three computational layers:

Layer 1: Input → Hidden Layer 1

Layer 2: Hidden Layer 1 → Hidden Layer 2

Layer 3: Hidden Layer 2 → Output Layer

The input image contains:

Image = 16 × 10 pixels

Total Input Pixels = 16 × 10 = 160

Therefore, the ANN receives 160 input values.

2. Three-Layer ANN Architecture

INPUT

x₁

x₂

x₃

...

x₁₆₀

→

HIDDEN 1

H₁

H₂

H₃

H₄

→

HIDDEN 2

G₁

G₂

G₃

→

OUTPUT

0

1

2

...

9

16 × 10 Image ↓ 160 Input Neurons ↓ Hidden Layer 1 ↓ Hidden Layer 2 ↓ 10 Output Neurons ↓ Predicted Digit 0–9

3. Small Numerical Example

For easy classroom calculation, we will temporarily use 4 input pixels. The real system has 160 inputs.

Input: X = [1, 0, 1, 0]

x₁ = 1
x₂ = 0
x₃ = 1
x₄ = 0

Architecture for our mathematical demonstration:

4 Inputs → 3 Neurons in Hidden Layer 1 → 2 Neurons in Hidden Layer 2 → 3 Output Neurons

The three output neurons represent:

O₀ → Digit 0
O₁ → Digit 1
O₂ → Digit 2

1 Input Layer

X = [1, 0, 1, 0]

The four values are sent to every neuron in Hidden Layer 1.

2 Input → Hidden Layer 1

Assume three neurons:

H₁, H₂, H₃

Input	H₁ Weight	H₂ Weight	H₃ Weight
x₁	0.5	0.2	0.4
x₂	0.3	0.4	0.1
x₃	0.8	0.6	0.7
x₄	0.1	0.7	0.3

Bias values:

b₁ = -0.5
b₂ = -0.4
b₃ = -0.3

4. Calculate H₁

Step 1 — Weighted Sum

z₁ = (1 × 0.5) + (0 × 0.3) + (1 × 0.8) + (0 × 0.1) - 0.5

z₁ = 0.5 + 0 + 0.8 + 0 - 0.5

z₁ = 0.8

Step 2 — Sigmoid Activation

H₁ = 1 / (1 + e⁻⁰·⁸)

H₁ ≈ 0.690

5. Calculate H₂

z₂ = (1 × 0.2) + (0 × 0.4) + (1 × 0.6) + (0 × 0.7) - 0.4

z₂ = 0.2 + 0 + 0.6 + 0 - 0.4

z₂ = 0.4

H₂ = 1 / (1 + e⁻⁰·⁴)

H₂ ≈ 0.599

6. Calculate H₃

z₃ = (1 × 0.4) + (0 × 0.1) + (1 × 0.7) + (0 × 0.3) - 0.3

z₃ = 0.4 + 0 + 0.7 + 0 - 0.3

z₃ = 0.8

H₃ = 1 / (1 + e⁻⁰·⁸)

H₃ ≈ 0.690

7. Hidden Layer 1 Output

H = [H₁,H₂,H₃]

H = [0.690, 0.599, 0.690]

Hidden Layer 1 Output = [0.690, 0.599, 0.690]

3 Hidden Layer 1 → Hidden Layer 2

Now the three values from Hidden Layer 1 become inputs to Hidden Layer 2.

Hidden Layer 2 contains two neurons:

G₁
G₂

Input	G₁ Weight	G₂ Weight
H₁	0.6	0.3
H₂	0.5	0.8
H₃	0.7	0.4

Bias:

b₄ = -0.6
b₅ = -0.5

8. Calculate G₁

z₄ = (H₁ × 0.6) + (H₂ × 0.5) + (H₃ × 0.7) - 0.6

Substitute the values:

z₄ = (0.690 × 0.6) + (0.599 × 0.5) + (0.690 × 0.7) - 0.6

z₄ = 0.414 + 0.2995 + 0.483 - 0.6

z₄ ≈ 0.5965

Activation:

G₁ = 1 / (1 + e⁻⁰·⁵⁹⁶⁵)

G₁ ≈ 0.645

9. Calculate G₂

z₅ = (H₁ × 0.3) + (H₂ × 0.8) + (H₃ × 0.4) - 0.5

z₅ = (0.690 × 0.3) + (0.599 × 0.8) + (0.690 × 0.4) - 0.5

z₅ = 0.207 + 0.4792 + 0.276 - 0.5

z₅ ≈ 0.4622

G₂ = 1 / (1 + e⁻⁰·⁴⁶²²)

G₂ ≈ 0.614

10. Hidden Layer 2 Output

G = [G₁,G₂]

G = [0.645,0.614]

4 Hidden Layer 2 → Output Layer

The output layer contains 10 neurons in the actual digit-recognition system.

O₀ → 0
O₁ → 1
O₂ → 2
O₃ → 3
O₄ → 4
O₅ → 5
O₆ → 6
O₇ → 7
O₈ → 8
O₉ → 9

For demonstrating the calculation, we will calculate three outputs:

Hidden	O₀	O₁	O₂
G₁	0.7	0.4	0.2
G₂	0.3	0.8	0.6

Bias:

b₀ = -0.4
b₁ = -0.3
b₂ = -0.2

11. Calculate O₀ — Digit 0

z₀ = (G₁ × 0.7) + (G₂ × 0.3) - 0.4

z₀ = (0.645 × 0.7) + (0.614 × 0.3) - 0.4

z₀ = 0.4515 + 0.1842 - 0.4

z₀ ≈ 0.2357

O₀ = 1 / (1 + e⁻⁰·²³⁵⁷)

O₀ ≈ 0.559

12. Calculate O₁ — Digit 1

z₁ = (G₁ × 0.4) + (G₂ × 0.8) - 0.3

z₁ = (0.645 × 0.4) + (0.614 × 0.8) - 0.3

z₁ = 0.258 + 0.4912 - 0.3

z₁ ≈ 0.4492

O₁ = 1 / (1 + e⁻⁰·⁴⁴⁹²)

O₁ ≈ 0.610

13. Calculate O₂ — Digit 2

z₂ = (G₁ × 0.2) + (G₂ × 0.6) - 0.2

z₂ = (0.645 × 0.2) + (0.614 × 0.6) - 0.2

z₂ = 0.129 + 0.3684 - 0.2

z₂ ≈ 0.2974

O₂ = 1 / (1 + e⁻⁰·²⁹⁷⁴)

O₂ ≈ 0.574

14. Final Prediction

Output	Digit	Activation
O₀	0	0.559
O₁	1	0.610
O₂	2	0.574

Highest Activation:

max(0.559, 0.610, 0.574)

= 0.610

Corresponding Output Neuron = O₁

Therefore:
Prediction = Digit 1

🔢 ANN PREDICTION = 1

15. Scaling the Mathematics to the Real 16 × 10 Image

The classroom example used only four input values. For the actual image:

16 × 10 = 160 pixels

X = [x₁,x₂,x₃,...,x₁₆₀]

Suppose the network contains:

Input Layer = 160 neurons
Hidden Layer 1 = 32 neurons
Hidden Layer 2 = 16 neurons
Output Layer = 10 neurons

Layer 1:

Z¹ = W¹X + B¹

W¹ = 32 × 160
X = 160 × 1
B¹ = 32 × 1

H¹ = Activation(Z¹)

Layer 2:

Z² = W²H¹ + B²

W² = 16 × 32
H¹ = 32 × 1
B² = 16 × 1

H² = Activation(Z²)

Layer 3:

Z³ = W³H² + B³

W³ = 10 × 16
H² = 16 × 1
B³ = 10 × 1

Y = Activation(Z³)

16. Complete Mathematical Flow

16 × 10 Image
↓
160 Pixel Values
↓
Layer 1
Z¹ = W¹X + B¹
H¹ = Activation(Z¹)
↓
Layer 2
Z² = W²H¹ + B²
H² = Activation(Z²)
↓
Layer 3
Z³ = W³H² + B³
Y = Activation(Z³)
↓
10 Output Neurons
↓
argmax(Y)
↓
Predicted Digit 0–9

17. Interactive Step-by-Step Explanation

① Input Layer

16 × 10 = 160 pixels

X = [x₁,x₂,...,x₁₆₀]

② Hidden Layer 1

Z¹ = W¹X + B¹

H¹ = Activation(Z¹)

The first hidden layer detects basic patterns in the pixels, such as lines, curves and edges.

③ Hidden Layer 2

Z² = W²H¹ + B²

H² = Activation(Z²)

The second hidden layer combines simpler patterns into larger digit structures.

④ Output Layer

Z³ = W³H² + B³

Y = Activation(Z³)

Y = [Y₀,Y₁,Y₂,...,Y₉]

⑤ Final Prediction

Prediction = argmax(Y)

Largest output ↓ Corresponding digit

🔢 Predicted Digit = 1

18. Important ANN Terms

160 Input Neurons Hidden Layer 1 Hidden Layer 2 Output Layer Weight Bias Weighted Sum Sigmoid Activation Neuron Pixel Digit Recognition Classification Prediction argmax