๐ง Two-Layer ANN — Step-by-Step Mathematics
Example of handwritten digit recognition using a two-layer Artificial Neural Network.
1. What Does Two-Layer ANN Mean?
A two-layer neural network contains two computational layers:
Layer 2: Hidden Layer → Output Layer
For digit recognition, the input image is converted into numerical pixel values.
= 160 input values
Input: X = [x₁, x₂, x₃, ... , x₁₆₀]
2. Two-Layer ANN Architecture
Output 0 → Digit 0
Output 1 → Digit 1
...
Output 9 → Digit 9
3. Small Numerical Example
Suppose our small input image is:
X = [1, 0, 1, 0]
This means:
| Pixel | Value |
|---|---|
| x₁ | 1 |
| x₂ | 0 |
| x₃ | 1 |
| x₄ | 0 |
1 Layer 1 — Input to Hidden Layer
Assume the hidden layer contains two neurons:
H₂
The weights are:
| Input | Weight → H₁ | Weight → H₂ |
|---|---|---|
| x₁ | 0.5 | 0.2 |
| x₂ | 0.3 | 0.4 |
| x₃ | 0.8 | 0.6 |
| x₄ | 0.1 | 0.7 |
Bias values:
b₂ = -0.4
4. Calculate Hidden Neuron H₁
Step 1 — Weighted Sum
Substitute the input values:
z₁ = 0.5 + 0 + 0.8 + 0 - 0.5
z₁ = 0.8
Step 2 — Activation
Using the sigmoid activation function:
H₁ = 1 / (1 + e⁻⁰·⁸)
H₁ ≈ 0.690
5. Calculate Hidden Neuron H₂
Step 1 — Weighted Sum
Substitute the input:
z₂ = 0.2 + 0 + 0.6 + 0 - 0.4
z₂ = 0.4
Step 2 — Activation
H₂ ≈ 0.599
6. Hidden Layer Output
After applying the activation function, the original four inputs have been transformed into two hidden-layer values.
[1, 0, 1, 0]
↓
Hidden Layer:
[0.690, 0.599]
These two values now become the input to Layer 2.
7. Layer 2 — Hidden Layer to Output Layer
For a complete digit classifier, we use 10 output neurons.
O₁ → Digit 1
O₂ → Digit 2
...
O₉ → Digit 9
For simplicity, we calculate three output neurons here:
O₁ → Digit 1
O₂ → Digit 2
Suppose the weights are:
| Hidden | O₀ | O₁ | O₂ |
|---|---|---|---|
| H₁ | 0.8 | 0.3 | 0.2 |
| H₂ | 0.4 | 0.9 | 0.5 |
Bias values:
b₁ = -0.2
b₂ = -0.4
8. Calculate Output Neuron O₀
z₀ = (0.690 × 0.8) + (0.599 × 0.4) - 0.3
z₀ = 0.552 + 0.2396 - 0.3
z₀ ≈ 0.4916
Apply sigmoid:
O₀ ≈ 0.621
9. Calculate Output Neuron O₁
z₁ = (0.690 × 0.3) + (0.599 × 0.9) - 0.2
z₁ = 0.207 + 0.5391 - 0.2
z₁ ≈ 0.5461
O₁ ≈ 0.633
10. Calculate Output Neuron O₂
z₂ = (0.690 × 0.2) + (0.599 × 0.5) - 0.4
z₂ = 0.138 + 0.2995 - 0.4
z₂ ≈ 0.0375
O₂ ≈ 0.509
11. Select the Highest Output
| Output Neuron | Digit | Activation |
|---|---|---|
| O₀ | 0 | 0.621 |
| O₁ | 1 | 0.633 |
| O₂ | 2 | 0.509 |
max(0.621, 0.633, 0.509)
= 0.633
Corresponding Neuron = O₁
12. Now Expand the Same Mathematics to 16 × 10 Pixels
The small example used only four inputs so that every calculation could be written manually. For the actual 16 × 10 digit image:
X = [x₁,x₂,x₃,...,x₁₆₀]
If we use 32 hidden neurons:
Hidden Layer = 32 neurons
Output Layer = 10 neurons
The first hidden neuron is calculated as:
H₁ = Activation(z₁)
The second layer then calculates:
O₀ = Activation(z₀)
The same calculation is performed for all ten output neurons.
13. Matrix Form of the Two-Layer ANN
The complete calculation can be written compactly using matrices.
Z¹ = W¹X + B¹
H = Activation(Z¹)
Layer 2:
Z² = W²H + B²
Output:
Y = Activation(Z²)
For our 160-pixel digit recognizer:
W¹ = 32 × 160
B¹ = 32 × 1
H = 32 × 1
W² = 10 × 32
B² = 10 × 1
Y = 10 × 1
14. Complete Two-Layer ANN Calculation
↓
160 Pixel Values
↓
X = [x₁,...,x₁₆₀]
↓
Layer 1
Z¹ = W¹X + B¹
↓
H = Activation(Z¹)
↓
Layer 2
Z² = W²H + B²
↓
Y = Activation(Z²)
↓
10 Output Values
↓
argmax(Y)
↓
Predicted Digit 0–9
15. Interactive Step-by-Step Summary
Step 1 — Input
X = [x₁,x₂,...,x₁₆₀]
Step 2 — Input → Hidden Layer
Each hidden neuron calculates a weighted sum of all input pixels.
Step 3 — Hidden Activation
For example:
H₁ = 0.690
H₂ = 0.599
Step 4 — Hidden → Output
Y = Activation(Z²)
The ten output neurons produce scores for digits 0 through 9.
Step 5 — Final Prediction
Largest Output ↓ Corresponding Digit
No comments:
Post a Comment