🤖 Support Vector Machine (SVM)

Support Vector Machine (SVM) is a supervised machine-learning algorithm used mainly for classification. It can also be used for regression through Support Vector Regression (SVR).

The central idea is to find a decision boundary that separates classes while maintaining an appropriate margin from the nearest observations.

1. What is SVM?

SVM searches for a decision boundary that separates observations belonging to different classes.

SVM separates observations using a decision boundary.

2. Hyperplane

The hyperplane is the mathematical decision boundary.

w · x + b = 0

3. Margin

The margin represents the separation between the decision boundary and the closest observations.

Margin = 2 / ||w||

4. Support Vectors

Support vectors are the observations closest to the decision boundary.

5. Maximum-Margin Principle

Several boundaries may separate the classes. SVM searches for an appropriate boundary with maximum separation from the nearest examples.

6. Hard Margin SVM

Hard-margin SVM requires the training observations to satisfy the separation constraints.

7. Soft Margin SVM

Soft-margin SVM allows some observations to violate the ideal margin.

Minimize ½ ||w||² + C Σ ξᵢ

8. C Parameter

C controls the penalty associated with margin violations.

9. Linear SVM

Linear SVM uses a straight decision boundary.

10. Non-Linear SVM

When a straight line cannot separate the classes effectively, SVM can use a non-linear kernel.

11. Kernel Trick

Original Data

→

Kernel

→

Higher Feature Space

→

Linear Separation

12. Linear Kernel

K(x,z) = x · z

13. Polynomial Kernel

K(x,z) = (γ x · z + r)^d

14. RBF / Gaussian Kernel

K(x,z) = exp(-γ ||x-z||²)

15. Sigmoid Kernel

K(x,z) = tanh(γ x · z + r)

16. Gamma Parameter

17. SVM Workflow

18. Feature Scaling

SVM models can be affected when features have very different numerical scales.

z = (x - μ) / σ

19. Spam Detection

Email

→

Text Processing

→

TF-IDF

→

SVM

→

Spam / Ham

20. Classification Metrics

Accuracy

Precision

Recall

21. Confusion Matrix

22. Iris Classification

23. Advantages of SVM

📐 Maximum Margin

Uses a margin-based decision principle.

📊 High Dimensions

Can work with high-dimensional feature representations.

🔄 Kernels

Can model non-linear relationships.

📝 Text Data

Useful with sparse text feature representations.

24. Limitations of SVM

• Large datasets can increase computational cost.
• Kernel and hyperparameter selection may require experimentation.
• Feature scaling is often important.
• Interpretability may be lower than simple rule-based models.

25. Applications of SVM

26. SVM Classification vs Regression

27. Important SVM Terminology

28. Kernel Comparison

Kernel	Visual Boundary	Main Idea
Linear	────────	Straight boundary
Polynomial	∿∿∿	Polynomial relationship
RBF	◉	Radial similarity
Sigmoid	Ｓ	Sigmoid-shaped relationship

29. Practical SVM Workflow

30. SVM Parameters at a Glance

31. SVM Decision Process

32. Hard Margin vs Soft Margin

Property	Hard Margin	Soft Margin
Training violations	Not allowed	Allowed
Slack variables	No	Yes
Noise tolerance	Low	Higher
Parameter C	Not the main formulation	Important

33. Classification Pipeline

Training Data

→

Feature Engineering

→

Scaling

→

SVM

→

Metrics

34. SVM vs Other Classification Models

35. SVM Mathematical View

yᵢ(w · xᵢ + b) ≥ 1

36. Support Vector Geometry

37. SVM Prediction

38. Advantages and Limitations Summary

39. Interview Concept Map

40. Complete SVM Concept Diagram

Core SVM idea: SVM finds an appropriate separating boundary, identifies the observations that most strongly determine that boundary, and balances margin size with training violations through its optimization parameters.

41. Important Interview Questions

Q1. What is SVM?

A supervised machine-learning algorithm that constructs a decision boundary for classification and can also be extended to regression.

Q2. What is a hyperplane?

A mathematical decision boundary separating observations in feature space.

Q3. What are support vectors?

Observations closest to the decision boundary that strongly influence the fitted SVM boundary.

Q4. What is margin?

The separation between the decision boundary and the nearest observations.

Q5. What is the kernel trick?

A technique that allows SVM to model non-linear relationships using kernel similarity functions.

Q6. What does C do?

C controls the penalty associated with observations violating the desired margin constraints.

Q7. What does gamma do?

Gamma controls the locality of influence for kernels such as RBF.

Q8. Why is scaling important?

Because the geometry used by SVM can be affected when features have very different numerical scales.

CREATED BY BIJAN KRISHNA PAUL