Classical ML Models

Trees · Ensembles · SVMs

1:00

A tour of the classical machine-learning toolkit — decision trees, random forests, gradient boosting, kNN, and SVMs — with the intuition behind each model and when to reach for it.

By the end, you can

Explain why gradient-boosted trees often outperform deep nets on tabular data.
Describe how a decision tree selects splits using Gini impurity and information gain.
Compute Gini impurity and information gain for a simple two-class split.
Distinguish overfitting from underfitting in decision trees and name two fixes.
Contrast bagging and boosting: what each reduces on the bias-variance trade-off, and which ensemble families each produces.
Explain why random forests add two sources of randomness and why more trees never hurt.
Trace the sequential error-correction loop of one boosting round.
Describe kNN as a lazy learner and explain the effect of changing k.
State what the SVM margin is and why only support vectors define the boundary.
Explain the kernel trick and why it enables non-linear boundaries without building the high-dimensional space explicitly.
Map any classical model to its bias-variance profile and choose the right technique for a given problem.

Up next in AI, Machine Learning & Course Review