Every algorithm here follows the same skeleton: data goes in,
a model transforms it, a prediction
comes out, an error signal corrects the model, and a
target defines what "correct" means. The colors above are
the vocabulary used across the figures and equations.
Read in order, or skip to whatever is bothering you. Each article stands alone.
Optimization & Learning
Neural Networks
Probabilistic Reasoning
Classical ML
The Vocabulary
Every figure in the series colors its elements by the role they play in the pipeline
below. Here's what each color stands for and where you'll see it.
Raw values flowing into the algorithm, what the model sees.
samples in gradient descent · tokens in attention · states in Markov · contexts in word2vec · pixels in convolution
What the algorithm is learning, the adjustable part.
θ in gradient descent · weights in backprop · splits in trees · centroids in clustering · principal axes in PCA · kernels in convolution · Q-table in RL
What the model outputs when given data.
ŷ · softmax probabilities · cluster assignments · projected coordinates · feature maps · Q-values
The error signal, how wrong the prediction was, and the direction to fix it.
∇L in gradient descent · chain-rule gradients in backprop · impurity in trees · reconstruction error in PCA · TD-error in RL
Ground truth or goal, what "correct" looks like.
labels · prior in Bayes · reward/goal in RL · true distribution in word2vec