Data, model, training, inference: the basic terms on a single example

The whole machine learning path on estimating the price of a flat: data and how to split them, a model with parameters, training by gradient descent, hyperparameters and the learning rate, evaluation, inference and a glossary of terms with links to the parts of the series.

9 minutes Matěj Žalmánek 2026-10-16

Series AI basics Part 3 of 52

Full series syllabus 52 of 83 topics published

What you will take away

What data, model, training and inference mean and how they relate to each other.
The difference between parameters and hyperparameters and why the learning rate matters.
Why training is expensive once and inference cheap, but frequent.

Useful to know first: Map of artificial intelligence: what is what and how it all fits together

Four words keep coming back throughout the series: data, model, training and inference. Whether it is digit recognition, a spam filter or a large language model, the path is always the same: a function is learned from examples and then answers new questions. In this part we walk the whole path on the smallest possible example — estimating the price of a flat from its floor area — and at each step we say what it looks like in large models. At the end there is a glossary of terms with links to the parts where you can learn more about them.

It builds on the parts Map of artificial intelligence and A brief history of AI.

Training and inference

1. Data: examples with correct answers

Definition — data, features and labels

Data are the examples the model learns from. Each example consists of

features — the inputs we know about it (here the floor area of the flat),
and, in supervised learning, also a label — the correct answer (here the price of the flat).

The data are split into three mutually disjoint parts, each with a different purpose:

training — the model learns from them (the parameters are searched for using them);
validation — we use them to tune the learning settings and to choose between models;
test — used only once, at the end, to give an honest estimate of how the model will do on new data.

The precise notation is in the definition “a dataset formally” below.

In plain words

A textbook with exercises and the answers at the back. You learn on the practice exercises, a mock exam tells you whether you have got it, and the real exam (which you have never seen before) shows where you really stand.

Our data are made up: 60 flats, each with its floor area (the feature) and its price in millions of Czech crowns (the label). The price grows with the area, but not exactly — two flats of the same size differ in location, condition and floor. We split them into 40 training, 10 validation and 10 test flats.

Definition — a dataset formally

Let d, n ∈ ℕ, d, n ≥ 1, and let Y be the set of possible labels (Y = ℝ in regression, a finite set of classes in classification). A dataset is a finite sequence of pairs

D = ((x⁽¹⁾, y⁽¹⁾), …, (x⁽ⁿ⁾, y⁽ⁿ⁾)), where x⁽ⁱ⁾ ∈ ℝ^d and y⁽ⁱ⁾ ∈ Y for i = 1, …, n.

The vector x⁽ⁱ⁾ contains the d features of the i-th example and y⁽ⁱ⁾ is its label. It is assumed that there is an (unknown) probability distribution on ℝ^d × Y from which the pairs are drawn mutually independently and from which future queries come as well.

A split of the data is given by a partition of the set of indices into three mutually disjoint sets

{1, …, n} = I_tr ∪ I_val ∪ I_test, I_tr ∩ I_val = I_tr ∩ I_test = I_val ∩ I_test = ∅;

the parts of the data are D_tr = ((x⁽ⁱ⁾, y⁽ⁱ⁾))_{i ∈ I_tr} and similarly D_val, D_test. So each example belongs to exactly one part. Here d = 1 (floor area), Y = ℝ (price in millions of CZK), n = 60 and the parts have 40, 10 and 10 examples.

In plain words

A table where each row is one flat: columns with the features and a last column with the price. Disjoint means that no flat is in two parts at once — otherwise the “real exam” would contain examples the model has already seen. And the same-distribution assumption says that the test flats come from the same market as the ones we will ask about later; without it, the error on the test would say nothing about the future.

For large language models the data are trillions of tokens of text from the web, books and code; the feature is the text so far and the label is the next token (the part on pretraining). The principle is the same.

2. Model: a function with parameters

Definition — model and parameters

Let X be a set of inputs, Y a set of outputs and Θ ⊆ ℝ^p, p ≥ 1, the set of admissible parameter values. A model is a family of functions

{f_θ : X → Y | θ ∈ Θ},

i.e. for each choice of θ = (θ₁, …, θ_p) ∈ Θ one function that computes an output f_θ(x) ∈ Y from an input x ∈ X. The numbers θ₁, …, θ_p are the parameters of the model (in neural networks, weights). We choose the shape of the function (and with it X, Y, Θ); learning finds the values of the parameters.

Our model is a straight line: X = Y = ℝ, Θ = ℝ², θ = (w, b) and f_w,b(x) = w · x + b, where x is the floor area, w says how much the price rises with each unit of area, and b where the line starts.

In plain words

A model is a machine with knobs. Ours has two knobs. A language model has billions of them — but it is still a machine that computes an output from an input, and its “knowledge” is nothing more than the setting of its knobs.

ŷ = w · x + b

ŷ: the predicted price; the hat distinguishes it from the actual price y, which we then compare it with
w: slope — how many millions each extra square metre adds; without it the price would not depend on the area at all
x: floor area in m², the only feature; the only thing the model knows about the flat
b: intercept — “the price of a zero-sized flat”; without it the line would have to pass through the origin, and flat prices do not head towards zero

In general one writes ŷ = f_θ(x): a function f with parameters θ = (w, b). In a language model, f is a transformer and θ is billions of weights.

3. Training: tuning the parameters

Definition — loss function and training

Let {f_θ : θ ∈ Θ}, Θ ⊆ ℝ^p, be a model and D_tr = ((x₁, y₁), …, (x_n, y_n)), n ≥ 1, the training data. A loss function is a function

J : Θ → [0, ∞),

that assigns to each choice of parameters θ a number J(θ) saying how wrong the model f_θ is on D_tr — here the mean of the squared errors (MSE), written out in full in the breakdown below. Training is the search for parameters with the smallest possible loss, ideally θ* ∈ argmin_{θ ∈ Θ} J(θ).

If J is differentiable and Θ = ℝ^p, denote by ∇J(θ) = (∂J/∂θ₁, …, ∂J/∂θ_p) its gradient. Gradient descent is the algorithm:

Input: initial parameters θ₀ ∈ ℝ^p, learning rate η > 0, number of steps T ∈ ℕ.
Output: parameters θ_T ∈ ℝ^p.

For t = 0, 1, …, T − 1 set θ_t+1 = θ_t − η · ∇J(θ_t).
After T steps, stop and return θ_T.

So in every step the descent computes in which direction the loss grows fastest and moves the parameters a small step the opposite way. An epoch is one pass through all the training data; in our descent, where every step computes the gradient from all n examples, one epoch is exactly one step.

In plain words

Fog on a hill and the goal of getting down into the valley. You only see the slope under your feet, so you take a step downhill, look again, take another step. Each step is one adjustment of the knobs; the valley is the knob setting with which the model is least wrong.

Our loss function, the mean squared error (MSE), written out in full:

J(w, b) = 1/n ∑_i=1ⁿ ( w x_i + b − y_i )²

J(w, b): the loss as a function of the parameters — the data are fixed, we may only change w and b, which is why only they are in the brackets
1/n ∑: the average over all n = 40 training flats — so that the loss is comparable for datasets of different sizes and no flat takes precedence
w x_i + b − y_i: the error on the i-th flat: the prediction ŷ_i minus the actual price; positive when the model overshoots, negative when it undershoots
²: the square — so that positive and negative errors do not cancel out, large errors weigh more than small ones, and the loss can be differentiated smoothly

The loss is zero only if the line passes exactly through all the points; otherwise it is positive.

Theorem — the gradient of MSE for a straight line

Let n ≥ 1, let (x₁, y₁), …, (x_n, y_n) ∈ ℝ² be the training data and J : ℝ² → ℝ the MSE of a straight line,

J(w, b) = 1/n ∑_i=1ⁿ (w x_i + b − y_i)².

Denote ŷ_i = w x_i + b for i = 1, …, n. Then for all (w, b) ∈ ℝ²

∂J/∂w = 2/n ∑_i=1ⁿ (ŷ_i − y_i) · x_i, ∂J/∂b = 2/n ∑_i=1ⁿ (ŷ_i − y_i).

Proof. The derivative of a sum is the sum of the derivatives, and the constant factor 1/n is preserved by differentiation. For a single term, by the chain rule: ∂/∂w (w x_i + b − y_i)² = 2 (w x_i + b − y_i) · x_i, because the inside of the bracket has derivative x_i with respect to w; with respect to b its derivative is 1. ∎

In plain words

Every flat “pulls” on the parameters in proportion to how wrong the model was on it. For w the pull is also multiplied by the floor area: an error on a large flat says more about the slope than an error on a studio.

One step of gradient descent then moves both parameters against the gradient:

w ← w − η · ∂J/∂w

w: the parameter before and after the step — the arrow ← means “overwrite with the new value”, not equality
−: we go against the gradient, i.e. downhill: we want to make the loss smaller, not larger
η: learning rate — how long a step we take; the gradient only holds near the point, so the step must be small (see the next section)
∂J/∂w: the gradient — in which direction and how steeply the loss grows when we increase w

The same for b, only with ∂J/∂b. Both parameters change at once, from the gradient computed at the old point.

The whole training as an algorithm (exactly how the figure below computes it):

w, b = 0, 0                                  # initial parameters
for epocha in range(200):                    # hyperparameter: number of epochs
    gw = gb = 0
    for plocha, cena in trenovaci_data:      # one epoch = all the data
        x = plocha / 100                     # area in hundreds of m²
        chyba = (w * x + b) - cena           # ŷ − y
        gw += 2 * chyba * x / n              # ∂J/∂w
        gb += 2 * chyba / n                  # ∂J/∂b
    w, b = w - eta * gw, b - eta * gb        # step downhill, eta = learning rate

We plug in the area in hundreds of m² (52 m² → 0.52). In metres, the gradient with respect to w would come out dozens of times larger than with respect to b, and a single learning rate would not suit both parameters at once. Such feature scaling is common in practice. That is why w below is in millions per 100 m².

Example — the first step of descent

At the point w = b = 0 the model predicts 0 everywhere, so the error on every flat is −y_i. The gradient is therefore

∂J/∂w = −2 · 1/n ∑_i=1ⁿ x_i y_i = −2 · 5.678 = −11.36, ∂J/∂b = −2 · ȳ = −2 · 6.88 = −13.76,

where 5.678 is the mean of the products of area (in hundreds of m²) and price, and ȳ = 6.88 is the mean price in millions. With learning rate η = 0.5 (the colours match the terms of the step formula above):

w ← 0 − 0.5 · (−11.36) = 5.68, b ← 0 − 0.5 · (−13.76) = 6.88.

(w is in millions per 100 m² here.) The loss drops from 54.2 to 17.7. Both gradients were negative (the model undershot everywhere), so both parameters increased.

We start with w = 0 and b = 0 (the model says “everything costs nothing”, loss 54.2) and run 200 epochs of gradient descent. The result: w = 0.0910 million per m², that is roughly CZK 91,000 for every extra metre, and b = 0.35 million. The exact least-squares solution (for a straight line it can be computed with a formula, see the part on linear regression) differs only in the fourth decimal place. The descent got where it was supposed to.

Flats by floor area and price, the learned line and the prediction for a new flat training validation test model

4. Hyperparameters: what we set ourselves

Definition — hyperparameters

Training is a procedure that computes the parameters θ* from the training data D_tr and from a setting h (e.g. gradient descent above). Hyperparameters are the components of the setting h: quantities that affect the shape of the model or the course of learning, are not part of θ, are not changed by training and are chosen by us in advance. They include:

the shape and size of the model (the shape of the function f_θ and the set Θ);
the learning rate η > 0 — how big a step of gradient descent is;
the number of epochs;
the batch size B ∈ {1, …, n} — how many training examples are used to compute the gradient in one step.

Hyperparameters are tuned by the results on the validation data D_val, never on the test data.

In plain words

Parameters are what the pupil learns. Hyperparameters are how we teach them: how fast, how long, how many examples at once.

Loss on the training data during learning for three learning rates learning rate 0.05learning rate 0.5learning rate 1.6

With a rate of 0.05 the model learns slowly — after 200 epochs its loss is 0.48 and it is still not there. With 0.5 it reaches 0.165 within a few dozen epochs. With 1.6 the steps are so large that the model overshoots the valley further and further and the loss explodes — after five epochs it is 46,310,590. Choosing the right learning rate is one of the most common kinds of tuning in all of machine learning.

For a straight line and MSE, one can compute exactly where the boundary between “slow” and “explodes” lies. The loss is a quadratic function of the parameters, so its curvature is the same everywhere and is described by two numbers — the curvature in the steep and in the flat direction of the valley (the eigenvalues λ of the Hessian matrix, the table of second derivatives).

e_t+1 = (1 − η λ) · e_t

e_t+1: the distance of the parameters from the best values after the next epoch (in one of the two directions) — this is what we want to bring to zero
1: without a step the distance would stay as it was; the step takes away a share η λ of it. If |1 − η λ| < 1, the distance shrinks; if it is greater than 1, it grows as a geometric sequence
η: learning rate — the only thing here we choose
λ: the curvature of the loss in the given direction — given by the data, not by us; in the steep direction the gradient is large, and so the step is long
e_t: the distance before the epoch; the formula repeats, so after t epochs e_t = (1 − η λ)^t e₀

Theorem — when gradient descent converges

Let p ≥ 1, let H ∈ ℝ^p×p be a symmetric matrix with eigenvalues λ₁, …, λ_p, where λ_min = min_j λ_j > 0 and λ_max = max_j λ_j, let θ* ∈ ℝ^p, c ∈ ℝ, and let the loss be the quadratic function

J(θ) = c + 1/2 (θ − θ*)^T H (θ − θ*), θ ∈ ℝ^p,

with Hessian matrix H and minimum at θ*. Let η ∈ ℝ and θ_t+1 = θ_t − η ∇J(θ_t) for t = 0, 1, 2, … Then θ_t → θ* for every starting point θ₀ ∈ ℝ^p if and only if

0 < η < 2/λ_max.

Proof. We have ∇J(θ) = H(θ − θ*), so the deviation e_t = θ_t − θ* satisfies e_t+1 = (I − ηH) e_t. In an orthonormal basis of eigenvectors of H, the j-th component e_t^(j) evolves as e_t+1^(j) = (1 − η λ_j) e_t^(j), so e_t^(j) = (1 − η λ_j)^t e₀^(j). All components go to zero for every e₀ if and only if |1 − η λ_j| < 1 for all j, i.e. 0 < η λ_j < 2; since λ_j > 0, the strictest condition is the one for λ_max. ∎

Example — the limit for our flats

For the MSE of a straight line, H = 2/n · [[∑x_i², ∑x_i], [∑x_i, n]] = [[1.193, 1.435], [1.435, 2]] (area in hundreds of m²). The eigenvalues come out as λ_max = 3.087 and λ_min = 0.106, so the descent converges for η < 2/3.087 = 0.648. The colours match the terms of the formula e_t+1 = (1 − η λ) · e_t above.

η = 0.05: the multiplier in the flat direction is 1 − 0.05 · 0.106 =0.9947; after 200 epochs what remains is 0.9947²⁰⁰ ≈ 35% of the original deviation — which is why the model is not there yet.

η = 0.5: the multipliers are |1 − 0.5 · 3.087| = 0.544 and 1 − 0.5 · 0.106 = 0.947; after 200 epochs what remains is 0.947²⁰⁰ ≈ 2 · 10⁻⁵ of the deviation — practically nothing.

η = 1.6 > 0.648: in the steep direction |1 − 1.6 · 3.087| = 3.94. The deviation roughly quadruples every epoch, and the loss, which grows with its square, 15.5-fold — in the chart from 801 to 12,395.

In plain words

The valley is a narrow trough: steep across, almost flat along. The learning rate must be small enough for the steep walls, otherwise you jump onto the opposite slope higher than you were. But then you creep along the trough ever so slowly. This conflict between the steep and the flat direction is the reason why large models use smarter variants of descent.

5. How good the model is

Let D′ be any part of the data with m ≥ 1 examples (x, y) and ŷ(x) = f_θ*(x) the prediction of the trained model. We measure the error on D′ by the root mean squared error (RMSE):

RMSE = √1/m ∑_{(x, y) ∈ D′} (ŷ(x) − y)²

√: the square root brings the result from (million CZK)² back to millions of crowns — the number can be read as the “typical error”
1/m ∑_D′: the average over the chosen part — training, validation or test; each answers a different question
(ŷ(x) − y)²: the squared error on one flat, just as in the loss — we measure with what we minimised

On the test part: RMSE = √0.256 = 0.51 million (under the root is the mean of the squared errors on the test flats).

The average error (the root of the MSE) is 0.41 million on the training data, 0.60 on the validation data and 0.51 million on the test data, i.e. roughly half a million crowns. With a single feature it cannot get better: the noise we added to the prices (location, condition) is exactly that large and cannot be guessed from the area. Anyone who wants a more accurate estimate needs more features — not a smarter line. If, on the other hand, we made the model too flexible, it would memorise the noise in the training data (overfitting) and be more wrong on new flats.

6. Inference: the model in operation

Definition — inference

Let {f_θ : θ ∈ Θ} be a model and θ* ∈ Θ the parameters found by training. Inference is the computation of the prediction

ŷ = f_θ*(x)

for a new input x ∈ X, with the parameters θ* kept fixed. The model does not learn during inference (unless we explicitly train it again). For language models every answer is inference — the model generates token by token and makes one pass through the network for each token.

In plain words

Training is studying, inference is the exam — or the job. You study once and for a long time, then you work daily and quickly. When you chat with a chatbot, you are not teaching it (that happens at another time and in another way), you are just using it.

For a flat of 65 m² our model estimates 6.27 million. That costs 3 arithmetic operations. Training cost roughly 72,000 operations. In large models the ratio is similar, just on a different scale: training takes months on thousands of GPUs, a single answer takes seconds. But there are billions of answers, so for the most widely used models inference can in total cost more computation than their training — which is why so much effort goes into making models smaller and faster.

Example — inference and the number of operations

We plug in the learned parameters w = 9.0957 million per 100 m² (i.e. 0.0910 per m²) and b = 0.3538; the area in hundreds of m², x = 0.65:

ŷ = 9.0957 · 0.65 + 0.3538 = 5.9122 + 0.3538 = 6.2660 ≈ 6.27 million

(the colours match the terms of the formula ŷ = w · x + b in the section on the model).

That is 3 operations: a unit conversion (division by a hundred), a multiplication and an addition. One epoch of training makes, for each of the 40 flats, a prediction, an error and both contributions to the gradient — we count roughly 9 operations per flat (the exact count depends on what we consider a single operation), so 40 · 9 · 200 = 72,000 operations for the whole training. The ratio of training to a single prediction is 24,000 : 1.

7. Glossary

The terms from this part and where in the series you can learn more about them:

data: examples the model learns from — here 60 flats with floor area and price — in detail
feature: an input value describing an example (floor area); an image has thousands of features — its pixels — in detail
label: the correct answer we want to predict (the price) — in detail
training, validation and test data: the model learns on the first, the settings are tuned on the second, the third is used once at the end — in detail
model: a function with adjustable parameters that computes an output from an input — in detail
parameter (weight): a number inside the model that is learned; language models have billions of them — in detail
loss function: a number that measures how wrong the model is; learning makes it smaller — in detail
training (learning): searching for parameters that make the loss on the training data small — in detail
gradient: the direction in which the loss grows fastest; training goes the opposite way — in detail
epoch: one pass through all the training data — in detail
batch: how many examples are used for one learning step — in detail
hyperparameter: a setting we choose ourselves and the model does not learn (learning rate, number of epochs, model size) — in detail
overfitting: the model memorises the training data and fails on new data — in detail
inference: using a trained model on a new input — what happens when you ask a chatbot — in detail
token: a piece of text that a language model works with — in detail
embedding: a vector of numbers that represents a word, an image or another object — in detail
context and prompt: all the text the model receives during inference — in detail

8. Summary

Data = examples with features and labels, split into a training, a validation and a test part.
Model = a function with parameters; training searches for the parameters with the smallest loss, typically by gradient descent.
Our model learned CZK 91,000 per m² — the same as the exact calculation.
We choose the hyperparameters: with a small learning rate training is slow, with a large one it falls apart.
The test error (0.51 million) matches the noise in the data; a better model needs more features.
Inference = using a trained model; it is cheap per query, but there is an enormous number of queries.

Want to get hands-on? Ask an AI agent to download real data on flat prices in your city, split them into three parts and train the same line — and then have it add a second feature (say, the distance from the centre) and compare the test error.

Do you see it differently? Write to support@coden-app.com.

Data, model, training, inference: the basic terms on a single example

1. Data: examples with correct answers

2. Model: a function with parameters

3. Training: tuning the parameters

4. Hyperparameters: what we set ourselves

5. How good the model is

6. Inference: the model in operation

7. Glossary

8. Summary

An Agent on your computer, driven from your phone