Functions and mappings: why every AI model is a function

Domain, image, injectivity, composition, inverses, functions of several variables and convexity — with graphs of the sigmoid, ReLU and a loss landscape.

13 minutes Matěj Žalmánek 2026-10-15

Series AI basics Part 6 of 52

Full series syllabus 52 of 83 topics published

What you will take away

What a function, its domain and its image are — and which functions you will meet most often in AI.
What injective, surjective and bijective mean, why a neural network is a composition of functions and when an inverse exists.
Why a loss function is a landscape and why convexity matters when searching for its bottom.

Useful to know first: How to read mathematical notation: symbols, indices, sums and quantifiers, Sets and relations: the building blocks of definitions in AI

Every model in artificial intelligence is a function. A spam classifier takes an e-mail and returns “spam” or “not spam”, a language model takes text and returns probabilities of the next word, and a neural network is a function composed of thousands of smaller functions. Learning is then nothing more than searching for the right function.

The previous part ended by defining a function as a special relation. Here we pick up from there: what functions look like, what properties they have, how they are composed and why it matters in AI.

1. What a function is

Definition — function (mapping)

Let X and Y be sets and X × Y = {(x, y) : x ∈ X, y ∈ Y} their Cartesian product (see the part Sets and relations). A function (mapping) from X to Y is a relation f ⊆ X × Y such that

∀x ∈ X ∃! y ∈ Y : (x, y) ∈ f

(∃! is read “there exists exactly one”). This unique y is denoted f(x), and we write f : X → Y, x ↦ f(x). So a function assigns to every element x ∈ X exactly one element f(x) ∈ Y.

For a function f : X → Y we introduce:

the domain is the set X;
the codomain (target set) is the set Y;
the image (range) of the function is the set of values f actually takes: f(X) = {y ∈ Y : ∃x ∈ X : f(x) = y} = {f(x) : x ∈ X} ⊆ Y;
the graph of the function is the set of pairs {(x, f(x)) : x ∈ X} ⊆ X × Y.

Two functions f, g : X → Y are equal if and only if f(x) = g(x) for all x ∈ X.

In plain words

A function is a vending machine: you put in an input and exactly one output drops out — the same one every time. The domain says what you are allowed to put into the machine, the codomain what can come out of it. The image is what actually comes out.

The difference between codomain and image shows on the sigmoid σ : ℝ → ℝ: it may return any real number, but it actually returns only numbers between 0 and 1. That is why it can be read as a probability.

Example — codomain and image of the function x²

f : ℝ → ℝ, f(x) = x². The codomain is ℝ, but the image is only f(ℝ) = [0, ∞).

Why: x² ≥ 0 for every real x, so negative numbers never appear. And every y ≥ 0 does appear: f(√y) = y, for example f(3) = 9.

2. Functions you will meet in AI

A function of one variable is best understood from its graph. These keep showing up in artificial intelligence over and over — as models, as activation functions of neural networks or as loss functions:

linear: f(x) = ax + b

quadratic: x²

exponential: e^x

logarithm: ln x

sigmoid: 1/1 + e^−x

hyperbolic tangent: tanh x

ReLU: max(0, x)

The functions of one variable you will meet most often

A linear function is the simplest model — linear regression is nothing else.
A quadratic function is the basis of the mean squared error: the penalty grows with the square of the deviation.
The exponential and the logarithm are inverse to each other; the logarithm turns multiplication into addition, which is why you will find it in every probability-based loss.
Sigmoid and tanh “squash” any number into the interval (0, 1) and (−1, 1), respectively.
ReLU lets positive numbers through and sets negative ones to zero. It is so simple that for a long time nobody believed it would work — today it is the most widely used activation function.

Definition — sigmoid, hyperbolic tangent and ReLU

Let e ≈ 2.718 be Euler’s number (the base of the natural logarithm). All three functions are defined for every x ∈ ℝ.

The sigmoid (logistic function) is the function σ : ℝ → ℝ, σ(x) = 1/1 + e^−x.
The hyperbolic tangent is the function tanh : ℝ → ℝ, tanh x = e^x − e^−x/e^x + e^−x.
ReLU (rectified linear unit) is the function ReLU : ℝ → ℝ, ReLU(x) = max(0, x) = x for x > 0, ReLU(x) = 0 for x ≤ 0.

The denominators in 1 and 2 are always positive (e^t > 0 for every t ∈ ℝ), so there is never division by zero. The image of the sigmoid is the interval (0, 1), the image of tanh the interval (−1, 1) and the image of ReLU the interval [0, ∞). The hyperbolic tangent is just a shifted and stretched sigmoid: tanh x = 2σ(2x) − 1 for all x ∈ ℝ.

σ(x) = 1/1 + e^−x

σ(x): a result between 0 and 1 — we want to turn any score into something that can be read as a probability
e^−x: always positive; for large x it approaches 0, for very negative x it grows without bound — this is the “lever” that moves the whole result
1 +: so the denominator is always greater than 1: the result never exceeds 1 and there is never division by zero
1: the one in the numerator and the fraction bar turn things around: a large denominator (very negative x) gives almost 0, a denominator close to 1 (large x) gives almost 1

The sigmoid is increasing and squashes the whole real line into the interval (0, 1).

Example — the sigmoid at a few points

The colours match the terms of the formula above.

σ(0) = 1/1 + e⁰ = 1/1 + 1 = 0.5 — a score of 0 means “I don’t know”.

σ(2) = 1/1 + e⁻² ≈ 1/1 + 0.1353 ≈ 0.881

σ(−2) = 1/1 + e² ≈ 1/1 + 7.389 ≈ 0.119

Notice that σ(2) + σ(−2) = 1. This holds in general, σ(−x) = 1 − σ(x): the opposite score gives the opposite probability.

And tanh 1 = 2σ(2) − 1 ≈ 2 · 0.8808 − 1 ≈ 0.762.

In AI the quadratic function shows up most often in the mean squared error (MSE). The model estimates n numbers ŷ₁, …, ŷ_n, and the correct results were y₁, …, y_n:

MSE = 1/n ∑_i=1ⁿ (y_i − ŷ_i)²

y_i − ŷ_i: the deviation on the i-th example — by how much the model was wrong and in which direction
²: the square makes every deviation a positive number (errors up and down do not cancel out) and penalizes large errors more than proportionally
∑_i=1ⁿ: adds up the penalties over all examples — we judge the model on the whole data, not on one point
1/n: turns the total into an average, so that the error on 10 and on 10,000 examples can be compared

Example — the MSE of three estimates

Correct: y = (3, 5, 2), model: ŷ = (2.5, 5, 4).

The colours match the terms of the formula above.

Deviations: 3 − 2.5 = 0.5; 5 − 5 = 0; 2 − 4 = −2.

Squares: 0.25; 0; 4. Sum 4.25.

MSE = 1/3 · 4.25 = 4.25/3 ≈ 1.42. The single large error (−2) makes up 4 of the 4.25 — exactly the “penalizes large errors more”.

Theorem — the logarithm turns a product into a sum

Theorem. Let ln : (0, ∞) → ℝ be the natural logarithm, i.e. the inverse function of x ↦ e^x (see the section on inverse functions below), and let a, b ∈ ℝ, a > 0, b > 0. Then

ln(a · b) = ln a + ln b.

Proof. By the rule for powers e^{s + t} = e^s · e^t we have e^{ln a + ln b} = e^{ln a} · e^{ln b} = a · b. Taking the logarithm of both sides, since ln is the inverse of e^x, gives ln a + ln b = ln(ab). ∎

Example. A model gave three correct answers the probabilities 0.9, 0.8 and 0.5. The probability of all three (assuming independence) is 0.9 · 0.8 · 0.5 = 0.36. In logarithms: ln 0.9 + ln 0.8 + ln 0.5 ≈ −0.105 + (−0.223) + (−0.693) ≈ −1.02 = ln 0.36. With a million examples the product would underflow to zero, while the sum of logarithms stays a reasonable number — that is why losses work with logarithms.

3. Injective, surjective and bijective

Definition — one-to-one, onto and bijective function

Let X, Y be sets and f : X → Y a function.

f is injective (one-to-one) if and only if ∀x₁, x₂ ∈ X : f(x₁) = f(x₂) ⇒ x₁ = x₂. Equivalently: x₁ ≠ x₂ ⇒ f(x₁) ≠ f(x₂) — different inputs give different outputs.
f is surjective (onto) if and only if ∀y ∈ Y ∃x ∈ X : f(x) = y, i.e. if and only if f(X) = Y — the function hits every element of Y.
f is bijective (a one-to-one correspondence) if and only if it is both injective and surjective, i.e. if and only if ∀y ∈ Y ∃! x ∈ X : f(x) = y.

In plain words

Injective: no two inputs share an output — from the output you can tell where it came from. Surjective: nothing in the target set is left out. Bijective: every input corresponds to exactly one output and vice versa — like a cloakroom where every coat has its own ticket.

injective injective: yes surjective: no

surjective injective: no surjective: yes

bijective injective: yes surjective: yes

neither injective: no surjective: no

Four mappings between small sets

In AI almost every model is non-injective: a classifier sends thousands of different cat photos into a single class “cat”. The information about which photo it was is lost in the process — and that is fine, it is exactly what we want from a classifier. Compression models (autoencoders), on the other hand, try to be “almost injective”, so that the input can be reconstructed again from its compressed form.

Example — how a property is verified or refuted

f(x) = x² on ℝ is not injective: one counterexample is enough, f(−2) = f(2) = 4, yet −2 ≠ 2.

On [0, ∞) it is injective: x₁² = x₂² implies (x₁ − x₂)(x₁ + x₂) = 0, and since for non-negative numbers x₁ + x₂ = 0 only when x₁ = x₂ = 0, we always get x₁ = x₂.

f(x) = 2x + 1 on ℝ is bijective: for any y we find the unique x = y − 1/2.

Theorem — how many functions, injections and bijections there are

Theorem. Let X, Y be finite sets, |X| = m and |Y| = n, where |M| denotes the number of elements of a set M and m, n ∈ ℕ, m, n ≥ 1. Then

there are n^m functions f : X → Y in total;
there are n · (n − 1) · … · (n − m + 1) injective functions f : X → Y for m ≤ n, and 0 for m > n;
if m = n, a function f : X → Y is injective if and only if it is bijective, and there are n! = n · (n − 1) · … · 1 bijections.

Proof. 1. Each of the m inputs independently chooses one of the n outputs. 2. The first input has n options, the second only n − 1 (it must not hit an occupied output), and so on; the m-th input has n − m + 1 options. For m > n the (n + 1)-th input would have no free option left — more pigeons than pigeonholes. 3. An injective function hits m = n different elements of the n-element set Y, i.e. all of them, and so it is surjective too; substituting m = n into 2 gives n!. ∎

Checked by brute force (by going through all functions):

\|`X`\| → \|`Y`\|	all	injective	surjective	bijective
3 → 3	3³ = 27	3 · 2 · 1 = 6	6	6
3 → 4	4³ = 64	4 · 3 · 2 = 24	0	0
4 → 3	3⁴ = 81	0	36	0

There are 36 surjections 4 → 3: we choose which two of the four inputs share an output (6 options), and then distribute the three outputs among the three groups (3! = 6 options); 6 · 6 = 36.

4. Composing functions

Definition — composite function

Let X, Y, Z be sets and f : X → Y, g : Y → Z functions (the codomain of f is the domain of g). The composite function (“g after f”) is the function

g ∘ f : X → Z, (g ∘ f)(x) = g(f(x)) for all x ∈ X.

So f is applied first, then g. Properties:

associativity: if moreover W is a set and h : Z → W a function, then h ∘ (g ∘ f) = (h ∘ g) ∘ f, because both sides send every x ∈ X to h(g(f(x)));
in general not commutative: even if X = Y = Z and both compositions exist, g ∘ f ≠ f ∘ g is possible (example below).

In plain words

Composition is an assembly line: the output of one machine goes straight into the next. The order matters — getting dressed and then washing turns out differently than the other way round.

Take f(x) = 2x + 1 and g(x) = x²:

(g ∘ f)(3) = g(f(3)) = g(7) = 49

(f ∘ g)(3) = f(g(3)) = f(9) = 19

This is exactly how a neural network is built. Every layer is a function — a linear step followed by an activation function — and the whole network is their composition: f = f₃ ∘ f₂ ∘ f₁. A “deep” network is simply a long composition. And when we learn to differentiate in later parts, the chain rule will tell us how to differentiate precisely composite functions.

h = φ(W x + b)

h: the output of the layer (a vector) — it will go as input into the next layer; this is how the composition arises
x: the input of the layer: the data, or the output of the previous layer
W: the weight matrix: every output is a weighted sum of all inputs and the weights decide which ones matter — these are what the network learns
b: the bias: without it a zero input would have to give a zero sum; the bias lets us set the threshold at which the neuron “switches on”
φ: the activation function, applied component-wise (ReLU, sigmoid, tanh) — the only non-linear part; without it the whole network would collapse into a single linear function

A linear step Wx + b, then the non-linearity φ; a network is this pair repeated and composed.

Example — one layer with two neurons

W = ( 1 −1 ; 0.5 2 ) (two rows, one per neuron), b = (−2, −1), x = (2, 1), φ = ReLU (colours as in the formula above).

Neuron 1: 1 · 2 + (−1) · 1 + (−2) = −1, ReLU(−1) = 0 — the neuron stays silent.

Neuron 2: 0.5 · 2 + 2 · 1 + (−1) = 2, ReLU(2) = 2.

The output of the layer h = (0, 2) goes as input into the next layer.

def sit(x, vrstvy):                     # vrstvy = [(W1, b1), (W2, b2), ...]
    h = x
    for W, b in vrstvy:                  # f = f_k ∘ ... ∘ f_2 ∘ f_1
        h = relu(W @ h + b)              # the output of one layer is the input of the next
    return h

Theorem — a composition of linear functions is a linear function

Theorem. Let a, b, c, d ∈ ℝ and let f, g : ℝ → ℝ be the functions f(x) = ax + b and g(x) = cx + d. Then for all x ∈ ℝ

(g ∘ f)(x) = (ca)x + (cb + d),

i.e. g ∘ f is again a function of the form x ↦ a′x + b′ with a′ = ca and b′ = cb + d.

Proof. g(f(x)) = c(ax + b) + d = cax + cb + d. ∎

The same holds for vectors and matrices: let W₁ ∈ ℝ^k×n, b₁ ∈ ℝ^k, W₂ ∈ ℝ^m×k and b₂ ∈ ℝ^m. Then for every x ∈ ℝⁿ, W₂(W₁x + b₁) + b₂ = (W₂W₁)x + (W₂b₁ + b₂), where W₂W₁ ∈ ℝ^m×n.

Example. f(x) = 2x + 1, g(x) = 3x − 4 ⇒ (g ∘ f)(x) = 6x − 1. Check at the point 3: g(f(3)) = g(7) = 17 and 6 · 3 − 1 = 17.

That is why networks need activation functions: ten linear layers without φ can do exactly the same as one.

5. Inverse function

Definition — inverse function

Let X, Y be sets and f : X → Y a function. A function g : Y → X is the inverse of f if and only if both conditions hold:

∀x ∈ X : g(f(x)) = x (i.e. g ∘ f = id_X),
∀y ∈ Y : f(g(y)) = y (i.e. f ∘ g = id_Y),

where id_X : X → X, x ↦ x is the identity on X. The inverse function is denoted f⁻¹. In words: g takes every output of f back to the input it came from, and vice versa.

Theorem. Let f : X → Y be a function. The inverse of f exists if and only if f is bijective; if it exists, it is unique, and for every y ∈ Y, f⁻¹(y) is the unique x ∈ X for which f(x) = y.

In plain words

An inverse function takes a step back: from the output it works out the input. That only works when no two inputs lead to the same output (otherwise it would be unclear where to go back to) and when every output really leads somewhere.

The exponential and the logarithm are a pair of inverse functions: ln(e^x) = x. For the sigmoid, understood as a bijection σ : ℝ → (0, 1), the inverse function is the logit : (0, 1) → ℝ, logit(p) = ln p/1 − p — it turns a probability back into an arbitrary real number. That is why the outputs of a network before the sigmoid or softmax are called “logits”.

Example — deriving the logit step by step

We find the inverse by solving the equation p = σ(x) for x (for p ∈ (0, 1)):

p = 1/1 + e^−x

1 + e^−x = 1/p (taking the reciprocal of both sides)

e^−x = 1/p − 1 = 1 − p/p

−x = ln 1 − p/p (ln is the inverse of e^x)

x = −ln 1 − p/p = ln p/1 − p = logit(p)

In numbers: logit(0.8) = ln 0.8/0.2 = ln 4 ≈ 1.386. And back (colours as in the breakdown of the sigmoid above): σ(ln 4) = 1/1 + 1/4 = 0.8, because e^{−ln 4} = 1/4. The ratio p/1 − p = 4 is the odds (“4 to 1”); the logit is their logarithm.

6. Functions of several variables

Definition — function of several variables

Let n, m ∈ ℕ, n, m ≥ 1, and let ℝⁿ = ℝ × … × ℝ (n times) be the set of all n-tuples of real numbers (vectors) x = (x₁, …, x_n).

A (real) function of n variables is a function f : ℝⁿ → ℝ, (x₁, …, x_n) ↦ f(x₁, …, x_n), i.e. it assigns exactly one real number to every vector.
A vector-valued function is a function f : ℝⁿ → ℝ^m, x ↦ (f₁(x), …, f_m(x)), where f_i : ℝⁿ → ℝ for i = 1, …, m are its components; it assigns exactly one vector to every vector.

In plain words

Most real functions take several inputs at once: the price of a flat depends on its floor area, location and age. A layer of a neural network takes a whole vector and returns another vector.

A function of two variables can be drawn as a map: we colour every point (x, y) according to the value f(x, y). Here is Himmelblau's function, a classic test for optimization algorithms. The dark areas are valleys with a small value, and the white dots are the four minima where the function equals zero:

A function of two variables as a landscape: four equally deep valleys

Example — Himmelblau’s function and why its minima are at zero

f(x, y) = (x² + y − 11)² + (x + y² − 7)².

Both brackets are squared, so f ≥ 0 everywhere. Zero comes out exactly where both brackets are zero at the same time — where the system x² + y = 11 and x + y² = 7 holds. It has four solutions, hence four valleys.

At the point (3, 2): (9 + 2 − 11)² + (3 + 4 − 7)² = 0 + 0 = 0 — a minimum.

At the point (0, 0): (−11)² + (−7)² = 121 + 49 = 170 — high up on a hill.

This is exactly how to picture a model's loss function: its input is the model's parameters, its output the error. Training is searching for the lowest point in the landscape — except that instead of two dimensions, the landscape of a real model has millions.

7. Monotonicity and convexity

Definition — increasing and convex function

Let I ⊆ ℝ be an interval and f : I → ℝ a function.

f is increasing if and only if ∀x₁, x₂ ∈ I : x₁ < x₂ ⇒ f(x₁) < f(x₂). An increasing function is always injective (x₁ ≠ x₂ implies x₁ < x₂ or x₂ < x₁, and hence f(x₁) ≠ f(x₂)).
f is convex if and only if ∀x₁, x₂ ∈ I ∀t ∈ [0, 1] : f(tx₁ + (1 − t)x₂) ≤ t f(x₁) + (1 − t) f(x₂). (The point tx₁ + (1 − t)x₂ lies between x₁ and x₂, and so in the interval I.)
A point x* ∈ I is a local minimum of f if and only if ∃δ > 0 ∀x ∈ I : |x − x*| < δ ⇒ f(x*) ≤ f(x), and a global minimum if and only if ∀x ∈ I : f(x*) ≤ f(x).

Theorem. Let I ⊆ ℝ be an interval, f : I → ℝ a convex function and x* ∈ I a local minimum of f. Then x* is a global minimum of f.

In plain words

A convex function is shaped like a bowl: the line segment between two points of the graph always lies above the graph. A bowl has only one bottom — whoever keeps walking downhill will find it. With a non-convex function you can end up in a smaller pit and never learn about the deeper one next to it.

f(tx₁ + (1 − t)x₂) ≤ t f(x₁) + (1 − t) f(x₂)

tx₁ + (1 − t)x₂: a point between x₁ and x₂; the weights t and 1 − t add up to 1, so as t goes from 1 to 0 we pass through the whole segment from x₁ to x₂
f( … ): the height of the graph at that point
t f(x₁) + (1 − t) f(x₂): the height of the line segment (chord) between the points of the graph above x₁ and x₂ — with the same weights, so it is above the same point
≤: the graph never rises above the chord — that is the bowl shape; if it rose somewhere, there would be a “hump” behind which another valley could hide

Example — the convexity inequality in numbers

The colours match the terms of the inequality above.

f(x) = x², x₁ = −1, x₂ = 3, t = 1/2: the point in the middle is 1, left side f(1) = 1, right side 1 + 9/2 = 5. 1 ≤ 5 ✓.

f(x) = x⁴ − 3x² + x, for x₁, x₂ we take both minima −1.30 and 1.13, t = 1/2: the point in the middle is −0.085,

left side f(−0.085) ≈ −0.11, right side −3.51 + (−1.07)/2 = −2.29.

−0.11 > −2.29 — the inequality ≤ does not hold, the function is not convex. The “hump” between the minima is exactly what separates the two valleys.

convex: x², a single minimum

non-convex: x⁴ − 3x² + x

Convex and non-convex function

The non-convex function x⁴ − 3x² + x has two minima: a global one at x ≈ −1.30 with value ≈ −3.51 and a local one at x ≈ 1.13 with value ≈ −1.07. Whoever starts searching on the right and keeps going down ends up in the worse one. The loss functions of linear and logistic regression are convex — that is why their training reliably finds the best solution. The losses of neural networks are not convex, and how to live with that will be the subject of the part on gradient descent.

We found the minima in the chart above numerically, without derivatives: with a dense grid of candidates, which are then refined by the golden-section search. In every step it discards the part of the interval where the minimum cannot be, and the interval shrinks to ≈ 0.618 of its previous length.

def minima(f, a, b, n=2000):
    krok = (b - a) / n
    vysledek = []
    for i in range(1, n):
        x = a + i * krok
        if f(x) < f(x - krok) and f(x) < f(x + krok):   # candidate: lower than its neighbours
            l, p = x - krok, x + krok
            z = (5 ** 0.5 - 1) / 2                        # ≈ 0.618
            for _ in range(60):                           # golden section
                c, d = p - z * (p - l), l + z * (p - l)
                if f(c) < f(d): p = d                     # the minimum is to the left of d
                else:           l = c                     # the minimum is to the right of c
            vysledek.append((l + p) / 2)
    return vysledek

8. Functions in artificial intelligence

Now the whole of machine learning can be said in the language of functions:

A model is a function f_θ : X → Y with parameters θ — a whole family of functions, one for every choice of parameters.
A loss function L : Y × Y → ℝ says how much the answer ŷ costs when the correct one is y.
Learning is picking one function from the family: the argmin over θ of the average loss — a function of the parameters, which we just drew as a landscape.
Activation functions (sigmoid, tanh, ReLU) turn linear layers into a non-linear network; without them a composition of linear functions would again be just a linear function.

θ* = argmin_θ 1/n ∑_i=1ⁿ L(f_θ(x_i), y_i)

θ*: the learned parameters — they pick one function f_θ* out of the whole family
argmin_θ: we do not want the smallest error itself, but the place where it occurs — parameters, not a number
1/n ∑_i=1ⁿ: the average over all n training examples — the model should be good on the data as a whole
L: the loss function — turns the difference between the answer and the truth into a single number that can be minimized (for example the square of the deviation, as in MSE)
f_θ(x_i): the model’s answer to the i-th input; the only place in the formula where θ appears
y_i: the correct answer from the data, which the model’s answer is compared with

Learning = finding the lowest point of a landscape whose coordinates are the parameters and whose height is the average loss.

9. Summary

A function assigns exactly one output to every input; the image is where it actually lands.
An injective function never mixes up two inputs, a surjective one covers the whole target, a bijective one has an inverse.
Composing functions is an assembly line — and a neural network is a composition of layers.
A loss function is a landscape over the parameter space; a convex landscape has a single bottom, a non-convex one can have several.

Do you see it differently? Write to support@coden-app.com.

Functions and mappings: why every AI model is a function

1. What a function is

2. Functions you will meet in AI

3. Injective, surjective and bijective

4. Composing functions

5. Inverse function

6. Functions of several variables

7. Monotonicity and convexity

8. Functions in artificial intelligence

9. Summary

An Agent on your computer, driven from your phone