Linear algebra for machine learning: vectors, matrices and why AI mostly multiplies

Vectors, the dot product and cosine similarity, matrices as transformations, the determinant, a neural network layer as Wx + b and eigenvectors — with figures computed by code.

14 minutes Matěj Žalmánek 2026-10-15

Series AI basics Part 9 of 52

Full series syllabus 52 of 83 topics published

What you will take away

What a vector and the dot product are — and how they give you the similarity of words.
Why a matrix is a transformation of space, what the determinant tells you and why matrix multiplication is composition.
That a neural network layer is Wx + b with a nonlinearity, and what the eigenvectors behind PCA are.

Useful to know first: How to read mathematical notation: symbols, indices, sums and quantifiers, Functions and mappings: why every AI model is a function

If you took any artificial intelligence model apart down to the core, you would mostly find one operation: matrix multiplication. To a model, an image is a table of numbers, a word is a list of numbers, a layer of a neural network is a matrix, and graphics cards matter so much because they can multiply matrices faster than anything else. Linear algebra is the language in which AI computes.

We won't cover the whole breadth of the university course. We'll go through only what you will need in the next parts — and for every concept we'll show where it sits in AI.

1. Vectors: an arrow and a list of numbers

Definition — vector and operations on it

Let n ∈ ℕ, n ≥ 1. The set of all ordered n-tuples of real numbers is

ℝⁿ = {(u₁, …, u_n) : u_i ∈ ℝ for i = 1, …, n}.

Its elements u = (u₁, …, u_n) ∈ ℝⁿ are called vectors and the numbers u_i ∈ ℝ their components. Two vectors u, v ∈ ℝⁿ are equal if and only if u_i = v_i for all i = 1, …, n.

Let u, v ∈ ℝⁿ and c ∈ ℝ. The sum of vectors and the scalar multiple of a vector are vectors in ℝⁿ defined component-wise:

u + v = (u₁ + v₁, …, u_n + v_n), cu = (cu₁, …, cu_n).

The difference is u − v = u + (−1)v and the zero vector is 0 = (0, …, 0) ∈ ℝⁿ.

Let k ∈ ℕ, k ≥ 1, let v₁, …, v_k ∈ ℝⁿ be vectors (here the index numbers the vectors, not the components) and c₁, …, c_k ∈ ℝ. The linear combination of the vectors v₁, …, v_k with coefficients c₁, …, c_k is the vector

c₁v₁ + … + c_kv_k = ∑_j=1^k c_jv_j ∈ ℝⁿ.

So we add vectors and multiply them by a number by doing the same with each component separately.

In plain words

A vector has two faces. To a computer it is a list of numbers — the price, floor area and age of a flat. To the imagination it is an arrow from the origin to a point. Adding vectors means placing arrows one after another: whoever walks first along u and then along v ends up at u + v.

The sum of vectors is the diagonal of a parallelogram

Example — sum, multiple and linear combination

For u = (3, 1) and v = (1, 2) from the figure:

u + v = (3 + 1, 1 + 2) = (4, 3)

2u = (2 · 3, 2 · 1) = (6, 2)

2u − v = (6 − 1, 2 − 2) = (5, 0)

The last line is a linear combination with coefficients c₁ = 2, c₂ = −1. Every vector of the plane can be put together from u and v as c₁u + c₂v, because they don't point in the same direction — how to find the coefficients is shown in the section on systems of equations.

In AI, almost everything is a vector. A black-and-white image of 28 × 28 pixels is a vector in ℝ⁷⁸⁴. A word in a language model is a vector with hundreds to thousands of components — a so-called embedding. And the parameters of the whole model are one giant vector θ.

2. Dot product, length and similarity

Definition — dot product, norm, angle

Let n ≥ 1 and u, v ∈ ℝⁿ. The dot product (scalar product) of the vectors u and v is the real number

u · v = ∑_i=1ⁿ u_iv_i = u₁v₁ + … + u_nv_n,

also written u^⊤v (see the transpose below).

The length (Euclidean norm) of a vector u ∈ ℝⁿ is the non-negative number

‖u‖ = √u · u = √u₁² + … + u_n².

Let moreover u ≠ 0 and v ≠ 0. The angle of the vectors u and v is the unique number φ ∈ [0, π] for which

cos φ = u · v/‖u‖ ‖v‖;

such a φ exists because the fraction lies in [−1, 1] (the Cauchy–Schwarz inequality below). Vectors u, v ∈ ℝⁿ are perpendicular (orthogonal) if and only if u · v = 0.

In plain words

The dot product multiplies matching numbers and adds them up. The result tells you how much two vectors “point the same way”: positive and large — similar, zero — perpendicular, negative — opposite.

For u = (3, 1) and v = (1, 2), u · v = 3 · 1 + 1 · 2 = 5, ‖u‖ = √10 ≈ 3.16, ‖v‖ = √5 ≈ 2.24, so ‖u‖ ‖v‖ = √50, and cos φ = 5/√50 ≈ 0.71 — the vectors make an angle of 45° (colours as in the formula below).

cos φ = u · v/‖u‖ ‖v‖

cos φ: cosine similarity — a number from −1 to 1 that depends only on the direction: 1 the same direction, 0 perpendicular, −1 opposite
u · v: the dot product — it grows when large components meet with the same signs; that is the very core of “agreement”
‖u‖ ‖v‖: the product of the lengths — dividing by it removes the effect of size; without it a long vector (a long document) would look similar to everything

The dot product measures agreement and size at the same time; dividing by the lengths leaves the pure direction.

Theorem — the Cauchy–Schwarz inequality

Let n ≥ 1 and u, v ∈ ℝⁿ. Then

|u · v| ≤ ‖u‖ ‖v‖.

So for u, v ≠ 0, −1 ≤ u · v/‖u‖ ‖v‖ ≤ 1: cosine similarity never comes out outside the interval [−1, 1] and the angle φ is well defined.

Proof. For v = 0 both sides are zero. Otherwise ‖v‖ > 0 and for every t ∈ ℝ, 0 ≤ ‖u − tv‖² = ‖u‖² − 2t(u · v) + t²‖v‖². A quadratic polynomial in t that is never negative has a discriminant ≤ 0: 4(u · v)² − 4‖u‖²‖v‖² ≤ 0, i.e. (u · v)² ≤ ‖u‖²‖v‖². Taking square roots (both sides are non-negative) gives the claim. ∎

In AI, the cosine of the angle is famous as cosine similarity. Imagine a toy word embedding with three components “animal, domestic, has wheels”: cat (0.9, 0.8, 0), dog (0.85, 0.9, 0.05), car (0, 0.3, 0.95). The similarity of cat and dog comes out as 0.995, of cat and car only 0.20. This is exactly how search engines find “similar” documents and language models “similar” words — just in hundreds of dimensions instead of three.

Example — cosine similarity step by step

cat · dog = 0.9 · 0.85 + 0.8 · 0.9 + 0 · 0.05 = 0.765 + 0.72 + 0 = 1.485

‖cat‖ = √0.81 + 0.64 + 0 = √1.45 ≈ 1.204, ‖dog‖ = √0.7225 + 0.81 + 0.0025 = √1.535 ≈ 1.239

cos = 1.485/1.204 · 1.239 ≈ 1.485/1.492 ≈ 0.995

cat · car = 0 + 0.8 · 0.3 + 0 = 0.24, ‖car‖ = √0.9925 ≈ 0.996

cos = 0.24/1.204 · 0.996 ≈ 0.24/1.200 ≈ 0.20

The colours match the terms of the cosine similarity formula above: the dot product, the product of the lengths, the result. Cat and car share only the “domestic” component (0.8 · 0.3), and even that is small for the car.

3. Matrices and multiplying by a vector

Definition — matrix and matrix–vector product

Let m, n ∈ ℕ, m, n ≥ 1. An m × n matrix is a collection of m · n real numbers A_ij ∈ ℝ, i = 1, …, m, j = 1, …, n, written in a table with m rows and n columns; A_ij is the entry in row i and column j. Formally it is a mapping A : {1, …, m} × {1, …, n} → ℝ, (i, j) ↦ A_ij. The set of all such matrices is denoted ℝ^m×n. In the text we write a matrix row by row, with rows separated by a semicolon: [2 1; 1 3] has the first row (2, 1) and the second (1, 3).

Let A ∈ ℝ^m×n and x ∈ ℝⁿ. The matrix–vector product is the vector Ax ∈ ℝ^m with components

(Ax)_i = ∑_j=1ⁿ A_ijx_j, i = 1, …, m.

The i-th component of the result is thus the dot product of the i-th row (A_i1, …, A_in) with the vector x. The product is defined if and only if x has as many components as A has columns.

In plain words

A matrix is a machine for vectors: you put a vector in and get a different one out. Each row of the matrix takes the dot product with the input, and the result is one component of the output.

A = [2 1; 1 3], x = (1, 2)

Ax = (2 · 1 + 1 · 2, 1 · 1 + 3 · 2) = (4, 7)

(colours as in the formula below)

(Ax)_i = ∑_j=1ⁿ A_ij x_j

(Ax)_i: the i-th component of the output — each is computed separately from one row, which is why the output has as many components as the matrix has rows
∑_j=1ⁿ: the sum over the columns — it collects the contributions of all inputs; that is why x must have exactly as many components as A has columns
A_ij: a weight — how much the j-th component of the input contributes to the i-th component of the output (in a network it is the weight of a connection)
x_j: the j-th component of the input — what gets weighted

Each output is a weighted sum of all inputs; the matrix is the table of those weights.

Definition — transpose and identity matrix

Let m, n ≥ 1 and A ∈ ℝ^m×n. The transpose of A is the matrix A^⊤ ∈ ℝ^n×m with entries

(A^⊤)_ij = A_ji, i = 1, …, n, j = 1, …, m.

It is thus obtained by swapping rows and columns. We identify a vector u ∈ ℝⁿ with an n × 1 matrix (a column); then u^⊤ ∈ ℝ^1×n is a row, and for u, v ∈ ℝⁿ, u^⊤v = u · v (a row times a column gives the dot product).

A square matrix A ∈ ℝ^n×n is symmetric if and only if A^⊤ = A, i.e. A_ij = A_ji for all i, j = 1, …, n (for example [2 1; 1 3]).

Let n ≥ 1. The identity matrix I = I_n ∈ ℝ^n×n is the matrix with entries

I_ij = 1 for i = j, I_ij = 0 for i ≠ j (i, j = 1, …, n),

i.e. ones on the diagonal and zeros elsewhere. For every x ∈ ℝⁿ, (Ix)_i = ∑_j I_ijx_j = x_i, so Ix = x — it is the “do nothing” transformation.

Example: [0.5 −1 2; 1 0 −0.5]^⊤ = [0.5 1; −1 0; 2 −0.5].

4. A matrix as a transformation of space

The best way to “see” a matrix is to look at what it does to the plane. Every 2 × 2 matrix carries a grid to another grid — lines stay lines and the origin stays in place. That is why it is called a linear map. Grey is the original grid, green the grid after the transformation, orange the image of the unit square:

Definition — linear map; theorem on the columns of a matrix

Let m, n ≥ 1. A map f : ℝⁿ → ℝ^m is linear if and only if it satisfies both conditions:

(additivity) for all u, v ∈ ℝⁿ, f(u + v) = f(u) + f(v);
(homogeneity) for all c ∈ ℝ and u ∈ ℝⁿ, f(cu) = c f(u).

Condition 2 for c = 0 gives f(0) = f(0 · 0) = 0 · f(0) = 0 — the origin stays in place.

Theorem. Let f : ℝⁿ → ℝ^m be linear and for j = 1, …, n let e_j ∈ ℝⁿ be the j-th unit vector (1 in the j-th place, 0 elsewhere). Then there is exactly one matrix A ∈ ℝ^m×n such that f(x) = Ax for all x ∈ ℝⁿ, and its j-th column is f(e_j). Conversely, every map x ↦ Ax is linear.

Proof: Every x ∈ ℝⁿ is x = x₁e₁ + … + x_ne_n, so by linearity f(x) = x₁f(e₁) + … + x_nf(e_n), and that is exactly Ax for the matrix with columns f(e_j). Uniqueness: Ae_j is the j-th column of A, so the columns are determined by the values f(e_j). Linearity of x ↦ Ax follows directly from the component-wise definition of the product. ∎

Example: rotation by 90° sends e₁ = (1, 0) to (0, 1) and e₂ = (0, 1) to (−1, 0). These two vectors placed as columns give R = [0 −1; 1 0].

rotation by 90°
[0 -1; 1 0] · area × 1

scaling 1.5×
[1.5 0; 0 1.5] · area × 2.25

shear
[1 1; 0 1] · area × 1

projection onto the x-axis
[1 0; 0 0] · area × 0

Four matrices and what they do to the plane

Definition — determinant of a 2 × 2 matrix

Let a, b, c, d ∈ ℝ and A = [a b; c d] ∈ ℝ^2×2. The determinant of the matrix A is the real number

det A = ad − bc.

Geometric meaning (a statement, not part of the definition): the map x ↦ Ax sends the unit square [0, 1] × [0, 1] to a parallelogram with sides (a, c) and (b, d) and area |det A|; in general it sends every shape with area S to a shape with area |det A| · S. |det A| is therefore the area scaling factor.

If det A < 0, the transformation also mirrors the plane (it swaps “clockwise” and “anticlockwise”); if det A = 0, it flattens the plane onto a line or a point.

Example — the determinants of the four matrices from the figure

rotation: det [0 −1; 1 0] = 0 · 0 − (−1) · 1 = 1

scaling: det [1.5 0; 0 1.5] = 1.5 · 1.5 − 0 · 0 = 2.25

shear: det [1 1; 0 1] = 1 · 1 − 1 · 0 = 1

projection: det [1 0; 0 0] = 1 · 0 − 0 · 0 = 0

In plain words

The determinant tells you how many times the matrix enlarges areas. Rotation and shear don't change area (det = 1), scaling 1.5× enlarges it 2.25×, and the projection flattens the plane onto a line — both the area and the determinant are zero, and what has once been flattened can't be stretched back.

5. Matrix multiplication is composition

Definition — matrix product

Let m, k, n ≥ 1, A ∈ ℝ^m×k and B ∈ ℝ^k×n (the number of columns of A equals the number of rows of B). The matrix product of A and B is the matrix AB ∈ ℝ^m×n with entries

(AB)_ij = ∑_l=1^k A_ilB_lj, i = 1, …, m, j = 1, …, n.

The entry (i, j) is thus the dot product of the i-th row of A with the j-th column of B. This product satisfies:

(composition) for every x ∈ ℝⁿ, (AB)x = A(Bx) — the matrix product is the composition of maps, first B, then A;
(associativity) for every C ∈ ℝ^n×p, p ≥ 1, (AB)C = A(BC);
(non-commutativity) in general AB ≠ BA: there are A, B ∈ ℝ^2×2 with AB ≠ BA (the example R, Z below).

In plain words

Multiplying two matrices means joining two machines one after another into one. And just as with functions, order matters: rotating and then shearing ends up differently than shearing and then rotating.

R = [0 −1; 1 0] (rotation), Z = [1 1; 0 1] (shear)

RZ = [0 −1; 1 1], ZR = [1 −1; 1 0]

Example — the product RZ entry by entry

The entry (i, j) is the dot product of the i-th row of R with the j-th column of Z:

(1, 1): (0, −1) · (1, 0) = 0 · 1 + (−1) · 0 = 0 (1, 2): (0, −1) · (1, 1) = 0 − 1 = −1

(2, 1): (1, 0) · (1, 0) = 1 (2, 2): (1, 0) · (1, 1) = 1

Check on the vector x = (1, 2): Zx = (3, 2), R(Zx) = (−2, 3), and likewise (RZ)x = (0 − 2, 1 + 2) = (−2, 3).

Straight from the definition, the product is computed with three nested loops:

def soucin(A, B):                 # A is m × k, B is k × n
    m, k, n = len(A), len(B), len(B[0])
    C = [[0] * n for _ in range(m)]
    for i in range(m):                # each row of the result
        for j in range(n):            # each column of the result
            for l in range(k):        # dot product of a row of A with a column of B
                C[i][j] += A[i][l] * B[l][j]
    return C

That is m · k · n multiplications: a billion for two 1000 × 1000 matrices. But they are all independent of each other, so a graphics card can do thousands of them at once.

6. A neural network layer is a matrix

Now we can say exactly what one layer of a neural network does:

y = ReLU(W x + b)

y ∈ ℝ^m: the output of the layer — m numbers that the next layer gets as its input
ReLU: max(0, ·) component-wise — the only nonlinear part; without it the whole network would merge into a single matrix (see the theorem below)
W ∈ ℝ^m×n: the weights — each row is one neuron that weights the input in its own way; this is what the network learns
x ∈ ℝⁿ: the input — data or the output of the previous layer
b ∈ ℝ^m: the bias — Wx is always zero for x = 0; the bias lets a neuron “switch on” somewhere other than at the origin

A linear step (weights and bias) followed by a nonlinearity — that is the whole layer.

The matrix W (weights) and the vector b (bias) are the parameters the network learns. For W = [0.5 −1 2; 1 0 −0.5], input x = (1, 2, 3) and b = (0, −1) (colours as in the formula above):

Wx = (0.5 · 1 + (−1) · 2 + 2 · 3, 1 · 1 + 0 · 2 + (−0.5) · 3) = (0.5 − 2 + 6, 1 + 0 − 1.5) = (4.5, −0.5)

Wx + b = (4.5 + 0, −0.5 + (−1)) = (4.5, −1.5), ReLU → y = (4.5, 0)

The layer turned three numbers into two and zeroed out the negative one. A deep network is a series of such steps one after another — and without a nonlinear function (ReLU, sigmoid) the whole thing would collapse into a single matrix, because a composition of linear maps is again a linear map. When the network processes several inputs at once (a batch), it stacks them into a matrix X, and the whole batch is computed by a single multiplication WX — which is exactly the operation graphics cards do fastest.

Theorem — two linear layers without a nonlinearity are one

Let n, k, m ≥ 1, W₁ ∈ ℝ^k×n, b₁ ∈ ℝ^k, W₂ ∈ ℝ^m×k and b₂ ∈ ℝ^m. Put W = W₂W₁ ∈ ℝ^m×n and b = W₂b₁ + b₂ ∈ ℝ^m. Then for every x ∈ ℝⁿ

W₂(W₁x + b₁) + b₂ = (W₂W₁)x + (W₂b₁ + b₂) = Wx + b.

Proof: expanding the bracket (distributivity) and associativity of matrix multiplication. ∎ By induction the same holds for any number of layers — a hundred linear layers can do exactly what one can.

7. The inverse matrix and systems of equations

Definition — inverse matrix

Let n ≥ 1, A ∈ ℝ^n×n, and let I ∈ ℝ^n×n be the identity matrix. The matrix A is invertible (regular) if and only if

∃ B ∈ ℝ^n×n : BA = AB = I.

There is at most one such matrix B (if also B′A = AB′ = I, then B′ = B′I = B′AB = IB = B); it is called the inverse matrix of A and denoted A⁻¹. A matrix that is not invertible is singular.

Theorem. Let A = [a b; c d] ∈ ℝ^2×2. Then A is invertible if and only if det A = ad − bc ≠ 0, and in that case

A⁻¹ = 1/ad − bc · [d −b; −c a].

(For general n the same criterion det A ≠ 0 holds, with the determinant of an n × n matrix, which this part does not introduce.) If A ∈ ℝ^n×n is invertible and y ∈ ℝⁿ, the system Ax = y has exactly one solution x = A⁻¹y.

In plain words

The inverse matrix undoes the transformation. That is only possible if the matrix hasn't flattened the space — and that is exactly what the determinant keeps watch over.

A⁻¹ = 1/ad − bc · [d −b; −c a]

A⁻¹: the inverse — the matrix that takes the result Ax back to x
1/ad − bc: one over the determinant, restores the area — A enlarged areas |det A| times, the inverse must shrink them by the same factor; with det A = 0 we would divide by zero, which is why the inverse does not exist
[d −b; −c a]: the diagonal swapped and the signs flipped — exactly the matrix whose product with A is (ad − bc) · I; all that is left is to divide

Check: [d −b; −c a] · [a b; c d] = [ad − bc 0; 0 ad − bc].

Example — the inverse of [2 1; 1 3] and solving a system

det A = 2 · 3 − 1 · 1 = 5

A⁻¹ = 1/5 · [3 −1; −1 2] = [0.6 −0.2; −0.2 0.4] (colours as in the formula above)

the system 2x₁ + x₂ = 4, x₁ + 3x₂ = 7:

x = A⁻¹(4, 7) = (0.6 · 4 − 0.2 · 7, −0.2 · 4 + 0.4 · 7) = (2.4 − 1.4, −0.8 + 2.8) = (1, 2)

Check: 2 · 1 + 2 = 4 and 1 + 3 · 2 = 7. In practice large systems are not solved via the inverse but by Gaussian elimination — it is faster and numerically more stable; the result is the same.

For A = [2 1; 1 3], det A = 2 · 3 − 1 · 1 = 5 and A⁻¹ = [0.6 −0.2; −0.2 0.4]. From the known result (4, 7) we thus get back the input (1, 2). This is also how linear regression is computed in closed form — the least squares method leads to the system (X^⊤X)θ = X^⊤y (the normal equations).

8. Eigenvalues and eigenvectors

Definition — eigenvalue and eigenvector

Let n ≥ 1 and let A ∈ ℝ^n×n be a square matrix. A number λ ∈ ℝ is an eigenvalue of the matrix A if and only if

∃ v ∈ ℝⁿ, v ≠ 0 : Av = λv.

Every such nonzero vector v is an eigenvector of the matrix A belonging to the eigenvalue λ. (In general complex λ ∈ ℂ and v ∈ ℂⁿ are also allowed; in this part real ones are enough.)

It holds that λ is an eigenvalue of A if and only if det(A − λI) = 0 (for n = 2 with the determinant defined above). Spectral theorem: if A is symmetric, it has n real eigenvalues (counted with multiplicity) and there are n mutually perpendicular eigenvectors of length 1.

In plain words

A matrix turns most vectors somewhere else. Eigenvectors are the exceptions: the matrix only stretches or shrinks them, the direction stays. The eigenvalue says by how much. They are the “natural axes” of the transformation.

The matrix A = [2 1; 1 3] turns a circle into an ellipse; the orange axes are the eigenvectors unit circle → its image

The eigenvalues of the matrix [2 1; 1 3] are 5 ± √5/2, that is ≈ 3.62 and ≈ 1.38. The ellipse is stretched 3.62× in the direction of the first eigenvector and 1.38× in the direction of the second. The grey arrows in a general direction turn after the transformation, the orange axes don't. This is the basis of principal component analysis (PCA): the eigenvectors of the covariance matrix of the data show the directions in which the data vary the most, and the eigenvalues show how much.

Example — eigenvalues and eigenvectors of [2 1; 1 3] by hand

The equation Av = λv is the same as (A − λI)v = 0. A nonzero solution exists only when A − λI flattens the plane, that is, when its determinant is zero:

det [2 − λ 1; 1 3 − λ] = (2 − λ)(3 − λ) − 1 = λ² − 5λ + 5 = 0

λ = 5 ± √25 − 20/2 = 5 ± √5/2 ≈ 3.618 and 1.382

for λ₁: the first row (2 − λ₁)v₁ + v₂ = 0 gives v = (1, λ₁ − 2) ≈ (1, 1.618), after normalization ≈ (0.526, 0.851)

Check: the sum of the eigenvalues 3.618 + 1.382 = 5 is the sum of the diagonal (2 + 3), and their product 5 is the determinant.

Large matrices have no formula for the roots, and nobody computes the determinant for them. The largest eigenvalue and its vector are found by the simple power method: it repeatedly multiplies a vector by the matrix and scales it to length 1. The component in the direction of the strongest eigenvector grows the fastest, so it gradually outgrows the others:

def mocninna_metoda(A, v, kroky):
    v = v / norma(v)
    for _ in range(kroky):
        Av = A @ v
        odhad = v @ Av                # Rayleigh quotient v·Av: estimate of the eigenvalue
        v = Av / norma(Av)            # scale to length 1 so the numbers don't run away
    return odhad, v

Step	Estimate of `λ`₁	`v` after the step
1	2	(0.894, 0.447)
2	3	(0.707, 0.707)
3	3.5	(0.600, 0.800)
4	3.6	(0.555, 0.832)
5	3.615	(0.537, 0.844)
6	3.618	(0.530, 0.848)

Starting from v = (1, 0) with the matrix [2 1; 1 3]: after six steps the estimate is 3.618 and the vector approaches (0.526, 0.851). The deviation of the vector shrinks at each step roughly by the ratio λ₂/λ₁ ≈ 0.38; for a symmetric matrix the deviation of the estimate shrinks even by its square, ≈ 0.15. Google originally computed PageRank with the same idea — the eigenvector of a giant matrix of links between pages.

9. Linear algebra in artificial intelligence

Data are matrices: a row is an example, a column a feature.
Embeddings are vectors; similarity is measured by the dot product or the cosine.
A network layer is Wx + b plus a nonlinearity; training changes the entries of the matrices W.
Attention in transformers computes the dot products of all queries with all keys at once — the matrix multiplication QK^⊤.
PCA and compression look for eigenvectors, recommender systems factor the “user × movie” matrix into a product of smaller matrices.

10. Summary

A vector is both a list of numbers and an arrow; it is added component-wise.
The dot product measures how much two vectors point the same way — cosine similarity is its normalized version.
A matrix is a linear transformation of space; the determinant says how many times it changes area, and zero means the transformation can't be undone.
Matrix multiplication is composition of transformations, and a neural network layer is Wx + b with a nonlinearity.
Eigenvectors are directions that the matrix only stretches — PCA is built on them.

Do you see it differently? Write to support@coden-app.com.

Linear algebra for machine learning: vectors, matrices and why AI mostly multiplies

1. Vectors: an arrow and a list of numbers

2. Dot product, length and similarity

3. Matrices and multiplying by a vector

4. A matrix as a transformation of space

5. Matrix multiplication is composition

6. A neural network layer is a matrix

7. The inverse matrix and systems of equations

8. Eigenvalues and eigenvectors

9. Linear algebra in artificial intelligence

10. Summary

An Agent on your computer, driven from your phone