A brief history of artificial intelligence: from Turing to language models

The Turing test, the Dartmouth workshop, the golden years, two AI winters and expert systems, statistical learning, deep learning and language models — a timeline with links to parts of the series and the bitter lesson of seventy years of research.

10 minutes Matěj Žalmánek 2026-10-16

Series AI basics Part 2 of 52

Full series syllabus 52 of 83 topics published

What you will take away

How AI developed and why enthusiasm alternated with winters.
Where the ideas today’s models rest on come from.
What the bitter lesson is and why general methods with a lot of compute win.

Useful to know first: Map of artificial intelligence: what is what and how it all fits together

Artificial intelligence is not new. Its history is older than the personal computer and resembles a roller coaster: periods of enthusiasm and big promises, followed by disillusionment and money drying up — the “winters” — and then another rise when it turned out that an old idea suddenly works. Most of the ideas today’s language models rest on were invented decades ago. In this part we go through them in the order in which they came, and for each one the series explains you will find a link to the relevant part.

It follows on from the part Map of artificial intelligence.

Timeline of artificial intelligence (booms in green, winters in purple)

1. Beginnings: can a machine think?

Definition — Turing test

In 1950 Alan Turing replaced the vague question “can machines think?” with a test he himself called the “imitation game”. The Turing test has three participants:

the judge — a human who decides;
the human — a hidden counterpart;
the machine — a hidden counterpart that is being tested.

and it proceeds as follows:

the judge communicates with both counterparts only through written text and does not know which is which (knows them only by the labels X and Y);
for a limited time the judge asks them any questions;
at the end the judge declares whether the machine is X or Y.

The machine holds up in the test if and only if the judge identifies it correctly no more often than by mere guessing; the next definition makes this condition precise. If the machine passes itself off as a human this way just as well as a human does, then according to Turing we have no reason to deny it intelligence — that is Turing’s philosophical conclusion, not part of the definition of the test.

In plain words

Instead of a philosophical debate, a practical test: if you are texting someone and cannot tell whether it is a person or a program, then for you it is intelligence. Today’s language models often pass such a test in a short conversation — and it turns out that this says less about intelligence than people expected.

Definition — the Turing test as a statistical test

Let n ∈ ℕ, n ≥ 1, be the number of independent conversations (runs of the test above). For i = 1, …, n let X_i ∈ {0, 1} be a random variable with X_i = 1 if and only if the judge identified the machine correctly in the i-th conversation, and let

S = ∑_i=1ⁿ X_i

be the number of correct identifications. The null hypothesis is: the machine is indistinguishable from a human, i.e. the judge is only guessing, X₁, …, X_n are independent and P(X_i = 1) = 1/2. Then S ~ Bin(n, 1/2), the expected value is ES = n/2 and the standard deviation is σ = √n/2. The test statistic is

z = S − n/2/√n/2.

The machine passes the test if and only if |z| < 1.96, i.e. if and only if a two-sided test at the 5% level (with the normal approximation of the binomial distribution) does not reject the null hypothesis. In words: the judge’s result does not differ statistically from guessing.

Note. This is a modern formalization, not Turing’s. In 1950 Turing described the test in words and instead of a threshold gave a prediction: around the year 2000, an average judge after five minutes of conversation would identify the machine correctly with a probability of no more than 70%.

In plain words

A single conversation proves nothing — the judge can get it right by chance. That is why many conversations are held and we ask whether the judge recognizes the machine more often than if they were tossing a coin. If not, the machine has passed.

z = S − n/2/√n/2

S: how many times the judge recognized the machine — the only thing the test measures
n/2: how many times they would be right by pure guessing; we subtract it because only what is above chance is interesting
√n/2: the typical spread of guessing; dividing by it compares the excess with ordinary noise — 6 extra out of 100 conversations is something different from 6 extra out of 10,000

How many “ordinary deviations” the judge is better than a coin.

Example — 100 conversations

The judge recognized the machine in 56 conversations out of 100 (the colours match the terms of the formula above). Guessing would give 100/2 = 50, the deviation is √100/2 = 5, so

z = 56 − 50/5 = 1.2.

That is less than 1.96 — it does not differ from guessing and the machine passed. It would have failed if the judge had been right in at least 50 + 1.96 · 5 = 59.8, that is 60 conversations.

1943 Artificial neuron. Warren McCulloch and Walter Pitts described the neuron as a logic element with a threshold. → part of the series
1950 Turing test. Alan Turing, in the paper Computing Machinery and Intelligence, proposed the imitation game in place of the question “can machines think?”.
1950 Chess by search. Claude Shannon described how a computer would play chess: search moves to a certain depth and score positions with a function. → part of the series

2. The golden years (1956–1974)

At a workshop at Dartmouth in the summer of 1956, people gathered who believed that “every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it”. Two decades of optimism followed. Programs proved theorems, solved word problems, played checkers and talked about blocks on a table. In 1965 Herbert Simon predicted that within twenty years machines would be capable of doing any work a human can do.

1956 Dartmouth workshop. John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon convened a summer workshop; the name “artificial intelligence” stuck.
1956 Logic Theorist. Allen Newell and Herbert Simon: a program that proved mathematical theorems by search. → part of the series
1958 Perceptron. Frank Rosenblatt built a neuron that learns; newspapers wrote about a machine that would walk and talk. → part of the series
1959 Machine learning. Arthur Samuel: a checkers program that improved by playing against itself. The term machine learning comes from him. → part of the series
1960 Kalman filter. Rudolf Kálmán: estimating a state from noisy measurements; it flew with Apollo. → part of the series
1965 Fuzzy sets. Lotfi Zadeh introduced membership in a set as a number between 0 and 1. → part of the series
1966 ELIZA. Joseph Weizenbaum wrote a program that imitated a psychotherapist with a handful of rules — and people confided their secrets to it.
1969 The book Perceptrons. Marvin Minsky and Seymour Papert showed what a single neuron cannot do (XOR); interest in neural networks faded. → part of the series
1971 Shakey and STRIPS. The robot Shakey from Stanford planned its steps in the STRIPS language. → part of the series
1973 Lighthill report. The British government had the results of AI assessed; the report was critical and funding was cut.

The perceptron: the first neuron that learns

Definition — perceptron

Let d ∈ ℕ, d ≥ 1, let w = (w₁, …, w_d) ∈ ℝ^d be the weights and b ∈ ℝ the bias. For x ∈ ℝ^d denote by w · x = w₁x₁ + … + w_dx_d the dot product. The perceptron with weights w and bias b is the function

h_w,b : ℝ^d → {−1, +1}, h_w,b(x) = +1 if w · x + b > 0, and h_w,b(x) = −1 otherwise.

Let (x, y) be an example with input x ∈ ℝ^d and correct class y ∈ {−1, +1}. The perceptron makes a mistake on it if and only if

y · (w · x + b) ≤ 0.

The case w · x + b = 0 (the point lies right on the boundary) also counts as a mistake here, even though h_w,b assigns it to −1. The boundary of the perceptron is the set {x ∈ ℝ^d : w · x + b = 0}; for w ≠ 0 it is a hyperplane (for d = 2 a line).

In plain words

Each input gets a weight according to how strongly it speaks for “yes”. The neuron adds up the inputs with their weights and if the sum comes out positive, it says “yes”. Learning means tuning the weights so that it says “yes” correctly.

Rosenblatt’s rule corrects the weights only when the perceptron makes a mistake:

w ← w + η · y · x

w: the weights — the only thing that is learned; the new value is the old one plus a correction, so learning proceeds gradually, one example after another
η: learning rate — how big a step; for a perceptron starting from zeros it only scales all weights equally, and so does not change the boundary
y: the correct class ±1 — sets the direction: a positive example pulls the weights towards itself, a negative one pushes them away
x: the input it got wrong — the correction points exactly there; an input with value 0 did not contribute to the error, and so its weight does not change

The bias is corrected the same way, b ← b + η · y. After the correction, y · (w · x + b) grows by exactly η · (‖x‖² + 1) > 0: the answer on that example moves in the right direction.

perceptron(data, η):
    w ← (0, …, 0);  b ← 0
    repeat:
        chyb ← 0
        for each example (x, y) in data:
            if y · (w · x + b) ≤ 0:              # a mistake (or it sits on the boundary)
                w ← w + η · y · x
                b ← b + η · y
                chyb ← chyb + 1
    while chyb > 0                               # a whole epoch without a mistake: done
    return w, b

Example — a perceptron learns AND

Data: (0, 0) → −1, (0, 1) → −1, (1, 0) → −1, (1, 1) → +1; η = 1, start w = (0, 0), b = 0. The colours match the terms of the rule w ← w + η · y · x above.

(0, 0), y = −1: −1 · (0 + 0) = 0 ≤ 0, mistake. w = (0, 0) + 1 · (−1) · (0, 0) = (0, 0), b = 0 + 1 · (−1) = −1.
(0, 1) and (1, 0): −1 · (0 − 1) = 1 > 0, correct, nothing changes.
(1, 1), y = +1: 1 · (0 + 0 − 1) = −1 ≤ 0, mistake. w = (0, 0) + 1 · (+1) · (1, 1) = (1, 1), b = −1 + 1 · (+1) = 0.

And so it goes on: 18 corrections in total, the last one in epoch 8, and the result is w = (3, 2), b = −4. Check: 3 · 1 + 2 · 1 − 4 = 1 > 0, and for the other inputs −4, −2, −1 < 0.

Theorem — a perceptron cannot do XOR (Minsky and Papert, 1969)

Let d = 2 and let h_w,b be the perceptron from the definition above. Then there exist no w = (w₁, w₂) ∈ ℝ² and b ∈ ℝ such that

h_w,b(0, 0) = −1 ∧ h_w,b(1, 1) = −1 ∧ h_w,b(0, 1) = +1 ∧ h_w,b(1, 0) = +1,

i.e. no perceptron classifies all four points of the XOR function correctly.

Proof. Correct classification would mean b ≤ 0, w₁ + w₂ + b ≤ 0, w₂ + b > 0 and w₁ + b > 0. Adding the last two inequalities gives w₁ + w₂ + 2b > 0, so w₁ + w₂ + b > −b ≥ 0 — a contradiction with the second inequality. ∎

In plain words

The XOR points lie as on a chessboard: the positive ones on one diagonal, the negative ones on the other. A single line cannot separate them. Two layers of neurons can — but in 1969 nobody knew how to train them. More in the part Perceptron.

3. Two winters and expert systems (1974–1993)

Definition — AI winter

An AI winter is a period of time for which all three conditions hold:

it was preceded by a period of great expectations and promises that were not fulfilled;
funding of AI research drops sharply (government grants as well as private investment);
interest in the field drops — fewer projects, companies and attention from experts and the public.

It is a historical notion, not a mathematical one: it has no precise quantitative bounds and the dates are approximate. Usually two winters are mentioned: roughly 1974–1980 (after the Lighthill report in the United Kingdom and cuts by the American grant agency DARPA) and 1987–1993 (after the collapse of the market for specialized computers for the Lisp language and disillusionment with expert systems).

In plain words

Programs that mastered toy worlds of blocks ran into the real world: too many possibilities, too many exceptions, too slow computers. Whoever had promised a machine that “will think within ten years” had to explain why it did not. The money went elsewhere.

Between the winters came a wave of expert systems: instead of general intelligence, a narrow field and hundreds of rules written down with experts (“if temperature > 38 and … then …”). They worked and made money. But the rules were hard to maintain, the systems could do nothing outside their field and were unable to learn. In the shadow of that era, though, neural networks quietly returned: backpropagation (1986) showed how to train networks with several layers.

1975 Genetic algorithms. John Holland summarized the evolution of a population of solutions with crossover and mutation. → part of the series
1980 Expert systems. XCON (R1) configured DEC computers with rules from experts and saved millions of dollars a year. → part of the series
1986 Backpropagation. David Rumelhart, Geoffrey Hinton and Ronald Williams showed how to train multi-layer networks. → part of the series
1988 Bayesian networks. Judea Pearl: probabilistic reasoning on a graph of dependencies. → part of the series
1989 Convolutional networks. Yann LeCun trained a convolutional network to read handwritten ZIP codes. → part of the series
1989 Q-learning. Christopher Watkins: learning the value of actions without knowing the rules of the world. → part of the series

Backpropagation: how to train several layers

Definition — backpropagation

Let L ≥ 1 be the number of layers, n₀, …, n_L ≥ 1 their widths, and for l = 1, …, L let W_l ∈ ℝ^n_l×n_l−1 and b_l ∈ ℝ^n_l be the weights and biases. Let φ : ℝ → ℝ be a differentiable activation function (applied to vectors component-wise) and let the network compute

h₀ = x, z_l = W_lh_l−1 + b_l, h_l = φ(z_l) for l = 1, …, L.

Let 𝓛 : ℝ^n_L → ℝ be a differentiable loss of the output h_L for a given correct answer (we denote the loss 𝓛 here so that it is not confused with the number of layers L). The symbol ⊙ denotes the component-wise product of vectors. Backpropagation is the algorithm:

Input: the network input x ∈ ℝ^n₀, the parameters W_l, b_l (l = 1, …, L) and the loss 𝓛.
Output: the gradients ∂𝓛/∂W_l ∈ ℝ^n_l×n_l−1 and ∂𝓛/∂b_l ∈ ℝ^n_l for all l = 1, …, L.

Forward pass: for l = 1, …, L compute and store z_l and h_l.
Output error: δ_L = ∂𝓛/∂h_L ⊙ φ′(z_L) (in general δ_l = ∂𝓛/∂z_l).
Backward pass: for l = L, L − 1, …, 1 set ∂𝓛/∂W_l = δ_l h_l−1^T and ∂𝓛/∂b_l = δ_l; if l > 1, compute δ_l−1 = (W_l^Tδ_l) ⊙ φ′(z_l−1).
End: after processing layer l = 1, return all the gradients.

Step 3 is the rule for differentiating a composite function (the chain rule), and every δ_l is reused for the layer below, so the whole gradient costs about the same as one forward pass through the network.

In plain words

The network made a mistake. How much of that mistake is the fault of which weight? Backpropagation divides it up: it starts at the output and sends the blame backwards, layer by layer, according to how much each weight influenced the result.

For a single neuron with input x ∈ ℝ, weight w ∈ ℝ, bias b ∈ ℝ and correct answer y ∈ ℝ, i.e. z = wx + b, ŷ = σ(z) with the sigmoid σ(z) = 1/1 + e^−z, and with the loss L = 1/2(ŷ − y)², the chain looks like this:

∂L/∂w = ∂L/∂ŷ · ∂ŷ/∂z · ∂z/∂w

∂L/∂w: what we want: how the error changes when we move the weight — the sign tells which way to shift the weight
∂L/∂ŷ: = ŷ − y: how much the error depends on the output; this is where the backward pass starts
∂ŷ/∂z: = ŷ(1 − ŷ) for the sigmoid: how sensitive the activation is; for a saturated neuron it is almost 0 and learning stops
∂z/∂w: = x: how much the weight contributed to the sum; in a multi-layer network the output of the previous layer stands here and the chain continues further back

The gradient is the product of local derivatives along the path from the error to the weight.

Example — one learning step

x = 1, w = 0.5, b = 0, correct answer y = 1. Forward: z = 0.5, ŷ = σ(0.5) = 1/1 + e^−0.5 = 0.6225, L = 1/2 · (0.6225 − 1)² = 0.0713.

Backward (the colours match the terms of the formula above):

∂L/∂ŷ = ŷ − y = −0.3775,
∂ŷ/∂z = ŷ(1 − ŷ) = 0.6225 · 0.3775 = 0.2350,
∂z/∂w = x = 1.

Product: ∂L/∂w = −0.3775 · 0.2350 · 1 = −0.0887.

A step with η = 1: w ← 0.5 − 1 · (−0.0887) = 0.5887. The gradient is negative, so the weight grows and the output moves closer to 1.

zpětné_šíření(x, y):
    h₀ ← x                                   # forward: remember the intermediate results
    for l = 1 … L:
        zₗ ← Wₗ · hₗ₋₁ + bₗ
        hₗ ← φ(zₗ)
    δ ← ∂Ztráta/∂h_L                         # backward: start at the output error
    for l = L … 1:
        δ ← δ ⊙ φ′(zₗ)                       # through the activation (element-wise)
        ∂Ztráta/∂Wₗ ← δ · hₗ₋₁ᵀ               # gradient of this layer's weights
        ∂Ztráta/∂bₗ ← δ
        δ ← Wₗᵀ · δ                          # send the blame one layer down
    return all ∂Ztráta/∂Wₗ, ∂Ztráta/∂bₗ

In depth: A neural network from scratch.

4. Statistical learning (1993–2012)

The field changed. Instead of “let’s program intelligence” came “let’s learn it from data” — and with it probability, statistics and careful measurement. Speech recognition, spam filters, recommendation and search worked thanks to hidden Markov models, Bayesian methods, SVMs and ensembles of trees. The term “artificial intelligence” was not used much at the time — people said machine learning. Deep Blue beat Kasparov in 1997, but not by learning: by brute-force search and a hand-tuned evaluation function.

1995 SVM. Corinna Cortes and Vladimir Vapnik: classification with the largest margin and the kernel trick. → part of the series
1997 Deep Blue. An IBM computer beat world chess champion Garry Kasparov — by search, not by learning. → part of the series
1997 LSTM. Sepp Hochreiter and Jürgen Schmidhuber: a recurrent network with gates that remembers for a long time. → part of the series
2001 Random forest. Leo Breiman: an ensemble of decision trees votes. → part of the series

5. Deep learning (2012–2018)

In 2012 the ImageNet image recognition competition was won by the deep convolutional network AlexNet (Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton) by such a margin that within a few years the field moved over to neural networks. It was not a new idea — convolutional networks had existed since the late 1980s. Three things were new: large labelled data (ImageNet had over a million images), graphics cards that multiply matrices a thousand times faster, and a few small tricks (ReLU, dropout). An avalanche followed: speech, translation, games, generation.

2012 AlexNet. A deep convolutional network trained on graphics cards won the ImageNet competition by a wide margin. → part of the series
2013 word2vec. Tomáš Mikolov et al.: words as vectors in which you can compute analogies. → part of the series
2014 GAN. Ian Goodfellow: two networks pitted against each other generate images. → part of the series
2014 Attention. Dzmitry Bahdanau et al.: a translation network looks at the important words of the sentence. → part of the series
2015 DQN. DeepMind taught a single network to play dozens of Atari games from the screen. → part of the series
2016 AlphaGo. DeepMind’s program beat Lee Sedol at go 4–1; the game had been considered the domain of human intuition.
2017 Transformer. The paper “Attention Is All You Need” (Vaswani et al., Google) — the architecture of today’s language models. → part of the series
2017 PPO. John Schulman et al. (OpenAI): stable policy learning. → part of the series

Example — why ReLU helped

The derivative of the sigmoid is σ′(z) = σ(z)(1 − σ(z)) ≤ ¼ (maximum at z = 0). In backpropagation these derivatives are multiplied across all layers, so after ten sigmoid layers the gradient is at most (¼)¹⁰ = 9.5 · 10^-7 of its original size (the weights can make it larger or smaller, but this factor remains). ReLU(z) = max(0, z) has derivative 1 for z > 0 — the gradient passes through active neurons undiminished, and so deep networks can be trained.

6. Language models (since 2018)

The transformer of 2017 scaled better than anything before it. GPT models pretrained on a large part of the internet showed that a bigger model and more data bring a predictable improvement. In 2020 GPT-3 handled tasks nobody had trained it for, from just a few examples in the prompt. Fine-tuning it to human preferences produced ChatGPT (November 2022) — the first language model tried out by hundreds of millions of people. Today models are becoming agents that work on a computer on their own.

2020 GPT-3. A language model with 175 billion parameters handles new tasks from just examples in the prompt. → part of the series
2020 Diffusion models. Jonathan Ho et al.: an image emerges from noise step by step. → part of the series
2020 AlphaFold 2. DeepMind predicts the three-dimensional shape of proteins with accuracy close to experiment.
2022 ChatGPT. A language model fine-tuned to human preferences; within two months an estimated 100 million people were using it. → part of the series
2024 Nobel Prizes. Physics to John Hopfield and Geoffrey Hinton (neural networks), chemistry to Demis Hassabis and John Jumper (AlphaFold) together with David Baker.
2024 Agents. Language models with tools program, search and work on a computer on their own. → part of the series

Attention: the formula the transformer rests on

Let Q ∈ ℝ^m×d_k be the queries (each row is one query), K ∈ ℝ^n×d_k the keys and V ∈ ℝ^n×d_v the values (the i-th row of V belongs to the i-th key). Softmax is computed for each row separately: for s ∈ ℝⁿ, softmax(s)_j = e^s_j/∑_i=1ⁿ e^s_i. The result is a matrix in ℝ^m×d_v:

Attention(Q, K, V) = softmax(QK^T/√d_k) V

QK^T: dot products of every query with every key — large when the vectors point in the same direction; this is how a word “looks for” the words that concern it
√d_k: the dot product of d_k random components has a variance proportional to d_k; without the division, softmax would lock onto a single word as the dimension grows and the gradients would vanish
softmax: turns arbitrary scores into positive weights that sum to 1 — and unlike “pick the maximum” it can be differentiated, so it can be learned
V: values — the information that is actually passed on; the output is their weighted average

Each word builds a new vector as a weighted average of the others, with weights according to similarity.

Example — one query, two keys

q = (1, 0), k₁ = (1, 0), k₂ = (0, 1), v₁ = (10, 0), v₂ = (0, 10), d_k = 2.

The colours match the terms of the formula above.

Scores: q · k₁/√2 = 1/√2 = 0.707 and q · k₂/√2 = 0/√2 = 0.

Softmax: e^0.707/e^0.707 + e⁰ = 0.670 and e⁰/e^0.707 + e⁰ = 0.330.

Output: 0.670 · (10, 0) + 0.330 · (0, 10) = (6.70, 3.30). The query “looked” more at the first word because its key is similar to it — but it did not ignore the second one either.

In depth: Attention and Transformers.

The scaling law: why “more” works predictably

Let N > 0 be the number of model parameters, D > 0 the number of training tokens and E, A, B, α, β > 0 constants fitted to measurements. The scaling law (in the form of Hoffmann et al.) estimates the loss of the trained model as

L(N, D) = E + A/N^α + B/D^β

E: the irreducible loss: text itself is uncertain and not even an infinite model knows the next word for sure
A/N^α: the tax for a small model (N parameters); it falls as a power law — doubling the parameters lowers it by only 21%, which is why models grow by orders of magnitude
B/D^β: the tax for too little data (D tokens); it stands on its own, because a huge model on little data runs into this term

L is the average loss (cross-entropy) per token. The training compute is approximately C ≈ 6 · N · D operations; the law says how to split it between model size and data.

Example — Gopher versus Chinchilla

The colours match the terms of the formula above. Constants fitted on hundreds of models (Hoffmann et al., 2022): E = 1.69, A = 406.4, B = 410.7, α = 0.34, β = 0.28.

Gopher: N = 280 bn, D = 300 bn: L = 1.69 + 0.052 + 0.251 =1.993; compute 5.0 · 10²³.
Chinchilla: N = 70 bn, D = 1,400 bn: L = 1.69 + 0.083 + 0.163 =1.937; compute 5.9 · 10²³.

With comparable compute, the four times smaller Chinchilla is better: for Gopher the data term clearly outweighed the parameter term (0.251 versus 0.052), so it paid off to remove parameters and add data.

7. What history tells us

Definition — the bitter lesson

The bitter lesson is a thesis that Richard Sutton drew from seventy years of AI research in a 2019 essay. It is an empirical observation, not a mathematical statement. It compares two kinds of methods:

general methods, whose results improve with growing computing power — search and learning;
methods with built-in knowledge, into which people put their knowledge of the particular task by hand.

The thesis says: because computation keeps getting cheaper in the long run (exponentially), methods of kind 1 eventually outperform methods of kind 2. Hand-crafted knowledge helps in the short term, but eventually it is overtaken by a method that learns the knowledge it needs by itself from data and compute.

In plain words

Chess was won by search, not by rules from grandmasters. Images were won by a network that learned by itself what an edge is, not by hand-designed edge detectors. Language was won by a model that read the internet, not by a grammar written down by linguists. It is “bitter” because people try again and again to put their clever ideas into machines — and every time they are overtaken by brute force with a good general method.

And one more thing becomes visible when we look at where the ideas we explain in the series come from. Of the 37 milestones on the timeline, 30 have a part of their own:

Which decade the milestones with their own part in the series come from

Most of them (7) are from the 2010s — but the 1950s and 1980s are almost as important. The perceptron, backpropagation, convolutional networks, LSTM and Q-learning are decades old; they were waiting for data and computing power. Most of what looks like a miracle today is a clever combination of old ideas at a large scale. And every wave of enthusiasm so far has ended in disillusionment — the only question is how deep.

8. Summary

1950 the Turing test, 1956 Dartmouth (the name “artificial intelligence” comes from the 1955 proposal for the workshop), 1958 the perceptron.
The golden years were followed by two winters (roughly 1974–1980 and 1987–1993), with expert systems in between.
From the 1990s statistical learning: from data instead of rules, with measurement and probability.
2012 AlexNet and deep learning; 2017 the transformer; 2022 ChatGPT; today agents.
The bitter lesson: in the long run, general methods that exploit compute win — search and learning.
Most of today’s ideas are old; what is new is data, compute and scale.

Want to get hands-on? Ask an AI agent to write you an ELIZA program with twenty rules and chat with it. Then compare the same conversation with a modern language model — and think about how much of the difference is “intelligence”.

Do you see it differently? Write to support@coden-app.com.

A brief history of artificial intelligence: from Turing to language models

1. Beginnings: can a machine think?

2. The golden years (1956–1974)

The perceptron: the first neuron that learns

3. Two winters and expert systems (1974–1993)

Backpropagation: how to train several layers

4. Statistical learning (1993–2012)

5. Deep learning (2012–2018)

6. Language models (since 2018)

Attention: the formula the transformer rests on

The scaling law: why “more” works predictably

7. What history tells us

8. Summary

An Agent on your computer, driven from your phone