Genetic algorithms: population, crossover, mutation and selection on a backpack for a hike
Genome and fitness, tournament selection, one-point crossover, mutation and elitism, the schema theorem and premature convergence — on a knapsack problem with a known optimum, from the tent to FunSearch and AlphaEvolve.
Series AI basics Part 27 of 52
- Map of artificial intelligence: what is what and how it all fits together
- A brief history of artificial intelligence: from Turing to language models
- Data, model, training, inference: the basic terms on a single example
- How to read mathematical notation: symbols, indices, sums and quantifiers
- Sets and relations: the building blocks of definitions in AI
- Functions and mappings: why every AI model is a function
- Logic and proofs: the rules AI reasons by
- Graphs and trees: the structure that both search and neural networks stand on
- Linear algebra for machine learning: vectors, matrices and why AI mostly multiplies
- Derivatives, the gradient and the chain rule: how a model finds out which way to learn
- Gradient descent: how a model learns
- What probability is: from rolling dice to the Kolmogorov axioms
- Conditional probability and Bayes’ theorem: how to update your estimate when you learn something new
- Random variables and the cumulative distribution function: one function that tells you everything about a distribution
- Probability density and probability mass functions: why a continuous variable has zero probability at a point
- Expected value, variance and covariance: where a distribution lies, how wide it is and what goes with what
- Overview of probability distributions: Bernoulli, binomial, Poisson, uniform, normal and exponential
- Law of large numbers and central limit theorem: why averages behave so predictably
- Statistics and parameter estimation: maximum likelihood, MAP and confidence intervals
- Information theory: entropy, cross-entropy and KL divergence — where loss functions come from
- State-space search: BFS, DFS, UCS and A* with animations
- Games and adversaries: minimax, alpha-beta pruning and Monte Carlo tree search
- Constraint satisfaction: backtracking, constraint propagation and AC-3 on sudoku and a map of Czech regions
- Logic and knowledge systems: forward and backward chaining, resolution and expert systems
- Planning: STRIPS, the blocks world, heuristics from relaxation and planning in AI agents
- Local search and simulated annealing: a travelling salesman through the Czech regional capitals
- Genetic algorithms: population, crossover, mutation and selection on a backpack for a hike
- Evolution strategies and genetic programming: from the one-fifth rule to Kepler’s law
- Ant colony optimization: how a pheromone trail finds a short path
- Particle swarm optimisation: inertia, personal experience and the swarm’s experience
- Neuroevolution: evolving the weights and shape of neural networks, and the NEAT algorithm
- Fuzzy sets and membership functions: how to write down “somewhat warm”
- Fuzzy logic and probability: why “somewhat warm” does not mean “probably warm”
- Fuzzy logic: t-norms, implications and IF–THEN rules
- Fuzzy control: Mamdani and Sugeno from a fan to a pole on a cart
- Fuzzy clustering: the fuzzy c-means algorithm and points between clusters
- Neuro-fuzzy systems (ANFIS): fuzzy rules that learn from data
- Markov chains: the transition matrix, the stationary distribution and PageRank
- Hidden Markov models: the Viterbi algorithm and the dishonest casino
- Bayesian networks: why the grass is wet and what it means for the sprinkler
- The Kalman filter: how to estimate where a car is and how fast it goes from imprecise measurements
- Types of machine learning: supervised, unsupervised, semi-supervised and more
- Prompts, context and RAG: how to give a model the right information
- Variational autoencoders (VAE): new images from a continuous latent space
- Generative adversarial networks (GAN): the forger versus the detective
- Diffusion models: how pure noise becomes an image
- Markov decision processes: states, actions, rewards and the Bellman equations
- Q-learning and DQN: learning the value of actions, from a table to a neural network
- Policy gradient and PPO: learning the policy directly
- How an AI agent works: a language model in a loop with tools
- Programming with AI agents: the task, tests and review
- Evaluating and securing AI systems: how to measure and how to keep watch
Full series syllabus 52 of 83 topics published
- How to encode a solution as a genome and how selection, crossover and mutation improve a whole population.
- Why evolution converges prematurely without mutation, and what crossover is for.
- Where genetic algorithms are used today — even with a language model in the role of mutation.
Useful to know first: Local search and simulated annealing: a travelling salesman through the Czech regional capitals
Nature has no engineer, and yet it produced the eye, the wing and the brain. Three things were enough: individuals differ from one another, they pass their traits on to their offspring, and the fitter ones have more offspring. In the 1970s John Holland of the University of Michigan showed that the same three things work in a computer too — as a general method for finding good solutions to problems we do not know how to solve cleverly. A genetic algorithm needs no gradient and no formula; all it needs is to be able to say which of two solutions is better.
It follows on from the part Local search and simulated annealing. Annealing improved a single solution; a genetic algorithm improves a whole population of them at once.
1. The problem: a backpack for a hike
You are getting ready for a hike of several days. At home you have 24 things — a tent, a sleeping bag, a stove, a camera, a book… — and each of them weighs something and is good for something. The backpack may weigh at most 10 kg. What should you take so that the usefulness is as large as possible?
Definition — the (0/1) knapsack problem
Let n ∈ ℕ, n ≥ 1, be the number of items, w1, …, wn > 0 their weights, u1, …, un > 0 their values and W > 0 the capacity of the backpack. We write a selection of items as a vector x = (x1, …, xn) ∈ {0, 1}n, where xi = 1 if and only if we take the i-th item. A selection x is feasible if and only if ∑i=1n wixi ≤ W. The (0/1) knapsack problem is the problem of finding
x* ∈ argmax { ∑i=1n uixi : x ∈ {0, 1}n, ∑i=1n wixi ≤ W }.
So we look for a feasible selection with the largest total value; the “0/1” in the name means that each item can be taken only whole and at most once.
Every item is a question of “take it or leave it” — a one or a zero. There are 224 possibilities, that is 16,777,216 backpacks. The greedy strategy “most useful per kilo first” often works, but it does not guarantee the optimum: one heavy item can push out three light ones that were worth more together.
For integer weights the optimum can be computed exactly by dynamic programming — with all 24 items and a capacity of 1,000 decagrams (1 dag = 10 g) it takes 24,000 steps. The best backpack has 727 points of value and weighs 9.9 kg; the extra food, the camera, the book, the sandals and the sunscreen stay at home. Thanks to that we will see exactly how close evolution gets. A genetic algorithm, however, is useful mainly where no such exact method exists.
Dynamic programming fills in a table T[i][c] — the best value that can be reached using only the first i items when they may weigh at most c. Every cell is computed from the row above it:
T[i][c] = max( T[i − 1][c] , T[i − 1][c − wi] + ui )
- T[i][c]
- the best value from the first i items with capacity c; the cell T[n][W] is the answer to the whole problem
- T[i − 1][c]
- the option “I leave the i-th item” — then the best is whatever could be put together from the previous items with the same capacity
- T[i − 1][c − wi]
- the option “I take it” — the item takes up wi, so only c − wi is left for the rest; the best use of the remainder is already in the table (allowed only for c ≥ wi)
- ui
- the value of the item we packed in that second option
Example — the table for four items and a capacity of 50 dag
A headlamp (10 dag, 25 points), a knife (10 dag, 18), a stove (40 dag, 30) and a sleeping mat (50 dag, 45). All the weights are multiples of ten, so columns in steps of 10 dag are enough:
| items | c = 0 | 10 | 20 | 30 | 40 | 50 |
|---|---|---|---|---|---|---|
| none | 0 | 0 | 0 | 0 | 0 | 0 |
| + headlamp | 0 | 25 | 25 | 25 | 25 | 25 |
| + knife | 0 | 25 | 43 | 43 | 43 | 43 |
| + stove | 0 | 25 | 43 | 43 | 43 | 55 |
| + sleeping mat | 0 | 25 | 43 | 43 | 43 | 55 |
Knife, c = 20: T[2][20] = max(T[1][20], T[1][10] + 18) = max(25, 25 + 18) = 43
Stove, c = 40: T[3][40] = max(T[2][40], T[2][0] + 30) = max(43, 0 + 30) = 43 — the stove is not worth it
Stove, c = 50: T[3][50] = max(T[2][50], T[2][10] + 30) = max(43, 25 + 30) = 55 — the stove and the headlamp
Sleeping mat, c = 50: T[4][50] = max(T[3][50], T[3][0] + 45) = max(55, 0 + 45) = 55
(Colours as in the formula above.)
The greedy strategy would take the items by value per decagram: the headlamp (2.5 points/dag), the knife (1.8), and then neither the sleeping mat (0.9) nor the stove (0.75) would fit into the remaining 30 dag — only 43 points. In the large problem with 24 items the greedy strategy ends at 702 points: it fills the backpack with small things and there is no room left for the tent (220 dag).
def batoh(w, u, W): # w, u: weights and values, W: capacity
T = [0] * (W + 1) # the row for “no item”
for i in range(len(w)):
for c in range(W, w[i] - 1, -1): # backwards: T[c − w[i]] is still from row i − 1
T[c] = max(T[c], T[c - w[i]] + u[i])
return T[W] # n · W steps: 24 · 1000 = 24,000
2. Individual, genome and fitness
Definition — genome, population, fitness
Let a knapsack problem with n items, weights wi and capacity W in decagrams and values ui (i = 1, …, n; here n = 24) be given.
- A genome (chromosome) is an encoded solution; for the backpack it is a string x = (x1, …, xn) ∈ {0, 1}n. Its components xi — one bit per item — are called genes.
- A population of size μ ∈ ℕ is a μ-tuple of genomes P = (x(1), …, x(μ)), x(j) ∈ {0, 1}n; the same genome may appear in it several times (the order does not matter, so it is a multiset with μ elements). The elements of the population are called individuals.
- Fitness is a function F : {0, 1}n → ℝ that the algorithm maximizes. For the backpack we define it with a penalty for overloading: F(x) = ∑i=1n uixi, if ∑i=1n wixi ≤ W, F(x) = ∑i=1n uixi − 3 · (∑i=1n wixi − W), otherwise.
For a feasible backpack the fitness thus equals its value, and for every decagram over the capacity 3 points are subtracted.
An individual is one particular backpack written as a row of ones and zeros. Fitness says how good it is. We do not forbid an overloaded backpack, we only penalize it — three points off for every decagram over. The algorithm thus learns about the capacity limit gradually, and even an overloaded backpack can carry genes that will come in handy later.
F(x) = ∑ uixi − 3 · max(0, ∑ wixi − W)
- F(x)
- the fitness of backpack x — the single number by which selection compares individuals
- ∑ uixi
- the value of the packed items (xi = 1 counts the item, 0 leaves it out) — what we really want to maximize
- 3
- the price of one decagram over the limit — it is greater than the best ratio of value to weight (the headlamp, 2.5 points/dag), so packing anything into an already overloaded backpack always lowers its fitness
- max(0, ∑ wixi − W)
- the overload in decagrams; the max with zero makes sure that a backpack that fits gets no penalty and a lighter backpack is not favoured
Example — the fitness of two backpacks
Optimum: value 727, weight 990 dag ≤ 1,000 ⇒ F = 727 − 3 · 0 = 727
Optimum + camera (80 dag, 20 points): value 747, weight 1,070 dag ⇒ F = 747 − 3 · 70 = 537
(Colours as in the formula above.)
The camera adds 20 points, but the overload of 70 dag costs 210 — selection reliably pushes such a backpack aside.
3. Selection
Definition — tournament and roulette-wheel selection
Let P = (x(1), …, x(μ)) be a population of size μ and F : {0, 1}n → ℝ the fitness. Selection is a random procedure that returns one individual (a parent) from P:
- Tournament selection of size k ∈ ℕ, k ≥ 1: draw indices i1, …, ik ∈ {1, …, μ} independently and uniformly (with replacement, the same index may come up several times) and return the individual x(i*), where i* ∈ argmaxj ∈ {i1, …, ik} F(x(j)) (on a tie in fitness, any of the best).
- Roulette-wheel selection (assumes F(x(j)) > 0 for all j): return the individual x(i), i ∈ {1, …, μ}, with probability P(x(i) selected) = F(x(i))/∑j=1μ F(x(j)).
Selection pressure is a qualitative notion: how strongly selection favours fitter individuals. It can be measured, for example, by the ratio of the expected number of selections of the best and of an average individual (see the example below); for a tournament it grows with k.
A tournament: you draw three random backpacks and the best of them becomes a parent. Better backpacks thus have more children, but even an average one has a chance when it meets weaker ones. If only the very best reproduced, the population would quickly degenerate into copies of a single individual — and with it would vanish the diversity that evolution lives on.
How strong a tournament is can be computed exactly. Our program draws k individuals with replacement (the same individual can get into the tournament twice). Let us sort the population from the best (i = 1) to the worst (i = μ):
P(i-th wins) = (μ − i + 1)k − (μ − i)k/μk
- P(i-th wins)
- the chance that it is exactly the i-th best individual that becomes a parent
- (μ − i + 1)k
- the number of tournaments in which nobody is better than the i-th — all k draws fall on it or on worse ones
- (μ − i)k
- from these we subtract the tournaments in which the i-th is not present at all — those would be won by someone worse; the difference is the tournaments where the i-th is the best one present
- μk
- the number of all possible tournaments (k draws, each out of μ individuals) — dividing by it gives a probability
Example — a tournament of three among 40 backpacks
μk = 403 = 64,000. Each generation selects 38 parents (two places are taken by the elite individuals); colours as in the formula above:
the best: 403 − 393/64,000 = 64,000 − 59,319/64,000 ≈ 7.31%, expected 38 · 0.0731 ≈ 2.78 selections
the 20th best: 213 − 203/64,000 = 9,261 − 8,000/64,000 ≈ 1.97%, expected ≈ 0.75 selections
the worst: 13 − 03/64,000 ≈ 0.0016%, expected ≈ 0.0006 selections
An average individual gets 38/40 = 0.95 selections, the best almost three times as many. In general, for a large population the best one gets roughly k times more than the average — that is why selection pressure grows with k.
4. Crossover and mutation
Definition — one-point crossover
Let n ≥ 2, let a = (a1, …, an) and b = (b1, …, bn) ∈ {0, 1}n be the genomes of the parents and pc ∈ [0, 1] the crossover probability. One-point crossover creates two offspring as follows:
- With probability 1 − pc no crossover takes place and the offspring are copies of the parents a and b.
- Otherwise draw a crossover point c ∈ {1, …, n − 1} uniformly, and the offspring are a′ = (a1, …, ac, bc+1, …, bn), b′ = (b1, …, bc, ac+1, …, an).
The offspring thus take the first c genes from one parent and the remaining n − c from the other. Other variants are two-point crossover (two points, the section between them is swapped) and uniform crossover, where for every i = 1, …, n independently, with probability 1/2, a′i = ai, b′i = bi, and otherwise a′i = bi, b′i = ai.
You cut two backpacks at the same place and swap the ends. The child gets the beginning of the list from one parent and the end from the other. If one parent solved sleeping well and the other food, the child has a chance to inherit both.
Definition — bit-flip mutation
Let x = (x1, …, xn) ∈ {0, 1}n be a genome and pm ∈ [0, 1] the mutation probability. Bit-flip mutation creates a genome x′ ∈ {0, 1}n so that for every i = 1, …, n, independently of the other genes,
x′i = 1 − xi with probability pm, x′i = xi with probability 1 − pm.
Each gene is thus flipped (0 ↔ 1) with probability pm. Usually pm = 1/n is chosen, so on average n · pm = 1 gene per individual is flipped.
Now and then a child differs from its parents in one small detail: it packs an item that neither parent had, or takes one out. Mutation is the only source of completely new genes — crossover can only rearrange what is already in the population.
Example — how many genes mutation flips
The number of flipped genes has a binomial distribution: P(j genes) = (n choose j) · pmj · (1 − pm)n−j. For n = 24 and pm = 1/24:
expected number: n · pm = 24 · 1/24 = 1
no change: (23/24)24 ≈ 0.360 (for large n it approaches 1/e ≈ 0.368)
exactly one gene: 24 · 1/24 · (23/24)23 ≈ 0.376
So roughly a third of the offspring stay unchanged, a third differ in one gene and the rest in two or more.
The parents are the two best backpacks of the initial population (423 and 398 points), and the cut is after the twelfth item. Neither offspring is better than the better parent — they have 403 and 418 points. Crossover does not guarantee an improvement; it only offers new combinations, and selection then decides which survive. In the first offspring, mutation packed a water filter (15 dag, 32 points), and it suddenly has 435 points — more than both parents.
Example — recomputing the crossover from the figure
Value adds up gene by gene, so it is enough to add up the value of the beginning (items 1–12) and of the end (items 13–24) separately:
parent 1: 310 + 113 = 423, parent 2: 305 + 93 = 398
offspring 1 = beginning of parent 1 + end of parent 2: 310 + 93 = 403
offspring 2 = beginning of parent 2 + end of parent 1: 305 + 113 = 418
mutation packs the water filter: 403 + 32 = 435
Parent 1 is better in both halves (310 > 305 and 113 > 93), so with this cut no offspring can beat it — crossover has nothing to build a better whole from. The improvement only came with mutation.
5. The whole algorithm
Algorithm — genetic algorithm
Input: genome length n ≥ 2, fitness F : {0, 1}n → ℝ, population size μ ∈ ℕ, number of elite individuals e ∈ {0, …, μ − 1}, tournament size k ≥ 1, crossover probability pc ∈ [0, 1], mutation probability pm ∈ [0, 1] and number of generations G ∈ ℕ. Output: the fittest individual of the last generation; for e ≥ 1 it is also the fittest individual the algorithm has ever evaluated.
- Create a random initial population P0 of μ individuals (in our case each gene has the value 1 with probability 0.3), evaluate them with F and set t ← 0.
- If t = G, stop and return the individual of Pt with the highest fitness.
- Copy the e fittest individuals of Pt into the new generation Pt+1 unchanged (elitism).
- While Pt+1 has fewer than μ individuals: select two parents from Pt by a tournament of size k, cross them over by one-point crossover with probability pc, mutate both offspring by bit-flip mutation with pm and add them to Pt+1; if in the end there is one individual over μ, discard it.
- Evaluate the individuals of Pt+1, set t ← t + 1 and continue with step 2.
The algorithm stops in step 2 after exactly G generations.
Generation after generation: the best two survive directly, and the other places are filled by the children of fit parents, a little rearranged and a little altered. Elitism guarantees that the best backpack found is never lost.
import random
def evoluce(zdatnost, n, velikost=40, generaci=60, p_krizeni=0.9, elita=2):
pop = [[int(random.random() < 0.3) for _ in range(n)] for _ in range(velikost)]
for _ in range(generaci):
pop.sort(key=zdatnost, reverse=True)
dalsi = [g[:] for g in pop[:elita]] # elitism
while len(dalsi) < velikost:
a, b = turnaj(pop, zdatnost), turnaj(pop, zdatnost)
if random.random() < p_krizeni:
bod = random.randrange(1, n)
a, b = a[:bod] + b[bod:], b[:bod] + a[bod:]
dalsi += [mutuj(a), mutuj(b)]
pop = dalsi[:velikost]
return max(pop, key=zdatnost)
def turnaj(pop, zdatnost, k=3):
return max(random.choices(pop, k=k), key=zdatnost) # k draws with replacement
def mutuj(g):
return [1 - x if random.random() < 1 / len(g) else x for x in g]
We measure the cost of the algorithm by the number of fitness evaluations — in real problems (simulations, measurements) that is the most expensive step. The initial population is evaluated and then every generation: μ + G · μ = 40 + 60 · 40 = 2,440 evaluations for μ = 40 individuals and G = 60 generations. The elite individuals are evaluated again even though they have not changed; a smarter program would remember their fitness.
6. The evolution of a backpack
Let us run the algorithm: 40 backpacks, 60 generations, a tournament of three, crossover with probability 0.9, mutation 1/24 and two elite individuals. The figure shows the population in generations 0, 5, 15, 30 and 60 — each row is one backpack, each column one item, and at the top the optimum for comparison:
At the start the population is varied and the backpacks are rather half-empty: each item got into them with probability 0.3, so they weigh around four kilos (the expected weight is 0.3 · ∑ wi = 0.3 · 12.75 kg ≈ 3.8 kg). The best has 423 points, the average 231. After five generations 36 of the 40 backpacks carry the tent, 35 the water bottle, and the best backpack has 634 points. Around the fifteenth generation the population hits the capacity — 12 backpacks are overloaded — and starts learning what to sacrifice: the extra food and the sandals have almost disappeared, but the camera is still carried by 39 backpacks. The algorithm first finds the optimum, 727 points, in generation 26; by generation 30 the camera is left in only one backpack. The population never merges into a single solution, though — mutation keeps trying small deviations.
To get there the algorithm evaluated 2,440 backpacks — not quite two hundredths of a percent of all 16,777,216 possibilities. Random search with the same number of attempts did not find the optimum even once in a hundred repetitions and ended at 574 points on average.
Example — why random search does not find the optimum
A random backpack takes each item independently with probability 0.3. The optimum contains 19 items and leaves out 5, so
P(random backpack = optimum) = 0.319 · 0.75 ≈ 1.16 · 10−10 · 0.168 ≈ 1.95 · 10−11
expected number of hits in 2,440 attempts: 2,440 · 1.95 · 10−11 ≈ 4.8 · 10−8
Even if the draw were uniform (each item with probability 1/2), the chance would be one in 16,777,216 per attempt. A genetic algorithm does not draw the optimum all at once — it assembles it piece by piece from backpacks that are already good.
7. What crossover is for and what mutation is for
The orange line in the chart is the same run with mutation switched off. After eleven generations it gets stuck at 573 points and does not move any more. Why, a comparison of a hundred runs with different seeds shows:
| Variant | Found the optimum | Average value |
|---|---|---|
| Selection + crossover + mutation | 88 out of 100 | 726.6 |
| Selection + mutation (no crossover) | 70 out of 100 | 725.0 |
| Selection + crossover (no mutation) | 0 out of 100 | 596.8 |
| Random search, same number of attempts | 0 out of 100 | 573.5 |
Without mutation the population loses its diversity: when a gene is not in any individual, crossover cannot produce it, and on top of that selection quickly pushes out everything but the best. This is called premature convergence. Without crossover the algorithm works — it is really a population of hill climbers with selection — but it is slower. Crossover helps to put together good pieces that arose in different individuals.
Theorem — Holland’s schema theorem
Let n ≥ 2. A schema is a string H = (H1, …, Hn) ∈ {0, 1, ∗}n, where ∗ means “anything”. A genome x ∈ {0, 1}n matches the schema H if and only if xi = Hi for all i with Hi ≠ ∗. The positions i with Hi ≠ ∗ are fixed; assume at least one exists. Define
- the order o(H) = |{i : Hi ≠ ∗}| — the number of fixed positions,
- the defining length δ(H) = max{i : Hi ≠ ∗} − min{i : Hi ≠ ∗} — the distance between the first and the last fixed position.
Let a genetic algorithm with population size μ and positive fitness F > 0 form each whole new generation by roulette-wheel selection, one-point crossover with probability pc and bit-flip mutation with probability pm (without elitism). Let m(H, t) be the number of individuals of generation t that match H, and if m(H, t) > 0, let F(H) be their average fitness and F̄ the average fitness of the whole generation t. Then for the expected number in the next generation (given generation t)
E[m(H, t + 1)] ≥ m(H, t) · F(H)/F̄ · [1 − pc · δ(H)/n − 1 − o(H) · pm].
In words: selection multiplies the number of carriers of the schema by the ratio of their fitness to the average, and crossover and mutation subtract at most the losses shown.
A schema is a “building block”, for example “I take the tent and the sleeping bag, the rest does not matter”. The theorem says that short building blocks with above-average fitness spread in the population — and exponentially fast, because every generation their number is roughly multiplied by the ratio F(H)/F̄. Short, because crossover cuts them less often. The theorem is only a lower bound, and how well the algorithm works is still debated today.
E[m(H, t + 1)] ≥ m(H, t) · F(H)/F̄ · [1 − pc · δ(H)/n − 1 − o(H) · pm]
- m(H, t)
- how many individuals carry the building block now — the next generation starts from them, which is why it is multiplied by this
- F(H)/F̄
- how much fitter than average the carriers of the schema are — the roulette wheel gives them proportionally more offspring; above 1 the schema multiplies, below 1 it dies out
- pc · δ(H)/n − 1
- the chance (an upper bound) that crossover cuts the schema: crossover happens with pc and the cut point falls between the fixed positions with probability δ(H)/n − 1 — that is why long schemata are a problem
- o(H) · pm
- the chance (an upper bound) that mutation flips one of the o(H) fixed positions — that is why high-order schemata are a problem
Example — a short and a long schema on the backpack
The square bracket for the settings of our run (pc = 0.9, n = 24, pm = 1/24). Both schemata have order 2; colours as in the formula above:
“the tent and the sleeping bag” (genes 1 and 2, δ = 1): 1 − 0.9 · 1/23 − 2 · 1/24 ≈ 1 − 0.039 − 0.083 = 0.878
“the tent and spare socks” (genes 1 and 24, δ = 23): 1 − 0.9 · 23/23 − 2 · 1/24 ≈ 1 − 0.9 − 0.083 = 0.017
For the short schema it is enough for its carriers to be 14% fitter than average (1/0.878 ≈ 1.14), and according to the theorem it already spreads. The long schema would have to be sixty times fitter — the theorem promises essentially nothing about it. Our algorithm uses a tournament, not a roulette wheel, so only the square bracket applies to it; the first two factors would be computed differently.
8. Genetic algorithms in practice
- Designing shapes. The antenna of NASA’s ST5 satellites (launched in 2006) was designed by an evolutionary algorithm: an oddly twisted piece of wire that would not have occurred to an engineer, but met the specification at least as well as a hand-designed antenna (and in later experiments even better).
- Timetables and logistics. Timetables, shift plans or warehouse layouts — problems with many constraints, where fitness can be computed but an exact algorithm would run for ages.
- Machine learning. Evolution tunes the hyperparameters of models and, in neuroevolution, which has its own part in the syllabus, even the weights and architectures of neural networks.
- Evolution with language models. Systems such as FunSearch (2023) and AlphaEvolve (2025) from Google DeepMind maintain a population of programs and use as mutation a language model that modifies a program. Selection is provided by automatic evaluation. This way new mathematical constructions and faster algorithms have been found.
9. Summary
- A genetic algorithm improves a whole population of solutions: by selecting the fit ones, by crossover and by mutation.
- Fitness is the only thing it needs to know about the problem; violated constraints can be handled with a penalty.
- Tournament selection keeps up the selection pressure, and elitism does not allow the best solution to be lost.
- On a backpack with 16.7 million possibilities it found the optimum in 88 out of 100 runs after evaluating 2,440 backpacks; random search with the same budget not even once.
- Mutation supplies diversity — without it the population converges prematurely. Crossover assembles good building blocks and speeds evolution up.
Want to get your hands on it? Have an AI agent write a genetic algorithm for your own backpack for a hike — and then try switching off mutation, enlarging the tournament or shrinking the population, and watch what happens to the diversity.
Do you see it differently? Write to support@coden-app.com.