Neuroevolution: evolving the weights and shape of neural networks, and the NEAT algorithm

Neurons and networks in a nutshell, evolving a controller for the cart-pole, why evolution instead of the gradient, XOR and hidden neurons, NEAT with historical markings and speciation, what happens without speciation, and successors from HyperNEAT to AmoebaNet.

15 minutes Matěj Žalmánek 2026-10-15

Series AI basics Part 31 of 52

Full series syllabus 52 of 83 topics published

What you will take away

How evolution finds the weights of a network when we only learn about success at the end.
Why tuning weights is not enough for XOR and how NEAT lets a network grow from a minimum.
Why NEAT barely works without speciation and where neuroevolution leads today.

Useful to know first: Genetic algorithms: population, crossover, mutation and selection on a backpack for a hike, Evolution strategies and genetic programming: from the one-fifth rule to Kepler’s law

Today, neural networks are trained almost exclusively by gradient descent: the network makes an error, we compute which way to nudge each weight, and we nudge it. But what if the error cannot be differentiated — when we only learn about success at the end of the game, or when we do not even know what the network should look like? That is where evolution comes in. Neuroevolution lets evolutionary algorithms breed the weights of networks and, in its most interesting form, their shape as well. This part closes the module on nature-inspired algorithms and at the same time opens the door to neural networks.

It follows on from Genetic algorithms and Evolution strategies and genetic programming.

1. Neural networks in a nutshell

Definition — artificial neuron and network

Let n ∈ ℕ, n ≥ 1, let w = (w₁, …, w_n) ∈ ℝⁿ be the weights, b ∈ ℝ the bias and σ : ℝ → ℝ a function we call the activation function. A neuron with these parameters is the function y : ℝⁿ → ℝ that assigns to inputs x = (x₁, …, x_n) the value

y(x) = σ(w₁x₁ + … + w_nx_n + b) = σ(∑_i=1ⁿ w_ix_i + b).

Common activation functions are the sigmoid σ(z) = 1/1 + e^−z and the step σ(z) = 1 for z > 0, σ(z) = 0 for z ≤ 0.

A neural network is a directed graph G = (V, E), where

the set of nodes V = V_in ∪ V_N is the disjoint union of the network’s inputs V_in and its neurons V_N, and no edge leads into an input;
every edge (u, v) ∈ E leads into a neuron v ∈ V_N and has a weight w_uv ∈ ℝ;
every neuron v ∈ V_N has a bias b_v ∈ ℝ and an activation function σ_v, and its value is y_v = σ_v(∑_{u : (u, v) ∈ E} w_uv y_u + b_v), where for an input u ∈ V_in, y_u is the given input value;
a non-empty set V_out ⊆ V_N are the output neurons; their values are the output of the network.

A network is feedforward if and only if the graph G has no directed cycle — then the values of the neurons can be computed one after another in topological order. The neurons in V_N ∖ V_out (neither inputs nor outputs) are hidden.

In plain words

A neuron adds up its inputs, each multiplied by its weight, and depending on the sum it either “lights up” or not. The weights decide what it cares about. A network is many neurons wired one after another. The whole behaviour of the network lies in the numbers of its weights and in who is connected to whom — and both can be bred by evolution. Neural networks and their training by gradient have a whole separate module in the syllabus.

y = σ( ∑_i=1ⁿ w_i x_i + b )

x_i: inputs — what the neuron “sees” (for the pole: position, velocity, angle); the neuron does not change them, it only computes with them
w_i: weights — the only thing that is learned or evolved; the sign says whether an input excites or inhibits the neuron, the magnitude how much it matters
b: bias — shifts the decision boundary; without it, zero inputs would always give the same σ(0) and the boundary would have to pass through the origin
σ: activation function — the nonlinearity; without it, even a network with many layers would compute only a linear function, because a composition of linear functions is again linear

The sum ∑ w_ix_i + b is a weighted vote of the inputs; σ turns it into the output. The boundary between “lights up” and “does not” is the set where the sum is zero — a line (in the plane), in general a hyperplane.

2. Evolving weights: the pole on a cart

A classic control task: a pole stands on a hinge on a cart, and the cart has to keep it upright by moving left and right — like balancing a broom on the palm of your hand.

Definition — the cart-pole task

The state is a quadruple s = (x, v, θ, ω) ∈ ℝ⁴: the position x (m) and velocity v (m/s) of the cart, the angle θ (rad) of the pole from the vertical and its angular velocity ω (rad/s). The cart has a mass of 1 kg, the pole 0.1 kg and a length of 1 m.
Action: in every step t = 0, 1, 2, … (a step lasts 0.02 s) the controller chooses a force F_t ∈ {−10 N, +10 N}.
Transition: s_t+1 is computed from s_t and F_t by the equations of motion below.
End of a trial: the trial ends after step t if and only if |θ_t+1| > 12° ∨ |x_t+1| > 2.4 m (the pole fell or the cart ran off the track), or when 1,000 steps (20 s) have passed. The goal is to last 1,000 steps.

The controller will be a single neuron with a step activation: for weights (w₁, w₂, w₃, w₄, b) ∈ ℝ⁵

u(s) = w₁x + w₂v + w₃θ + w₄ω + b, F(s) = +10 N for u(s) > 0, F(s) = −10 N for u(s) ≤ 0.

So the controller pushes the cart in the direction of positive x if and only if the weighted sum of the state is positive. The fitness of the controller (the average number of steps from three different starts) is defined below.

In plain words

Five numbers decide when to jerk the cart to the left and when to the right. No teacher tells us which force was right at which moment — we only learn how long the pole stayed up. Exactly this kind of situation is hard to handle with a gradient and easy with evolution.

Definition — equations of motion of the cart-pole

Let s = (x, v, θ, ω) be the state (the angle θ is measured from the vertical) and F ∈ {−10, +10} N the controller’s force; a positive force pushes the cart in the direction of positive x. Constants: the mass of the cart m_v = 1 kg, the mass of the pole m_t = 0.1 kg, their sum M = m_v + m_t = 1.1 kg, l = 0.5 m half the length of the pole (the distance of its centre of mass from the hinge) and the gravitational acceleration g = 9.8 m/s². Without friction (the equations in the form given by Barto, Sutton and Anderson, 1983, and still used by most implementations), the auxiliary acceleration a_F, the angular acceleration of the pole ε and the acceleration of the cart a are given by

a_F = F + m_t l ω² sin θ/M

ε = g sin θ − cos θ · a_F/l (4/3 − m_t cos² θ/M)

a = a_F − m_t l ε cos θ/M

The simulation proceeds by the Euler method with step Δt = 0.02 s: from the state s_t and the force F_t, a and ε are computed and the new state is

x_t+1 = x_t + Δt v_t, v_t+1 = v_t + Δt a, θ_t+1 = θ_t + Δt ω_t, ω_t+1 = ω_t + Δt ε,

i.e. the new position is the old one plus Δt times the old velocity, and the new velocity is the old one plus Δt times the acceleration.

ε = g sin θ − cos θ · a_F/l (4/3 − m_t cos² θ/M)

g sin θ: gravity — the reason the pole falls: the more it leans, the harder gravity tips it further; in the upright position it is zero, so upright is an equilibrium, but an unstable one
a_F: the acceleration of the cart from the controller’s force (plus a small centrifugal force of the pole) — the only way the controller acts on the pole: the cart slides under the pole and the hinge “nudges” it back
cos θ: how much of the horizontal nudge rotates the pole — almost all of it for an upright pole, none for a lying one; that is why a pole tilted by more than a few degrees is hard to save
l (4/3 − …): the inertia of the pole — 4/3 comes from the moment of inertia of a uniform rod about its end, 1/3 m(2l)² = 4/3 ml²; the subtracted term corrects for the fact that the cart beneath also jerks when the pole rotates

The angular acceleration of the pole is a tug of war between two terms: gravity tips the pole over, the acceleration of the cart brings it back.

Example — one simulation step

Start (0, 0, 0.05, 0), i.e. the pole tilted by 0.05 rad ≈ 2.9°; cos 0.05 ≈ 0.9988 and sin 0.05 ≈ 0.0500. Colours as in the breakdown of the formula for ε above. Denominator: 0.5 · (4/3 − 0.1 · 0.9988²/1.1) ≈ 0.621.

no force (F = 0, hence also a_F = 0): ε = 9.8 · 0.0500/0.621 ≈ +0.79 rad/s² — the pole starts to fall

force +10 N: a_F = 10/1.1 ≈ 9.09 m/s², ε = 0.490 − 0.999 · 9.09/0.621 ≈ −13.8 rad/s²

after the step: ω = 0 + 0.02 · (−13.8) ≈ −0.28 rad/s, v ≈ 0 + 0.02 · 9.72 ≈ 0.19 m/s

A force of ±10 N is huge compared with gravity — a single jerk reverses the direction of the fall. So the task is not about force but about timing: when to stop pushing so that the pole does not swing over to the other side.

How far does chance get us? Out of a thousand controllers with random weights, only one lasted all 20 seconds; the median is 9 steps — the pole falls in less than a fifth of a second. A straightforward controller that simply moves wherever the pole leans lasts 39 steps: it does catch the pole, but sets it oscillating. The (5, 20) evolution strategy from the part on evolution strategies found a controller that lasts the full 20 seconds from all three starts in generation 15 — after trying roughly three hundred controllers:

The pole on a cart: the first 8 seconds of the simulation

The evolved controller gives the largest weight to the angular velocity — it does not catch the pole only once it is falling, but already when it starts to tip. The angle and the velocity of the cart also carry non-negligible weights, so the cart does not oscillate and does not run off the track. Nobody wrote such a “rule”; it fell out of evolution.

Definition — controller fitness and weight evolution

Let w = (w₁, …, w₄, b) ∈ ℝ⁵ be the weights of the controller from the cart-pole task and s⁽¹⁾ = (0, 0, 0.05, 0), s⁽²⁾ = (0, 0, −0.08, 0), s⁽³⁾ = (0.5, 0, 0.02, 0.1) three starting states. For k ∈ {1, 2, 3} denote by T_k(w) ∈ {1, …, 1,000} the number of steps a trial from the start s^(k) lasts with this controller (until the pole falls or the cart runs off, at most 1,000). The fitness of the controller is

Z(w) = T₁(w) + T₂(w) + T₃(w)/3 ∈ [1, 1,000].

Three starts instead of one keep evolution from finding a controller that can handle only one initial position. Z is piecewise constant — a small change of the weights usually does not change the number of steps at all, so the gradient is zero almost everywhere. Hence evolution.

Weight evolution by the (5, 20) evolution strategy (see the part Evolution strategies and genetic programming): the input is the function Z and a number of generations G ∈ ℕ, the output the best weights found.

Choose 5 parents w⁽¹⁾, …, w⁽⁵⁾ ∈ ℝ⁵, each of their components independently and uniformly from [−1, 1].
In every generation g = 1, …, G create 20 offspring: for each, choose a parent w uniformly at random from the five and set w′ = w + 0.5 · z, where z ∈ ℝ⁵ has independent components with the standard normal distribution N(0, 1) (see the part Overview of probability distributions).
The new parents are the 5 offspring with the largest fitness Z; the old parents do not pass on to the next generation.
After G generations return the weights with the largest fitness that appeared during the run.

import random

def regulator(w, s):                 # s = (x, v, θ, ω)
    u = w[0]*s[0] + w[1]*s[1] + w[2]*s[2] + w[3]*s[3] + w[4]
    return 10 if u > 0 else -10       # step activation: force ±10 N

def zdatnost(w):                     # simuluj(): steps until the fall, at most 1000
    return sum(simuluj(regulator, w, start) for start in STARTY) / 3

rodice = [[random.uniform(-1, 1) for _ in range(5)] for _ in range(5)]
for generace in range(30):
    deti = [[wi + 0.5 * random.gauss(0, 1) for wi in random.choice(rodice)]
            for _ in range(20)]
    deti.sort(key=zdatnost, reverse=True)
    rodice = deti[:5]                # (5, 20): only the offspring survive

Example — how the evolved controller decides

The weights found by evolution (rounded): w = (0.05, 1.01, 1.37, 4.23), b = 0.09 for (x, v, θ, ω). Colours as in the breakdown of the neuron at the beginning of the part: weights, inputs, bias.

pole tilted, at rest: (0, 0, 0.05, 0) → u = 1.37 · 0.05 + 0.09 ≈ +0.16 → +10 N, the cart slides under the pole

pole tilted, but already coming back: (0, 0, 0.05, −0.1) → u = 1.37 · 0.05 + 4.23 · (−0.1) + 0.09 ≈ 0.07 − 0.42 + 0.09 ≈ −0.26 → −10 N, the cart brakes

In the second case the pole is tilted just as much, but the controller pushes the other way: the angular velocity −0.1 rad/s with weight 4.23 outweighs the angle 0.05 rad with weight 1.37. The pole is coming back on its own, and if the cart kept pushing, it would swing over. The straightforward “angle only” controller would push the same way in both states — which is why it sets the pole oscillating.

3. Why evolution and not the gradient

Sparse reward. With a pole, a game or a robot, quality only shows after a whole episode. Evolution needs just one number at the end.
Non-differentiable parts. A step activation, a physics simulator, the rules of a game — evolution does not need to differentiate them.
Parallelism. Each individual can be evaluated independently on a different computer.
Structure. A gradient can tune weights, but it cannot add a neuron. Evolution can change what the network looks like, too.

The drawback is the appetite: instead of one gradient that gives the direction for all weights at once, evolution tries random changes. For networks with millions of weights the gradient tends to be far more economical — if it can be computed at all.

4. When weights are not enough: XOR

The XOR function returns 1 when exactly one of its two inputs is 1. As the part Logic and proofs showed, a single neuron cannot compute it: its output splits the input plane with a line, and no line separates the points (0, 1), (1, 0) from (0, 0), (1, 1). XOR needs at least one hidden neuron. And that raises a question that evolving weights alone cannot answer: how many neurons, and how should they be wired?

Theorem — XOR is not linearly separable

Let XOR : {0, 1}² → {0, 1} be the function given by XOR(x₁, x₂) = 1 ⇔ x₁ ≠ x₂. Then there are no numbers w₁, w₂, b ∈ ℝ such that for all (x₁, x₂) ∈ {0, 1}²

w₁x₁ + w₂x₂ + b > 0 ⇔ XOR(x₁, x₂) = 1.

So no line separates the points where XOR equals 1 from the points where it equals 0.

Proof. If such numbers existed, we would have

(0, 0): b ≤ 0 (1, 1): w₁ + w₂ + b ≤ 0

(1, 0): w₁ + b > 0 (0, 1): w₂ + b > 0

Adding the first row gives w₁ + w₂ + 2b ≤ 0, adding the second gives w₁ + w₂ + 2b > 0. Contradiction. ∎

The proof does not use the shape of the activation function — it holds for every single neuron whose output depends only on the sign of the weighted sum (a step, or a sigmoid thresholded at 0.5).

The question can be answered by hand — design an architecture and let evolution tune only the weights. But for harder tasks we have no idea which architecture is right. Why not let it evolve too?

5. NEAT: evolving growing networks

Attempts to evolve the structure of networks ran into three problems. How do you cross over two networks of different shapes? How do you give a chance to a new structure that is worse at first because its new weights are not yet tuned? And how do you keep networks from bloating needlessly? In 2002 Kenneth Stanley and Risto Miikkulainen answered with the NEAT algorithm (NeuroEvolution of Augmenting Topologies), built on three ideas.

Definition — the NEAT genome and historical markings

Let the nodes of the network (inputs, bias, outputs and hidden neurons, see the definition of a neural network above) be labelled with natural numbers. A connection gene is a quintuple (u, v, w, e, i), where u is the source and v the target node of the connection u → v, w ∈ ℝ its weight, e ∈ {0, 1} a flag (e = 1: the connection is enabled and counts in the network, e = 0: disabled) and i ∈ ℕ its innovation number. A genome is a finite set of connection genes in which different genes have different innovation numbers; it determines the network whose edges are the enabled connections.

Innovation numbers are assigned by a single counter for the whole population: whenever a mutation creates the connection u → v for the first time, it receives the next free number; if the same structural change (a connection between the same nodes) arises in another individual, it receives the same number.

There are three mutations of a genome:

changing weights — the weights w of genes change (by adding a random number or by a new random value);
adding a connection — a gene (u, v, w, 1, i) with a random weight w is added for two nodes u, v not yet connected (our program adds a connection only if no cycle arises, so the network stays feedforward);
adding a neuron — for a chosen enabled gene (a, b, w, 1, i) a new hidden neuron h is created, the gene is disabled (e ← 0) and the genes (a, h, 1, 1, i′) and (h, b, w, 1, i″) with new innovation numbers are added.

Crossover of two parents lines up their genes by innovation number. The offspring receives each matching gene (the innovation number is in both parents) from a parent chosen at random with equal probability, and each other gene (it is in only one of the parents) only if it is in the genome of the fitter parent.

In plain words

The innovation number is a connection’s birth certificate. When two networks both have a connection numbered 7, it is “the same” connection, even if each got there by a different route — and in crossover they can be paired up as the same gene in two parents. A new neuron is inserted so that the network initially computes much as it did before; nothing essential breaks, only a new possibility appears.

Example — adding a neuron on a connection with weight 2

Like the original paper, our NEAT uses a steeper sigmoid σ(z) = 1/1 + e^−4.9z. The connection x₁ → y with weight 2 contributed 2x₁ to the output. After the split, x₁ → h has weight 1 and h → y has weight 2:

x₁ = 1: 2 · σ(1) = 2 · 0.993 ≈ 1.985 (previously 2)

x₁ = 0: 2 · σ(0) = 2 · 0.5 = 1 (previously 0)

So “much as before” holds only in part: for input 1 the change is negligible, for input 0 a one is added to the sum, because the sigmoid returns one half at zero. The network gets slightly worse — and that is exactly why a new structure needs the protection discussed next.

Definition — speciation and fitness sharing

Let G₁, G₂ be non-empty genomes (see the definition of a genome above) and I₁, I₂ the sets of innovation numbers of their genes. Denote m = min(max I₁, max I₂) and

matching genes: the innovation numbers in I₁ ∩ I₂;
disjoint genes: the numbers i ∈ (I₁ ∖ I₂) ∪ (I₂ ∖ I₁) with i ≤ m (missing from one of the genomes “in the middle”); D is their count;
excess genes: the numbers i ∈ (I₁ ∖ I₂) ∪ (I₂ ∖ I₁) with i > m (beyond the end of the shorter genome); E is their count;
W̄ is the average of |w_i⁽¹⁾ − w_i⁽²⁾| over the matching genes i (the difference of the weights of the same gene in the two genomes; if there are no matching genes, W̄ = 0);
N = max(|G₁|, |G₂|) is the number of genes of the larger genome if it is at least 20, otherwise N = 1.

For constants c₁, c₂, c₃ ≥ 0 the compatibility distance of the genomes G₁, G₂ is the number

δ(G₁, G₂) = c₁ E/N + c₂ D/N + c₃ W̄.

Speciation: let δ_t > 0 be a threshold and let every species have a representative. A genome belongs to the first species whose representative Z satisfies δ(genome, Z) < δ_t; if there is none, it founds a new species. The species S₁, …, S_K thus divide a population of size P into disjoint non-empty parts.

Fitness sharing: if an individual i ∈ S_k has fitness f_i ≥ 0, its shared fitness is f′_i = f_i/|S_k|, species k has F̄_k = ∑_{i ∈ S_k} f′_i (the average fitness of its members) and receives

n_k = P · F̄_k/∑_j=1^K F̄_j

offspring (rounded down; the remainder goes to the species with the best network). So similar genomes form a species, an individual’s fitness is divided by the size of its species, and offspring are allotted to species according to their average fitness.

In plain words

Networks with a similar build compete mainly among themselves, not with the whole population. A new neuron thus gets a few generations for its weights to be tuned, instead of being swept away at once by established networks. And because fitness is shared within a species, no species can take over the whole population.

δ = c₁ E/N + c₂ D/N + c₃ W̄

c₁ E: excess genes — structure that one genome acquired only after the last shared “invention”; it signals a younger, divergent branch of development
c₂ D: disjoint genes — inventions from a time when both already existed that only one of them has; another measure of structural difference
N: the size of the larger genome — so that two large genomes differing in a few genes out of many do not come out as far apart as two small ones that differ in almost everything; for genomes under 20 genes, N = 1
c₃ W̄: the average weight difference of matching genes — identically wired networks with very different weights compute different functions; the coefficient c₃ is smaller because structure changes more rarely and means more

The constants c₁ = c₂ = 1, c₃ = 0.4 and the threshold δ_t = 3 are the values Stanley and Miikkulainen (2002) used for XOR; our program uses the same.

Example — the distance after adding a neuron

Genome A has genes 1–5. Genome B descends from a similar ancestor: it also has genes 1–5 (gene 3 disabled, because a new neuron split it) plus genes 6 and 7 from adding the neuron. The weights of the matching genes differ by 0.2, 0.6, 0, 0.9 and 0.8.

matching: 1–5 → W̄ = 0.2 + 0.6 + 0 + 0.9 + 0.8/5 = 0.5

excess: 6, 7 (beyond the end of the shorter genome, which ends with gene 5) → E = 2; disjoint: none → D = 0; the genomes have fewer than 20 genes → N = 1

δ = 1 · 2/1 + 1 · 0/1 + 0.4 · 0.5 = 2.2 < 3 → same species (colours as in the formula above)

If B acquired one more connection (gene 8), we would get E = 3 and δ = 3.2 ≥ 3 — B would found a new species. So a single new neuron does not push a network out of its species, but a new neuron plus a new connection does.

n_k = P · F̄_k/∑_j F̄_j, F̄_k = ∑_{i ∈ S_k} f_i/|S_k|

n_k: the number of offspring of species k in the next generation — this is how NEAT rewards success, rather than by selecting individuals from the whole population
P: the population size (150) — offspring are divided up like a cake, the population neither grows nor shrinks
|S_k|: the number of members of the species — fitness sharing: each member’s fitness is divided by the size of its species, f′_i = f_i/|S_k|; a large species thus cannot gain more merely by being large
F̄_k: the sum of the species’ shared fitnesses, i.e. its average fitness — the share of the cake is decided by quality, not numbers
∑_j F̄_j: the sum over all species — dividing by it gives shares that add up to 1, so exactly P offspring are allotted (after rounding down, the remainder goes to the best species)

Example — why sharing protects a new species

Population 150: the established species A has 120 networks with average fitness 9.0, the new species B with a hidden neuron has 30 networks with average fitness 7.5.

with sharing (colours as in the formula above): n_A = 150 · 9.0/16.5 ≈ 81.8 → 81 (+1 remainder) = 82, n_B = 150 · 7.5/16.5 ≈ 68.2 → 68

without sharing (by total fitness 120 · 9.0 = 1,080 and 30 · 7.5 = 225): A ≈ 150 · 1,080/1,305 ≈ 124, B ≈ 26

Without sharing, the new species would die out within a few generations, because it is worse and smaller. With sharing it gets more offspring than it has members, and has time to tune its weights.

The third idea is the simplest: start minimal. At the beginning, no network has any hidden neuron — the inputs lead straight to the output. Structure is added only when it pays off. The resulting networks therefore tend to be small.

Putting it all together — one generation of our simplified NEAT (the probabilities are the ones the program runs with for XOR):

population ← 150 minimal networks (x₁, x₂ and bias straight into y,
    weights random from [−2, 2])
for generation g = 0, 1, …, 100:
    evaluate each network’s fitness f
    if some network solves the task: stop
    divide the networks into species: a network joins the first species whose
                                      representative is at δ < 3,
                                      otherwise it founds a new one
    drop species that have not improved for 15 generations (except the species with the
        best network)
    allot n_k = 150 · F̄_k / ∑ F̄_j offspring to species k
    in each species:
        parents ← the best 20 % of members
        if the species has more than 5 members, its best network passes on unchanged
        each further offspring:
            with probability 0.75 crossover of two parents, otherwise a copy of one
            with probability 0.8 change weights
                (90 %: w ← w + N(0, 1), 10 %: new random value)
            with probability 0.1 add a connection
            with probability 0.03 add a neuron (split a random enabled connection)

6. NEAT on XOR

Our simplified NEAT has a population of 150 networks and at most 100 generations. In the run shown in the figure it solved XOR in generation 27 with a network with a single hidden neuron and seven connections — the smallest possible solution:

f = ( 4 − ∑ |y − t| )²

∑ |y − t|: the error — the sum of the deviations of the output y from the correct value t over all four inputs; absolute value, so that errors up and down do not cancel out
4: the largest possible error (four inputs, each deviation at most 1) — subtracting turns the error we want to shrink into a fitness that evolution increases, and which is always non-negative
²: squaring — amplifies differences close to the solution: a network with error 0.5 has fitness 12.25, one with error 1 only 9, so selection rewards the last few percent more

Fitness for XOR as in the original NEAT paper (Stanley and Miikkulainen, 2002); at most 16. A network that answers 0.5 to everything has fitness (4 − 2)² = 4. A network counts as a solution when, after rounding its output, it answers all four inputs correctly.

The XOR network grown by NEAT; its outputs on the right

The network can be read. The hidden neuron h lights up only when both inputs are 0 (both have a large negative weight, the bias is positive). The direct connections to the output would on their own compute “not both at once” — output 1 for everything except (1, 1) — but the strongly negative weight from h switches it off for (0, 0). The outputs 0.000, 0.855, 1.000 and 0.005 are exactly XOR after rounding. Meanwhile the number of species grew from one to about twenty.

Example — computing the network for all four inputs

Weights (rounded): into h from x₁, x₂ and the bias −3.37, −3.67, 0.95; into y from x₁, x₂, the bias and h −1.42, −2.96, 3.32, −5.69. The activation is σ(z) = 1/1 + e^−4.9z. First h is computed, then y (the table uses the unrounded weights, so a hand calculation may differ in the last digit):

(`x`₁, `x`₂)	`z`_h	`h`	`z`_y	`y`	\|`y` − `t`\|
(0, 0)	0.95	0.990	3.32 − 5.69 · 0.990 = −2.32	0.000	0.000
(0, 1)	−2.72	0.000	−2.96 + 3.32 = 0.36	0.855	0.145
(1, 0)	−2.43	0.000	−1.42 + 3.32 = 1.89	1.000	0.000
(1, 1)	−6.09	0.000	−1.42 − 2.96 + 3.32 = −1.06	0.005	0.005

The error is 0.150 and the fitness f = (4 − 0.150)² ≈ 14.82 out of 16 (colours as in the fitness formula above). The network does worst on (0, 1): the output 0.855 is correctly above one half, but not close to one — that is enough for a solution, and further tuning of the weights would raise the fitness still more.

Out of fifty runs with different seeds, NEAT solved XOR in 47, on average in 31 generations, with 2.6 hidden neurons that actually do something. The original paper reports 32 generations and 2.35 hidden neurons — so our simplified NEAT comes close.

7. What NEAT rests on

The same 50 runs with one part of the algorithm switched off:

Variant	Solved XOR	Average generation
NEAT	47 of 50	31.1
without speciation (one species)	1 of 50	—
without crossover	50 of 50	47.3

Without speciation, NEAT barely works. A network with a hidden neuron is, when it appears, worse than the established networks without one, because its new weights are random — and in a single shared competition, selection eliminates it immediately. Speciation protects it until it matures. Without crossover, XOR gets solved too, only more slowly and with larger networks (3.1 active hidden neurons): crossover helps combine partial discoveries of different individuals.

8. NEAT’s successors

HyperNEAT (Stanley et al., 2009) evolves a small network that only then generates the weights of a large network according to the geometry of the task — like a genome that does not describe every cell but the rules of growth.
Novelty search (Lehman and Stanley, 2011) rewards not fitness but new behaviour. In a maze with deceptive dead ends, it found the goal more often than fitness-driven search.
Deep neuroevolution. In 2017 the Uber AI team showed that a plain genetic algorithm can train networks with millions of weights on Atari games comparably to reinforcement learning.
Evolutionary neural architecture search. The AmoebaNet network (Google, 2019) arose by evolving building blocks and was among the best at image recognition on ImageNet at the time. Today architectures are more often designed by people or by other methods, but evolution remains one of the options.

9. Summary

Neuroevolution breeds neural networks with evolutionary algorithms — it is useful where the gradient cannot be computed or where the reward only comes at the end.
Evolving the weights of a single neuron keeps the pole on a cart up for 20 seconds after roughly three hundred attempts; random weights give a median of 9 steps.
A single neuron is not enough for XOR — the structure of the network has to change.
NEAT evolves both weights and structure: historical markings make crossover possible, speciation protects new structures, and networks grow from a minimum.
Without speciation NEAT solved XOR in 1 of 50 runs, with speciation in 47 — and the best solution had a single hidden neuron.

Want to try it yourself? Have an AI agent write the cart-pole with an evolved controller, and then try two poles of different lengths on one cart — the famous harder version on which NEAT once excelled.

Do you see it differently? Write to support@coden-app.com.

Neuroevolution: evolving the weights and shape of neural networks, and the NEAT algorithm

1. Neural networks in a nutshell

2. Evolving weights: the pole on a cart

3. Why evolution and not the gradient

4. When weights are not enough: XOR

5. NEAT: evolving growing networks

6. NEAT on XOR

7. What NEAT rests on

8. NEAT’s successors

9. Summary

An Agent on your computer, driven from your phone