Bayesian networks: why the grass is wet and what it means for the sprinkler

Joint distribution and its factorisation, Bayesian network and conditional probability tables, inference by enumeration, explaining away, d-separation and approximate inference by sampling — on the network cloudy, sprinkler, rain, wet grass.

13 minutes Matěj Žalmánek 2026-10-15

Series AI basics Part 40 of 52

Full series syllabus 52 of 83 topics published

What you will take away

How a Bayesian network shrinks a huge probability table into a few small tables.
How a network reasons from effects to causes and what explaining away is.
How to read from the graph which variables are independent, and how probabilities are estimated by sampling.

Useful to know first: Conditional probability and Bayes’ theorem: how to update your estimate when you learn something new, Graphs and trees: the structure that both search and neural networks stand on

You step outside in the morning and the grass is wet. Did it rain, or was the sprinkler on? And when a neighbour then tells you it rained during the night — does that change what you think about the sprinkler? People reason like this without thinking. A machine can do it too, as long as it knows what is connected to what. A Bayesian network is a picture of these connections from which you can compute the answer to any question. Judea Pearl received the 2011 Turing Award for it and for the theory of causality built on top of it.

It builds on the parts Conditional probability and Bayes’ theorem and Graphs and trees.

1. The problem of the big table

Complete information about several random variables is their joint distribution — the probability of every combination of values. For four yes/no variables there are 16 combinations, so 15 independent numbers. For 37 variables (that is how many the well-known ALARM medical network for monitoring patients under anaesthesia has) it would be over 137 billion numbers, even if they were all just yes/no. Nobody can build such a table, store it or estimate it from data. What saves us is that most variables are related only indirectly.

Definition — joint distribution and the number of its parameters

Let (Ω, 𝓕, P) be a probability space (see the part What is probability), n ≥ 1, and let X₁, …, X_n be discrete random variables on it (see the part Random variables); the variable X_i takes values in a finite or countable set 𝒳_i, i = 1, …, n. The joint distribution of the variables X₁, …, X_n is the function 𝒳₁ × ⋯ × 𝒳_n → [0, 1] that assigns to every combination of values (x₁, …, x_n) the number

P(x₁, …, x_n) = P({ω ∈ Ω : X₁(ω) = x₁ ∧ … ∧ X_n(ω) = x_n}),

also written P(x₁, …, x_n) = P(X₁ = x₁, …, X_n = x_n). From the axioms of probability it follows that:

P(x₁, …, x_n) ≥ 0 for all (x₁, …, x_n) ∈ 𝒳₁ × ⋯ × 𝒳_n;
∑_{(x₁, …, x_n) ∈ 𝒳₁ × ⋯ × 𝒳_n} P(x₁, …, x_n) = 1.

Number of parameters. If all the variables are two-valued (|𝒳_i| = 2 for i = 1, …, n, values yes/no), the domain has 2ⁿ elements. Any 2ⁿ − 1 values can be chosen independently (non-negative, with a sum of at most 1), and the last one is determined by condition 2 as 1 minus their sum. The joint distribution of n two-valued variables therefore has

2ⁿ − 1

independent parameters — that many numbers are needed to determine it uniquely.

Calculation. 2⁴ − 1 = 15; 2³⁷ − 1 = 137,438,953,471.

Theorem — the chain rule

Let (Ω, 𝓕, P) be a probability space, n ≥ 2, X₁, …, X_n discrete random variables with values in 𝒳₁, …, 𝒳_n, and (x₁, …, x_n) ∈ 𝒳₁ × ⋯ × 𝒳_n a combination of values for which P(x₁, …, x_n−1) > 0. By P(x_i | x₁, …, x_i−1) we mean the conditional probability (see the part on conditional probability) of the event {X_i = x_i} given {X₁ = x₁, …, X_i−1 = x_i−1}. Then

P(x₁, …, x_n) = P(x₁) · P(x₂ | x₁) · P(x₃ | x₁, x₂) ⋯ P(x_n | x₁, …, x_n−1) = ∏_i=1ⁿ P(x_i | x₁, …, x_i−1),

where the factor for i = 1 is P(x₁) (empty condition).

Proof: for k ≤ n − 1 the event {X₁ = x₁, …, X_k = x_k} is a superset of the event {X₁ = x₁, …, X_n−1 = x_n−1}, so it has positive probability and all the conditional probabilities in the formula are defined. By the definition of conditional probability

P(x_n | x₁, …, x_n−1) = P(x₁, …, x_n)/P(x₁, …, x_n−1),

so P(x₁, …, x_n) = P(x_n | x₁, …, x_n−1) · P(x₁, …, x_n−1). We apply the same to the second factor and so on, until only P(x₁) is left. ∎

The chain rule saves nothing: the last factor still depends on all the preceding variables, so it is just another way of writing the same huge table. The savings come once we drop some of the conditions.

2. Bayesian network

Definition — Bayesian network

Let n ≥ 1 and let X₁, …, X_n be variables, X_i with values in a finite non-empty set 𝒳_i, i = 1, …, n. A Bayesian network over these variables is a pair (G, Θ), where

G = (V, H) is a directed acyclic graph (DAG, see the part Graphs and trees) with vertex set V = {X₁, …, X_n} and edge set H ⊆ V × V; the parents of the vertex X_i are parents(X_i) = {X_j ∈ V : (X_j, X_i) ∈ H};
Θ assigns to every vertex X_i a conditional probability table: for every combination of values u of its parents a distribution P( · | u) on 𝒳_i, i.e. numbers P(x_i | u) ≥ 0 for x_i ∈ 𝒳_i such that ∑_{x_i ∈ 𝒳_i} P(x_i | u) = 1. A vertex without parents has a single distribution P(x_i).

The network determines the joint distribution of the variables X₁, …, X_n by the rule

P(x₁, …, x_n) = ∏_i=1ⁿ P(x_i | parents(X_i)) for all (x₁, …, x_n) ∈ 𝒳₁ × ⋯ × 𝒳_n,

where in the i-th factor we substitute for parents(X_i) the values the parents have in the combination (x₁, …, x_n). The product is non-negative and sums to 1 over all combinations (if we first sum over the values of the vertex that comes last in topological order, its factor gives 1 and the network without it remains), so it really is a joint distribution.

In plain words

Arrows lead from causes to effects. For each variable it is enough to know how it depends on its direct causes — nothing more. The probability of the whole situation is the product of these local numbers. Instead of one huge table we have a few small ones.

Why may the network write just P(x_i | parents) and not the whole history like the chain rule? Because it assumes conditional independences about the graph.

Definition — conditional independence

Let (Ω, 𝓕, P) be a probability space and A, C, E discrete random variables on it with values in 𝒜, 𝒞, ℰ (each of them may also be a group of variables, i.e. a vector; E may be an empty group). For short we write P(a | c, e) = P(A = a | C = c, E = e) and so on. The variables A and C are conditionally independent given E, written A ⊥ C | E, if and only if

∀a ∈ 𝒜 ∀c ∈ 𝒞 ∀e ∈ ℰ : P(c, e) > 0 ⇒ P(a | c, e) = P(a | e).

In words: once we know the value of E, knowing the value of C does not change the probability of any value of A. The equality in the definition:

P(a | c, e) = P(a | e)

P(a | c, e): what we know about A when we know E and also C
P(a | e): what we know about A from E alone; the equality says that C adds nothing more — the whole connection between A and C runs through E

Equivalently, P(a, c | e) = P(a | e) · P(c | e) for all a, c and all e with P(e) > 0. For an empty E it is ordinary independence.

Theorem — factorisation is the chain rule plus local independences

Let G = (V, H) be a directed acyclic graph with vertices V = {X₁, …, X_n}, which are discrete random variables on a probability space (Ω, 𝓕, P), and let the vertices be numbered topologically: (X_j, X_i) ∈ H ⇒ j < i. The descendants of a vertex X_i are the vertices other than X_i to which a directed path leads from X_i; non-descendants(X_i) = V ∖ ({X_i} ∪ descendants(X_i)). Assume the local Markov condition:

∀i ∈ {1, …, n} : X_i ⊥ (non-descendants(X_i) ∖ parents(X_i)) | parents(X_i),

i.e. every variable is, given its parents, conditionally independent of all other variables that are not its descendants. Then for every combination of values (x₁, …, x_n) with P(x₁, …, x_n−1) > 0

P(x₁, …, x_n) = ∏_i=1ⁿ P(x_i | parents(X_i)).

Proof: in the chain rule in topological order we have P(x_i | x₁, …, x_i−1). Among X₁, …, X_i−1 are all the parents of X_i (a parent has a smaller number) and none of its descendants (a descendant comes after it in the order), so {X₁, …, X_i−1} ⊆ non-descendants(X_i). By the Markov condition we can drop all variables except the parents from the condition, and what remains is P(x_i | parents(X_i)). ∎

It also holds the other way round: every distribution given by a product of tables satisfies the local Markov condition. That is why a Bayesian network can be defined in either of the two ways.

Our network: when it is cloudy, rain is more likely and it is less likely that someone turns on the sprinkler. Both rain and the sprinkler make the grass wet. The numbers come from Kevin Murphy’s popular example:

The “wet grass” network with conditional probability tables + yes, − no

The network has 1 + 2 + 2 + 4 = 9 numbers instead of 15. With four variables the saving is small, with hundreds it is huge: if every vertex has at most k parents, the number of numbers grows with n linearly (n · 2^k), not exponentially (2ⁿ). The probability of the situation “cloudy, the sprinkler was off, it rained, the grass is wet” is 0.5 · 0.9 · 0.8 · 0.9 = 0.324 (colours as in the formula below).

P(z, p, d, t) = P(z) · P(p | z) · P(d | z) · P(t | p, d)

P(z): cloudy has no parents, its table is a single number; in the example 0.5
P(p | z): the sprinkler depends only on the clouds — here nothing differs from the chain rule; in the example it was off under a cloudy sky: 1 − 0.1 = 0.9
P(d | z): the chain rule would want P(d | z, p); the network assumes that once the sky is known, the sprinkler says nothing about rain, so p drops out of the condition; in the example 0.8
P(t | p, d): the chain rule would want P(t | z, p, d); the grass “knows” about the clouds only through rain and the sprinkler, so z drops out; in the example (sprinkler no, rain yes) 0.9

Every dropped term of a condition is one conditional independence the network assumes — exactly these cut 15 numbers down to 9.

3. Inference: from effect to cause

Definition — inference by enumeration

Let (G, Θ) be a Bayesian network over the variables X₁, …, X_n, and let these variables be split into three disjoint groups: the query Q (one variable with values in 𝒬), the evidence E = (E₁, …, E_m) with observed values e = (e₁, …, e_m), and the hidden variables Y (all the others) with the set of value combinations 𝒴. Assume P(e) > 0. Then for every q ∈ 𝒬

P(q | e) = ∑_{y ∈ 𝒴} P(q, e, y)/∑_{q′ ∈ 𝒬} ∑_{y ∈ 𝒴} P(q′, e, y),

where each P(q, e, y) is a value of the joint distribution of the network — a product of numbers from the tables according to the factorisation. Inference by enumeration is the algorithm that computes by this formula:

Input: the network (G, Θ), the query variable Q, evidence E = e with P(e) > 0. Output: the numbers P(q | e) for all q ∈ 𝒬. Steps:

For every q ∈ 𝒬 set S(q) := 0.
For every q ∈ 𝒬 and every y ∈ 𝒴 compute the product P(q, e, y) from the tables and add it to S(q).
After going through all (finitely many) pairs (q, y) set Z := ∑_{q′ ∈ 𝒬} S(q′) = P(e) and return P(q | e) = S(q)/Z for every q ∈ 𝒬.

In plain words

We go through all possible worlds that agree with what we know (the grass is wet) and look at how much weight the ones where it rained have among them. It is Bayes’ theorem from the part on conditional probability, only summed over more variables at once.

P(q | e) = ∑_y P(q, e, y)/∑_{q′, y} P(q′, e, y)

e: evidence — the values we observed (in our case wet grass); they are fixed in all the worlds being summed
∑_y: we do not know the hidden variables (in our case cloudy and the sprinkler) and do not care about them, so we sum over all their possibilities — marginalisation; the worlds are mutually exclusive, so their probabilities add up
P(q, e, y): the probability of one complete world in which both the query and the evidence hold — a product of numbers from the tables according to the factorisation
∑_{q′, y} P(q′, e, y): = P(e), the weight of all worlds that agree with the evidence; the division is the definition of conditional probability P(q | e) = P(q, e)/P(e) and guarantees that ∑_q P(q | e) = 1 (for a yes/no query P(Q | e) + P(¬Q | e) = 1)

Example — P(rain | wet grass) by enumeration

The evidence is T = yes, the query is D, and Z and P are hidden. There are 2³ = 8 worlds with wet grass; each is the product P(z) · P(p | z) · P(d | z) · P(t | p, d):

rain	cloudy	sprinkler	product	`P`(world)
yes	yes	yes	0.5 · 0.1 · 0.8 · 0.99	0.0396
yes	no	yes	0.5 · 0.5 · 0.2 · 0.99	0.0495
yes	yes	no	0.5 · 0.9 · 0.8 · 0.9	0.3240
yes	no	no	0.5 · 0.5 · 0.2 · 0.9	0.0450
no	yes	yes	0.5 · 0.1 · 0.2 · 0.9	0.0090
no	no	yes	0.5 · 0.5 · 0.8 · 0.9	0.1800
no	yes	no	0.5 · 0.9 · 0.2 · 0	0.0000
no	no	no	0.5 · 0.5 · 0.8 · 0	0.0000

The rows where it rained are the worlds P(q, e, y) for q = rain yes; their sum over the hidden Z, P gives P(D, T) = 0.4581, the sum of all eight P(T) = 0.6471. The last two rows are zero: without rain and without the sprinkler the grass is not wet. Therefore (colours as in the formula above)

P(D | T) = 0.4581/0.6471 = 0.708.

def dotaz(Q, dukazy, sit):
    citatel = jmenovatel = 0.0
    for svet in vsechna_prirazeni(sit):              # 2^n complete worlds
        if any(svet[k] != v for k, v in dukazy.items()):
            continue                                 # disagrees with the evidence
        p = 1.0
        for uzel in sit:                             # factorisation of the network
            p *= P(uzel, svet[uzel], svet[rodice(uzel)])
        jmenovatel += p                              # → P(e)
        if svet[Q]:
            citatel += p                             # → P(Q, e)
    return citatel / jmenovatel

Beforehand (without any observation) the probability of rain is 0.5 and of the sprinkler 0.3. Wet grass raises both: P(rain | wet grass) = 0.708 and P(sprinkler | wet grass) = 0.430. Rain is the more likely explanation, because it rains more often than the sprinkler runs. Wet grass also raises the probability that it was cloudy — from 0.5 to 0.576 — even though the grass and the clouds are not directly connected. Information flows through the network against the arrows.

4. Explaining away

Now we learn that it really did rain during the night. What happens to the sprinkler?

Sprinkler and rain: beforehand, given wet grass, and given wet grass when we know about the other cause beforehand the grass is wet and we know about the other cause

The probability of the sprinkler drops from 0.430 to 0.194. Rain has already explained the wet grass, so there is no reason left for the sprinkler. Conversely, when we know the sprinkler was on, the probability of rain falls from 0.708 to 0.320. This phenomenon is called explaining away. Two causes that were beforehand only weakly related start “pushing each other out” as soon as we observe their common effect. And if we knew that it had not rained, the sprinkler would have to have been on: P(sprinkler | wet grass, no rain) = 1, because in our network the grass cannot be wet by itself.

Example — explaining away in numbers

From the table in the previous example: the worlds where it rained, the sprinkler was on and the grass is wet together have P(P, D, T) = 0.0396 + 0.0495 = 0.0891. Hence

P(P | T, D) = P(P, D, T)/P(D, T) = 0.0891/0.4581 = 0.194,

P(D | T, P) = P(P, D, T)/P(P, T) = 0.0891/0.2781 = 0.320

(the denominator is the probability of the evidence P(e), colours as in the formula for inference by enumeration above).

Formally, explaining away is the inequality P(P | T, D) < P(P | T): confirming one cause lowers the probability of the other (0.194 < 0.430).

5. What the network tells you: d-separation

The graph does not only say how to compute — it also says which variables are independent of each other, and without a single number. All possibilities are made up of three basic shapes:

Chain A → B → C: A and C are related, but once we know B, they are not. (Cloudy → rain → wet grass: if you know whether it rained, the clouds tell you nothing about the grass.)
Fork A ← B → C: a common cause makes A and C dependent; once we know it, the dependence disappears. (The sprinkler and rain are related beforehand through the clouds: P(sprinkler | rain) = 0.18, but P(sprinkler | no rain) = 0.42. When we know whether it is cloudy, they are independent.)
Collider A → B ← C: the causes are independent until we observe the common effect — then they become dependent. That is explaining away.

Definition — d-separation

Let G = (V, H) be a directed acyclic graph, A, C ∈ V distinct vertices and E ⊆ V ∖ {A, C} a set of vertices (the observed variables).

An undirected path between A and C is a sequence of mutually distinct vertices (V₀, V₁, …, V_k), k ≥ 1, with V₀ = A, V_k = C, such that (V_j−1, V_j) ∈ H ∨ (V_j, V_j−1) ∈ H for j = 1, …, k (neighbouring vertices are joined by an edge, the direction of the arrow does not matter).
An inner vertex V_j of the path (0 < j < k) is the middle of a chain if and only if V_j−1 → V_j → V_j+1 or V_j−1 ← V_j ← V_j+1; a fork if and only if V_j−1 ← V_j → V_j+1; a collider if and only if V_j−1 → V_j ← V_j+1 (U → W means (U, W) ∈ H). Every inner vertex is exactly one of these three.
A path is blocked by the set E if and only if it has an inner vertex V_j for which at least one of the following holds: (a) V_j is the middle of a chain or a fork ∧ V_j ∈ E, (b) V_j is the middle of a collider ∧ ({V_j} ∪ descendants(V_j)) ∩ E = ∅, where descendants(V_j) are the vertices to which a directed path leads from V_j.
The vertices A and C are d-separated by the set E if and only if every undirected path between A and C is blocked by E.

Theorem (Verma and Pearl). Let (G, Θ) be a Bayesian network, P the joint distribution it determines, and A, C, E as above. If A and C are d-separated by the set E, then A ⊥ C | E with respect to P — and this for every choice of the tables Θ.

In plain words

Information flows through the graph along paths. A known variable in the middle of a chain or a fork stops the flow — it is already “explained”. With a collider it is the other way round: as long as we do not know the effect, the flow stands still there; as soon as we observe it, it opens. From the picture you can thus read off what needs to be measured and what is unnecessary.

In our network, cloudy and wet grass are d-separated when we know both the sprinkler and the rain — the calculation confirms it: the probability of a cloudy sky then does not depend on the grass at all.

Example — three shapes verified by calculation

Fork sprinkler ← cloudy → rain. Without knowing the sky: P(P | D) = 0.18 ≠ P(P) = 0.3, dependent. With the sky known: P(P | Z, D) = 0.1 = P(P | Z), so P ⊥ D | Z.
Collider sprinkler → wet grass ← rain. With the sky known, P and D are independent (P(P | Z, D) = P(P | Z, ¬D) = 0.1), but as soon as we observe the grass, they are not: P(P | Z, D, T) = 0.109, whereas P(P | Z, ¬D, T) = 1.
D-separation of the cloudy sky and the wet grass by the set {P, D}: both paths Z → P → T and Z → D → T are chains with a known middle, so they are blocked. And indeed: P(Z | P, D) = 0.444 and P(Z | P, D, T) = 0.444; P(Z | P, ¬D) = 0.048 and P(Z | P, ¬D, T) = 0.048.

6. Approximate inference by sampling

Inference by enumeration goes through all combinations of the hidden variables — in a general network exponentially many. Exact inference in Bayesian networks is NP-hard. For large networks the probabilities are therefore estimated by simulation.

Algorithms — rejection sampling and likelihood weighting

Both algorithms have the same input: a Bayesian network (G, Θ) over the variables X₁, …, X_n numbered topologically (a parent has a smaller number than its child), a query Q = q, evidence E = e and a number of samples N ∈ ℕ, N ≥ 1; and the same output: an estimate P̂(q | e) of the number P(q | e). To draw x_i means to choose a random value from 𝒳_i according to the distribution P( · | u) from the table of the vertex X_i, where u are the values of its parents already determined in this sample.

Rejection sampling:

Set N_acc := 0, N_q := 0.
Repeat N times: for i = 1, …, n draw x_i. If the sample agrees with the evidence (every evidence variable has its observed value in it), increase N_acc by 1, and if moreover Q = q holds in it, increase N_q by 1 too; otherwise throw the sample away.
After N repetitions stop and return P̂(q | e) = N_q/N_acc (for N_acc = 0 the estimate is undefined).

Likelihood weighting:

Set W := 0, W_q := 0.
Repeat N times (the k-th sample): set w_k := 1 and for i = 1, …, n: if X_i is an evidence variable with observed value e_i, set x_i := e_i (do not draw) and w_k := w_k · P(e_i | u), where u are the values of the parents in the sample; otherwise draw x_i. Then W := W + w_k, and if Q = q holds in the sample, also W_q := W_q + w_k.
After N repetitions stop and return P̂(q | e) = W_q/W (for W = 0 the estimate is undefined).

In plain words

Rejection is honest but wasteful: when the evidence is rare, it throws away almost all samples. Likelihood weighting throws away nothing — it just gives each sample a weight according to how well it fits what we know.

def zamitani(Q, dukazy, n, sit):
    prijato = ano = 0
    for _ in range(n):
        svet = {}
        for uzel in sit:                             # from causes to effects
            svet[uzel] = nahoda() < P(uzel, True, svet[rodice(uzel)])
        if any(svet[k] != v for k, v in dukazy.items()):
            continue                                 # into the bin
        prijato += 1
        ano += svet[Q]
    return ano / prijato

def vazena(Q, dukazy, n, sit):
    s = s_ano = 0.0
    for _ in range(n):
        svet, w = {}, 1.0
        for uzel in sit:
            p = P(uzel, True, svet[rodice(uzel)])
            if uzel in dukazy:                       # we do not draw the evidence…
                svet[uzel] = dukazy[uzel]
                w *= p if dukazy[uzel] else 1 - p    # …but we weigh how likely it is
            else:
                svet[uzel] = nahoda() < p
        s += w
        s_ano += w * svet[Q]
    return s_ano / s

P̂(Q | e) = ∑_k w_k [Q holds in sample k]/∑_k w_k, w_k = ∏_j P(e_j | parents_k)

w_k: the weight of the sample (green like the likelihood-weighting curve in the chart) — the product of the probabilities of all pieces of evidence e_j given the values of their parents that came up in sample k: we did not draw the evidence but imposed it, so we must take into account how likely it would be given the other drawn values — otherwise the estimate would be biased
[Q holds in sample k]: 1 if the query holds in the sample, otherwise 0 — we count only the weight of samples in which it rained
∑_k w_k: the total weight; dividing by it turns the sum into a fraction, just like dividing by P(e) in enumeration. Thanks to it, it does not matter that the weights do not sum to 1

Rejection is a special case: the weight is 1 for samples that agree with the evidence and 0 for the others.

Example — weights given the evidence “the grass is wet”

The only piece of evidence is T = yes, and its parents are the sprinkler and rain. The weight of a sample is therefore simply P(T = yes | p, d) from the wet grass table:

sprinkler and rain: w = 0.99,
only one of them: w = 0.9,
neither: w = 0 — such a sample contributes nothing, just as rejection would throw it away.

(Colour as in the formula above.)

The weight does not depend on the sky: cloudy is not a parent of the grass.

Estimate of P(rain | wet grass) by number of samples rejection likelihood weighting exact 0.708

With a hundred samples, rejection gave the estimate 0.686 and used only 70 of them — the remaining 30 had dry grass and went into the bin. With 10,000 samples (of which 6,524 were left) the estimate is 0.703, close to the exact value 0.708. The error decreases as 1/√N, as the central limit theorem says: for an estimate ten times more precise you need a hundred times more samples.

σ_P̂ = √p(1 − p)/N_acc, N_acc ≈ N · P(e)

p(1 − p): the variance of one yes/no sample (Bernoulli distribution with probability p); it is largest for p = 0.5, when a single sample is least predictable
N_acc: the number of samples that actually count; in rejection only those that passed, and on average they make up the fraction P(e) — with rare evidence almost none. The variance of the mean of N independent samples is N times smaller

The square root converts the variance back into units of probability — that is why the error decreases only as 1/√N. (The equality is approximate: the number of accepted samples is itself random.)

Example — how big an error to expect

Exactly, p = 0.708 and P(e) = P(T) = 0.6471, so out of 10,000 samples we expect about 6,471 accepted (in reality there were 6,524).

N_acc = 70: σ = √0.708 · 0.292/70 = 0.054; actual error |0.686 − 0.708| = 0.022.
N_acc = 6,524: σ = 0.0056; actual error 0.0050.

Both actual errors are smaller than one standard deviation — consistent with about two thirds of the estimates falling within ±σ (normal distribution by the central limit theorem).

7. Bayesian networks in practice

Diagnosis. Medical networks (ALARM, QMR-DT with hundreds of diseases and thousands of symptoms) and fault diagnosis of machines and networks: infer the most likely cause from the observed symptoms — and suggest what to measure next.
Learning from data. The tables can be estimated from data by counting (by the maximum likelihood method from the part on statistics), even when some values are missing — with the EM algorithm, a relative of the Baum–Welch algorithm from the previous part.
Causality. Pearl showed that a network read as a graph of causes can also answer questions like “what would happen if we intervened” — turned on the sprinkler ourselves — which cannot be read from correlations alone. Today’s causal inference in epidemiology and economics rests on this.
Relatives. The naive Bayes classifier is a Bayesian network with one cause and many independent effects. Hidden Markov models from the previous part are a Bayesian network unrolled in time.

8. Summary

A Bayesian network is a directed acyclic graph with tables P(X | parents); the joint distribution is the product of these tables. Our network needs 9 numbers instead of 15.
Inference also works against the arrows: wet grass raises the probability of rain to 0.708 and of the sprinkler to 0.430.
Explaining away: when we know it rained, the probability of the sprinkler drops to 0.194.
D-separation reads independences straight from the graph — a chain and a fork are closed by observation, a collider is opened.
Exact inference is NP-hard in general; sampling gives estimates with an error decreasing as 1/√N.

Want to get your hands on it? Have an AI agent build a Bayesian network for a fault in your own car or computer (“it won’t start — the battery or the starter?”) and see how the suspicion changes as you add observations.

Do you see it differently? Write to support@coden-app.com.

Bayesian networks: why the grass is wet and what it means for the sprinkler

1. The problem of the big table

2. Bayesian network

3. Inference: from effect to cause

4. Explaining away

5. What the network tells you: d-separation

6. Approximate inference by sampling

7. Bayesian networks in practice

8. Summary

An Agent on your computer, driven from your phone