Mutation testing: a practical guide with Stryker, mutmut and PIT
What mutation testing is, why code coverage is not enough, how to read the mutation score — and how to run it in JavaScript, Python and Java without waiting all day.
Your test suite is green and coverage says 90 %. But would the tests actually notice if the code were wrong? Coverage cannot tell you — it only says which lines ran, not whether anything checked what they did. Mutation testing answers the real question by deliberately breaking your code and watching whether the tests fail. Here is how it works, how to read the results, and how to run it in JavaScript, Python and Java.
1. What is mutation testing
A mutation testing tool makes many small, deliberate changes to your source code — each one a mutant — and runs your tests against every mutant. If a test fails, the mutant is killed: your tests noticed the change. If all tests still pass, the mutant survived: you just found behaviour that nothing checks.
In other words, mutation testing tests your tests. It is one of the most honest measures of test quality you can get, because it cannot be gamed by tests that run code without asserting anything.
2. Why code coverage is not enough
function canVote(age) {
return age >= 18;
}
test("adults can vote", () => {
expect(canVote(30)).toBe(true);
});
Coverage: 100 %. Every line ran. Now change >= to > — the
test still passes, because 30 is far from the boundary. A 17-year-old voting or an
18-year-old refused would both go unnoticed. A mutation tool makes exactly that change,
sees the test survive, and points you at the missing test:
canVote(18) must be true and canVote(17)
false.
3. Mutants, operators and the mutation score
The kinds of change a tool makes are called mutation operators. Typical ones:
| Operator | Original | Mutant |
|---|---|---|
| conditional boundary | a >= b | a > b |
| negated condition | if (ok) | if (!ok) |
| arithmetic | a + b | a - b |
| logical | a && b | a || b |
| return value | return x | return null |
| removed call | save(order) | (call removed) |
The mutation score is the share of mutants your tests killed:
mutation score = killed mutants / (all mutants − equivalent mutants)An equivalent mutant changes the code without changing its behaviour — for example a change in a branch that can never run. No test can kill it, so it is left out of the score once identified.
4. Mutation testing in JavaScript and TypeScript: Stryker
StrykerJS is the standard tool for JavaScript and TypeScript. It works with Jest, Vitest, Mocha, Jasmine, Karma and others:
# installs Stryker and runs an interactive setup
npm init stryker@latest
# run it
npx stryker run
The initializer creates stryker.config.mjs. After a run you get a report
listing every surviving mutant with the exact change, file and line.
5. Mutation testing in Python: mutmut
pip install mutmut
mutmut run
mutmut browse # interactive view of the results
By default mutmut runs pytest on the tests in tests or test.
Configure it in pyproject.toml:
[tool.mutmut]
source_paths = [ "src/" ]
pytest_add_cli_args_test_selection = [ "tests/" ]
To mutate only part of the code: mutmut run "my_module*". On Windows,
mutmut has to run inside WSL. Before you use mutmut apply to write a mutant
into your files, commit your work.
6. Mutation testing in Java: PIT
PIT (pitest) is the mutation tool for the JVM. With Maven:
mvn test-compile org.pitest:pitest-maven:mutationCoverage
It writes an HTML report to target/pit-reports/. Limit what it mutates
with targetClasses and targetTests in the plugin
configuration, and add -DwithHistory to reuse results between runs so
unchanged code is not mutated again. Gradle users have a community plugin.
7. How to read the results and fix surviving mutants
- Look at survivors, not the score. Each surviving mutant is a concrete question: “if this line were wrong, would anyone notice?”
- Write the test the mutant describes. A survived
>=→>means a missing boundary test. A survived removed call means nobody checks its side effect. - Mark equivalent mutants instead of writing absurd tests to kill them.
- Prioritize. A survivor in payment logic matters; one in a log message usually does not.
Surviving mutants are also excellent tasks for an AI coding agent: each one is a precise, verifiable request — “write a test that kills this mutant” — and the next mutation run tells you whether it succeeded.
8. Mutation testing in practice: speed and CI
The cost is time. Every mutant means another test run, so a full run on a big project can take hours. What keeps it practical:
- Start small. One module with real logic — branching, arithmetic, comparisons — not glue code.
- Incremental runs. Mutate only what changed since the last run (PIT's history, Stryker's incremental mode).
- Fast tests first. Mutation testing amplifies a slow suite; keep unit tests quick.
- In CI, run it on changed files in pull requests and the full run nightly.
- Do not chase 100 %. Aim for a rising score on the code that matters; around 80 % on critical business logic is a sensible first target.
Mutation score is one of several ways to judge a test suite, next to coverage criteria such as branch or condition coverage. In coden you can follow your test runs, their history and the documentation of each test case on your phone — and hand the agent the next surviving mutant as a task.
Do you see it differently? Write to support@coden-app.com.