mutate_only_covered_lines gates against stale line numbers on re-runs, silently dropping mutants

[chwiese]

Summary

When mutate_only_covered_lines = true, coverage is measured against the file in mutants/, but the resulting covered-line set is applied to node positions in the pristine source. Those two line numberings only agree on the first run, when mutants/<file> is still a byte-identical copy. After generation the copy contains an injected trampoline import (and a full copy of every function per mutant), so every line below the injection point is shifted.

Any later regeneration of that file therefore filters its nodes against misaligned line numbers. In the worst case every mutable node is rejected and the function silently produces zero mutants — no warning, no no tests marker, it simply disappears from the run.

The bug needs a file to be regenerated after it has been generated once, so it doesn't show up in a first-run or clean-checkout test.

Reproduction

repro/
├── pyproject.toml
├── mylib/
│   ├── __init__.py      # empty
│   └── thing.py
└── tests/
    └── test_mylib.py

pyproject.toml:

[project]
name = "repro"
version = "0.1.0"

[tool.mutmut]
paths_to_mutate = ["mylib/"]
mutate_only_covered_lines = true

[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["."]

mylib/thing.py:

import os


def helper(a, b):
    return a + b


def other(a, b):
    return a - b

Step 1 — start with neither function exercised, so that this file legitimately generates zero mutants:

# tests/test_mylib.py
import mylib.thing  # imported, but nothing in it is called


def test_placeholder():
    assert True

$ mutmut run

0 mutants for thing.py — correct. But mutants/mylib/thing.py is now shifted by the injected import:

1: import os
2:
3:
4: from mutmut.mutation.trampoline import wrap_in_trampoline as _mutmut_mutated, MutantDict
5:
6:
7: def helper(a, b):
8:     return a + b          # <- source line 5
9:
10:
11: def other(a, b):
12:     return a - b          # <- source line 9

Step 2 — now cover both functions, and touch the source so the mtime check forces regeneration:

# tests/test_mylib.py
from mylib.thing import helper, other


def test_helper():
    assert helper(1, 2) == 3


def test_other():
    assert other(5, 2) == 3

$ touch mylib/thing.py
$ mutmut run
    done in 135ms (1 files mutated, 0 ignored, 1 unmodified)

The file is regenerated, and both functions are now fully covered — but:

$ grep -c "__mutmut_[0-9]" mutants/mylib/thing.py
0

Step 3 — identical source, identical tests, cold cache:

$ rm -rf mutants && mutmut run
$ grep -c "__mutmut_[0-9]" mutants/mylib/thing.py
2

0 mutants warm vs 2 mutants cold, from the same inputs.

Why

Coverage reports lines 8 and 12 as covered — those are the body positions in mutants/mylib/thing.py. The gate then tests the pristine source, where the bodies are at lines 5 and 9. Neither is in {8, 12}, so both are rejected.

gather_coverage() measures the mutants/ copy:

# mutmut/code_coverage.py
abs_filename = str((mutants_path / filename).absolute())
lines = set(coverage_data.lines(abs_filename) or [])
covered_lines[abs_filename] = lines

and _should_mutate_node() applies it to source positions:

# mutmut/mutation/file_mutation.py
if self._covered_lines is not None and position.start.line not in self._covered_lines:
    return False

MutantLineSpans records the mapping that would be needed here, but it's only consumed by the mutmut show diff renderer (__main__.py:1639) — nothing maps coverage lines back to source lines.

There's a second-order effect once generation is fixed: mutmut-stats.json's tests_by_mangled_function_name is built from trampoline hits during the stats run, so a function whose trampoline didn't exist at that point comes back no tests (exit 33) and never executes even after its mutants appear. Deleting mutants/mutmut-stats.json alone doesn't clear it.

Impact

On a real project (~40 source files, 834 mutants) the two runs disagree materially:

	mutants	killed	survived
warm mutants/	822	684	139
rm -rf mutants	834	700	133

12 mutants never generated at all, and results misreported for others. Since the whole point of the cache is that warm runs are the normal development loop, the numbers reported during day-to-day use are the untrustworthy ones — and there's no signal that anything was skipped.

Environment

• mutmut version: 3.7.0
• Python: 3.14.4
• OS: macOS 26.6.2

[Otto-AA]

Thanks for the report, I don't have an idea of the top of my head how to solve this.

I'd summarize the issue like this: We run coverage currently in the ./mutants/ directory but use it like it is coverage of the non-mutated source code. Previously this was also true, because we always regenerated the files, only with the caching of mutated code this changed.

For the mutate_only_covered_lines feature, we want to know which lines of the original source code are covered, so we only mutate those. For this, I think we need to run it on the original source code. With #479 we changed it to run in ./mutants/ (not sure exactly why anymore, but there's an E2E test that should cover the reason).

Maybe, we can move it back to run in the main folder (not ./mutants/). The main issue is likely that pytest may fail when ./mutants/ already exists (at least if there's no testpaths = ["tests"] setting, then pytest looks for tests in the main folder and in the mutants folder). And I'm not sure if there are cases, where we want to run in the mutmut generated setup (with a subset of files), but probably not?