Heap corruption in Document.tobytes() on 1.24.11/1.24.14 surfaces only under jemalloc (fixed in 1.25.5+)

[pborges-conduit]

Summary

Document.tobytes() with the flag combination below appears to corrupt the process heap in 1.24.11 and 1.24.14 when jemalloc is the active allocator. It is already resolved in 1.25.5 and 1.26.4, so this is filed for documentation rather than as a fix request — we found nothing describing this failure mode when searching, and the allocator dependence makes it easy to misdiagnose.

We hit it in production as a daily gunicorn worker SIGSEGV and initially chased entirely the wrong causes, so the details may save someone else the same detour.

Environment

• python:3.11.10-slim, CPython 3.11.10, linux/amd64 (also reproduced on linux/arm64)
• PyMuPDF 1.24.11 (MuPDF 1.24.10) and 1.24.14 — both fail
• PyMuPDF 1.25.5 and 1.26.4 (MuPDF 1.26.7) — both clean
• LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2, MALLOC_ARENA_MAX=2

Reproducer

Single-threaded. No threads involved anywhere.

import fitz  # PyMuPDF 1.24.11

data = open("scan.pdf", "rb").read()   # ~21MB, 67 pages, JPEG page images
for i in range(50):
    print(i, flush=True)
    d = fitz.open(stream=data, filetype="pdf")
    d.subset_fonts()
    out = d.tobytes(deflate=True, garbage=4, clean=True,
                    deflate_images=True, deflate_fonts=True, use_objstms=1)
    d.close()

FROM python:3.11.10-slim
ENV PYTHONFAULTHANDLER=1 MALLOC_ARENA_MAX=2
ENV LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
RUN apt-get update && apt-get install -y --no-install-recommends libjemalloc2 \
 && rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir PyMuPDF==1.24.11

The input is a synthetic scanned-style document (one grayscale JPEG per page, generated with new_page() + insert_image()), and it validates cleanly — pypdf and fitz both read all 67 pages before the loop starts.

Observed behaviour

Fails within roughly 5–15 iterations, non-deterministically, in one of several ways:

• SIGSEGV
• an apparent infinite loop (100% CPU, no progress, >60s on an operation that normally takes ~0.03s)
• pymupdf.mupdf.FzErrorFormat: code=7: cycle in page tree raised from pdf_subset_fonts2
• Fatal Python error: Segmentation fault with Current thread ... Garbage-collecting
• MuPDF error: format error: non-page object in page tree on stderr

The key detail: these occur on a freshly opened document, constructed from the same immutable bytes that succeeded on earlier iterations. Nothing carries over at the Python level, which points at process-global state corruption rather than a bad input document.

Faulthandler traceback from one run:

Current thread 0x... (most recent call first):
  File ".../pymupdf/__init__.py", line 3694 in close

and from another, after the loop had already printed OK:

Fatal Python error: Segmentation fault
Current thread 0x... (most recent call first):
  Garbage-collecting
  <no Python frame>

Allocator dependence

This is the part that made it hard to find:

build	jemalloc LD_PRELOAD	result
1.24.11	yes	fails within ~5–15 iterations
1.24.11	no (glibc malloc)	200 iterations clean
1.24.14	yes	fails
1.25.5	yes	clean
1.26.4	yes	clean

Under glibc malloc the corruption appears to stay latent — freed regions are not reused aggressively enough to fault. jemalloc's faster reuse turns it into a hard crash. So on glibc this looks like a stable library, and only the allocator change exposes it.

Flag bisect on 1.24.11 with jemalloc (10 iterations each):

flags	result
deflate	clean
+garbage=4	clean
+clean	clean
+deflate_images/deflate_fonts	clean at 10, SIGSEGV at 15
full set +use_objstms=1	hang >60s

use_objstms=1 is not required — it changes how fast the failure appears, not whether it happens. Longer runs fail without it too, so I would not treat any single flag as the trigger.

Downstream impact, for context

In our service the corruption did not surface inside PyMuPDF at all. It detonated on the next allocation-heavy call — a boto3 S3 upload immediately after compression returned — producing a worker SIGSEGV with no Python traceback and a 502 to the caller. Because memory utilisation was flat (~2%) it did not look like a memory bug, and because only large files took this code path it looked file-specific. It took a while to land on the allocator.

Two things that might be worth a documentation note, if you think it's warranted:

1. Document.tobytes()/save() with garbage/clean/deflate_images on image-heavy documents was not safe in 1.24.x.
2. Behaviour can differ materially under a non-glibc allocator. jemalloc via LD_PRELOAD is common in containerised Python services for fragmentation reasons, so others may be carrying this latently on 1.24.x without ever seeing a crash.

Workaround

Upgrading to 1.26.4 resolved it for us — no code changes, same flags. Validated with jemalloc enabled: 150 iterations of our full compression pipeline on a 34MB/67-page scan, plus a 56.5MB/300-page document through compression and splitting, all clean. Confirmed on both amd64 and arm64.

Not asking for anything here — just leaving a record in case it helps someone bisecting the same symptoms, or if you consider it worth a changelog/documentation mention for 1.24.x users.

[JorjMcKie]

Thank you for this information!