Skip to content

Sitemaps: image sitemap build accumulates memory linearly, OOMs cron on 100k+ media #49786

Description

@gin0115

Impacted plugin

Jetpack

Quick summary

On large media libraries, Jetpack's image sitemap build (jp_sitemap_cron_hookjp_img_sitemap / image-sitemap-N.xml) grows resident memory ~10 MB per chunk and never frees it within the request, so it walks up to / past the PHP memory_limit and OOMs during WP-Cron. Regular sync crons stay flat — the cost is entirely the sitemap builder.

Steps to reproduce

  1. Site with a large media library (here: over 113,000 media items, over 16,000 posts, several CPTs, WooCommerce + Subscriptions) so the image sitemap spans 100+ chunks.
  2. Default PHP memory_limit (512M).
  3. Let jp_sitemap_cron_hook run (or trigger it).
  4. Watch memory climb ~10 MB per image-sitemap-N.xml chunk (~390 KB each) and approach/exceed 512M → OOM fatal (surfaces in wp-includes/class-wpdb.php, the victim of the next allocation).

Site owner impact

Fewer than 20% of the total website/platform users

Severity

Minor

What other impact(s) does this issue have?

No revenue impact

If a workaround is available, please outline it here.

Per-chunk climb in one run:

cron START: jp_sitemap_cron_hook   166 MB
image-sitemap-96.xml  (397 KB)     176 MB
image-sitemap-98.xml               196 MB
image-sitemap-100.xml              214 MB
image-sitemap-103.xml              244 MB
…                                  → 404 MB

Peak per sitemap-cron run (UTC):

Run Peak
Jun 11 (from 12:22) did not run
Jun 12 21:07 412 MB
Jun 13 09:06 513 MB (over limit)
Jun 13 21:07 511 MB
Jun 14 21:06 434 MB
Added mitigation
Jun 15 did not run
Jun 16 09:07 404 MB
Jun 17 did not run
Jun 18 09:06 312 MB
Jun 19 (to 09:14) did not run
  • Jun 13 21:07 hit 511 MB writing zero new chunks — the build holds the memory regardless of writes.
  • Mitigation tried: add_filter( 'jetpack_sitemap_suspend_cache_addition', '__return_true' ) (the 15.0 filter). Stopped the hard OOMs, but the run still climbs to ~300–400 MB — so it's not just the object cache; the builder/data itself accumulates.
  • Suggested fix: stagger and resume — cap work per run by item count (filterable, e.g. ~75,000 media items), persist state, continue next cron run so memory is reclaimed between runs; and/or free per-chunk data (DOMDocument/buffer, primed caches) after each chunk.

Platform (Simple and/or Atomic)

Atomic

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugWhen a feature is broken and / or not performing as intendedNeeds triageTicket needs to be triaged

    Type

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions