Version: v10.3.16 (self-hosted, community). Code references below are against main.
⚠️ Disclaimer / provenance: this investigation and write-up were done by Claude (Anthropic) — "vibe coding" while a user was setting up Plex → Ryot on Unraid. The debugging is empirical (RUST_LOG=debug + direct DB inspection on a live instance) and the source reasoning is from the public repo. Line numbers are best-effort; please sanity-check. Posting in case it's useful — feel free to close/split as you see fit. Related ongoing work: the TEMP(1611) instrumentation in these files.
Not a duplicate of #814. That issue shared the surface symptom ("Plex webhook not working") but was a username-mismatch config error and is resolved. The three problems below are code-level bugs that reproduce with a correct username; cross-linking only so searchers who land on that popular issue can find these.
While wiring up a Plex Sink integration, nothing was ever recorded despite the webhook endpoint returning 202. Enabling backend logging (the backend logs nothing by default at the shipped level) revealed three separate problems on the path from "webhook received" to "seen row written". Each is independently reproducible.
Bug 1 — Option::unwrap() panic when an integration has NULL minimum_progress / maximum_progress
This is the primary blocker. The webhook job panics and aborts, so no seen row is written.
Backend log:
thread 'main' panicked at crates/services/integration/src/webhook_handler.rs:63:79:
called `Option::unwrap()` on a `None` value
...apalis... Worker encountered an error: AbortError: PanicError: called `Option::unwrap()` on a `None` value
Source (crates/services/integration/src/webhook_handler.rs, integration_progress_update):
.retain(|update| match update.progress {
Some(progress) if progress < integration.minimum_progress.unwrap() => { ... } // line ~63
_ => true,
});
...
if let Some(progress) = update.progress
&& progress > integration.maximum_progress.unwrap() // line ~72
integration.minimum_progress / maximum_progress are Option<Decimal> and can be NULL in the DB. They are optional in CreateOrUpdateUserIntegrationInput, so any integration created via the GraphQL API without explicitly setting them (or, presumably, any row where they end up null) makes every incoming webhook panic.
Repro:
createOrUpdateUserIntegration(input: { provider: PLEX_SINK, providerSpecifics: { plexSinkUsername: "x" }, extraSettings: { disableOnContinuousErrors: false } }) — note no minimumProgress/maximumProgress.
POST /_i/<integration_id> with a valid Plex payload.
payload len: N is logged, an ImportResult with 1 item is built, then panic at the line above.
Fix: unwrap_or with the intended defaults (e.g. .unwrap_or(dec!(2)) and .unwrap_or(dec!(100))), or skip the threshold comparison when the field is None.
Workaround on our instance: UPDATE integration SET minimum_progress=2, maximum_progress=95 WHERE minimum_progress IS NULL OR maximum_progress IS NULL; — after this the webhook processes without panicking and a seen row is written.
Bug 2 — Webhook endpoint returns HTTP 400 when the multipart contains Plex's thumb part
Plex always sends webhooks as multipart/form-data with a payload field plus a thumb image part. The endpoint rejects any request that includes the thumb part.
Plex Media Server log:
WARN - Webhook: Error delivering payload to http://.../_i/<id>: 400
(with Webhook: Delivering media.play event by user ... to 1 hooks immediately before — Plex does fire and reach the server, including for Plex Home users under the owner's webhook.)
Reproduction (from any host that can reach the instance):
Request to /_i/<id> |
Result |
-F payload=<valid json> (payload only) |
202 "Webhook queued for processing" |
-F payload=<valid json> -F thumb=@img.jpg (1 KB) |
400 |
-F payload=<valid json> -F thumb=@img.jpg (150 KB) |
400 |
-d <json> with Content-Type: application/json |
202 |
It is not a body-size limit (1 KB thumb also fails; our reverse proxy allows 2000m). It looks like the multipart extractor doesn't tolerate the extra thumb field that Plex always includes, so native Plex delivery can never succeed even when everything else is correct. (We are working around this by feeding clean JSON from Tautulli instead of Plex's native webhook.)
Bug 3 — A single media.scrobble (progress = 100) never completes; the first event discards the reported progress
After fixing Bug 1, a single scrobble records a seen row but leaves it at progress = 0, state = in_progress instead of completed.
Source (crates/utils/dependent/progress/src/lib.rs, commit_import_seen_item, is_import == false branch):
- If
in_progress_cache is None (first event for the item), it builds MetadataProgressUpdateChange::CreateNewInProgress { data: common, started_on: now } — input.progress is not used at all, so a scrobble carrying progress = 100 still creates a 0%/in-progress seen.
- Only a subsequent event takes the
ChangeLatestInProgress(progress) path, where progress >= 100 finally transitions to completed.
Backend log for a single scrobble: ... in_progress_cache is None, creating new in-progress seen → seen stays in_progress, progress = 0.
Two events confirm the completion path:
#1 in_progress_cache is None, creating new in-progress seen -> progress=0, in_progress
#2 Updating in-progress seen ... with progress 100
Progress >= 100, expiring in-progress cache and setting completed cache -> progress=100, completed
Impact: consumers that emit a single "watched"/scrobble event (Plex's media.scrobble, or a Tautulli "watched" webhook) leave the item permanently at 0% in-progress. In normal Plex playback multiple events arrive (media.play … media.scrobble), so it often works by accident, but a lone completion event does not.
Fix: honor input.progress in the CreateNewInProgress branch — if progress >= 100, create it completed (or apply the progress and run the same completion check as the update branch).
Sub-note — in-progress/completed cache can desync from the DB
commit_import_seen_item gates on application_cache (MetadataProgressUpdateInProgressCache / ...CompletedCache). If the underlying seen rows are removed out-of-band (we deleted test rows directly), the stale cache makes the next event log No in-progress seen found ... when trying to change progress and silently do nothing, or short-circuit as "already completed". Worth a fallback that reconciles cache misses against the DB.
Summary
| # |
Symptom |
Root cause |
Suggested fix |
| 1 |
Webhook panics, nothing recorded |
unwrap() on null minimum_progress/maximum_progress (webhook_handler.rs:~63/72) |
unwrap_or defaults / handle None |
| 2 |
Plex native delivery → 400 |
multipart with thumb rejected |
tolerate/ignore extra thumb part |
| 3 |
Single scrobble stuck at in-progress 0% |
first event ignores input.progress (progress/src/lib.rs CreateNewInProgress) |
honor progress, complete if >= 100 |
Version: v10.3.16 (self-hosted, community). Code references below are against
main.While wiring up a Plex Sink integration, nothing was ever recorded despite the webhook endpoint returning
202. Enabling backend logging (the backend logs nothing by default at the shipped level) revealed three separate problems on the path from "webhook received" to "seen row written". Each is independently reproducible.Bug 1 —
Option::unwrap()panic when an integration has NULLminimum_progress/maximum_progressThis is the primary blocker. The webhook job panics and aborts, so no seen row is written.
Backend log:
Source (
crates/services/integration/src/webhook_handler.rs,integration_progress_update):integration.minimum_progress/maximum_progressareOption<Decimal>and can beNULLin the DB. They are optional inCreateOrUpdateUserIntegrationInput, so any integration created via the GraphQL API without explicitly setting them (or, presumably, any row where they end up null) makes every incoming webhook panic.Repro:
createOrUpdateUserIntegration(input: { provider: PLEX_SINK, providerSpecifics: { plexSinkUsername: "x" }, extraSettings: { disableOnContinuousErrors: false } })— note nominimumProgress/maximumProgress.POST /_i/<integration_id>with a valid Plex payload.payload len: Nis logged, anImportResultwith 1 item is built, then panic at the line above.Fix:
unwrap_orwith the intended defaults (e.g..unwrap_or(dec!(2))and.unwrap_or(dec!(100))), or skip the threshold comparison when the field isNone.Workaround on our instance:
UPDATE integration SET minimum_progress=2, maximum_progress=95 WHERE minimum_progress IS NULL OR maximum_progress IS NULL;— after this the webhook processes without panicking and a seen row is written.Bug 2 — Webhook endpoint returns HTTP 400 when the multipart contains Plex's
thumbpartPlex always sends webhooks as
multipart/form-datawith apayloadfield plus athumbimage part. The endpoint rejects any request that includes thethumbpart.Plex Media Server log:
(with
Webhook: Delivering media.play event by user ... to 1 hooksimmediately before — Plex does fire and reach the server, including for Plex Home users under the owner's webhook.)Reproduction (from any host that can reach the instance):
/_i/<id>-F payload=<valid json>(payload only)202"Webhook queued for processing"-F payload=<valid json> -F thumb=@img.jpg(1 KB)400-F payload=<valid json> -F thumb=@img.jpg(150 KB)400-d <json>withContent-Type: application/json202It is not a body-size limit (1 KB thumb also fails; our reverse proxy allows 2000m). It looks like the multipart extractor doesn't tolerate the extra
thumbfield that Plex always includes, so native Plex delivery can never succeed even when everything else is correct. (We are working around this by feeding clean JSON from Tautulli instead of Plex's native webhook.)Bug 3 — A single
media.scrobble(progress = 100) never completes; the first event discards the reported progressAfter fixing Bug 1, a single scrobble records a seen row but leaves it at
progress = 0,state = in_progressinstead of completed.Source (
crates/utils/dependent/progress/src/lib.rs,commit_import_seen_item,is_import == falsebranch):in_progress_cacheisNone(first event for the item), it buildsMetadataProgressUpdateChange::CreateNewInProgress { data: common, started_on: now }—input.progressis not used at all, so a scrobble carryingprogress = 100still creates a 0%/in-progress seen.ChangeLatestInProgress(progress)path, whereprogress >= 100finally transitions to completed.Backend log for a single scrobble:
... in_progress_cache is None, creating new in-progress seen→ seen staysin_progress,progress = 0.Two events confirm the completion path:
Impact: consumers that emit a single "watched"/scrobble event (Plex's
media.scrobble, or a Tautulli "watched" webhook) leave the item permanently at 0% in-progress. In normal Plex playback multiple events arrive (media.play…media.scrobble), so it often works by accident, but a lone completion event does not.Fix: honor
input.progressin theCreateNewInProgressbranch — ifprogress >= 100, create it completed (or apply the progress and run the same completion check as the update branch).Sub-note — in-progress/completed cache can desync from the DB
commit_import_seen_itemgates onapplication_cache(MetadataProgressUpdateInProgressCache/...CompletedCache). If the underlyingseenrows are removed out-of-band (we deleted test rows directly), the stale cache makes the next event logNo in-progress seen found ... when trying to change progressand silently do nothing, or short-circuit as "already completed". Worth a fallback that reconciles cache misses against the DB.Summary
unwrap()on nullminimum_progress/maximum_progress(webhook_handler.rs:~63/72)unwrap_ordefaults / handleNonethumbrejectedthumbpartinput.progress(progress/src/lib.rsCreateNewInProgress)progress, complete if>= 100