TorBox requestdl 502 during streaming poisons rclone/FUSE mount (Socket not connected until restart)

[0oAstro]

Is there an existing issue for this?

• I have searched the existing open and closed issues

Related but not exact duplicates: #92, #179, #300, #302, PR #217, PR #303.

Current Behavior

When using TorBox via Decypharr's rclone/FUSE mount, transient TorBox requestdl failures with HTTP 502 during streaming/probing poison the mount. After the 502, clients start seeing filesystem errors such as:

• Socket not connected
• Transport endpoint is not connected
• Input/output error
• stuck reads/ffprobe in uninterruptible D state

This affects Jellyfin library scans/playback and *arr stat/import checks against symlinked media. Restarting/remounting Decypharr clears it temporarily, but the issue recurs when Jellyfin scans/probes many files or large remuxes.

Important: this is not the old TorBox / vs ? requestdl URL bug from PR #217. This deployment already includes that fix, and also includes a local build with PR #303 (CDN URL caching) and PR #305.

Expected Behavior

A transient TorBox requestdl HTTP 502 should not poison the FUSE/rclone mount or leave consumers stuck.

Ideally Decypharr should:

1. Treat TorBox requestdl 502/5xx as transient.
2. Retry with backoff/jitter before returning failure to rclone/WebDAV.
3. Invalidate any cached download link for that file when CDN/link fetch fails.
4. Return a clean retryable error if all retries fail, without wedging the mount.
5. Avoid leaving rclone/FUSE reads stuck in a state where future stat/read calls return Socket not connected until container restart.

Steps To Reproduce

1. Configure Decypharr with TorBox and rclone/FUSE mount.
2. Import symlinked media into Jellyfin/Radarr/Sonarr.
3. Let Jellyfin scan/probe several FUSE-backed files, especially anime episodes or large remuxes.
4. Eventually TorBox returns 502 for a requestdl/download-link fetch.
5. After this, symlinked files that previously resolved can fail with Socket not connected or hang in reads/ffprobe until Decypharr is restarted/remounted.

Environment

- OS: Linux / Docker
- Decypharr version: 2.3.1 beta local build
- Image: beta + PR #303 + PR #305 (`decypharr-local:beta-pr303-pr305`)
- Debrid provider: TorBox
- Mount type: rclone/FUSE
- Consumers: Jellyfin, Radarr, Sonarr
- TorBox throttling in config: low/conservative (rate/download/repair roughly 1/minute, workers 1)

Relevant Logs

Decypharr WebDAV/link-fetch failures:

2026-05-25 23:58:30 | ERROR | [webdav] Error streaming file: Frieren Beyond Journeys End S01E07 ... error="failed to get download link: 502: unknown error code: 502"
2026-05-26 00:03:33 | ERROR | [webdav] Error streaming file: Frieren Beyond Journeys End S01E01-E04 ... error="failed to get download link: 502: unknown error code: 502"
2026-05-26 00:04:32 | ERROR | [webdav] Error streaming file: Frieren Beyond Journeys End S01E05 ... error="failed to get download link: 502: unknown error code: 502"
2026-05-26 00:09:32 | ERROR | [webdav] Error streaming file: Frieren Beyond Journeys End S01E09 ... error="failed to get download link: 502: unknown error code: 502"
2026-05-26 00:10:33 | ERROR | [webdav] Error streaming file: Frieren Beyond Journeys End S01E10 ... error="failed to get download link: 502: unknown error code: 502"
2026-05-26 00:12:32 | ERROR | [webdav] Error streaming file: Frieren.Beyond.Journeys.End.S01E12... error="failed to get download link: 502: unknown error code: 502"

After these errors, clients see filesystem failures:

stat: cannot statx '/data/media/movies/The Devil Wears Prada (2006)/The.Devil.Wears.Prada.2006.EUR.BDRemux.1080p.mkv': Socket not connected
stat: cannot statx '/data/media/movies/F1 (2025)/F1.The.Movie.2025.2160p...mkv': Socket not connected
stat: cannot statx '/data/media/movies/Project Hail Mary (2026)/Project.Hail.Mary.2026...mkv': Socket not connected

Jellyfin can leave ffprobe stuck in D state while probing a FUSE-backed file:

/usr/lib/jellyfin-ffmpeg/ffprobe ... -i file:/data/media/anime/Frieren - Beyond Journey's End/Season 1/Frieren - Beyond Journey's End - S01E13 - Aversion to One's Own Kind WEBDL-1080p.mkv ...

Decypharr/rclone restart/remount clears the issue temporarily:

[rclone] Successfully mounted rclone filesystem

After restart/remount, the same file can stat/read successfully again:

stat rc 0 out 35909270356 regular file
read4k rc 0

Trace Logs have been provided as applicable

• Logs above include the relevant failing path and symptoms. Full trace can be gathered if needed, but the issue is intermittent and reproduces during TorBox 502s under Jellyfin probing/streaming.

[Razor5284]

+1
Same failure mode on Synology NAS (DFS / hanwen), with healthcheck and host-level wedge
I’m seeing the same pattern described here on a Synology NAS running Decypharr in Docker with TorBox and mount.type: dfs (hanwen backend, mount_path: /mnt/decypharr). This is not the old requestdl / vs ? bug - I’m on 2.3-stable.

Symptoms (match this issue closely)

1. Transient TorBox / mount errors:

• ls / reads on /mnt/decypharr/__all__/...: Input/output error (partial listing sometimes continues).
• Sonarr/Plex: Transport endpoint is not connected on stale rslave mounts after Decypharr restart (fixed by docker restart consumers, not Decypharr alone).

2. HTTP UI vs mount health

• curl http://127.0.0.1:8282/ still returns 200 while mount paths are broken.
• Symlinks in library point at /mnt/decypharr/__all__/...; playback/import fails even when UI is up.

3. Image healthcheck amplifies failure

• When mount is bad, docker exec decypharr … healthcheck hangs.
• Docker/Synology then logs: "Timed out starting health check for container" every ~30s, FailingStreak in hundreds, for every other container with a healthcheck on my system.
• All containers with a healthcheck show orange in Container Manager, which looks like a fleet outage but is actually caused by Decypharr mount and docker exec timeouts.
  Workaround: healthcheck: disable: true in compose + --force-recreate.

4. Host-level wedge (worst case)

• After overnight load, NAS became unresponsive: SSH Connection reset / Connection closed, sensors in Home Assistant unavailable, had to hard reset (pkill & graceful shutdown also hung!!).
• Before wedge: docker compose stop decypharr hung 400s+; docker kill also ineffective until reboot.
• Decypharr web UI still responded briefly while SSH and Sonarr were dead — similar to “light HTTP up, system dying.”

5. Recovery

• Only reboot or SIGKILL + lazy umount /mnt/decypharr + recreate Decypharr reliably clears it.
• Restarting Sonarr/Radarr/Lidarr/Plex after Decypharr is required for rslave consumers.

Extra info:

Environment

• Host: Synology DSM 7, Docker / Container Manager
• Image: cy01/blackhole:latest (Decypharr ~2.3-stable)
• Mount: DFS (hanwen), mount_path=/mnt/decypharr
• Providers: TorBox (primary for TV), Real-Debrid also enabled → all aggregation
• Consumers: Sonarr, Radarr, Lidarr, Plex (linuxserver/plex)
• Compose: /mnt:/mnt:rshared on decypharr; *arr + Plex bind /mnt/decypharr:rslave
• Cache: /volume2/decypharr-cache:/cache (moved out of BTRFS-snapshotted docker share)
• PUID/PGID: 1026/101

What I’ve already ruled out / mitigated

• PR fix(torbox): fix download link URL construction (/ vs ?) #217 - on stable; Torbox paths did work at times... Until eventually the above happens.
• Torbox temporarily removed - currently running an A/B test for stability.

Overlap with this issue
Strong correlation with heavy read/probe activity (Plex library, Sonarr import/ffprobe, all + RD).
all readdir often prints I/O error once per directory even when some entries list — consistent with one bad provider entry poisoning aggregation.
I have not tested a custom build with PR #303 + PR #305 yet; planning beta + #303 when re-enabling Torbox. Reporting now because stable + #217 alone is not sufficient on this setup.

Request (aligns with OP)
Would be very helpful to have in upstream:

1. Retry/backoff on TorBox requestdl / CDN fetch 5xx before error reaches DFS/FUSE.
2. Invalidate cached download link on failure.
3. Ensure FUSE/hanwen returns retryable errors without leaving the mount in “Socket not connected” state for unrelated paths.
4. Document that healthcheck can block on bad mounts - or make healthcheck HTTP-only / timeout-bounded.
5. Happy to provide Decypharr logs around 502 / [webdav] Error streaming / [dfs] if needed; I can also test a beta + cache TorBox CDN download URLs to reduce API call frequency #303 image when re-enabling Torbox and report back here.

Thanks for filing this - it matches production pain on Synology + arr + Plex better than any single “Torbox config” tweak.

(written with help of AI due to the complexities and extensiveness of my debugging - this has been a nightmare on my NAS setup since it's basically broken all containers when this issue presents. Also apols for the long message.)

[TwistedRat]

I have not had the time to set myself in to the detail if the project code myself but I had some AI tokens to burn so thought I might as well put them to a possible use. Take any suggestions here by the AI as just that, a possible solution only.

Root cause analysis and fix for the mount-poisoning 502 bug
Three independent failure points combined to make a single transient 502 from TorBox's CDN break all subsequent file reads until container restart.

Failure point 1 — 502 and 504 classified as permanent errors
ErrorCodeToLinkError had no case for 502 or 504, so both fell through to the default branch which returns CategoryPermanent. A single 502 from a HEAD validation request permanently blacklisted the CDN URL in the s.validated cache. Every future GetLink call for that file returned an unrecoverable error from cache without ever hitting the network again.

Failure point 2 — Retryable validation errors written to the validation cache
Even after fixing the category, any retryable validation error (429, 502, 503, 504) was still stored in s.validated. The first transient failure would block all reads for that file until the cache was explicitly cleared (via account-disable or restart). Retryable errors should never be cached — they need to re-run validation fresh on every attempt.

Failure point 3 — CDN 5xx responses not triggering the retry loop in doRequest
doRequest only retried when streamClient.Do itself returned an error (network-level failure). A valid HTTP 502 response left doErr == nil, so retry.Do exited without retrying. The CDN failure was never recovered.

Root cause of excessive API calls (related — #300)
fetchDownloadLink stored the requestdl?redirect=true proxy URL as the download link. Every byte-range GET — one per chunk from rclone/FUSE — hit the TorBox API to obtain the CDN redirect. A single 2-hour file generates ~375 API calls, and bulk operations could reach tens of thousands, causing persistent 429 rate-limit errors that fed back into the poisoning loop above.

---

Fix 1 — Correctly classify 502 and 504 as retryable

--- a/pkg/manager/link/errors.go
+++ b/pkg/manager/link/errors.go
@@ -94,7 +94,9 @@ var (
 var (
      Err404 = errors.New("HTTP 404 Not Found")
      Err429 = errors.New("HTTP 429 Too Many Requests")
+     Err502 = errors.New("HTTP 502 Bad Gateway")
      Err503 = errors.New("HTTP 503 Service Unavailable")
+     Err504 = errors.New("HTTP 504 Gateway Timeout")
 )

@@ -145,8 +147,12 @@ func ErrorCodeToLinkError(code string) *Error {
      case "429":
              return NewRetryableError(Err429, code)
+     case "502":
+             return NewRetryableError(Err502, code)
      case "503":
              return NewRetryableError(Err503, code)
+     case "504":
+             return NewRetryableError(Err504, code)
      default:
              return NewPermanentError(fmt.Errorf("unknown error code: %s", code), code)
      }

Fix 2 — Don't cache retryable validation errors

--- a/pkg/manager/link/service.go
+++ b/pkg/manager/link/service.go
@@ -145,8 +145,13 @@ func (s *Service) fetchAndValidate(...) {
-     // Store validation result
-     s.validated.Store(link.DownloadLink, validationErr)
+     // Only cache permanent/refetchable failures and successes.
+     // Retryable errors (502, 503, 429) must not be cached: a single transient
+     // failure would block all future reads for the file until the cache is
+     // explicitly cleared (account-disable or restart).
+     if linkErr := GetLinkError(validationErr); linkErr == nil || !linkErr.ShouldRetry() {
+             s.validated.Store(link.DownloadLink, validationErr)
+     }

Fix 3 — Treat CDN 5xx responses as retryable errors in doRequest

--- a/pkg/manager/stream.go
+++ b/pkg/manager/stream.go
@@ -368,6 +368,16 @@ func (m *Manager) doRequest(...) {
                      return retry.Unrecoverable(StreamError{Err: doErr, Retryable: true})
              }
+
+             // A 5xx from the CDN is a transient upstream failure, not a
+             // connection error, so doErr is nil and the retry loop would exit
+             // without retrying. Treat it as retryable here so retry.Do fires.
+             if resp.StatusCode >= 500 {
+                     status := resp.StatusCode
+                     resp.Body.Close()
+                     resp = nil
+                     return fmt.Errorf("CDN returned HTTP %d", status)
+             }
              return nil
      },

Fix 4 — Resolve the actual CDN URL from requestdl once instead of per chunk

--- a/pkg/debrid/providers/torbox/torbox.go
+++ b/pkg/debrid/providers/torbox/torbox.go
@@ -479,29 +479,44 @@ func (tb *Torbox) fetchDownloadLink(...) {
-     query := url.Values{}
-     query.Set("token", account.Token)
-     query.Set("torrent_id", id)
-     query.Set("file_id", file.Id)
-     query.Set("redirect", "true")
-
-     downloadURL := fmt.Sprintf("%s/api/torrents/requestdl?%s", tb.Host, query.Encode())
+     var res DownloadLinksResponse
+
+     resp, err := tb.doGet("/api/torrents/requestdl", map[string]string{
+             "token":      account.Token,
+             "torrent_id": id,
+             "file_id":    file.Id,
+     }, &res)
+     if err != nil {
+             return types.DownloadLink{}, err
+     }
+     if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+             return types.DownloadLink{}, fmt.Errorf("torbox requestdl error: HTTP %d", resp.StatusCode)
+     }
+     if !res.Success || res.Data == nil || *res.Data == "" {
+             return types.DownloadLink{}, fmt.Errorf("torbox: empty CDN URL from requestdl: %s", res.Detail)
+     }
+
+     // TorBox CDN URLs are valid for 3 hours. Never cache them longer than that
+     // regardless of the user's auto_expire_links_after setting, otherwise we
+     // serve stale URLs that return 403/404 from the CDN.
+     const torboxCDNExpiry = 3 * time.Hour
+     expiry := torboxCDNExpiry
+     if tb.autoExpiresLinksAfter > 0 && tb.autoExpiresLinksAfter < torboxCDNExpiry {
+             expiry = tb.autoExpiresLinksAfter
+     }
+
      now := time.Now()
-     dl := types.DownloadLink{
+     return types.DownloadLink{
              Filename:     file.Name,
              Size:         file.Size,
              Token:        tb.APIKey,
              Link:         file.Link,
-             DownloadLink: downloadURL,
+             DownloadLink: *res.Data,
              Debrid:       tb.config.Name,
              Id:           file.Id,
              Generated:    now,
-             ExpiresAt:    now.Add(tb.autoExpiresLinksAfter),
-     }
-     return dl, nil
+             ExpiresAt:    now.Add(expiry),
+     }, nil
 }

Fix 5 — Honour CDN URL expiry in the validation cache

--- a/pkg/manager/link/service.go
+++ b/pkg/manager/link/service.go
@@ -107,6 +108,13 @@ func (s *Service) fetchAndValidate(...) {
      if validationErr, exists := s.validated.Load(link.DownloadLink); exists {
              if validationErr == nil {
+                     // Re-fetch if the cached CDN URL has passed its declared expiry.
+                     // Without this check a 3-hour TorBox CDN URL would be served from
+                     // s.validated forever — bypassing the HEAD validation that would
+                     // otherwise catch the expired URL.
+                     if !link.ExpiresAt.IsZero() && time.Now().After(link.ExpiresAt) {
+                             return s.invalidateAndRefetch(ctx, entry, link, attempt)
+                     }
                      return link, nil // Already validated successfully
              }

Result

Scenario	Before	After
TorBox CDN returns 502	URL permanently blacklisted in cache, mount broken until restart	Classified retryable, not cached, retried with existing backoff budget
First 502 during HEAD validation	All future reads for that file fail from cache	Validation reruns fresh on every retry
CDN 5xx during streaming	retry.Do exits immediately (no network error), stream stuck	Detected and returned as a plain error so retry.Do fires
Byte-range chunk GET	One requestdl API call per chunk (~375 calls per file)	One requestdl call per file per 3-hour CDN expiry
Expired CDN URL in validation cache	Stale URL served indefinitely, 403/404 on every read	Re-fetched automatically when ExpiresAt passes