[pull] main from LibreChat-AI:main - #16
Merged
Merged
Conversation
Deployments currently have to track main, which advances whenever an internal snapshot is merged. Cut versioned tags instead, each carrying the packaged Helm chart so a deployment can pin one. The tag is the app version and must match helm/codeapi/Chart.yaml appVersion, so a deployed chart cannot report a version no release ever carried. `latest` moves only for the highest stable tag, and the chart is packaged before the tag is created so a rate-limited subchart pull leaves the version unused and the run retryable. Closes #63
* feat: add local NsJail runtime profile * fix: refresh stale local runtimes * fix: surface local runtime state loss
* fix(lambda-microvm): give each PTC replay iteration a distinct clientToken PTC replay reuses one executionId across every stateless iteration, but the launch clientToken was derived from that executionId alone. Each iteration relaunches with a changed sandbox payload (a fresh _ptc_history.json), so AWS rejected the second launch with "The provided clientToken was used with different request parameters" and LibreChat surfaced the generic "Bash programmatic execution failed" (#59). Fold the launch inputs and the per-iteration request body into the token so each distinct launch gets a distinct token while an identical retry stays idempotent. Reuses runtimeSessionLaunchRequestFingerprint rather than restating the launch inputs. Reported with a working patch by @snapydziuba. * fix(lambda-microvm): key the stateless launch token to the queued job Addresses codex review on #82. Hashing the request body made the token move between attempts of the same job: workers.ts rebuilds the request on every attempt with a fresh egress grant (random IV and sandbox session id) and a re-signed manifest, so a replacement worker taking over a stalled job would derive a different token, launch a second VM, and leave the accepted one burning capacity until its maximum duration expired. Use the queued job id instead. Each PTC replay iteration is enqueued as its own job, so it is distinct per iteration and stable across attempts of the same job -- and it carries no capability-bearing material. The launch configuration stays in the digest so a worker with a different config cannot reuse another's token.
* fix: preserve uploaded filenames without s3 metadata * fix: preserve filenames through hardened egress
Guest clock drift past the 30s execution-manifest tolerance makes every /v1/exec fail with "not_yet_valid" while both health endpoints keep reporting healthy, so the stack looks fine while nothing runs (#37). The healthcheck already detects this, but it stays disabled unless an orchestrator opts in, and the Compose files never did -- only the Helm chart set it. Opt in there too, at the same 10s the chart uses. The 2s probe timeout keeps the check inside both files' healthcheck timeouts (3s and 5s) and leaves headroom under the 30s tolerance.
* fix(helm): resolve Bitnami subcharts over OCI Bitnami distributes charts OCI-only. The classic charts.bitnami.com index still lists redis 24.1.0 and minio 17.0.21, but resolves them to an oci:// download URL that HTTP-repository getters cannot follow, so FluxCD's source-controller fails dependency resolution outright with 'unsupported protocol scheme "oci"' (#21). Point both dependencies at the OCI registry directly. Requires Helm >= 3.8. Reported by @meroo36. * docs(helm): require Helm >= 3.8 for OCI subchart resolution Addresses codex review on #83. The README's "Helm 3.x" prerequisite and setup-local.sh's existence-only check both allowed 3.0-3.7, where OCI dependency references are not resolved without an experimental flag -- so the documented setup flow would fail at dependency resolution rather than with a clear message. State the real minimum, and reject older Helm in setup-local.sh before it gets that far. Also drop the classic bitnami repo registration, which the OCI references no longer use; verified 'helm dependency update' resolves both subcharts with that repo removed from the local Helm config. * chore(helm): bump chart to 0.3.1 for the dependency source change Addresses codex review on #83. Changing where the subcharts resolve from is a chart-level change, and Chart.yaml's own version comment asks for a bump. Leaving 0.3.0 in place lets consumers reconciling on chart version treat the corrected chart as the already-seen 0.3.0 artifact and keep the broken HTTP dependency metadata. Matches 4b72e9d, which bumped the chart for the same reason.
* feat: add networkless BYOM file relay * fix: harden managed file relay * fix: fence relay lifecycle races * fix: order relay handoffs by registration * fix: gate relay workers on runtime readiness * fix: preserve legacy marker listings * fix: recover reclaimed relay staging
* 🗑️ fix: Make Code Environment File Deletion Work Object deletion has never removed anything, and the failure was silent at every layer. The client (LibreChat `deleteCodeEnvFile`) issues DELETE against `/v1/sessions/:session_id/objects/:fileId`, the file-server's own path, which is not exposed on `/v1` — only GET is mounted there. Every deletion 404'd, and a 404 is indistinguishable from "already gone", so the caller cleared its state and the bucket only ever grew (13 GiB / 29k objects on a six-week-old deployment, per danny-avila/LibreChat#15511). Mount DELETE on that path as an alias of `/v1/files/:session_id/:fileId`, so deployments running a client older than LibreChat v0.8.6 — before the fallback to `/files/...` landed — delete successfully. Pass the file-server's 404 through instead of collapsing it into a 500: a 500 reads as retryable, and a client sweeping its retention window re-issues the same DELETE hourly, forever, for an object that no longer exists. Correcting the route is not sufficient on its own. `sessionAuth` authorizes deletion against `session:<session_id>`, whose `SESSION_CACHE_TTL` is 24h and is not refreshed by use, so an object was deletable only for the day following upload and stranded permanently after that — unreadable, unusable as an execution input, and undeletable through every route. Clients are typically far outside that window when they get there; LibreChat's default retention is 30 days. Record ownership twice: `session:<id>` stays the hot-path cache bounding read access, and a durable `session-owner:<id>` record (`SESSION_OWNER_TTL`, 90 days, never shorter than the cache TTL) backs deletion once the cache key has lapsed. The fallback applies to DELETE only — reads keep the window they have always had — and a live cache key naming a different owner remains authoritative, so a re-registered session is never deletable by its previous owner. The recovery script restores both records, so a rehydrated session stays deletable rather than stranding again a day later. * fix: Close Codex review findings on session ownership Four P2 findings from the review of daae56c: - The 404 deletion path cleared the upload key with a bare `await` inside the catch block. A Redis failure there rejects with no handler above it, and Express 4 does not forward async rejections, so the request would hang instead of answering 404. Make the cleanup best effort and log it. - The blocking PTC path discarded the registration promise with `void`, preserving the previous fire-and-forget behavior. That now spans two keys: a partial write (cache key stored, durable record refused by a Redis ACL scoped to `session:*`) would produce exactly the undeletable files this change exists to prevent. Await it; the caller turns a rejection into a 500 before anything is enqueued. - Recovery treated a durable owner record naming someone else as a log line while still counting the session as restored or matching, so an apply could exit 0 having recovered nothing usable. Reconcile the owner record before touching the cache key and report the disagreement as a conflict, in dry run as well as apply. The cache key is no longer restored for those sessions either — the manifest's claimant should not get a day of access the service never granted it. - `SET NX` cannot extend an expiry, so a matching owner record could carry less remaining TTL than the cache key being restored and lapse first, stranding the session again just as recovery reported success. Top up the expiry when it is shorter than the target, leaving longer ones alone. * fix: Settle session ownership before recovery writes anything Two findings from the review of 816dbb5: - Reconciling the durable owner record first meant creating it before the live cache key had been consulted. For a session whose durable record was absent and whose cache key named a different owner, recovery wrote a durable record for the manifest's claimant, then reported the cache conflict and moved on — leaving the record behind. It outlives the cache key by design, so once that expired, `sessionAuth` would authorize the manifest owner to delete the real owner's files. Split the read from the write. The durable record is now inspected read-only up front, where a disagreement still settles the session before anything is written, and is created or extended only once the cache key has been confirmed to name the same owner. - `/exec` registered ownership before entering the route's `try`. Express 4 does not forward a rejected async handler to the error middleware, so a Redis failure there would hang the request rather than answering. Guard it and return a controlled 500. * fix: Harden recovery's durable owner handling Three findings from the review of 47b36e0, all in the recovery script: - A `SET NX` that lost the race to a key which then expired before the follow-up read left no record and no conflict, and the session was reported as recovered while its durable half did not exist. Retry once, and report anything past that as missing so an apply exits nonzero instead of claiming success. - When the owner commit conflicted on a session whose cache key this run had just created, the cache key stayed. That grant authorizes reads and deletes for its full TTL while the durable record names somebody else, so roll it back. A durable record that merely could not be created is left alone: the session is no worse off than before the run, and removing the grant would leave the operator with nothing. - A dry run reported a session whose cache key already matched as `matching` even when its owner record was absent or short-lived, hiding the work an apply would do and inviting operators to skip it. Pending owner repairs now count as missing. * fix: Answer Redis failures instead of hanging on them Two findings from the review of d1de664: - `sessionAuth` awaited the ownership lookup unguarded. Express 4 does not forward a rejected async middleware, so an unavailable Redis — or an ACL granting `session:*` but not `session-owner:*` — would hang a DELETE rather than answering it. Catch and return a controlled 500. - The recovery script's TTL top-up read, extended and returned across three round trips, reporting success on evidence it had not rechecked. The record can lapse in between, in which case it is now created fresh, or name somebody else, in which case the session is a conflict. Extending a record that turns out to belong to another owner prolongs a claim the service itself wrote and grants nothing new, but reporting the session as recovered on that basis would not be true.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )