JITSU-108: build console as a production dist in the dev chart - #1515
sahiltyagi-jitsu wants to merge 2 commits into
Conversation
The dev chart ran the console as `next dev` behind a background rsync loop that re-synced webapps/, libs/ and types/ from the host mount every 2 seconds. That made it the largest memory consumer in the dev stack, and the rsync hot-deploy was unreliable — races, missed changes, stale .next — on top of an apt-get install in every console pod. Build the console from the mounted source with `next build` in an init container and serve it with `next start`, the same model bulker and rotor already use: compile in an init container, run the output in the main container. Rebuild happens on redeploy. The init container is named `build`, so `./dev-deploy.sh build-logs console` now works for console too. Measured on minikube, before and after, same cluster and session: cgroup current 2,165 MiB -> 415 MiB (-81%) cgroup peak 2,499 MiB -> 416 MiB (-83%) next-server RSS 1,418 MiB -> 333 MiB (-76%) The cost moves from runtime to deploy rather than simply growing: pod created -> Ready 1m 01s -> 2m 20s first hit on / 3.48s -> 0.004s first hit on /signin 0.40s -> 0.021s `next dev` compiles routes lazily, so Ready only meant the healthcheck route had compiled; every other page paid a compile on first visit, after every restart. Build time falls as .next warms on the node-cache PVC: 2m 39s cold, then 1m 58s, then 1m 15s. In-cluster hot reload is gone. For console UI iteration use `pnpm console:dev` on the host with `scaling.console.replicas: 0`, which is faster and lighter than any in-cluster dev server — documented in helm/README.md. In that mode the console Deployment is not rendered at all, so no build runs either. The seed retry and the events-log-init loop are both kept: nothing else in the dev chart creates the ClickHouse events-log tables that power Live Events. Also drops the now-unused /project hostPath mount from the console pod. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The install job is a pre-install/pre-upgrade helm hook, so it only runs on deploy or upgrade. `dev-deploy.sh restart console` is a plain rollout restart and never triggers it, which left /cache/workspace holding whatever source was last copied. The rsync loop this PR removes had been keeping it fresh, so dropping it meant a restart silently rebuilt stale code — and the README told people to use exactly that command after editing. Sync webapps, libs and types from /project before building, the same reason bulker's init container copies from /project before it compiles. Dotfiles are excluded, so the cached .next survives and a stale host build is never copied over it. Verified on minikube: wrote a marker file into webapps/console, ran `./dev-deploy.sh restart console`, and read it back from /cache/workspace/webapps/console in the new pod. Caught by jitsu-code-review on #1515. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Good catch from the review — that was a real bug, now fixed in
The init container now syncs Verified on minikube rather than reasoned about: wrote a marker file into |
There was a problem hiding this comment.
Reviewed the console Helm deployment change and its developer documentation, including the rendered template and the production build/start flow. I also checked the existing review thread; the source re-sync in the build init container addresses that concern. No additional actionable findings.
Closes JITSU-108.
The dev chart ran the console as
next devbehind a background loop that rsyncedwebapps/,libs/andtypes/from the host mount every 2 seconds. That made it the largest memory consumer in the dev stack, and the hot-deploy was unreliable — races, missed changes, stale.next— on top of anapt-get install rsyncin every console pod.This builds the console from the mounted source with
next buildin an init container and serves it withnext start— the same model bulker and rotor already use: compile in an init container, run the output in the main container. Rebuild happens on redeploy.Measured
Minikube, before and after, same cluster and session.
next dev)next start)next-serverRSSThe trade
The cost moves from runtime to deploy rather than simply growing:
//signinnext devcompiles routes lazily, so "Ready" only meant the healthcheck route had compiled — every other page paid a compile on first visit, after every restart.Build time falls as
.nextwarms on thenode-cachePVC: 2m 39s cold, then 1m 58s, then 1m 15s.You only pay it when console is redeployed, and not at all in host-UI mode (
scaling.console.replicas: 0) where the Deployment isn't rendered.What this gives up
In-cluster hot reload. For console UI iteration use
pnpm console:devon the host — faster and lighter than any in-cluster dev server. Documented in a new "Developing the console" section inhelm/README.md.One behaviour change worth knowing: a build failure now fails the init container, so the pod won't start rather than showing the error in the browser.
./dev-deploy.sh build-logs consoleworks for console now — the init container is namedbuild, matching bulker/rotor.Kept deliberately
The seed retry and the
events-log-initloop. Nothing else in the dev chart creates the ClickHouse events-log tables that power Live Events — in prod theadmin/events-log-initroute does it.Also drops the now-unused
/projecthostPath mount from the console pod.Verified
next buildsucceeds in the init container with the console envevents-log-initboth still run (events-log ClickHouse tables ready)/api/healthcheck→ 200,Ready in 75ms./dev-deploy.sh restart consolerebuilds and comes back healthy.nextpersists across restarts, so rebuilds are incrementalNotes
Standalone output (
NEXTJS_STANDALONE_BUILD=1) was left unexplored — at an 81% reduction it isn't worth the extra moving part.The ticket also asks for the onboarding doc to point console iteration at
pnpm console:dev. That lives outside this repo and isn't covered here.Distinct from JITSU-48 (prod mode = published images for self-hosting); that one runs console from an image rather than building it. Both need
next build, so this lands the build plumbing first.🤖 Generated with Claude Code