Repository navigation
feat(infra): add a capped staging stack on the production box - #841
Open
anneoneone wants to merge 1 commit into
Open
anneoneone wants to merge 1 commit into
anneoneone wants to merge 1 commit into
Conversation
Staging runs dev/flow images at staging.live.moafunk.de before a release reaches production. For now it shares the production box as a separate compose project with hard CPU and memory caps; it moves to its own server later. staging.sh refuses to start when a setting could reach production or real accounts (buckets, harbor, SoundCloud, GitHub dispatch, Telegram, Instagram, OpenAI, recovery mail), when production is on air or a show starts within 2 hours, or when disk or memory is low. seed-db makes a copy of the production DB without sessions, stored tokens or usable passwords. staging.test.sh covers every refusal and runs in CI. Refs: #840
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a staging stack for
dev/flowimages at https://staging.live.moafunk.de. For now it runs on the production box as a separate compose project with hard CPU and memory caps. It moves to its own server later. Nothing changes for production until someone runs the setup on the box.Issue: #840
Validation:
bash backend/staging/staging.test.sh(macOS bash 3.2): 19 of 19 pass. Every unsafe setting is refused, the safe one passes, and the untouched example is refused. The newstaging-tests.ymlruns it on Ubuntu.docker compose config(withSTAGING_TAG=sha-16c560e): valid. Caps resolve to API 0.75 CPU / 640 MB, Liquidsoap 0.5 / 320 MB, Icecast 0.25 / 96 MB, allrestart: "no", API only on127.0.0.1:8001. Without asha-*tag, compose refuses to start.nginx -tonnginx-staging.conf(nginx 1.27): ok.icecast.staging.xml: well-formed XML.setup,up,seed-dbagainst the real box. My SSH to the production box is blocked, so Anton runs them.Implementation and review
Not architecture-epic work, so the epic fields do not apply.
Epic: n/a
Executor: n/a
Lane: n/a
Reviewer: n/a
Leaf IDs: n/a
What
staging.shchecks before it starts anything.env:APP_ENV=staging,ADMIN_BASE_URL= the staging host,SOUNDCLOUD_AUTO_UPLOAD=false(the default is on),TELEGRAM_ENABLED=false, both bucket names are staging buckets (unset names fall back tomoafunk-prod), producer and status URLs point at the staging harbor 8015 and Icecast 8020. NoGITHUB_DISPATCH_TOKEN(it would start the production backup workflow), SoundCloud, Telegram, Instagram, Facebook, OpenAI orRECOVERY_*keys.<placeholders>.stream.env: ports 8015/8020, all passwords set, none equal to a production password from/etc/moafunk/stream.env(values are never printed)./api/stream/statussays not active, and no show is live or starts within 2 hours (read-only query of the production DB).Isolation
/opt/unheard-staging/data.seed-dbcopies production with the SQLite backup API, then deletes sessions, deletesapp_settingsrows with token or secret keys (the SoundCloud token lives there) and makes every password unusable.moafunk.liqunchanged (it reads its ports from the environment) and the existingmoafunk/icecast-kh:kh22image (pull_policy: never). Icecast's health check is overridden to port 8020.nginx -tpasses, is never overwritten on re-run (certbot edits it), and gets TLS only when DNS already points at the box.Box steps (Anton or phaabe): DNS A record, R2 buckets plus a token scoped to them, Bitwarden
STAGING_*values, copy the files,staging.sh setup, optionallyseed-db, staging test accounts. All inbackend/staging/README.md.Goal: #829.