Skip to content

feat(scripts): worktree-aware Docker isolation - #6479

Open
Fluf22 wants to merge 6 commits into
mainfrom
feat/worktree-better-support
Open

Fluf22 wants to merge 6 commits into
mainfrom
feat/worktree-better-support

Conversation

@Fluf22

@Fluf22 Fluf22 commented Jun 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Enable N worktrees to operate Docker simultaneously with full isolation.

What this does

  • Build once, run many: Images are tagged with a content-addressed hash of version inputs. First worktree builds, subsequent worktrees reuse the same images.
  • Container isolation: Each worktree gets its own Docker Compose project (via COMPOSE_PROJECT_NAME), so containers don't collide.
  • CTS port isolation: Each worktree claims a slot (0, 1, 2...) that offsets CTS test server ports by slot × 21, enabling parallel test execution.
  • Clean teardown: yarn docker:teardown stops containers and releases the slot.
  • Backwards compatible: Single-worktree setups and CI behave identically to before (slot 0, no offset, original ports).

Changes

File Change
docker-compose.yml Remove container_name, add image tags + extra_hosts
scripts/docker/setup.sh Content-addressed tags, worktree detection, slot claim, .env.docker
scripts/docker/teardown.sh New — stop containers, release slot
scripts/docker/slot.sh New — slot registry management
scripts/common.ts docker exec → docker compose --env-file .env.docker exec
scripts/config.ts getDockerImage → getDockerService
config/clients.config.json apic_base → base, apic_ruby → ruby, apic_swift → swift
generators/.../TestsClient.java Port offset for customHosts
scripts/cts/testServer/index.ts PORT_OFFSET in setupServer()
.gitignore Add .apic-worktree-slot, .env.docker
package.json Add docker:teardown script

How it works

Worktree 0 (main):  slot 0 → ports 6671–6691
Worktree 1:         slot 1 → ports 6692–6712
Worktree 2:         slot 2 → ports 6713–6733

Slot assignments are tracked in ~/.config/apic/worktree-slots.json. Stale entries (deleted worktrees) are auto-reclaimed.

Migration

Run yarn docker:setup — existing containers with hardcoded names (apic_base, etc.) can be removed manually when convenient.

@algolia-api-clients-automation-bot

algolia-api-clients-automation-bot Bot commented Jun 3, 2026 •

Copy link
Copy Markdown
Contributor

💥 Breaking changes detected !!

Either this PR or a previous PR not released yet introduced breaking changes, be careful when merging.
You can find the details in the 'client java@21' CI job.

✔️ Code generated!

Name Link
🪓 Triggered by 38f757b0f86e58a3cf52f8f01e35d99566453e27
🍃 Generated commit dba8c4865f86da133ea036c19ac8c3adccc3f063
🌲 Generated branch generated/feat/worktree-better-support
📊 Benchmark results

Benchmarks performed on the method using a mock server, the results might not reflect the real-world performance.

Language Req/s
javascript 2361
go 1890
php 1816
python 1378
csharp 1312
java 1128
ruby 967
swift 378
scala 23

@Fluf22 Fluf22 changed the title feat(docker): worktree-aware Docker isolation feat(scripts): worktree-aware Docker isolation Jun 3, 2026
@Fluf22
Fluf22 force-pushed the feat/worktree-better-support branch from f00115b to 036baab Compare June 3, 2026 21:03
@github-actions

github-actions Bot commented Jun 3, 2026 •

Copy link
Copy Markdown

@Fluf22 Fluf22 self-assigned this Jun 4, 2026
@Fluf22
Fluf22 force-pushed the feat/worktree-better-support branch 3 times, most recently from 2f94506 to 61a23d2 Compare June 4, 2026 18:28
@Fluf22
Fluf22 marked this pull request as ready for review June 9, 2026 13:53
@Fluf22
Fluf22 requested a review from a team as a code owner June 9, 2026 13:53
Fluf22 added 5 commits June 23, 2026 11:11
- Remove hardcoded container_name from docker-compose.yml
- Add content-addressed image tags for build-once-run-many
- Add COMPOSE_PROJECT_NAME scoping per worktree
- Switch from 'docker exec' to 'docker compose exec' in common.ts
- Add slot-based CTS port offsetting (Java generator + test server)
- Add .env.docker for persistent compose config
- Add docker:teardown command
- Add extra_hosts for Linux host.docker.internal support
- Fix clients.schema.json enum to match new service names
- Restore debug port mapping with per-slot offset (APIC_DEBUG_PORT)
- Add port offset to 7 hand-written manual test files
- Extract PORTS_PER_SLOT constant in TS with cross-language SYNC markers
- Remove redundant COMPOSE_PROJECT_NAME derivation from common.ts
- Source .env.docker in teardown.sh instead of re-deriving
- Warn on slot change to re-run CTS generation
- Fix offsetPorts() to use explicit types and copy-on-write
- Replace silent exception swallowing with targeted catches + warnings
…tralization

- Centralize port assignments in ports.ts, replace magic numbers across 14 test server files
- Switch from file-based to env-var-based port offset (CTS_PORT_OFFSET) in Java generator and all language test files
- Rename config field dockerImage -> dockerService to match semantics
- Derive PORTS_PER_SLOT from ports.ts in slot.sh (eliminates hardcoded copy)
- Add vitest CI check (portsSync.test.ts) to validate Java generator constants stay in sync
- Register LoggingTests.cs as hand-written in generation.config.mjs
- Fix operator precedence bug in slot.sh stale-entry cleanup
- Unify on CTS_PORT_OFFSET env var as single runtime input for port offset
  (index.ts now reads env var / .env.docker instead of .apic-worktree-slot)
- Extract offsetPortsInString helper in TestsClient.java
- Add CONTAINER_DEBUG_PORT constant in common.ts
- Add SYNC comments to hand-written test files
- Tighten slot.sh PORTS_PER_SLOT regex
- Guard against missing .env.docker in run()
@Fluf22
Fluf22 force-pushed the feat/worktree-better-support branch from 2e2ca5a to 9b1ba2a Compare June 23, 2026 09:11
@eric-zaharia
eric-zaharia deleted the feat/worktree-better-support branch June 23, 2026 10:13
@eric-zaharia
eric-zaharia restored the feat/worktree-better-support branch June 23, 2026 10:14
@eric-zaharia eric-zaharia reopened this Jun 23, 2026
ports.ts is now the one contiguous port list (debug + CTS servers); slot.sh
and the Java generator derive the per-slot offset from it. Folds the two
stray test servers and the debug port into the list (fixes the cross-slot
port collision), drops the local .apic-worktree-slot file for the central
registry, and keys the Compose project on the slot.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants