EasyDeploy execution worker

Long-running Go service that pulls deployment jobs from RabbitMQ, runs per-deployment work inside Docker containers, publishes build logs back to the broker, and drives the bash pipeline that talks to ECR, BuildKit, and Helm.

---

What this repository contains

Piece	Role
main.go	AMQP consumer, job routing, Docker API client, container log streaming to RabbitMQ.
build.sh	ENTRYPOINT of the worker image: clone/build/push (GitHub strategy), or Helm-only path (Docker image strategy), plus delete/restart flows.
helm-chart/	Chart baked into the worker image and used to install each user application (helm upgrade --install easydeploy-<deploymentId> …).
Dockerfile	Ubuntu-based image with Docker CLI, Railpack, kubectl, Helm, AWS CLI, build.sh, and the chart.
Dockerfile.engine	Multi-stage image that compiles main.go into a minimal Alpine binary (/app/engine) for the long-lived engine pod.

The Go process does not shell out to build.sh on the host. It uses the Docker Engine API to create and start containers from the worker image; those containers execute build.sh with environment variables derived from each message.

---

High-level flow

1. NestJS (or another producer) publishes a JSON message to the deployments queue.
2. main.go unmarshals the Nest-style envelope (pattern + data), then branches on operation.
3. For create/redeploy/restart paths it starts a one-shot container (image configurable in code, see below) with OPERATION, STRATEGY, repo URL, env JSON, etc.
4. build.sh reports progress to the control plane with curl against http://localhost:8080/deployment/status (expects an API reachable from that container).
5. While the job container runs, stdout/stderr lines are published to the logs.exchange topic exchange with routing key build.<deploymentId>.

---

RabbitMQ contract

Broker URL (current code): amqp://admin:admin@rabbitmq:5672/ — change in main.go for non-local clusters.

Queue: deployments (consumer auto-ack enabled).

Log exchange: logs.exchange (topic, durable). Routing key pattern: build.<deploymentId>.

Envelope (NestJS-style)

{
  "pattern": "<your-nestjs-pattern>",
  "data": { }
}

The worker only unmarshals data; pattern is not enforced in code. The inner data object maps to DeploymentMessage:

Field	JSON	Purpose
Deployment ID	deploymentId	Correlates logs, Helm release name prefix, image tag (GitHub strategy).
Repository URL	repoUrl	Git clone URL when strategy is github.
Strategy	strategy	github — clone, Railpack build, push to ECR, Helm install. docker — deploy existing image via Helm only.
Operation	operation	CREATE, REDEPLOY, or DELETE (see below).
Start command	startCommand	Passed through to Helm as the app container command.
Docker image	dockerImage	Used for docker strategy (repository:tag or repository + tag inference in build.sh).
Port	port	Application listen port (default 8080 if zero).
Environment	env	Array of { "key", "value" }; serialized to USER_ENV JSON for build.sh.
GitHub token	githubToken	Optional; injected into clone URL for private repositories.

Operations (as sent in the message)

operation	Behaviour in main.go
DELETE	Starts worker container with OPERATION=delete → build.sh runs helm uninstall for easydeploy-<deploymentId>.
REDEPLOY	github: full pipeline again with OPERATION=redeploy (same deployment ID; forced rollout after push). docker: OPERATION=restart → kubectl rollout restart / Helm touch.
CREATE (or anything else)	Full create path: GitHub pipeline or Docker Helm path with OPERATION=create.

---

Environment passed into job containers

buildWorkerEnv in main.go sets (among others):

• DEPLOYMENT_ID, REPO_URL, STRATEGY, OPERATION (create | redeploy | restart | delete)
• USER_ENV — JSON object of user env vars
• START_COMMAND, IMG (from dockerImage), GITHUB_TOKEN, PORT
• DOCKER_HOST=tcp://172.17.0.1:2375, BUILDKIT_HOST=tcp://172.17.0.1:1234 — expected DinD / BuildKit sidecars on the worker host/pod network

The job image must therefore run where that Docker endpoint and BuildKit address are reachable (typical pattern: engine pod with docker:dind and moby/buildkit).

---

build.sh pipeline (summary)

• GitHub strategy: init → optional authenticated git clone → railpack build (BuildKit remote) → tag → ECR login/push → kubectl kubeconfig → helm upgrade --install using chart under /app/helm-chart. On redeploy, forces a rollout so imagePullPolicy: Always pulls the new digest.
• Docker strategy: skips clone/build/push; Helm install/upgrade with image.repository / image.tag derived from IMG.
• Delete: helm uninstall for the release if it exists.

Ingress (when enabled) uses host {{ Release.Name }}.{{ .Values.ingress.baseHostIP }}.nip.io (see helm-chart/templates/ingress.yaml). Default baseHostIP in values.yaml should be replaced for your cluster.

---

Tech stack (accurate to this repo)

• Go 1.25 — worker daemon (github.com/rabbitmq/amqp091-go, github.com/docker/docker/client)
• Bash — build.sh pipeline
• Railpack — application image builds (invoked as railpack in build.sh)
• Docker / BuildKit — image build and registry push
• AWS CLI — ECR authentication and push
• kubectl / Helm 3 — EKS deploy and lifecycle

---

Building images

Worker image (contains build.sh, tooling, and Helm chart — what the engine starts per job):

docker build -f Dockerfile -t <registry>/easydeploy-worker:<tag> .
docker push <registry>/easydeploy-worker:<tag>

Engine image (compiled Go binary only — long-lived queue consumer):

docker build -f Dockerfile.engine -t <registry>/easydeploy-engine:<tag> .
docker push <registry>/easydeploy-engine:<tag>

The engine’s main.go currently references a concrete worker image name (ahmedharabi/easydeploy-worker); align that constant with the registry you push to, or make it configurable before production use.

After pushing, roll your Kubernetes deployment as appropriate, for example:

kubectl rollout restart deployment/easydeploy-engine -n easydeploy

---

Operational notes

• Secrets: githubToken and registry credentials travel in container env; treat the worker namespace, image, and RabbitMQ ACLs accordingly.
• Status API: build.sh posts to http://localhost:8080; the engine does not implement that server — run the API gateway or sidecar expected by your stack.
• Auto-ack: the AMQP consumer uses auto-ack; a crash after accepting a message can lose the job unless the broker layer adds idempotency or retries.