Skip to content

Configuration Reference

Jump to bottom
genesluna edited this page Mar 15, 2026 · 11 revisions

Configuration Reference

Complete reference for the k8s-ee.yaml configuration file used by ephemeral PR environments.

⚡ Quick Start

Minimal configuration requires only projectId:

# k8s-ee.yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/koder-cat/k8s-ephemeral-environments/main/.github/actions/validate-config/schema.json
projectId: myapp

💡 Tip: Add the schema comment for IDE autocompletion and validation in VS Code!

This creates PR environments at myapp-pr-{number}.k8s-ee.genesluna.dev with sensible defaults.

📄 Full Configuration Example

# k8s-ee.yaml - Full example with all options
projectId: myapp
trigger: automatic    # or "on-demand" for /deploy-preview command

app:
  port: 3000
  healthPath: /health
  metricsPath: /metrics

image:
  context: .
  dockerfile: Dockerfile
  repository: ghcr.io/myorg/myapp  # Optional, auto-generated if not set

resources:
  requests:
    cpu: 50m
    memory: 128Mi
  limits:
    cpu: 200m
    memory: 384Mi

env:
  NODE_ENV: production
  LOG_LEVEL: info

envFrom:
  - secretRef:
      name: my-secret
  - configMapRef:
      name: my-config

databases:
  postgresql: true
  mongodb: false
  redis: false
  minio: false
  mariadb: false

ingress:
  enabled: true
  annotations:
    custom.annotation/key: value

metrics:
  enabled: false
  interval: 30s

📑 Field Reference

projectId (required)

Unique identifier for this project in multi-tenant clusters.

Property Value
Type string
Required Yes
Min Length 1
Max Length 20
Pattern ^[a-z0-9]([a-z0-9-]{0,18}[a-z0-9])?$

Validation Rules:

  • Must be lowercase alphanumeric with hyphens
  • Must start and end with alphanumeric character
  • Maximum 20 characters (leaves room for -pr-{number} suffix)

Examples:

Value Valid Reason
myapp ✅ Lowercase alphanumeric
my-cool-app ✅ Hyphens allowed
my-app-123 ✅ Numbers allowed
MyApp ❌ Uppercase not allowed
my_app ❌ Underscores not allowed
-myapp ❌ Cannot start with hyphen
this-is-a-very-long-project-name ❌ Exceeds 20 characters

trigger

Controls how PR environments are created.

Property Value
Type string
Required No
Default automatic
Values automatic, on-demand

automatic (default): Environment is created automatically when a PR is opened or updated. This is the standard behavior.

on-demand: Environment is only created when someone comments /deploy-preview on the PR. After creation, subsequent pushes auto-redeploy. Use /destroy-preview to tear down the environment early.

trigger: on-demand

💡 Note: On-demand mode requires trigger: on-demand in your k8s-ee.yaml. The universal workflow template from Quick Start handles both modes — once you have it, switching is a one-line config change. See On-Demand Environments for details.

app

Application settings for the deployed container.

app.port

Property Value
Type integer
Default 3000
Minimum 1
Maximum 65535

Container port the application listens on. The platform automatically configures:

  • Deployment: Sets the container port
  • NetworkPolicy: Allows ingress traffic on this port from Traefik

Common Configurations:

app:
  port: 3000  # Node.js, Express, NestJS (default)
  port: 8080  # .NET, Go, Java Spring Boot
  port: 8000  # Python FastAPI, Django

app.healthPath

Property Value
Type string
Default "/health"
Pattern ^/.*

Health check endpoint path used for liveness and readiness probes.

app:
  healthPath: /api/health

app.metricsPath

Property Value
Type string
Default (none)
Pattern ^/.*

Metrics endpoint path for Prometheus scraping. Only needed if metrics.enabled: true.

app:
  metricsPath: /metrics

image

Docker image build configuration.

image.context

Property Value
Type string
Default "."

Docker build context path relative to repository root.

image:
  context: ./backend

image.dockerfile

Property Value
Type string
Default "Dockerfile"

Path to Dockerfile relative to the build context.

image:
  dockerfile: Dockerfile.prod

image.repository

Property Value
Type string
Default (auto-generated)

Custom image repository URL. If not set, auto-generated based on the registry type:

  • GHCR (default): ghcr.io/{owner}/{repo}/{project-id}
  • ECR: <account-id>.dkr.ecr.<region>.amazonaws.com/{owner}/{repo}/{project-id} 
image:
  repository: ghcr.io/myorg/custom-image

resources

Container resource requests and limits.

⚠️ Limit: Maximum 512Mi memory per container (cluster LimitRange).

resources.requests.cpu

Property Value
Type string
Default "50m"
Pattern ^[0-9]+(m|[0-9]*)?$

CPU request in millicores (e.g., 50m, 100m, 0.5).

resources.requests.memory

Property Value
Type string
Default "128Mi"
Pattern ^[0-9]+(Mi|Gi)$

Memory request (e.g., 128Mi, 256Mi, 1Gi).

resources.limits.cpu

Property Value
Type string
Default "200m"
Pattern ^[0-9]+(m|[0-9]*)?$

CPU limit in millicores.

resources.limits.memory

Property Value
Type string
Default "384Mi"
Pattern ^[0-9]+(Mi|Gi)$

Memory limit. Maximum allowed: 512Mi (cluster LimitRange).

resources:
  requests:
    cpu: 100m
    memory: 256Mi
  limits:
    cpu: 500m
    memory: 512Mi

env

User-defined environment variables injected into the application pod as key-value pairs. These are stored in a Kubernetes ConfigMap ({namespace}-app-config) and mounted via envFrom.

Property Value
Type object
Default {}
Key Pattern ^[A-Za-z_][A-Za-z0-9_]*$ 
env:
  NODE_ENV: staging
  LOG_LEVEL: info
  JWT_SECRET: "ephemeral-preview-secret-not-for-production"

How it works: The env values flow through the deployment pipeline:

  1. validate-config parses k8s-ee.yaml and outputs them as env-json
  2. The reusable workflow passes env-json to the deploy-app action
  3. deploy-app passes them to Helm via --set-json env=...
  4. The Helm chart injects them into the app ConfigMap alongside platform variables (PORT, PR_NUMBER, etc.)

Verification:

kubectl get configmap {namespace}-app-config -n {namespace} -o yaml

💡 Note: Database connection variables (DATABASE_URL, PGHOST, MINIO_ENDPOINT, etc.) are injected separately by database charts and do not need to be listed in env. Variables like MINIO_BUCKET that are configured via databases.minio.bucket are also injected automatically.

envFrom

Environment variables from Kubernetes secrets or configmaps.

Property Value
Type array
Default []

Each item must have exactly one of secretRef or configMapRef (not both).

envFrom:
  - secretRef:
      name: database-credentials
  - configMapRef:
      name: app-config

writableDirs

Directories that need to be writable at runtime. Each directory is mounted as an emptyDir volume. Required because containers run with readOnlyRootFilesystem: true for security.

Property Value
Type array of strings
Default []
Path Pattern Must start with / 
writableDirs:
  - /app/upload
  - /app/data

💡 /tmp is always writable (mounted automatically). Only declare directories outside of /tmp that your app needs to write to.

💾 databases

Database configuration. All databases are disabled by default (opt-in).

When enabled, databases are automatically deployed to your PR environment and connection details are injected as environment variables.

Each database can be configured as:

  • Boolean: true to enable with defaults, false to disable
  • Object: Enable with custom configuration

databases.postgresql

Property Value
Type boolean | object
Default false

PostgreSQL database using CloudNativePG operator.

# Simple enable
databases:
  postgresql: true

# With custom configuration
databases:
  postgresql:
    enabled: true
    version: "16"
    storage: 2Gi

# With bootstrap SQL (for table creation)
databases:
  postgresql:
    enabled: true
    bootstrap:
      postInitApplicationSQL:
        - |
          CREATE TABLE users (
            id SERIAL PRIMARY KEY,
            name VARCHAR(255)
          );
          GRANT ALL PRIVILEGES ON users TO app;
          GRANT USAGE, SELECT ON SEQUENCE users_id_seq TO app;

Object properties:

Property Type Default Description
enabled boolean true Enable/disable
version string "16" PostgreSQL version
storage string "1Gi" Storage size
bootstrap.postInitApplicationSQL array [] SQL to run after DB creation
bootstrap.initSQL array [] SQL to run on postgres DB (extensions)

📝 Note: Bootstrap SQL runs as postgres superuser, but your app connects as app user. Include GRANT statements for table access. Use $func$ instead of $$ for function delimiters.

Injected Environment Variables:

Variable Description
DATABASE_TYPE postgresql
DATABASE_URL Full connection string
PGHOST Hostname
PGPORT Port (5432)
PGDATABASE Database name
PGUSER Username
PGPASSWORD Password

databases.mongodb

Property Value
Type boolean | object
Default false

MongoDB database using MongoDB Community Operator.

databases:
  mongodb:
    enabled: true
    storage: 2Gi

Object properties:

Property Type Default Description
enabled boolean true Enable/disable
version string - MongoDB version
storage string "1Gi" Storage size

Injected Environment Variables:

Variable Description
MONGODB_URL Full connection string (uses /admin for auth)
MONGODB_URI Alias for MONGODB_URL
MONGODB_DATABASE Database name for application data (default: app)

💡 Tip: The connection string uses /admin for authentication. Use MONGODB_DATABASE for your collections database.

databases.redis

Property Value
Type boolean | object
Default false

Redis cache using simple deployment.

databases:
  redis: true

Object properties:

Property Type Default Description
enabled boolean true Enable/disable

Injected Environment Variables:

Variable Description
REDIS_HOST Hostname
REDIS_PORT Port (6379)
REDIS_PASSWORD Password

💡 Note: Your app should construct the Redis URL from these variables.

databases.minio

Property Value
Type boolean | object
Default false

MinIO object storage (S3-compatible).

databases:
  minio:
    enabled: true
    storage: 5Gi

Object properties:

Property Type Default Description
enabled boolean true Enable/disable
storage string "1Gi" Storage size

Injected Environment Variables:

Variable Description
MINIO_ENDPOINT Hostname
MINIO_PORT Port (9000)
MINIO_ACCESS_KEY Access key
MINIO_SECRET_KEY Secret key
MINIO_BUCKET Default bucket name
S3_* S3-compatible aliases

databases.mariadb

Property Value
Type boolean | object
Default false

MariaDB database using simple deployment.

databases:
  mariadb:
    enabled: true
    version: "11.4"
    storage: 2Gi

Object properties:

Property Type Default Description
enabled boolean true Enable/disable
version string "11.4" MariaDB version
storage string "1Gi" Storage size

Injected Environment Variables:

Variable Description
DATABASE_TYPE mariadb
MYSQL_URL Full connection string
MYSQL_HOST Hostname
MYSQL_PORT Port (3306)
MYSQL_DATABASE Database name
MYSQL_USER Username
MYSQL_PASSWORD Password

⚠️ Note: Only enable one SQL database (PostgreSQL OR MariaDB) at a time. MariaDB takes precedence for DATABASE_TYPE.

📊 Resource Requirements by Database

The platform automatically calculates ResourceQuota based on enabled databases. No manual configuration needed!

Service CPU Limit Memory Limit Storage
Application (base) 300m 512Mi 1Gi
+ PostgreSQL +500m +512Mi +2Gi
+ MongoDB +500m +512Mi +2Gi
+ Redis +200m +128Mi -
+ MinIO +500m +512Mi +2Gi
+ MariaDB +300m +256Mi +2Gi

Example Calculated Quotas:

Configuration CPU Limit Memory Limit Storage
App only 300m 512Mi 1Gi
App + PostgreSQL 800m 1Gi 3Gi
App + PostgreSQL + Redis 1000m 1.1Gi 3Gi
App + PostgreSQL + MongoDB 1300m 1.5Gi 5Gi
All databases enabled 2100m 2.4Gi 9Gi

📝 Note: Quotas are calculated at namespace creation. If you add databases to an existing PR, close and reopen the PR to recalculate.

ingress

Ingress configuration for external access.

ingress.enabled

Property Value
Type boolean
Default true

Enable/disable ingress creation.

ingress.annotations

Property Value
Type object
Default {}

Additional annotations for the ingress resource.

ingress:
  enabled: true
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 10m
    traefik.ingress.kubernetes.io/rate-limit: "100"

metrics

Prometheus metrics configuration.

metrics.enabled

Property Value
Type boolean
Default false

Enable ServiceMonitor for Prometheus scraping.

metrics.interval

Property Value
Type string
Default "30s"
Pattern ^[0-9]+(s|m|h)$

Scrape interval (e.g., 15s, 1m, 5m).

metrics:
  enabled: true
  interval: 15s

💡 Automatic Labeling: When metrics are enabled, the ServiceMonitor automatically adds a namespace label to all scraped metrics. This enables Grafana dashboards to filter by namespace without app changes.

Metrics Instrumentation Guide

When metrics.enabled: true, the platform deploys a ServiceMonitor that tells Prometheus to scrape GET /metrics on your app. Your app must expose this endpoint returning Prometheus text format — otherwise the Grafana "PR Developer Insights" dashboard will show App Status = DOWN and DB Connected = NO.

Required Metrics

These metrics power the Grafana dashboard panels:

Metric Name Type Labels Dashboard Panels
http_requests_total Counter method, route, status_code Request Rate, Error Rate, 5xx by Endpoint, Requests by Status
http_request_duration_seconds Histogram method, route, status_code P95 Latency, P95 by Endpoint, Slowest Endpoints
db_pool_connections_total Gauge — DB Connected, Connection Pool
db_pool_connections_idle Gauge — Connection Pool
db_pool_connections_waiting Gauge — Connection Pool
db_query_duration_seconds Histogram operation, success Query Duration, Failed Queries

Minimum requirements: A /metrics endpoint returning Prometheus text format, plus http_requests_total and http_request_duration_seconds. The database metrics are optional but recommended if your app uses a database.

Automatic metrics: The up metric (App Status panel) and kube_pod_status_phase (Pods Running panel) are provided automatically by Prometheus and kube-state-metrics — no app instrumentation needed.

Recommended Histogram Buckets
HTTP duration: [0.01, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10] (seconds)
DB query duration: [0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1] (seconds)
Node.js/Express Example (prom-client)
import * as client from 'prom-client';

const registry = new client.Registry();
registry.setDefaultLabels({
  app: 'my-app',
  pr: process.env.PR_NUMBER || 'unknown',
});
client.collectDefaultMetrics({ register: registry });

const httpRequestTotal = new client.Counter({
  name: 'http_requests_total',
  help: 'Total number of HTTP requests',
  labelNames: ['method', 'route', 'status_code'],
  registers: [registry],
});

const httpRequestDuration = new client.Histogram({
  name: 'http_request_duration_seconds',
  help: 'Duration of HTTP requests in seconds',
  labelNames: ['method', 'route', 'status_code'],
  buckets: [0.01, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10],
  registers: [registry],
});

// GET /metrics endpoint
app.get('/metrics', async (_req, res) => {
  res.set('Content-Type', registry.contentType);
  res.send(await registry.metrics());
});

📖 Reference implementation: See demo-app/apps/api/src/metrics/ for a complete example with HTTP middleware, database metrics, and pool monitoring.

🎨 Common Scenarios

Node.js API with PostgreSQL

projectId: my-api

app:
  port: 3000
  healthPath: /health

env:
  NODE_ENV: production

databases:
  postgresql: true

Python Flask with Redis Cache

projectId: flask-app

app:
  port: 5000
  healthPath: /healthz

resources:
  requests:
    memory: 256Mi
  limits:
    memory: 512Mi

databases:
  redis: true

Full-Stack with Multiple Databases

projectId: fullstack

app:
  port: 8080
  healthPath: /api/health
  metricsPath: /metrics

databases:
  postgresql: true
  redis: true
  minio:
    enabled: true
    storage: 2Gi

metrics:
  enabled: true
  interval: 30s

On-Demand Environments

Save resources by creating environments only when needed:

projectId: myapp
trigger: on-demand

app:
  port: 3000
  healthPath: /health

databases:
  postgresql: true

With this configuration, environments are only created when someone comments /deploy-preview on the PR. Use /destroy-preview to tear down the environment early. Requires the universal workflow template from Quick Start.

Monorepo Backend Service

projectId: backend-svc

image:
  context: ./services/backend
  dockerfile: Dockerfile

app:
  port: 4000
  healthPath: /ready

envFrom:
  - secretRef:
      name: shared-secrets

❌ Validation Errors

The schema validation provides clear error messages:

Error Cause Fix
projectId must match pattern Invalid characters or format Use lowercase alphanumeric + hyphens only
projectId must be <= 20 characters ID too long Shorten the project ID
app.port must be >= 1 Invalid port number Use port between 1-65535
resources.limits.memory must match pattern Invalid memory format Use format like 256Mi or 1Gi
databases.*.storage must match pattern Invalid storage format Use format like 1Gi, 500Mi, or 2Ti
env property name is invalid Invalid env var name Use pattern [A-Za-z_][A-Za-z0-9_]*
envFrom item must have secretRef or configMapRef Empty envFrom entry Specify either secretRef or configMapRef

⚙️ Computed Values

These values are automatically computed and available in the deployment:

Field Formula Example
_computed.namespace {projectId}-pr-{prNumber} myapp-pr-42
_computed.previewUrl https://{namespace}.{domain} https://myapp-pr-42.k8s-ee.genesluna.dev
_computed.prNumber From workflow input 42

🔌 Platform-Injected Environment Variables

In addition to user-defined env values, the platform automatically injects these environment variables into every app pod:

Variable Source Example
PORT app.port config 3000
PR_NUMBER Workflow input 42
COMMIT_SHA Git HEAD SHA a1b2c3d4...
BRANCH_NAME PR head branch feat-my-feature
APP_VERSION Image tag pr-42
PREVIEW_URL Computed preview URL https://myapp-pr-42.k8s-ee.genesluna.dev

Database charts inject additional variables when databases are enabled (e.g., DATABASE_URL, PGHOST, MINIO_ENDPOINT).

⚠️ CORS tip: If your application has a CORS allowlist, add process.env.PREVIEW_URL to it so the frontend served from the preview domain can make API calls. Example (Express.js):

const allowedOrigins = [
  'http://localhost:3000',
  process.env.PREVIEW_URL,  // k8s-ee preview domain
].filter(Boolean);

Related Pages

Home

Getting Started

User Guides

Troubleshooting

Operations

Architecture

Demo Applications

Development

Clone this wiki locally