OpenBKN Deploy

中文 | English

One-click deployment of OpenBKN onto a single-node Kubernetes cluster.

This repository provides scripts to install OpenBKN along with its dependencies including Kubernetes, infrastructure services, and data services.

For portable MariaDB and OpenSearch backups, restore commands, and initial snapshot directory setup, see the backup and restore guide (中文). Entry points: deploy.sh backup check|run and deploy.sh restore check|run.

Platforms: Linux is the recommended and fully documented install target (preflight.sh, k3s or kubeadm, data services). macOS is optional for local development only (Docker + kind + dev/mac.sh); see Mac install (dev) (中文) — not a substitute for Linux production installs.

Linux: default k8s (kubeadm) vs optional k3s

KUBE_DISTRO defaults to k8s (package-manager Kubernetes + kubeadm on a single node). k3s is an optional lighter single-node stack. If you install k3s with deploy.sh k3s install, keep preflight.sh / openbkn aligned by passing --distro=k3s before the module (or export KUBE_DISTRO=k3s) — otherwise preflight may flag a k3s/kubeadm mismatch and installs can disagree on bootstrap.

kubeadm / KUBE_DISTRO=k8s (default)

Single-node kubeadm flow is bash ./deploy.sh k8s install (deploy/scripts/services/k8s.sh). Product modules reuse an existing cluster when kubectl already works (ensure_k8s skips reinstall), then ensure_platform_prerequisites installs the bundled data-services layer (MariaDB, Redis, Kafka, OpenSearch, …) before Core. macOS kind skips host kubeadm but openbkn install still runs ensure_data_services first (see macOS section). Legacy kubeadm is still accepted as an alias for k8s.

deploy.sh global flags (--distro, -y, --force-upgrade, --config, …) must appear before the module name. Correct: bash ./deploy.sh --distro=k3s openbkn install. Wrong: bash ./deploy.sh openbkn install --distro=k3s (that --distro is not read as a global option). Equivalent without moving flags: export KUBE_DISTRO=k3s then bash ./deploy.sh openbkn install.

bash ./deploy.sh k8s install
bash ./deploy.sh openbkn install

k3s (optional — lightweight single-node)

Uses the upstream k3s installer (Traefik disabled; this stack still installs ingress-nginx for a consistent chart/accessAddress setup). Override K3S_INSTALL_URL, INSTALL_K3S_VERSION, or INSTALL_K3S_MIRROR if you need a mirror or air-gapped tuning.

cd openbkn-deploy

bash ./deploy.sh k3s install

# Align distro with k3s for preflight and platform bootstrap:
bash ./deploy.sh --distro=k3s openbkn install
# or: export KUBE_DISTRO=k3s && bash ./deploy.sh openbkn install

Check status: bash ./deploy.sh k3s status — remove: bash ./deploy.sh k3s uninstall.

accessAddress vs Kubernetes API (kubeconfig)

accessAddress in your install config is the HTTP(S) base URL users hit through Ingress (often a public IP or DNS on 80/443). It does not have to match how kubectl / helm talk to the control plane (6443).

On the same Linux host as k3s, use the file /etc/rancher/k3s/k3s.yaml for kubectl / helm (usually server: https://127.0.0.1:6443 after copying to ~/.kube/config with correct ownership). Do not rewrite the API server: to your Elastic IP / public address for on-host use unless you mean to call the API through the public interface with 6443 open and matching tls-san — hairpin/NAT setups often cause dial tcp …6443: i/o timeout. Remote admins may use a reachable server: if the API is exposed and trusted.

macOS (optional — local dev with kind)

Use this only for Mac validation; for real installs use Linux above. Local Kubernetes via kind — no preflight.sh / k3s on the Mac host. mac.sh sets OPENBKN_SKIP_PLATFORM_BOOTSTRAP (no host k3s/kubeadm bootstrap). openbkn install now runs ensure_data_services first — same Helm layer as data-services install (MariaDB, Redis, Kafka, OpenSearch); mac.sh defaults AUTO_INSTALL_INGRESS_NGINX=false so kind’s existing ingress is not duplicated. Set OPENBKN_SKIP_DATA_SERVICES_BUNDLE=true to skip bundled data installs (advanced / external infra). data-services install alone remains useful to pre-stage or refresh the data layer. Apple Silicon: kind nodes are arm64; use arm64/multi-arch images (see dev/conf/mac-config.yaml). Step order: dev/README.md.

cd openbkn-deploy   # repository root
bash ./dev/mac.sh doctor
# optional: install missing tools via Homebrew — bash ./dev/mac.sh doctor --fix (or -y doctor --fix to skip confirm)
bash ./dev/mac.sh cluster up
bash ./dev/mac.sh openbkn install       # full stack incl. mandatory bkn-safe (auth on); bundled data-services first (same as data-services install)
# optional: bash ./dev/mac.sh data-services install   # only if you want the data layer without Core, or to refresh it
# optional: bash ./dev/mac.sh openbkn download
# optional: bash ./dev/mac.sh onboard
# add leading -y for non-interactive (deploy.sh / onboard)

Config defaults: dev/conf/mac-config.yaml. isf is delegated to deploy.sh — see dev/README.md.

🚀 Quick Start

Host prerequisites

Run install commands as root or through sudo.

# 1. Disable firewall
systemctl stop firewalld && systemctl disable firewalld

# 2. Disable swap
swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab

# 3. Set SELinux to permissive if needed
setenforce 0

# 4. Install containerd.io
dnf install containerd.io

Keep firewalld enabled (optional)

The default flow keeps the compatibility mode that disables the host firewall. To retain or re-enable firewalld, explicitly enable OpenBKN firewall reconciliation before installation. The script starts and enables firewalld, opens only the live ingress HTTP/HTTPS ports, places the live Pod/Service CIDRs and discovered CNI interfaces in the internal trusted zone, and leaves Kubernetes API port 6443 closed to the public network by default.

export OPENBKN_FIREWALL_ENABLED=true
# Set only when remote Kubernetes administration is required; separate CIDRs with commas.
# export K8S_API_ALLOWED_CIDRS="10.10.0.0/16,192.168.1.50/32"
bash ./deploy.sh openbkn install

Ingress ports are read from the installed controller's hostPort or nodePort, so custom INGRESS_NGINX_HTTP_PORT and INGRESS_NGINX_HTTPS_PORT values are reconciled on every install or upgrade.

The enabled state is persisted at /etc/openbkn-deploy/firewall-enabled. Later upgrades continue reconciling OpenBKN firewall rules even when OPENBKN_FIREWALL_ENABLED is omitted. An explicit OPENBKN_FIREWALL_ENABLED=false skips firewall management for that invocation. Use OPENBKN_FIREWALL_ENABLED=forget to permanently stop OpenBKN management without stopping firewalld or deleting existing rules.

Install OpenBKN

# 1. Clone the repository
git clone https://github.com/openbkn-ai/deploy.git openbkn-deploy
cd openbkn-deploy

# 2. (Recommended) Pre-install host check / fix
sudo bash ./preflight.sh                # check-only (default)
sudo bash ./preflight.sh --fix          # check + interactive fixes
sudo bash ./preflight.sh --fix -y       # auto-approve every fix
sudo bash ./preflight.sh --list-fixes   # preview which fixes would run, no changes
sudo bash ./preflight.sh --help         # all flags (--role, --skip, --report, --output=json, …)
# Default checks match k8s/kubeadm; for single-node k3s use: sudo bash ./preflight.sh --distro=k3s
# (same as env KUBE_DISTRO=k3s — shared with deploy.sh)

# 3. Install OpenBKN (this also installs the install-status dashboard)
# Container-log viewing is enabled by default. The command waits until
# install-status is ready before it succeeds.
sudo bash ./deploy.sh openbkn install
# Default is kubeadm (k8s). For single-node k3s instead (--distro must appear before openbkn):
# sudo bash ./deploy.sh --distro=k3s openbkn install
# or: export KUBE_DISTRO=k3s, then run the installation command above
# The script will interactively prompt for the access address and auto-detect the API server address.

# Or specify addresses explicitly (skips interactive prompts):
#   --access_address       Address for clients to reach OpenBKN services (can be IP or domain)
#   --api_server_address   IP bound to a local network interface for K8s API server (must be a real NIC IP)
sudo bash ./deploy.sh openbkn install \
  --access_address=<your-ip> \
  --api_server_address=<your-ip>

# If you include a port in --access_address, that port is used for the matching
# ingress protocol when the script installs ingress-nginx (for example, https://...:8443
# updates the HTTPS ingress port, while http://...:8080 updates the HTTP port).

# (Optional) Customize ingress ports (default 80/443):
export INGRESS_NGINX_HTTP_PORT=8080
export INGRESS_NGINX_HTTPS_PORT=8443
# Re-run the same install command after changing these values; the script now
# detects port drift and upgrades the existing ingress-nginx release.

# 4. (Recommended) Post-install bootstrap
#    Registers an LLM + embedding (skips when already there), patches the BKN ConfigMap
#    only when the default actually changes, and on a full (bkn-safe) install creates the business
#    user `test`, assigns every role from `openbkn admin role list`, switches `openbkn` to
#    that user.
sudo bash ./onboard.sh        # interactive
sudo bash ./onboard.sh -y     # non-interactive (uses defaults)
sudo bash ./onboard.sh --help # all flags (--config=models.yaml, --enable-bkn-search, …)

Why sudo? onboard.sh reads $HOME/.openbkn-ai/config.yaml (written by sudo deploy.sh into /root/.openbkn-ai/) and writes the bkn auth token to $HOME/.bkn. Running it without sudo falls back to the in-repo template conf/config.yaml and may resolve a different access URL. macOS dev path (bash ./dev/mac.sh onboard) does not need sudo. The script also prints this hint at startup; silence with ONBOARD_SUDO_HINT_DISABLED=1.

Full preflight / onboard flow and Mermaid diagrams: see help/en/install.md — Post-install: onboard.sh.

onboard.sh runtime messages are English. The seeded admin must change its password on first login; onboard clears that automatically (self-service /api/safe/v1/auth/change-password) before the credential login. Chinese + English context: dev/README.md · dev/README.zh.md; product docs help/zh/install.md / help/en/install.md.

Dev/test: pick chart versions (--version_file)

Release installs pin exact chart versions in a committed manifest (release-manifests/0.1.5/openbkn.yaml) — a lockfile, reproducible.

For dev/test you usually want the newest builds, and CI only republishes the components a branch actually changed. scripts/gen-dev-manifest.sh resolves each chart's version from GHCR and writes a manifest you pass with --version_file:

# latest stable — every chart = highest clean semver (e.g. 0.1.0)
./scripts/gen-dev-manifest.sh --out=/tmp/m.yaml

# test a branch — components it rebuilt use the branch build; the rest fall back
# to latest stable, then to the --base branch (default: main)
./scripts/gen-dev-manifest.sh --branch=fix/my-thing --out=/tmp/m.yaml

# install the generated manifest
sudo bash ./deploy.sh --distro=k3s openbkn install --version_file=/tmp/m.yaml

An exact chart version in the composition template is preserved and must exist in GHCR. The main placeholder uses the resolution rules below. The sample service is pinned to 0.1.0-main.20261013062625.shaafa25ac, which includes dynamic sample uninstall (#125, backported to 0.2.0 by #130). Its chart pins the service image by digest. Keep this component version when assembling the 0.2.0 release lockfile; refreshing the sample catalog does not update the service image.

Per-chart resolution for placeholders (stable-first): --branch newest build → latest stable → --base newest build → error. The generated manifest annotates each chart's source (branch / stable / base). A release branch starts with a full build; later builds update only the charts that changed. Requires python3; see ./scripts/gen-dev-manifest.sh -h.

To install the newest builds for the checked-out branch, use --latest from main or release/X.Y.Z. On a release branch it selects each chart's newest X.Y.Z-release.<date>.sha… build, then falls back to stable and main for charts without a release build. Other branches and detached HEAD are rejected:

./scripts/gen-dev-manifest.sh --latest --out=/tmp/m.yaml

macOS note: the system python3 may lack CA certs and silently resolve every chart as NOT FOUND. Set SSL_CERT_FILE=/etc/ssl/cert.pem (or pip install certifi). --latest reads the branch of this repository's checkout. Run it from a main or release/X.Y.Z checkout.

Install behind a restricted network (CN / no docker.io / slow GHCR)

On clusters that can't reach docker.io or pull GHCR image blobs (read timeouts), openbkn install accepts three flags so the script handles it — no manual crictl pull/retag:

• --registry=<swr / ghcr / host/ns> — image registry for BKN images and the bundled data-service / ingress images (sugar for --set image.registry). swr → swr.cn-east-3.myhuaweicloud.com/openbkn-ai, ghcr → ghcr.io/openbkn-ai. Precedence: explicit --set image.registry=… > --registry > an image.registry already in your --config YAML (respected, e.g. dev/conf/mac-config.yaml) > default swr (when config file does not set image.registry). SWR mirrors the same …-main.<date>.sha… build tags as GHCR.
• --dockerhub-mirror=<auto / host / off> — containerd docker.io mirror for third-party images (otel/hydra/postgres/minio). Writes /etc/containerd/certs.d/docker.io/hosts.toml (needs root + a containerd config_path certs.d; else it warns and skips, never fails). Defaults to auto — probes a candidate list and picks the first mirror that serves this stack's docker.io images over the mirror (?ns=docker.io) protocol (sentinel oryd/hydra; docker.m.daocloud.io 403s namespaced repos there, so a fixed default isn't safe). Pass a host to pin one (e.g. docker.1panel.live); off disables. Candidate list overridable via OPENBKN_DOCKERHUB_MIRROR_CANDIDATES.
• --latest — when no --version_file is given, installs the newest builds for this checkout's main or release/X.Y.Z branch (requires git).

# Newest builds + BKN images from SWR + docker.io third-party via the default mirror:
sudo bash ./deploy.sh openbkn install --latest --registry=swr

# Or with a pre-generated manifest (e.g. built on a dev box, no git on the target):
sudo bash ./deploy.sh openbkn install --version_file=/tmp/m.yaml --registry=swr

Moving BKN images into an air-gapped cluster by hand: versions published from 0.1.5 on also carry single-architecture tags, <image>:<version>-amd64 and <image>:<version>-arm64 (earlier versions have only the multi-arch <image>:<version>). Pull the one matching the target node on any machine — no --platform, no skopeo — retag it to the plain version, and import it into the node's containerd (k3s ctr on k3s):

docker pull ghcr.io/openbkn-ai/bkn-backend:0.1.5-arm64
docker tag  ghcr.io/openbkn-ai/bkn-backend:0.1.5-arm64 ghcr.io/openbkn-ai/bkn-backend:0.1.5
docker save ghcr.io/openbkn-ai/bkn-backend:0.1.5 -o bkn-backend.tar
# on the target node:
ctr -n k8s.io images import bkn-backend.tar

The committed migrations fix any DB-schema drift (e.g. vega-backend 0.9.x), but only run when the data-migrator pre-install job runs — i.e. via openbkn install, not a bare kubectl set image.

Resource Configuration

You can set Kubernetes resource requests and limits for all Core services via environment variables:

# Set CPU and memory requests
OPENBKN_CORE_REQ_CPU=200m OPENBKN_CORE_REQ_MEM=512Mi \
  sudo bash ./deploy.sh openbkn install

# Set full resource limits
OPENBKN_CORE_REQ_CPU=200m OPENBKN_CORE_REQ_MEM=512Mi \
  OPENBKN_CORE_LIM_CPU=2 OPENBKN_CORE_LIM_MEM=2Gi \
  sudo bash ./deploy.sh openbkn install

Environment Variable	Description	Example Values
OPENBKN_CORE_REQ_CPU	CPU request value	200m, 1
OPENBKN_CORE_REQ_MEM	Memory request value	512Mi, 1Gi
OPENBKN_CORE_LIM_CPU	CPU limit value	2, 4
OPENBKN_CORE_LIM_MEM	Memory limit value	2Gi, 4Gi

Notes:

• These variables default to empty; chart defaults are used when unset
• In k3s mode, lightweight resource defaults are applied (CPU request 100m, memory request 128Mi); override with the variables above
• Settings apply uniformly to all bkn-core releases

📋 Prerequisites

System requirements

Item	Minimum configuration (testing and learning)	Recommended configuration (minimum for production)
Operating system	CentOS 8+, OpenEuler 23+, or compatible Linux	CentOS 8+
CPU	4 cores	16 cores or more
Memory	8 GB	32 GB or more
Disk	200 GB	500 GB or more
Privileges	root or a user with sudo access	root or a user with sudo access

Network requirements

The deployment scripts need access to these domains:

Domain	Purpose
mirrors.aliyun.com	RPM package mirrors
mirrors.tuna.tsinghua.edu.cn	containerd.io RPM mirror
registry.aliyuncs.com	Kubernetes component images
swr.cn-east-3.myhuaweicloud.com	OpenBKN application image registry
repo.huaweicloud.com	Helm binary download
openbkn-ai.github.io	OPenbkn Helm chart repository
rancher-mirror.rancher.cn	k3s install script / binary (k3s quickstart path; override with K3S_INSTALL_URL)

📦 Deployment Model

openbkn is the OpenBKN product-level entrypoint in this repository. The install flow is:

1. Install or repair single-node Kubernetes, local-path storage, and ingress-nginx.
2. Install or repair data services: MariaDB, Redis, Kafka, and OpenSearch.
3. Deploy the OpenBKN application charts.

The Core application layer includes charts for data services management, application deployment, and task orchestration.

🔧 Usage

Recommended commands

# Install OpenBKN
./deploy.sh openbkn install

# Show Core status
./deploy.sh openbkn status

# Uninstall Core
./deploy.sh openbkn uninstall

# Cluster and Pod status
kubectl get nodes
kubectl get pods -A

Install status & health

openbkn status prints a live, detailed table for operators on the server — for every release in the manifest it shows the expected vs deployed chart version, app version, helm revision/status, workload ready count (DRIFT/MISSING flagged), the bundled dependency services, and a per-service application health section (probed via the apiserver service proxy: /health/ready → /api/v1/health → /healthz → /health, classified up / degraded / no-endpoint).

# Detailed live status table (versions, ready, drift, service health)
./deploy.sh openbkn status

# (Re)publish the non-sensitive JSON snapshot + the /install-status endpoint
# without reinstalling. Runs automatically at the end of `openbkn install`.
./deploy.sh openbkn publish-status

For an offline environment, sync the latest images (including openbkn-ai/library/nginx:1.27-alpine) before republishing the endpoint, and pass the offline flag to publish-status:

./scripts/sync-k8s-images.sh <offline-registry>
./deploy.sh --offline=<offline-registry> openbkn publish-status

A dashboard is served through the administrator-only ingress/port (a tiny nginx serving ConfigMaps — see conf/install-status/). The administrator entry point redirects with a Studio access token in the URL fragment; there is no manual token-entry form:

• GET /install-status — an HTML page rendering releases, per-service health, and dependency topology (auto-refreshes; static, no build step / CDN).
• GET /install-status.json — the raw JSON snapshot the page consumes (admin token required).

# Administrator redirect target: https://<access-address>/install-status#token=<Studio-access-token>
# English UI: https://<access-address>/install-status?lang=en
# Chinese UI: https://<access-address>/install-status?lang=zh (default)

The token is read from the URL fragment and removed from the address bar immediately. A tab-scoped sessionStorage copy permits refresh for up to 15 minutes while the same-origin Studio access-token cookie still matches. Studio logout, cookie rotation, expiry, or a 401/403 response clears the copy. This dashboard requires Studio and /install-status on the same origin. Reopen Service operations from the Studio administrator menu when authorization is required. Every status and log request still uses the bkn-safe administrator check.

The existing /install-status page now shows whitelisted container waiting and termination reasons, exit codes, rollout counts for Deployments, StatefulSets, DaemonSets and Jobs, safe Warning Event summaries, and probe-method summaries. Each Pod also has an expandable sanitized Describe summary covering lifecycle, Pod Conditions, init containers, container state, and related Warning Events; init-container logs are available from that summary. It excludes nodes/IPs, annotations, environment, mounts, and raw Event messages. Version, dependency, and HTTP-health data comes from the latest publication; Pod, workload, and Event data refreshes every 30 seconds. It displays the last successful collection time for Pod, workload and Event snapshots, flags data older than 90 seconds, and distinguishes chart versions from the live image-tag source. The redirect token is read from the URL fragment (which is never sent to nginx), immediately removed from the address bar, and kept only in page memory. All snapshot, Pod, workload, Event and log endpoints independently validate it through bkn-safe's Studio is_admin rule. Logs are never fetched automatically or included in public JSON. Clicking a log button opens a modal dialog, where operators can switch between recent and previous logs or search loaded lines; closing it clears the displayed log content. The reader returns at most 200 lines / 64 KiB; page links request 100 lines. Access audit entries (Studio account, target, line bound and result) are written to the reader container log without log content or credentials; inspect them with kubectl -n <namespace> logs deployment/install-status -c log-reader.

Log access is enabled by default and requires the same administrator Token. Application logs can contain secrets that the status service cannot reliably redact; disable it and republish where this exposure is unacceptable:

INSTALL_STATUS_LOGS_ENABLED=false ./deploy.sh openbkn publish-status

Only the existing install-status Deployment and Service are used; there is no second workload or public operations API. A Python reader container is added to the same Pod, using swr.cn-east-3.myhuaweicloud.com/openbkn-ai/library/python:3.12-alpine online. Sync this image for offline deployment or override it with INSTALL_STATUS_PYTHON_IMAGE. Because this is a single-Pod design, an image pull failure can block the whole Deployment rollout. This change also grants namespace-scoped pods/log get to the existing ServiceAccount, and read access to Events, Deployments, StatefulSets, DaemonSets and Jobs in the namespace.

The authenticated status view carries product/release versions, ready counts, dependency-service connection topology, and classified per-service health — and deliberately no credentials: the collector (scripts/lib/install_status.py) whitelists fields (host/port/type only; passwords, users, keys, tokens are dropped) and never exposes raw health-endpoint bodies (those can carry internal versions/topology). The ConfigMap is refreshed per install; nginx reads the mounted file per request, so no pod restart is needed.

Studio one-click samples

The standard product composition includes bkn-sample-studio. The installer supplies the existing internal ingress-nginx HTTP gateway through the Chart baseURL value, independently of the public HTTPS access address; explicit operator --set overrides remain supported. Incorrect stored sample gateway values are reconciled even when the Chart version is unchanged. Published sample Charts pin the service/executor image digest. The installer does not infer API compatibility from Chart versions or component names.

With the sample service's standard-API runtime defaults, leave platform.version and platform.capabilities empty: the service reads backend version through GET /api/bkn-backend/v1/health and checks required API adapter names against its implemented installation profile. Target API availability, authorization and licensing are enforced by the actual installation calls, with failures recorded in the corresponding stages. GET /api/safe/v1/capabilities describes licensed capabilities and is not the sample API inventory. Older sample service builds still require explicit compatibility values; use a published build containing the runtime-defaults change. See the sample service deployment guide. Full fresh-product installation acceptance remains separate from configuration/preflight checks.

After signing in, open Studio Home → One-click experience. Click Refresh samples to scan official releases, inspect a version's release notes and install it. Without an online catalog, administrators can Import offline package; its fixed data image must be available to the cluster. The internal openbkn-samples Kubernetes namespace is managed by the Chart. The current scope is one OpenBKN instance per environment; in-place sample upgrades and uninstall are not supported. See the sample development guide.

📁 Project Structure

deploy/
├── deploy.sh                 # Main entry script
├── conf/                     # Bundled config and static manifests
│   └── install-status/       # /install-status endpoint manifests (nginx + ingress)
├── release-manifests/        # Versioned release bill of materials
├── scripts/
│   ├── lib/                  # Common helper functions (install_status.py: status collector)
│   ├── services/             # Product and dependency install scripts (status.sh: install status)
│   └── sql/                  # Versioned SQL initialization scripts
└── .tmp/charts/              # Local chart cache generated by download

🗑️ Uninstall

bash deploy.sh openbkn uninstall removes only the Core application layer.

# Remove the Core application layer
./deploy.sh openbkn uninstall

bash deploy.sh k8s reset resets the Kubernetes cluster, including data services and core.

# Reset Kubernetes cluster
./deploy.sh k8s reset

🔍 Troubleshooting

CoreDNS is not ready

# Check whether firewall is disabled
systemctl status firewalld

# Restart CoreDNS
kubectl -n kube-system delete pod -l k8s-app=kube-dns

Pods fail to pull images

# Check network connectivity
curl -I https://swr.cn-east-3.myhuaweicloud.com

# Check containerd config
cat /etc/containerd/config.toml

Kubernetes apt / yum source missing or 404

preflight.sh --check-only reports one of these in strict mode (default):

[FAIL] Deprecated Kubernetes apt source detected (packages.cloud.google.com) ...
[FAIL] apt has no install candidate for kubeadm — Kubernetes apt source missing or unreachable.
[FAIL] dnf/yum has no install candidate for kubeadm — Kubernetes yum repo missing or unreachable.

Recommended fix (one command):

sudo bash deploy/preflight.sh --fix --fix-allow=k8s-pkgs-repo
# or, to also pre-stage containerd / helm / Node, etc.:
sudo bash deploy/preflight.sh --fix -y

preflight --fix → k8s-pkgs-repo covers both scenarios (legacy alias k8s-apt-source still works in --fix-allow):

• Legacy packages.cloud.google.com source present → migrate it to pkgs.k8s.io.
• No K8s source configured at all → write /etc/apt/sources.list.d/kubernetes.list (or /etc/yum.repos.d/kubernetes.repo) pointing to pkgs.k8s.io/core:/stable:/<vX.Y>/deb|rpm/.

Pin a specific minor with PREFLIGHT_K8S_APT_MINOR=v1.28 (default: detected from installed kubeadm, falls back to v1.28).

Manual fallback (Ubuntu/Debian):

sudo apt-mark unhold kubeadm kubelet kubectl || true
sudo apt remove -y kubeadm kubelet kubectl
sudo rm -f /etc/apt/sources.list.d/kubernetes.list
sudo rm -f /etc/apt/keyrings/kubernetes-apt-keyring.gpg
sudo mkdir -p /etc/apt/keyrings

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key \
  | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /' \
  | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt update
sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Manual fallback (RHEL/CentOS/openEuler):

sudo tee /etc/yum.repos.d/kubernetes.repo > /dev/null <<'EOF'
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF

sudo dnf install -y --disableexcludes=kubernetes kubeadm kubelet kubectl   # or: yum

containerd cannot be installed (no containerd.io candidate)

On stock Ubuntu (no Docker CE repo) preflight.sh reports:

[FAIL] apt has no install candidate for containerd.io OR containerd.
[FAIL] containerd not found ...

preflight --fix → containerd-install now tries containerd.io first (Docker CE repo) and falls back to the distro containerd package automatically:

sudo bash deploy/preflight.sh --fix --fix-allow=containerd-install

If both fail, the apt/yum source itself is broken — fix apt-get update / dnf repolist first.

Strict mode and --lenient

preflight.sh defaults to strict mode (PREFLIGHT_STRICT=true). Items that block install AND are auto-fixable by --fix are reported as [FAIL] (so --check-only exits 1), not [WARN]:

• swap, net.ipv4.ip_forward, br_netfilter / overlay modules, bridge-nf-call-*
• vm.max_map_count, fs.inotify.*, ulimit -n soft
• containerd not found / socket missing, kubectl, helm, overlay fs
• broken apt-get update, dnf/yum repolist failures, missing kubeadm/containerd install candidate

To downgrade these back to [WARN] (e.g. on a low-spec lab box where you accept the risk):

sudo bash deploy/preflight.sh --check-only --lenient
# equivalent to: PREFLIGHT_STRICT=false PREFLIGHT_STRICT_SOURCES=false sudo bash deploy/preflight.sh

View component logs

kubectl logs -n <namespace> <pod-name>

OAuth login redirect_uri mismatch

Login fails with invalid_request ... 'redirect_uri' ... does not match ... pre-registered redirect urls: the callback the browser sent is not in the login client's (openbkn-studio, etc.) registered list. These clients live in Hydra and are registered by the bkn-safe chart's client-seed Job from accessAddress and clientSeed.*. The callback is the browser's address + /studio/callback (e.g. https://<access-address>/studio/callback).

# 1. Standard dev port (least effort): run the local studio dev server on
#    localhost:8000; http://localhost:8000/studio/callback is registered by default.

# 2. Permanent / production address -> add to chart values, then reinstall (the
#    seed Job re-registers it and it survives upgrades):
#    add an entry to clientSeed.extraWebRedirectUris in
#    bkn-safe/charts/bkn-safe/values.yaml. The server-side studio's gateway callback
#    is derived from accessAddress automatically — no need to list it.

# 3. Temporary / dev address -> no reinstall, add via the bkn-safe admin API
#    (super-admin required; wiped on the next upgrade):
export BKN_HOST=https://<access-address>
openbkn auth login "$BKN_HOST"        # must be a super-admin session
bash deploy/scripts/bkn-redirect.sh add  http://localhost:5173/studio/callback
bash deploy/scripts/bkn-redirect.sh list
bash deploy/scripts/bkn-redirect.sh del  http://localhost:5173/studio/callback

See bkn-safe/docs/oauth-redirect-uris.md for detail.

📄 License

Apache License 2.0

Internal Trace Admission protocol upgrade

This install profile targets the protocol in Foundry PR #2047. Policy, configuration, heartbeat and ACK use agent-observability-internal:8081 without Admission OAuth credentials or Ed25519 keys. Upgrade the server, publishers, Collector and Charts as a compatible set; there is no legacy client compatibility layer.

The complete install enables five Kafka Evidence publishers. Kafka SASL still requires credentials: the installer copies the first client password from resource/kafka-sasl into bkn-trace-evidence-kafka in the application namespace, using KAFKA_CLIENT_USER as the username. For external Kafka, provision the target Secret with username and password; OPENBKN_TRACE_KAFKA_SECRET overrides its name. Artifact credentials remain separate.

The install also enables agent-observability's Evidence Kafka consumer, using the same broker and credential Secret. A separate Admin Job (96 MiB Java heap, 256 MiB container limit) creates openbkn.evidence.v1 when absent with message.timestamp.type=LogAppendTime, then verifies the effective setting before deploying Core. It never runs Admin tools inside the broker. Existing topics are not altered: a CreateTime topic stops installation with an actionable error, because changing the setting does not rewrite old messages. Owners must review the migration and then verify newly produced MCP Evidence. The Job is cleaned up on success or failure.

For external Kafka, the client needs permission to create/describe this topic, or the operator must provision it first. OPENBKN_TRACE_KAFKA_ADMIN_IMAGE selects a Kafka CLI image; the default uses the bundled Kafka image. SASL defaults to PLAIN; use OPENBKN_TRACE_KAFKA_SASL_MECHANISM for SCRAM-SHA-256 or SCRAM-SHA-512. Topic creation defaults to one partition and replication factor one, matching the bundled broker; override OPENBKN_TRACE_TOPIC_PARTITIONS and OPENBKN_TRACE_TOPIC_REPLICATION_FACTOR for a larger cluster. An explicit --set kafkaConsumers.evidence.enabled=false skips the topic check. The indexed broker, Secret name and SASL --set overrides are also honored by the check.

Before upgrading, review networkPolicy.allowedClients for all six callers and the actual namespace/Pod labels. The resource-limit cleanup path exports old effective Helm values, so an old allowlist can survive even without --reuse-values. Explicitly update it. Same-version installs reconcile missing or outdated publisher/control values.

The default kubeadm installer uses Flannel, whose data plane does not enforce NetworkPolicy by itself. Configure a policy enforcement component and verify it in the target environment. See https://github.com/flannel-io/flannel/blob/master/Documentation/netpol.md . Verify enforcement on k3s and other distributions as well.

Acceptance requires all six callers to read policy/configuration and send heartbeat/ACK, an unlisted Pod to fail access to port 8081, public internal routes to return 404, and an enable/disable operation to converge with Evidence delivery. Test fresh installation and existing-version upgrades before closing Foundry #2043/#2046. Roll back the server, clients, Collector and Charts together, restoring the OAuth/signing configuration required by the old version.

Cluster-free regressions:

bash scripts/services/openbkn_trace_control_test.sh
bash scripts/services/openbkn_trace_profile_test.sh
bash scripts/services/openbkn_trace_prerequisites_test.sh
bash scripts/services/openbkn_trace_topic_test.sh
python3 scripts/lib/trace_kafka_topic_job_test.py

Set FOUNDRY_ROOT to a compatible checkout for additional index and data-migrator contract checks. A standalone checkout explicitly reports those cross-repository checks as skipped. These checks do not replace Chart rendering or cluster acceptance.

System Audit installation (Foundry #2061 / #2094)

The complete installer also enables Safe Audit/access publishing with the configured MQ broker and existing Kafka credential Secret. auditPublisher.environment uses current config/CLI, then the saved Safe environment, then the saved AO environment (default for a new deployment: development); Safe uses the flat credentialsSecretName, usernameSecretKey and passwordSecretKey fields. Missing/disabled publisher configuration is repaired even when the Safe Chart version is unchanged; a matching configuration skips the rollout. Verify new user-management and login/logout events through Kafka, Ledger and the log API/Studio; Pod Ready or historical records alone do not establish coverage.

This ports the installer wiring from Foundry PR #2097, not application code or Charts. The standalone Chart default kafkaConsumers.audit.enabled=false stays unchanged. The complete installer explicitly enables the Audit consumer and Core publisher in final agent-observability values and uses Core MariaDB with migration enabled. The default independent consumer group is bkn-trace-audit-ledger-v1; existing group and Secret references are preserved.

No extra --config file is required. Existing Trace prerequisites prepare bkn-trace-core-mariadb and reuse bkn-trace-evidence-kafka (OPENBKN_TRACE_KAFKA_SECRET overrides the Kafka Secret name). Brokers and SASL come from the existing MQ configuration. External Kafka still requires an existing credentials Secret. No Kafka users, passwords, ACLs, or broker authentication settings are changed.

Publisher environment defaults to development; a saved valid value or an explicit override is reused. No environment argument is required for the default installation on main:

./deploy.sh openbkn install --registry=swr --latest \
  --set resources.requests.cpu=0m --set resources.requests.memory=0Mi

The consumer and publisher may reuse the existing Kafka credentials Secret; there is no production-only requirement for a separate approved write Secret. Override auditPublisher.environment or auditPublisher.existingSecret.name only when needed. No account is provisioned automatically. Both components support existingSecret.usernameKey / passwordKey (defaults: username / password); credentials use Secret references, not Helm plaintext. The existing publisher requires PLAIN; unsupported mechanisms fail before application upgrades.

After existing Trace prerequisites, before the application release loop, client-only helm template validates the staged pinned AO Chart, final merged values, and required Secret keys. A bounded separate Kafka Admin Job checks openbkn.audit.v1. Only an absent topic is created with message.timestamp.type=LogAppendTime. Existing CreateTime stops installation and requires a separately reviewed migration: changing configuration cannot rewrite historical records. Existing topics, records, offsets, authentication and Evidence topic behavior are not modified. The Job is deleted on success/failure.

Same-version Audit repair preserves AO's Chart-owned user settings including image, resources, Evidence, OpenSearch and EE. Unintended non-Audit changes are rejected; current explicit inputs may change them. AO skips only when final configuration and original reconciliation checks match. Pending releases are not automatically uninstalled; concurrent revision changes stop the upgrade. Other release order remains unchanged. Failures return nonzero and stop the application release loop; earlier prerequisites are not automatically rolled back.

Real-cluster acceptance: both Audit flags true, valid environment/Secret references, unchanged Kafka authentication and other service settings, effective LogAppendTime, healthy consumer/publisher, and a real audited business operation visible in System Audit without audit_consumer_not_configured. An empty HTTP 200 alone is insufficient. Cover fresh install, same-version repair, repeat install and failures. Rollback the prior AO Helm revision only; do not delete database tables, Kafka messages or offsets.

Cluster-free regressions (Helm 3.18.6 and Python 3):

PYTHONPATH=scripts/lib python3 -m unittest test_audit_consumer test_audit_helm_render audit_kafka_topic_job_test
bash scripts/services/openbkn_audit_consumer_test.sh

The small Helm fixture and mocked cluster tests do not replace published product Chart rendering or real-cluster acceptance.

Trace/Evidence admission budget

Set observability.admissionBudget in the persistent --config YAML used for installation and upgrades. Config generation preserves the existing observability block. New configs supply collectorMetricsEndpoint: http://otelcol-contrib:8888/metrics for the bundled Collector; change it when using a different deployment topology. No Admission credentials or permissions are added by this configuration.

Fill all four deployment/SLO thresholds (opensearchCapacityThreshold, opensearchHeapThreshold, collectorQueueThreshold, storagePoolThreshold) with finite numbers in (0, 1]. They deliberately have no default. Leaving them empty keeps budget-dependent configuration reads and enable requests unavailable, with explicit diagnostic fields; it does not disable existing capture or stop unrelated APIs. The metrics endpoint must provide both otelcol_exporter_queue_size and a positive otelcol_exporter_queue_capacity.

Keep these values in the config used on every upgrade. A separate one-time Helm override or kubectl set env does not establish that persistent configuration source. Add the block explicitly to existing configurations before requesting Trace/Evidence enable. If applying config changes without changing Chart versions, use --force-upgrade so the installer does not skip the release.