From aa11a002135b03e7d9d72bc8df7744476a5883db Mon Sep 17 00:00:00 2001 From: stevyhacker Date: Tue, 20 Oct 2026 15:11:15 +0200 Subject: [PATCH 1/5] Add signed apt and dnf repositories for the Linux packages The .deb and .rpm from the release page never updated. Stable releases now publish to apt and dnf repositories at pkg.lokalbot.com (Cloudflare R2), and once the repository key is committed the packages add them themselves (deb822 sources + keyring; yum .repo with gpgcheck and repo_gpgcheck + key), so apt upgrade and dnf upgrade bring later versions. - linux-repos.yml: called by release.yml for stable tags, or by hand for a published one. GitHub-hosted only; skips without the LINUX_REPO_* secrets (fails once the key is committed). Signs with rpmsign and gpg via scripts/linux-repos.sh, keeps the newest 3 versions, uploads with rclone in a cache-safe order and checks what the domain serves. - On pull requests, scripts/ci/check-linux-repos.sh packages the latest release's binaries with a throwaway key and checks install, upgrade, kept edits and refusal of tampered or unsigned files in Ubuntu 24.04, Debian 13 and Fedora containers. - Docs: Linux README, RELEASING.md (setup, expiry, rotation, leaks), READMEs and the system requirements page. --- .github/workflows/linux-repos.yml | 282 +++++++++++++++++ .github/workflows/release.yml | 19 ++ LinuxWindows/README.md | 14 +- LinuxWindows/packaging/linux/lokalbot.repo | 12 + LinuxWindows/packaging/linux/lokalbot.sources | 9 + LinuxWindows/scripts/ci/check-linux-repos.sh | 285 ++++++++++++++++++ LinuxWindows/scripts/linux-repos.sh | 152 ++++++++++ LinuxWindows/scripts/package-linux-distros.sh | 40 ++- README.md | 2 +- README.zh-CN.md | 2 +- RELEASING.md | 150 +++++++++ Scripts/guide_pages.py | 2 +- web/system-requirements.html | 2 +- 13 files changed, 962 insertions(+), 9 deletions(-) create mode 100644 .github/workflows/linux-repos.yml create mode 100644 LinuxWindows/packaging/linux/lokalbot.repo create mode 100644 LinuxWindows/packaging/linux/lokalbot.sources create mode 100755 LinuxWindows/scripts/ci/check-linux-repos.sh create mode 100755 LinuxWindows/scripts/linux-repos.sh diff --git a/.github/workflows/linux-repos.yml b/.github/workflows/linux-repos.yml new file mode 100644 index 00000000..38864a9d --- /dev/null +++ b/.github/workflows/linux-repos.yml @@ -0,0 +1,282 @@ +# Publishes LokalBot's apt and dnf repositories at https://pkg.lokalbot.com +# (apt at /apt, rpm at /rpm, the public key at /lokalbot.gpg and +# /lokalbot.asc) from a stable release's .deb and .rpm, with +# LinuxWindows/scripts/linux-repos.sh. The packages add these repositories +# once their key is in LinuxWindows/packaging/linux (RELEASING.md, "Linux +# package repositories"). +# +# Release (release.yml) calls this for a stable v* tag after it attached the +# Linux packages. To publish a release that is already out: +# +# gh workflow run linux-repos.yml -f tag=v0.11.0 +# +# Pre-releases never reach the repositories. Without the secrets below it +# skips with a notice, unless the packages already point at the +# repositories: then a missing secret fails the run. +# +# LINUX_REPO_GPG_PRIVATE_KEY armored secret key that signs the +# repositories and RPMs +# LINUX_REPO_R2_ACCOUNT_ID Cloudflare account that has the bucket +# LINUX_REPO_R2_ACCESS_KEY_ID R2 API token with Object Read & Write +# LINUX_REPO_R2_SECRET_ACCESS_KEY on the lokalbot-packages bucket only +# +# The key decides what every subscribed computer installs as root, so +# signing runs only on GitHub's runners, never on a self-hosted one. +# +# On a pull request that changes the repositories or the packages, it +# packages the latest release's Linux binaries with a throwaway key and +# checks that apt and dnf upgrade from a local copy of the repositories and +# refuse tampered files (scripts/ci/check-linux-repos.sh). That publishes +# nothing and reads no secret. + +name: Linux package repositories + +on: + workflow_call: + inputs: + tag: + description: The stable release tag, e.g. v0.11.1 + required: true + type: string + secrets: + LINUX_REPO_GPG_PRIVATE_KEY: + required: false + LINUX_REPO_R2_ACCOUNT_ID: + required: false + LINUX_REPO_R2_ACCESS_KEY_ID: + required: false + LINUX_REPO_R2_SECRET_ACCESS_KEY: + required: false + workflow_dispatch: + inputs: + tag: + description: "Stable release tag to publish, e.g. v0.11.0" + required: true + type: string + pull_request: + branches: [master] + paths: + - ".github/workflows/linux-repos.yml" + - "LinuxWindows/scripts/linux-repos.sh" + - "LinuxWindows/scripts/ci/check-linux-repos.sh" + - "LinuxWindows/scripts/package-linux-distros.sh" + - "LinuxWindows/packaging/linux/lokalbot*" + +permissions: + contents: read + +defaults: + run: + shell: bash + working-directory: LinuxWindows + +env: + BUCKET: lokalbot-packages + KEY_FILE: packaging/linux/lokalbot-archive-keyring.asc + +jobs: + check: + name: Check apt and dnf upgrades (throwaway key) + if: github.event_name == 'pull_request' + runs-on: ubuntu-24.04 + timeout-minutes: 40 + concurrency: + group: linux-repos-check-${{ github.head_ref }} + cancel-in-progress: true + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + # The same nfpm as linux-windows-release.yml. + - name: Install nfpm + env: + NFPM_VERSION: "2.47.0" + NFPM_SHA256: 0660ca602b2d2d2ae4781a06c692b3eeb9d437ffea05b831d76e41f4a3188783 + run: | + archive="nfpm_${NFPM_VERSION}_Linux_x86_64.tar.gz" + curl -fsSL -o "$RUNNER_TEMP/$archive" "https://github.com/goreleaser/nfpm/releases/download/v${NFPM_VERSION}/$archive" + echo "$NFPM_SHA256 $RUNNER_TEMP/$archive" | sha256sum -c - + tar -xzf "$RUNNER_TEMP/$archive" -C "$RUNNER_TEMP" nfpm + + # Packaging needs binaries, not this branch's build: the latest + # release's are enough. + - name: Download the latest release's Linux binaries + env: + GH_TOKEN: ${{ github.token }} + run: | + mkdir -p "$RUNNER_TEMP/release" + cd "$RUNNER_TEMP/release" + gh release download --repo "$GITHUB_REPOSITORY" --pattern 'LokalBot-*-linux-x86_64.tar.gz*' + sha256sum -c LokalBot-*-linux-x86_64.tar.gz.sha256 + tar -xzf LokalBot-*-linux-x86_64.tar.gz + + - name: Check the repositories in Ubuntu, Debian and Fedora containers + run: NFPM="$RUNNER_TEMP/nfpm" scripts/ci/check-linux-repos.sh "$RUNNER_TEMP"/release/LokalBot-*-linux-x86_64/ + + ready: + name: Check the tag and repository secrets + if: github.event_name != 'pull_request' + runs-on: ubuntu-24.04 + timeout-minutes: 5 + outputs: + publish: ${{ steps.check.outputs.publish }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + sparse-checkout: LinuxWindows/packaging/linux + + - id: check + env: + TAG: ${{ inputs.tag }} + GH_TOKEN: ${{ github.token }} + LINUX_REPO_GPG_PRIVATE_KEY: ${{ secrets.LINUX_REPO_GPG_PRIVATE_KEY }} + LINUX_REPO_R2_ACCOUNT_ID: ${{ secrets.LINUX_REPO_R2_ACCOUNT_ID }} + LINUX_REPO_R2_ACCESS_KEY_ID: ${{ secrets.LINUX_REPO_R2_ACCESS_KEY_ID }} + LINUX_REPO_R2_SECRET_ACCESS_KEY: ${{ secrets.LINUX_REPO_R2_SECRET_ACCESS_KEY }} + run: | + prerelease=$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json isPrerelease --jq .isPrerelease) + if [[ "$TAG" == *-* || "$prerelease" == true ]]; then + echo "::notice title=Linux repositories::$TAG is a pre-release; the apt and dnf repositories take stable releases only." + echo "publish=false" >> "$GITHUB_OUTPUT" + exit 0 + fi + missing=() + for name in LINUX_REPO_GPG_PRIVATE_KEY LINUX_REPO_R2_ACCOUNT_ID \ + LINUX_REPO_R2_ACCESS_KEY_ID LINUX_REPO_R2_SECRET_ACCESS_KEY; do + [[ -n "${!name:-}" ]] || missing+=("$name") + done + if (( ${#missing[@]} )); then + if [[ -f "$KEY_FILE" ]]; then + echo "::error title=Linux repositories::Missing ${missing[*]}. This release's packages add the repositories at pkg.lokalbot.com, so they must get it (RELEASING.md, Linux package repositories)." + exit 1 + fi + echo "::notice title=Linux repositories not updated::Missing ${missing[*]}; see RELEASING.md, Linux package repositories." + echo "publish=false" >> "$GITHUB_OUTPUT" + exit 0 + fi + echo "publish=true" >> "$GITHUB_OUTPUT" + + publish: + name: Publish to pkg.lokalbot.com + needs: ready + if: needs.ready.outputs.publish == 'true' + runs-on: ubuntu-24.04 # GitHub-hosted: the signing key never reaches a self-hosted runner + timeout-minutes: 30 + # One publish at a time: each one rebuilds the indexes from the bucket. + concurrency: + group: linux-repos-publish + cancel-in-progress: false + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - name: Install the repository tools + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends apt-utils createrepo-c rpm rclone gnupg + + # The repositories serve the same .deb users download; the .rpm gets a + # signature on the way in. + - name: Download the release's packages + env: + TAG: ${{ inputs.tag }} + GH_TOKEN: ${{ github.token }} + run: | + mkdir -p "$RUNNER_TEMP/packages" + cd "$RUNNER_TEMP/packages" + gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \ + --pattern 'LokalBot-*-linux-amd64.deb*' --pattern 'LokalBot-*-linux-x86_64.rpm*' + sha256sum -c ./*.sha256 + version="${TAG#v}-1" + [[ "$(dpkg-deb --field LokalBot-*.deb Version)" == "$version" ]] || { echo "::error::The .deb is not lokalbot $version" >&2; exit 1; } + [[ "$(rpm -qp --queryformat '%{VERSION}-%{RELEASE}' LokalBot-*.rpm)" == "$version" ]] || { echo "::error::The .rpm is not lokalbot $version" >&2; exit 1; } + rm ./*.sha256 + + - name: Import the signing key + env: + LINUX_REPO_GPG_PRIVATE_KEY: ${{ secrets.LINUX_REPO_GPG_PRIVATE_KEY }} + run: | + umask 077 + export GNUPGHOME="$RUNNER_TEMP/gnupg" + mkdir -p "$GNUPGHOME" + printf '%s\n' "$LINUX_REPO_GPG_PRIVATE_KEY" | gpg --batch --quiet --import + keys=$(gpg --batch --with-colons --list-secret-keys | awk -F: '$1 == "sec"' | wc -l) + if (( keys != 1 )); then + echo "::error::LINUX_REPO_GPG_PRIVATE_KEY must hold exactly one secret key, not $keys" >&2 + exit 1 + fi + fpr=$(gpg --batch --with-colons --list-secret-keys | awk -F: '$1 == "fpr" {print $10; exit}') + expires=$(gpg --batch --with-colons --list-secret-keys | awk -F: '$1 == "sec" {print $7}') + if [[ -f "$KEY_FILE" ]]; then + if ! gpg --batch --show-keys --with-colons "$KEY_FILE" | awk -F: '$1 == "fpr" {print $10}' | grep -qx "$fpr"; then + echo "::error::The signing key $fpr is not in $KEY_FILE, which the packages install" >&2 + exit 1 + fi + else + echo "::warning::$KEY_FILE does not exist yet, so the packages do not add the repositories. Commit the public key (RELEASING.md, Linux package repositories)." + fi + if [[ -n "$expires" ]] && (( expires < $(date +%s) + 180 * 86400 )); then + echo "::warning::The repository key $fpr expires on $(date -u -d "@$expires" +%F). Extend it now (RELEASING.md, Linux package repositories)." + fi + echo "GNUPGHOME=$GNUPGHOME" >> "$GITHUB_ENV" + echo "FPR=$fpr" >> "$GITHUB_ENV" + + - name: Download the published packages + env: + RCLONE_CONFIG_R2_TYPE: s3 + RCLONE_CONFIG_R2_PROVIDER: Cloudflare + RCLONE_CONFIG_R2_ENDPOINT: https://${{ secrets.LINUX_REPO_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com + RCLONE_CONFIG_R2_ACCESS_KEY_ID: ${{ secrets.LINUX_REPO_R2_ACCESS_KEY_ID }} + RCLONE_CONFIG_R2_SECRET_ACCESS_KEY: ${{ secrets.LINUX_REPO_R2_SECRET_ACCESS_KEY }} + RCLONE_CONFIG_R2_NO_CHECK_BUCKET: "true" + run: rclone copy "r2:$BUCKET" "$RUNNER_TEMP/repo" --include '/apt/pool/**' --include '/rpm/*.rpm' + + - name: Add the release and sign the repositories + run: | + # Publish the keyring the packages install. + if [[ -f "$KEY_FILE" ]]; then export PUBLIC_KEY="$KEY_FILE"; fi + scripts/linux-repos.sh "$RUNNER_TEMP/repo" "$FPR" "$RUNNER_TEMP"/packages/* + find "$RUNNER_TEMP/repo" -type f | sed "s|^$RUNNER_TEMP/repo/||" | sort + + - name: Remove the signing key + if: always() + run: | + gpgconf --kill all || true + rm -rf "$RUNNER_TEMP/gnupg" + + # Packages and the indexes named by their hash go up first and never + # change, then the signed indexes that list them, which no cache keeps. + # Pruned packages go last. Old by-hash indexes and repodata stay (a few + # KB per release), so a client or cache holding the previous index + # still finds its files. + - name: Upload to the bucket + env: + RCLONE_CONFIG_R2_TYPE: s3 + RCLONE_CONFIG_R2_PROVIDER: Cloudflare + RCLONE_CONFIG_R2_ENDPOINT: https://${{ secrets.LINUX_REPO_R2_ACCOUNT_ID }}.r2.cloudflarestorage.com + RCLONE_CONFIG_R2_ACCESS_KEY_ID: ${{ secrets.LINUX_REPO_R2_ACCESS_KEY_ID }} + RCLONE_CONFIG_R2_SECRET_ACCESS_KEY: ${{ secrets.LINUX_REPO_R2_SECRET_ACCESS_KEY }} + RCLONE_CONFIG_R2_NO_CHECK_BUCKET: "true" + run: | + repo="$RUNNER_TEMP/repo" + rclone copy "$repo" "r2:$BUCKET" --checksum \ + --header-upload 'Cache-Control: public, max-age=31536000, immutable' \ + --filter '- /rpm/repodata/repomd.xml*' \ + --filter '+ /apt/pool/**' --filter '+ /apt/dists/**/by-hash/**' \ + --filter '+ /rpm/*.rpm' --filter '+ /rpm/repodata/**' --filter '- **' + rclone copy "$repo" "r2:$BUCKET" --checksum --header-upload 'Cache-Control: no-cache' + rclone sync "$repo" "r2:$BUCKET" --checksum \ + --filter '- /apt/dists/**/by-hash/**' --filter '- /rpm/repodata/**' + + - name: Check what pkg.lokalbot.com serves + run: | + repo="$RUNNER_TEMP/repo" + for path in apt/dists/stable/InRelease rpm/repodata/repomd.xml rpm/repodata/repomd.xml.asc lokalbot.gpg; do + if ! curl -fsS --retry 3 -o "$RUNNER_TEMP/served" "https://pkg.lokalbot.com/$path"; then + echo "::error::https://pkg.lokalbot.com/$path is unreachable. Is the bucket connected to the domain?" >&2 + exit 1 + fi + if cmp -s "$RUNNER_TEMP/served" "$repo/$path"; then + echo "https://pkg.lokalbot.com/$path is the new copy" + else + echo "::warning::https://pkg.lokalbot.com/$path is not the copy just uploaded; a cache may still hold the old one." + fi + done diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 5bf9de4a..6c89adb8 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -492,6 +492,25 @@ jobs: permissions: contents: read + # Adds a stable release's .deb and .rpm to the apt and dnf repositories at + # pkg.lokalbot.com once they are attached (linux-repos.yml, which lists its + # LINUX_REPO_* secrets and skips without them). GitHub-hosted, like every + # job that sees a secret. + linux-repos: + name: apt and dnf repositories + if: github.event_name == 'push' && !contains(github.ref_name, '-') + needs: linux-windows-publish + uses: ./.github/workflows/linux-repos.yml + with: + tag: ${{ github.ref_name }} + secrets: + LINUX_REPO_GPG_PRIVATE_KEY: ${{ secrets.LINUX_REPO_GPG_PRIVATE_KEY }} + LINUX_REPO_R2_ACCOUNT_ID: ${{ secrets.LINUX_REPO_R2_ACCOUNT_ID }} + LINUX_REPO_R2_ACCESS_KEY_ID: ${{ secrets.LINUX_REPO_R2_ACCESS_KEY_ID }} + LINUX_REPO_R2_SECRET_ACCESS_KEY: ${{ secrets.LINUX_REPO_R2_SECRET_ACCESS_KEY }} + permissions: + contents: read + # Runs only after the macOS job published the release (so its release gates # passed), signs the Linux and Windows update feeds with the Sparkle key the # DMG uses, checks them against SUPublicEDKey, and attaches archives and diff --git a/LinuxWindows/README.md b/LinuxWindows/README.md index c084e8fa..57caf657 100644 --- a/LinuxWindows/README.md +++ b/LinuxWindows/README.md @@ -74,8 +74,10 @@ release has none yet, use a CI run's artifacts (`lokalbot-linux-x86_64`, `lokalb The same release has packages that install into `/usr/bin` with their dependencies: `LokalBot--linux-amd64.deb` (Debian, Ubuntu: `sudo apt install ./LokalBot-…deb`), `LokalBot--linux-x86_64.rpm` (Fedora, openSUSE: `sudo dnf install ./LokalBot-…rpm`) and - `LokalBot--linux-x86_64.pkg.tar.zst` (Arch: `sudo pacman -U ./LokalBot-…zst`). A package - install tells you about updates and offers the download instead of replacing its files. + `LokalBot--linux-x86_64.pkg.tar.zst` (Arch: `sudo pacman -U ./LokalBot-…zst`). Installing the + `.deb`, or the `.rpm` on Fedora, also adds the LokalBot repository at `pkg.lokalbot.com`, so + `sudo apt upgrade` or `sudo dnf upgrade` brings later versions. With the Arch package, LokalBot tells you about updates and + offers the download instead of replacing its files. - **Windows** 10/11 (x64): run `LokalBot--windows-x86_64-setup.exe`. It installs for your user into `%LOCALAPPDATA%\Programs\LokalBot` without administrator rights, adds a Start menu shortcut (which also lets notifications show LokalBot's name) and an Installed apps entry; uninstalling keeps your library. Or @@ -120,6 +122,14 @@ its old file is removed on a later start. Your library is never touched. When Lo files (a package manager's copy, `/usr`, `/opt`, Program Files, a folder you cannot write), it offers the download from GitHub instead. Debug builds never update. +The `.deb` and `.rpm` update with your package manager, from the LokalBot repository they add: +`/etc/apt/sources.list.d/lokalbot.sources` with its key in `/usr/share/keyrings`, or +`/etc/yum.repos.d/lokalbot.repo` with its key in `/etc/pki/rpm-gpg`. apt and dnf check the repository's +signature and each package's. Your edits to those files survive upgrades; add `Enabled: no` or set +`enabled=0` to stop updates from it. The repository has the newest three stable versions, never pre-releases. +openSUSE's zypper doesn't read `/etc/yum.repos.d`, so there the `.rpm` is told about updates like the Arch +package. + On first launch LokalBot asks what to remember and downloads the models you choose: about 4 GB for the default transcription, Think and search models. diff --git a/LinuxWindows/packaging/linux/lokalbot.repo b/LinuxWindows/packaging/linux/lokalbot.repo new file mode 100644 index 00000000..d946c3d0 --- /dev/null +++ b/LinuxWindows/packaging/linux/lokalbot.repo @@ -0,0 +1,12 @@ +# LokalBot (alpha). The lokalbot package added this file so that +# `sudo dnf upgrade` installs new versions. Your edits are kept on upgrades; +# set enabled=0 to stop updates from this repository. +[lokalbot] +name=LokalBot (alpha) +baseurl=https://pkg.lokalbot.com/rpm +enabled=1 +gpgcheck=1 +repo_gpgcheck=1 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-lokalbot +metadata_expire=6h +skip_if_unavailable=True diff --git a/LinuxWindows/packaging/linux/lokalbot.sources b/LinuxWindows/packaging/linux/lokalbot.sources new file mode 100644 index 00000000..1a67a07f --- /dev/null +++ b/LinuxWindows/packaging/linux/lokalbot.sources @@ -0,0 +1,9 @@ +# LokalBot (alpha). The lokalbot package added this file so that +# `sudo apt upgrade` installs new versions. Your edits are kept on upgrades; +# add "Enabled: no" to stop updates from this repository. +Types: deb +URIs: https://pkg.lokalbot.com/apt +Suites: stable +Components: main +Architectures: amd64 +Signed-By: /usr/share/keyrings/lokalbot-archive-keyring.gpg diff --git a/LinuxWindows/scripts/ci/check-linux-repos.sh b/LinuxWindows/scripts/ci/check-linux-repos.sh new file mode 100755 index 00000000..2f05deea --- /dev/null +++ b/LinuxWindows/scripts/ci/check-linux-repos.sh @@ -0,0 +1,285 @@ +#!/usr/bin/env bash +# Checks LokalBot's apt and dnf repositories end to end with a throwaway key. +# scripts/package-linux-distros.sh packages STAGE's binaries as four +# versions, scripts/linux-repos.sh publishes them into a repository served +# over HTTPS as pkg.lokalbot.com on a private Docker network, and clean +# Ubuntu 24.04, Debian 13 and Fedora containers install the first version +# from the downloaded file, then upgrade with `apt upgrade` and `dnf upgrade` +# as later versions are published. It checks that: +# - the .deb and .rpm add the repository as config files, and an edited +# repository file survives an upgrade; the Arch package adds nothing; +# - apt refuses a tampered InRelease, a tampered Release (signed by +# Release.gpg) and a swapped .deb; dnf refuses a tampered repomd.xml, a +# swapped .rpm and an unsigned one; +# - each repository keeps its newest three versions. +# +# NFPM=/path/to/nfpm scripts/ci/check-linux-repos.sh STAGE +# +# STAGE is a staged release folder, LokalBot--linux-x86_64 (from +# scripts/package.sh, or extracted from a release's .tar.gz). Needs Docker +# and nfpm (a static binary, run inside a container). Writes into dist/ and +# removes its containers, network and files when it ends. +set -euo pipefail +STAGE=$(cd "$1" && pwd) +NFPM=$(command -v "${NFPM:-nfpm}") +cd "$(dirname "$0")/../.." +ROOT=$(pwd) +base=$(basename "$STAGE" | sed -E 's/^LokalBot-([0-9]+\.[0-9]+\.[0-9]+)-linux-x86_64$/\1/') +if ! [[ "$base" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "$STAGE is not a LokalBot--linux-x86_64 folder" >&2 + exit 2 +fi +IFS=. read -r major minor patch <<<"$base" +V=("$base") +for step in 1 2 3; do V+=("$major.$minor.$((patch + step))"); done + +ID="lokalbot-repos-$$" +WORK=$(mktemp -d "${RUNNER_TEMP:-${TMPDIR:-/tmp}}/lokalbot-repos.XXXXXX") +chmod 755 "$WORK" +CLIENTS=(ubuntu:24.04 debian:13 fedora:latest) +cleanup() { + docker ps -aq --filter "label=$ID" | xargs -r docker rm -f > /dev/null + docker network rm "$ID" > /dev/null 2>&1 || true + rm -rf "$WORK" +} +trap cleanup EXIT + +step() { printf '\n=== %s\n' "$*"; } +fail() { echo "FAILED: $*" >&2; exit 1; } +name_of() { echo "$ID-${1%%:*}"; } +# Runs a command in the publisher as this user, so WORK stays removable. +pub() { + docker exec --user "$(id -u):$(id -g)" -e HOME=/work -e GNUPGHOME=/work/gnupg -w /src "$ID-pub" "$@" +} +in_client() { docker exec -e DEBIAN_FRONTEND=noninteractive "$(name_of "$1")" bash -c "$2"; } +# Expects a client command to fail or to skip LokalBot's index (apt keeps +# the last good one and warns), and shows why. +refused() { + local client=$1 what=$2 command=$3 + if in_client "$client" "/work/run 'signature|BADSIG|hash sum|checksum|not signed|size|verif' bash -c '$command'"; then + fail "$client accepted $what" + fi + echo "ok: $client refused $what" +} +installed() { + case $1 in + fedora:*) in_client "$1" "rpm -q --queryformat '%{VERSION}' lokalbot" ;; + *) in_client "$1" "dpkg-query --showformat '\${Version}' --show lokalbot | sed 's/-1\$//'" ;; + esac +} +expect_version() { + local got + got=$(installed "$1") + [ "$got" = "$2" ] || fail "$1 has lokalbot $got, expected $2" + echo "ok: $1 has lokalbot $2" +} +publish() { + local args=() v + for v in "$@"; do + args+=("/work/packages/LokalBot-$v-linux-amd64.deb" "/work/packages/LokalBot-$v-linux-x86_64.rpm") + done + pub scripts/linux-repos.sh /work/repo "$FPR" "${args[@]}" +} + +step "Publisher and HTTPS server for pkg.lokalbot.com (Ubuntu 24.04, like the release runner)" +docker network create --label "$ID" "$ID" > /dev/null +docker run -d --name "$ID-pub" --label "$ID" --network "$ID" --network-alias pkg.lokalbot.com \ + -v "$ROOT:/src" -v "$WORK:/work" -v "$STAGE:/stage:ro" -v "$NFPM:/usr/local/bin/nfpm:ro" \ + ubuntu:24.04 sleep infinity > /dev/null +docker exec "$ID-pub" bash -c 'apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \ + apt-utils createrepo-c rpm gnupg openssl python3 zstd > /dev/null' +pub mkdir -p /work/gnupg /work/tls /work/packages /work/repo +pub chmod 700 /work/gnupg +pub gpg --batch --quiet --passphrase '' --quick-generate-key 'LokalBot repository check (throwaway)' rsa3072 sign 1d +FPR=$(pub gpg --batch --with-colons --list-secret-keys | awk -F: '$1 == "fpr" {print $10; exit}') +pub bash -c "gpg --batch --armor --export $FPR > /work/key.asc" +echo "throwaway key $FPR" +pub bash -c 'cd /work/tls && + openssl req -x509 -newkey rsa:2048 -nodes -days 1 -subj "/CN=LokalBot check CA" -keyout ca.key -out ca.crt \ + -addext basicConstraints=critical,CA:TRUE -addext keyUsage=critical,keyCertSign 2> /dev/null && + openssl req -newkey rsa:2048 -nodes -subj /CN=pkg.lokalbot.com -keyout server.key -out server.csr 2> /dev/null && + printf "subjectAltName=DNS:pkg.lokalbot.com\nextendedKeyUsage=serverAuth\n" > ext && + openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 1 -extfile ext -out server.crt 2> /dev/null && + chmod 644 server.key' +# HTTP/1.1, and TLS connections end with close_notify: Debian 13's apt +# rejects a TLS stream that just stops ("unexpected eof"). +cat > "$WORK/serve.py" <<'EOF' +import functools, http.server, ssl + +class Handler(http.server.SimpleHTTPRequestHandler): + protocol_version = "HTTP/1.1" + +class Server(http.server.ThreadingHTTPServer): + def shutdown_request(self, request): + try: + request.unwrap() + except (OSError, ValueError): + pass + super().shutdown_request(request) + +server = Server(("0.0.0.0", 443), functools.partial(Handler, directory="/work/repo")) +context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) +context.load_cert_chain("/work/tls/server.crt", "/work/tls/server.key") +server.socket = context.wrap_socket(server.socket, server_side=True) +server.serve_forever() +EOF +docker exec -d "$ID-pub" bash -c 'python3 -u /work/serve.py 2> /work/server.log' +# In a client: runs a command, shows its lines that match a pattern, and +# fails when the command fails or apt reports an error or a skipped index. +cat > "$WORK/run" <<'EOF' +#!/bin/bash +pattern=$1 +shift +"$@" > /tmp/run.log 2>&1 +status=$? +errors='^(Err|Ign):.*lokalbot|^(E|W|Error|Warning):' +grep -iE "$pattern" /tmp/run.log | grep -vE "$errors" +if grep -E "$errors" /tmp/run.log; then exit 1; fi +exit $status +EOF +chmod 755 "$WORK/run" + +step "Packages ${V[*]} with the throwaway key" +for v in "${V[@]}"; do + pub env PACKAGE_VERSION="$v" STAGE=/stage LINUX_REPO_PUBLIC_KEY=/work/key.asc NFPM=nfpm \ + scripts/package-linux-distros.sh > /dev/null + pub bash -c "mv dist/LokalBot-$v-linux-* /work/packages/" +done +deb="/work/packages/LokalBot-$base-linux-amd64.deb" +rpm="/work/packages/LokalBot-$base-linux-x86_64.rpm" +pub dpkg-deb --info "$deb" conffiles | grep -qx /etc/apt/sources.list.d/lokalbot.sources || + fail "the .deb does not mark lokalbot.sources as a conffile" +pub dpkg-deb --contents "$deb" | grep -q ' ./usr/share/keyrings/lokalbot-archive-keyring.gpg$' || + fail "the .deb has no keyring" +pub rpm -qp --queryformat '[%{FILEFLAGS:fflags} %{FILENAMES}\n]' "$rpm" | grep -qx 'cn /etc/yum.repos.d/lokalbot.repo' || + fail "the .rpm does not mark lokalbot.repo as %config(noreplace)" +pub rpm -qpl "$rpm" | grep -qx /etc/pki/rpm-gpg/RPM-GPG-KEY-lokalbot || fail "the .rpm has no key" +if pub tar --zstd -tf "/work/packages/LokalBot-$base-linux-x86_64.pkg.tar.zst" | grep -q '^etc/'; then + fail "the Arch package adds files under /etc" +fi +echo "ok: the .deb and .rpm add the repository as config files; the Arch package does not" + +step "Publish $base" +publish "$base" + +for client in "${CLIENTS[@]}"; do + step "$client: install $base from the downloaded file" + docker run -d --name "$(name_of "$client")" --label "$ID" --network "$ID" -v "$WORK:/work:ro" \ + "$client" sleep infinity > /dev/null + case $client in + fedora:*) + in_client "$client" 'cp /work/tls/ca.crt /etc/pki/ca-trust/source/anchors/lokalbot-check.crt && update-ca-trust && + dnf install -y -q --setopt=install_weak_deps=False /work/packages/LokalBot-'"$base"'-linux-x86_64.rpm > /dev/null && + cat /etc/yum.repos.d/lokalbot.repo && + /work/run "lokalbot|key|metadata" dnf makecache -y --repo lokalbot --setopt=lokalbot.skip_if_unavailable=False' + ;; + *) + in_client "$client" 'apt-get update -qq && apt-get upgrade -y -qq > /dev/null && + apt-get install -y -qq --no-install-recommends ca-certificates > /dev/null && + cp /work/tls/ca.crt /usr/local/share/ca-certificates/lokalbot-check.crt && update-ca-certificates > /dev/null && + apt-get install -y -qq --no-install-recommends /work/packages/LokalBot-'"$base"'-linux-amd64.deb > /dev/null && + cat /etc/apt/sources.list.d/lokalbot.sources && /work/run lokalbot apt update && + apt-cache policy lokalbot && apt-cache policy lokalbot | grep -q "500 https://pkg.lokalbot.com/apt stable/main"' + ;; + esac + expect_version "$client" "$base" +done + +step "Edit the repository files, publish ${V[1]}, upgrade" +publish "${V[1]}" +for client in "${CLIENTS[@]}"; do + case $client in + fedora:*) + in_client "$client" 'echo "# edited by the check" >> /etc/yum.repos.d/lokalbot.repo && + /work/run "lokalbot|^upgrad|signature|key" dnf upgrade -y --refresh && + test "$(tail -1 /etc/yum.repos.d/lokalbot.repo)" = "# edited by the check" && ! test -e /etc/yum.repos.d/lokalbot.repo.rpmnew && + rpm -q --queryformat "signature: %{RSAHEADER:pgpsig}\n" lokalbot' + ;; + *) + in_client "$client" 'echo "# edited by the check" >> /etc/apt/sources.list.d/lokalbot.sources && + /work/run "lokalbot|upgrad" apt update && /work/run "lokalbot|upgraded" apt upgrade -y && + test "$(tail -1 /etc/apt/sources.list.d/lokalbot.sources)" = "# edited by the check" && + ! test -e /etc/apt/sources.list.d/lokalbot.sources.dpkg-dist' + ;; + esac + expect_version "$client" "${V[1]}" + echo "ok: $client kept the edited repository file" +done + +step "Tampered indexes" +dist=/work/repo/apt/dists/stable +pub cp "$dist/InRelease" "$dist/Release" /work/ +pub sed -i '0,/^ [0-9a-f]/s/^ \([0-9a-f]\)/ f\1/' "$dist/InRelease" +for client in ubuntu:24.04 debian:13; do + refused "$client" "a tampered InRelease" 'apt-get update' +done +pub rm "$dist/InRelease" +pub sed -i '0,/^ [0-9a-f]/s/^ \([0-9a-f]\)/ f\1/' "$dist/Release" +for client in ubuntu:24.04 debian:13; do + refused "$client" "a tampered Release" 'apt-get update' +done +pub cp /work/InRelease "$dist/InRelease" +pub cp /work/Release "$dist/Release" +repomd=/work/repo/rpm/repodata/repomd.xml +pub cp "$repomd" /work/repomd.xml +pub sed -i 's||1|' "$repomd" +refused fedora:latest "a tampered repomd.xml" \ + 'dnf makecache -y --refresh --repo lokalbot --setopt=lokalbot.skip_if_unavailable=False' +pub cp /work/repomd.xml "$repomd" +for client in "${CLIENTS[@]}"; do + case $client in + fedora:*) in_client "$client" '/work/run lokalbot dnf makecache -y --refresh --repo lokalbot --setopt=lokalbot.skip_if_unavailable=False' ;; + *) in_client "$client" '/work/run lokalbot apt-get update' ;; + esac +done +echo "ok: restored indexes verify again" + +step "Publish ${V[2]} and ${V[3]}: the oldest version leaves" +publish "${V[2]}" "${V[3]}" +pool=$(pub find /work/repo/apt/pool -name '*.deb' -printf '%f\n' | sort -V | tr '\n' ' ') +rpms=$(pub find /work/repo/rpm -maxdepth 1 -name '*.rpm' -printf '%f\n' | sort -V | tr '\n' ' ') +echo "apt pool: $pool" +echo "rpm: $rpms" +[ "$pool" = "lokalbot_${V[1]}-1_amd64.deb lokalbot_${V[2]}-1_amd64.deb lokalbot_${V[3]}-1_amd64.deb " ] || + fail "the apt repository should keep ${V[1]}, ${V[2]} and ${V[3]}" +[ "$rpms" = "lokalbot-${V[1]}-1.x86_64.rpm lokalbot-${V[2]}-1.x86_64.rpm lokalbot-${V[3]}-1.x86_64.rpm " ] || + fail "the rpm repository should keep ${V[1]}, ${V[2]} and ${V[3]}" +pub grep -c '^Package: lokalbot$' /work/repo/apt/dists/stable/main/binary-amd64/Packages | grep -qx 3 || + fail "Packages should list three versions" + +step "Swapped and unsigned packages" +newest_deb="/work/repo/apt/pool/main/l/lokalbot/lokalbot_${V[3]}-1_amd64.deb" +newest_rpm="/work/repo/rpm/lokalbot-${V[3]}-1.x86_64.rpm" +pub cp "$newest_deb" "$newest_rpm" /work/ +pub cp "/work/repo/apt/pool/main/l/lokalbot/lokalbot_${V[2]}-1_amd64.deb" "$newest_deb" +pub cp "/work/repo/rpm/lokalbot-${V[2]}-1.x86_64.rpm" "$newest_rpm" +for client in ubuntu:24.04 debian:13; do + refused "$client" "a swapped .deb" 'apt-get update -qq && apt-get upgrade -y' + expect_version "$client" "${V[1]}" +done +refused fedora:latest "a swapped .rpm" 'dnf upgrade -y --refresh lokalbot' +expect_version fedora:latest "${V[1]}" +# Signed indexes listing an unsigned package: gpgcheck still refuses it. +pub cp "/work/packages/LokalBot-${V[3]}-linux-x86_64.rpm" "$newest_rpm" +pub bash -c "rm -rf /work/repo/rpm/repodata && createrepo_c --quiet --no-database /work/repo/rpm && + gpg --batch --yes --local-user $FPR --armor --detach-sign --output $repomd.asc $repomd" +refused fedora:latest "an unsigned .rpm" 'dnf upgrade -y --refresh lokalbot' +expect_version fedora:latest "${V[1]}" +pub cp "/work/lokalbot_${V[3]}-1_amd64.deb" "$newest_deb" +pub cp "/work/lokalbot-${V[3]}-1.x86_64.rpm" "$newest_rpm" +pub scripts/linux-repos.sh /work/repo "$FPR" + +step "Upgrade to ${V[3]}" +for client in "${CLIENTS[@]}"; do + case $client in + fedora:*) in_client "$client" '/work/run "lokalbot|^upgrad" dnf upgrade -y --refresh' ;; + *) in_client "$client" '/work/run lokalbot apt update && /work/run "lokalbot|upgraded" apt upgrade -y' ;; + esac + expect_version "$client" "${V[3]}" +done + +step "Files the clients fetched" +pub grep -oE '"GET [^ ]+' /work/server.log | sed 's/"GET //' | sed -E 's/[0-9a-f]{64}//g' | sort | uniq -c +pub grep -q 'by-hash/SHA256/' /work/server.log || fail "apt did not fetch indexes by hash" +echo +echo "All repository checks passed." diff --git a/LinuxWindows/scripts/linux-repos.sh b/LinuxWindows/scripts/linux-repos.sh new file mode 100755 index 00000000..e7913682 --- /dev/null +++ b/LinuxWindows/scripts/linux-repos.sh @@ -0,0 +1,152 @@ +#!/usr/bin/env bash +# Adds Linux packages to LokalBot's apt and dnf repositories and signs them. +# .github/workflows/linux-repos.yml publishes the tree at https://pkg.lokalbot.com: +# apt/ suite stable, component main, amd64: pool/main/l/lokalbot/ +# and dists/stable/ with InRelease, Release and Release.gpg +# (apt-ftparchive, SHA-256 only, indexes also by hash) +# rpm/ the RPMs, each signed with rpmsign, and repodata/ +# (createrepo_c) with repomd.xml.asc +# lokalbot.asc the public key, and lokalbot.gpg, the same key unarmored +# +# scripts/linux-repos.sh REPO KEY [PACKAGE.deb|PACKAGE.rpm ...] +# +# REPO holds the published packages (copied from the bucket, or empty for a +# new repository) and receives the new tree. KEY is the signing key's +# fingerprint; GNUPGHOME has its secret key. A package whose version is +# already published is left out: a published version's bytes never change. +# Each repository then keeps its newest KEEP (default 3) versions, and every +# index is rebuilt, signed and checked against the public key. With no +# packages, the indexes are only rebuilt and signed again. +# +# PUBLIC_KEY: the armored public keyring to publish (default: KEY's public +# key). It must contain KEY. +# +# Needs apt-ftparchive (apt-utils), dpkg-deb, createrepo_c, rpm, rpmsign and +# gpg, as on Ubuntu 24.04. +set -euo pipefail +if [ $# -lt 2 ]; then + echo "usage: $0 REPO KEY [PACKAGE.deb|PACKAGE.rpm ...]" >&2 + exit 2 +fi +mkdir -p "$1" +REPO=$(cd "$1" && pwd) +KEY=$2 +shift 2 +KEEP=${KEEP:-3} +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT + +APT="$REPO/apt" +POOL="$APT/pool/main/l/lokalbot" +DIST="$APT/dists/stable" +RPMS="$REPO/rpm" +mkdir -p "$POOL" "$RPMS" + +if [ -n "${PUBLIC_KEY:-}" ]; then + cp "$PUBLIC_KEY" "$REPO/lokalbot.asc" +else + gpg --batch --armor --export-options export-minimal --export "$KEY" > "$REPO/lokalbot.asc" +fi +gpg --dearmor < "$REPO/lokalbot.asc" > "$REPO/lokalbot.gpg" +if ! gpg --batch --show-keys --with-colons "$REPO/lokalbot.asc" | + awk -F: '$1 == "fpr" {print $10}' | grep -qix "$KEY"; then + echo "The published key does not contain $KEY." >&2 + exit 1 +fi +RPMDB="$WORK/rpmdb" +mkdir -p "$RPMDB" +rpmkeys --dbpath "$RPMDB" --import "$REPO/lokalbot.asc" + +for package in "$@"; do + case "$package" in + *.deb) + name=$(dpkg-deb --field "$package" Package) + version=$(dpkg-deb --field "$package" Version) + arch=$(dpkg-deb --field "$package" Architecture) + target="$POOL/lokalbot_${version}_amd64.deb" + ;; + *.rpm) + read -r name version arch < <(rpm -qp --queryformat '%{NAME} %{VERSION}-%{RELEASE} %{ARCH}\n' "$package") + target="$RPMS/lokalbot-$version.x86_64.rpm" + ;; + *) + echo "$package is not a .deb or .rpm" >&2 + exit 1 + ;; + esac + if [ "$name" != lokalbot ] || { [ "$arch" != amd64 ] && [ "$arch" != x86_64 ]; }; then + echo "$package is $name for $arch, not lokalbot for x86_64" >&2 + exit 1 + fi + if [ -e "$target" ]; then + echo "lokalbot $version is already published; keeping $(basename "$target")" + continue + fi + cp "$package" "$WORK/new" + if [[ "$package" == *.rpm ]]; then + # Ubuntu's rpm looks for gpg as gpg2, which it no longer ships. + rpmsign --addsign --define "__gpg $(command -v gpg)" --define "_gpg_name $KEY" \ + --define "_gpg_sign_cmd_extra_args --batch --digest-algo sha256" "$WORK/new" > /dev/null + fi + mv "$WORK/new" "$target" + echo "added $(basename "$target")" +done + +# The newest KEEP versions stay. +prune() { + local dir=$1 prefix=$2 suffix=$3 version + find "$dir" -maxdepth 1 -name "$prefix*$suffix" -printf '%f\n' | + sed "s/^$prefix//; s/$suffix\$//" | sort -V | head -n -"$KEEP" | + while read -r version; do + rm "$dir/$prefix$version$suffix" + echo "removed $prefix$version$suffix" + done +} +prune "$POOL" lokalbot_ _amd64.deb +prune "$RPMS" lokalbot- .x86_64.rpm + +# apt: one index, listed in Release by SHA-256 and also stored under its hash +# (Acquire-By-Hash), so a cached or half-uploaded index never mismatches. +SHA256_ONLY=(-o APT::FTPArchive::MD5=false -o APT::FTPArchive::SHA1=false -o APT::FTPArchive::SHA512=false) +BIN="$DIST/main/binary-amd64" +rm -rf "$DIST" +mkdir -p "$BIN" +(cd "$APT" && apt-ftparchive "${SHA256_ONLY[@]}" packages pool) > "$BIN/Packages" +gzip -9nc "$BIN/Packages" > "$BIN/Packages.gz" +apt-ftparchive "${SHA256_ONLY[@]}" \ + -o APT::FTPArchive::Release::Origin=LokalBot \ + -o APT::FTPArchive::Release::Label=LokalBot \ + -o APT::FTPArchive::Release::Suite=stable \ + -o APT::FTPArchive::Release::Codename=stable \ + -o APT::FTPArchive::Release::Architectures=amd64 \ + -o APT::FTPArchive::Release::Components=main \ + -o "APT::FTPArchive::Release::Description=LokalBot (alpha)" \ + -o APT::FTPArchive::Release::Acquire-By-Hash=yes \ + release "$DIST" > "$WORK/Release" +mv "$WORK/Release" "$DIST/Release" +awk '/^SHA256:/ {list = 1; next} /^[^ ]/ {list = 0} list {print $1, $3}' "$DIST/Release" | + while read -r hash file; do + mkdir -p "$DIST/$(dirname "$file")/by-hash/SHA256" + cp "$DIST/$file" "$DIST/$(dirname "$file")/by-hash/SHA256/$hash" + done +sign() { gpg --batch --yes --local-user "$KEY" --digest-algo SHA512 "$@"; } +sign --clearsign --output "$DIST/InRelease" "$DIST/Release" +sign --armor --detach-sign --output "$DIST/Release.gpg" "$DIST/Release" + +# dnf: repodata names each file by its hash; repomd.xml is signed. +rm -rf "$RPMS/repodata" +createrepo_c --quiet --no-database --general-compress-type=gz "$RPMS" +sign --armor --detach-sign --output "$RPMS/repodata/repomd.xml.asc" "$RPMS/repodata/repomd.xml" + +# Everything verifies with the public key alone. +gpgv --quiet --keyring "$REPO/lokalbot.gpg" "$DIST/InRelease" +gpgv --quiet --keyring "$REPO/lokalbot.gpg" "$DIST/Release.gpg" "$DIST/Release" +gpgv --quiet --keyring "$REPO/lokalbot.gpg" "$RPMS/repodata/repomd.xml.asc" "$RPMS/repodata/repomd.xml" +for rpm in "$RPMS"/*.rpm; do + [ -e "$rpm" ] || continue + if ! rpmkeys --dbpath "$RPMDB" --checksig "$rpm" | grep -q ' signatures OK$'; then + echo "$(basename "$rpm") is not signed by $KEY" >&2 + exit 1 + fi +done +echo "apt: $(grep -c '^Package:' "$BIN/Packages") versions; rpm: $(find "$RPMS" -maxdepth 1 -name '*.rpm' | wc -l) versions" diff --git a/LinuxWindows/scripts/package-linux-distros.sh b/LinuxWindows/scripts/package-linux-distros.sh index 864934b2..cd0c5775 100755 --- a/LinuxWindows/scripts/package-linux-distros.sh +++ b/LinuxWindows/scripts/package-linux-distros.sh @@ -7,15 +7,21 @@ # each with a .sha256 file. Run scripts/package.sh first. # # NFPM: the nfpm binary to use (default: nfpm on PATH). +# PACKAGE_VERSION, STAGE, LINUX_REPO_PUBLIC_KEY: another version, staged +# folder or repository key, for scripts/ci/check-linux-repos.sh. # # The packages install lokalbot and lokalbot-cli into /usr/bin, the desktop # entry and icons into /usr/share. LokalBot does not replace files it cannot # write, so a package install is told about updates and offered the download. +# Once the repository key is in packaging/linux, the .deb and .rpm also add +# the apt and dnf repositories at pkg.lokalbot.com (scripts/linux-repos.sh) +# with that key, so later versions come with `apt upgrade` and `dnf upgrade`. set -euo pipefail cd "$(dirname "$0")/.." -VERSION=$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1) +VERSION=${PACKAGE_VERSION:-$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)} ROOT="$(pwd)" -STAGE="$ROOT/dist/LokalBot-$VERSION-linux-x86_64" +STAGE="${STAGE:-$ROOT/dist/LokalBot-$VERSION-linux-x86_64}" +REPO_KEY="${LINUX_REPO_PUBLIC_KEY:-$ROOT/packaging/linux/lokalbot-archive-keyring.asc}" NFPM="${NFPM:-nfpm}" APP_ID=com.lokalbot.LokalBot if [ ! -x "$STAGE/lokalbot" ] || [ ! -x "$STAGE/lokalbot-cli" ]; then @@ -24,6 +30,11 @@ if [ ! -x "$STAGE/lokalbot" ] || [ ! -x "$STAGE/lokalbot-cli" ]; then fi CONFIG="$ROOT/dist/nfpm.yaml" +KEYRING="$ROOT/dist/lokalbot-archive-keyring.gpg" +mkdir -p "$ROOT/dist" +if [ -f "$REPO_KEY" ]; then + gpg --dearmor < "$REPO_KEY" > "$KEYRING" +fi { cat < "$OUT.sha256" echo "dist/$OUT" done -rm -f "$CONFIG" +rm -f "$CONFIG" "$KEYRING" diff --git a/README.md b/README.md index 81388a2e..5779fdcf 100644 --- a/README.md +++ b/README.md @@ -162,7 +162,7 @@ LokalBot runs on Apple Silicon Macs with macOS 15 or later. Windows and Linux ar | | macOS | Windows (alpha) | Linux (alpha) | | --- | --- | --- | --- | | Runs on | Apple Silicon (M1 or later), macOS 15+ | Windows 10 or 11, x64 | x86_64 with glibc 2.39 or later (Ubuntu 24.04, Debian 13, Fedora 40 or newer), PulseAudio or PipeWire, and a Vulkan or OpenGL driver | -| Install | [LokalBot.dmg](https://github.com/stevyhacker/lokalbot/releases/latest/download/LokalBot.dmg): drag LokalBot to Applications. Or `brew install --cask stevyhacker/tap/lokalbot`. | [`LokalBot--windows-x86_64-setup.exe`](https://github.com/stevyhacker/lokalbot/releases/latest): run it to install LokalBot for your user, with a Start menu shortcut and no administrator rights. It isn't code-signed yet, so SmartScreen asks first: **More info → Run anyway**. The same release has a portable zip. | [`LokalBot--linux-x86_64.tar.gz`](https://github.com/stevyhacker/lokalbot/releases/latest): extract it and run `./install.sh`, which adds LokalBot to your applications menu. Or install the `.deb` (Debian, Ubuntu), `.rpm` (Fedora, openSUSE) or Arch package from the same release; those tell you about updates instead of installing them. | +| Install | [LokalBot.dmg](https://github.com/stevyhacker/lokalbot/releases/latest/download/LokalBot.dmg): drag LokalBot to Applications. Or `brew install --cask stevyhacker/tap/lokalbot`. | [`LokalBot--windows-x86_64-setup.exe`](https://github.com/stevyhacker/lokalbot/releases/latest): run it to install LokalBot for your user, with a Start menu shortcut and no administrator rights. It isn't code-signed yet, so SmartScreen asks first: **More info → Run anyway**. The same release has a portable zip. | [`LokalBot--linux-x86_64.tar.gz`](https://github.com/stevyhacker/lokalbot/releases/latest): extract it and run `./install.sh`, which adds LokalBot to your applications menu. Or install the `.deb` (Debian, Ubuntu), `.rpm` (Fedora, openSUSE) or Arch package from the same release. The `.deb`, and the `.rpm` on Fedora, add the LokalBot repository, so `sudo apt upgrade` or `sudo dnf upgrade` brings later versions; with the Arch package, LokalBot tells you about updates instead of installing them. | | Your library | `~/Library/Application Support/me.dotenv.LokalBot` | `%LOCALAPPDATA%\LokalBot` | `~/.local/share/LokalBot` | A library recorded on one platform opens on the others. Not every Mac feature is on Windows and Linux yet; see [what works where](LinuxWindows/README.md#what-works-where). diff --git a/README.zh-CN.md b/README.zh-CN.md index 6c7ba22f..948b3482 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -166,7 +166,7 @@ LokalBot 支持 macOS 15 或更新版本的 Apple Silicon Mac;Windows 和 Linu | | macOS | Windows(alpha) | Linux(alpha) | | --- | --- | --- | --- | | 系统要求 | Apple Silicon(M1 或更新),macOS 15 及以上 | Windows 10 或 11,x64 | x86_64,glibc 2.39 或更新(Ubuntu 24.04、Debian 13、Fedora 40 或更新),PulseAudio 或 PipeWire,以及 Vulkan 或 OpenGL 驱动 | -| 安装 | [LokalBot.dmg](https://github.com/stevyhacker/lokalbot/releases/latest/download/LokalBot.dmg):把 LokalBot 拖进“应用程序”文件夹;或者运行 `brew install --cask stevyhacker/tap/lokalbot`。 | [`LokalBot--windows-x86_64-setup.exe`](https://github.com/stevyhacker/lokalbot/releases/latest):运行它即可为当前用户安装 LokalBot,会添加开始菜单快捷方式,不需要管理员权限。它还没有代码签名,SmartScreen 会先询问:选择**更多信息 → 仍要运行**。同一版本中也有免安装的 zip。 | [`LokalBot--linux-x86_64.tar.gz`](https://github.com/stevyhacker/lokalbot/releases/latest):解压后运行 `./install.sh`,它会把 LokalBot 添加到应用程序菜单。也可以安装同一版本中的 `.deb`(Debian、Ubuntu)、`.rpm`(Fedora、openSUSE)或 Arch 软件包;用软件包安装时,有新版本会提示你,但不会自动安装。 | +| 安装 | [LokalBot.dmg](https://github.com/stevyhacker/lokalbot/releases/latest/download/LokalBot.dmg):把 LokalBot 拖进“应用程序”文件夹;或者运行 `brew install --cask stevyhacker/tap/lokalbot`。 | [`LokalBot--windows-x86_64-setup.exe`](https://github.com/stevyhacker/lokalbot/releases/latest):运行它即可为当前用户安装 LokalBot,会添加开始菜单快捷方式,不需要管理员权限。它还没有代码签名,SmartScreen 会先询问:选择**更多信息 → 仍要运行**。同一版本中也有免安装的 zip。 | [`LokalBot--linux-x86_64.tar.gz`](https://github.com/stevyhacker/lokalbot/releases/latest):解压后运行 `./install.sh`,它会把 LokalBot 添加到应用程序菜单。也可以安装同一版本中的 `.deb`(Debian、Ubuntu)、`.rpm`(Fedora、openSUSE)或 Arch 软件包。`.deb` 和 Fedora 上的 `.rpm` 会添加 LokalBot 软件源,之后运行 `sudo apt upgrade` 或 `sudo dnf upgrade` 即可获得新版本;用 Arch 软件包安装时,有新版本会提示你,但不会自动安装。 | | 资料库位置 | `~/Library/Application Support/me.dotenv.LokalBot` | `%LOCALAPPDATA%\LokalBot` | `~/.local/share/LokalBot` | 在一个平台上录制的资料库可以在其他平台上打开。并非所有 Mac 功能都已移植到 Windows 和 Linux,详见[各平台功能对照(英文)](LinuxWindows/README.md#what-works-where)。 diff --git a/RELEASING.md b/RELEASING.md index 84d87a02..254c6c7e 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -14,6 +14,7 @@ tooling lives in `Scripts/`: | `Scripts/clean_local.sh` | wipe local build/derived/DMG artifacts | | `Scripts/bump-version.py` | set the version in every file one tag ships it from | | `LinuxWindows/scripts/release-feeds.sh` | sign the Linux/Windows archives and write their update feeds | +| `LinuxWindows/scripts/linux-repos.sh` | add the `.deb` and `.rpm` to the signed apt and dnf repositories ([Linux package repositories](#linux-package-repositories)) | --- @@ -547,6 +548,155 @@ account's normal password. CI passes `--repo "$GITHUB_REPOSITORY"`, so the `OWNER/REPO` placeholder is resolved automatically there. +## Linux package repositories + +The release's `.deb` and `.rpm` add LokalBot's own apt and dnf repositories, +the way Chrome's and VS Code's packages do, so installing the downloaded file +once means `sudo apt upgrade` or `sudo dnf upgrade` brings every later stable +version. The `.deb` installs `/etc/apt/sources.list.d/lokalbot.sources` and +`/usr/share/keyrings/lokalbot-archive-keyring.gpg`; the `.rpm` installs +`/etc/yum.repos.d/lokalbot.repo` (`gpgcheck=1`, `repo_gpgcheck=1`) and +`/etc/pki/rpm-gpg/RPM-GPG-KEY-lokalbot`. The repository files are config +files, so a user's edits (`Enabled: no`, `enabled=0`) survive upgrades. +`scripts/package-linux-distros.sh` adds these files only once +`LinuxWindows/packaging/linux/lokalbot-archive-keyring.asc` exists. The Arch +package has no repository. + +| URL | What it is | +| --- | --- | +| `https://pkg.lokalbot.com/apt` | apt repository: suite `stable`, component `main`, `amd64`; signed `InRelease` and `Release.gpg`, indexes also by hash | +| `https://pkg.lokalbot.com/rpm` | dnf repository: signed RPMs, `repodata/repomd.xml.asc` | +| `https://pkg.lokalbot.com/lokalbot.gpg`, `lokalbot.asc` | the public key, unarmored and armored | + +`release.yml`'s `linux-repos` job calls `.github/workflows/linux-repos.yml` +for stable tags after the packages are attached; pre-release tags never reach +the repositories. On a GitHub-hosted runner (never a self-hosted one: this key +decides what every subscribed computer installs as root) it downloads the +release's `.deb` and `.rpm` and checks their `.sha256`, copies the published +packages from the R2 bucket `lokalbot-packages`, and runs +`LinuxWindows/scripts/linux-repos.sh`. That adds the release unless its +version is already published (a published version's bytes never change), +keeps the newest 3 versions, signs new RPMs with `rpmsign`, rebuilds and signs +the indexes, and checks every signature against the public key. The job then +uploads packages and hash-named indexes first, then the signed indexes (sent +with `Cache-Control: no-cache`), deletes the pruned packages, and compares +what `pkg.lokalbot.com` serves with what it uploaded. The workflow owns the +bucket: anything else in it is deleted, except older hash-named indexes, kept +so a client holding the previous index still finds its files. Without the +secrets the job skips with a notice; once the key is committed, a missing +secret fails it, because that release's packages point at the repositories. + +To add a release that is already out (or to retry), run +`gh workflow run linux-repos.yml -f tag=vX.Y.Z`. Pull requests that change the +repositories or the packages run `scripts/ci/check-linux-repos.sh`: it +packages the latest release's binaries as four versions with a throwaway key, +serves the repositories as `pkg.lokalbot.com` on a private Docker network, and +checks that clean Ubuntu 24.04, Debian 13 and Fedora containers install the +first from the downloaded file, upgrade with `apt upgrade` and `dnf upgrade`, +keep an edited repository file, and refuse a tampered `InRelease`, `Release` +or `repomd.xml`, a swapped `.deb` or `.rpm`, and an unsigned `.rpm`. Run it on +any Linux machine with Docker: +`NFPM=/path/to/nfpm LinuxWindows/scripts/ci/check-linux-repos.sh LokalBot-X.Y.Z-linux-x86_64`. + +### One-time setup + +1. Generate the key on a trusted computer: RSA 4096 (every apt and rpm + version verifies it; Ed25519 needs a recent rpm), signing only, expiring in + 3 years. CI has no passphrase prompt, so the key has none; the backup is + what protects it. + + ```sh + export GNUPGHOME=$(mktemp -d) + gpg --batch --passphrase '' --quick-generate-key \ + 'LokalBot package signing ' rsa4096 sign 3y + FPR=$(gpg --with-colons --list-secret-keys | awk -F: '$1 == "fpr" {print $10; exit}') + gpg --armor --export-options export-minimal --export "$FPR" \ + > LinuxWindows/packaging/linux/lokalbot-archive-keyring.asc + gpg --armor --export-secret-keys "$FPR" | gh secret set LINUX_REPO_GPG_PRIVATE_KEY --repo stevyhacker/lokalbot + ``` + + Keep `gpg --armor --export-secret-keys "$FPR"` and the revocation + certificate `$GNUPGHOME/openpgp-revocs.d/$FPR.rev` in your password manager, + not on disk, then `rm -rf "$GNUPGHOME"`. Don't commit the public key yet. +2. In Cloudflare: create the R2 bucket `lokalbot-packages`; under its + Settings → Custom Domains, connect `pkg.lokalbot.com` (this adds the DNS + record and serves the bucket publicly there; leave the `r2.dev` URL off); + then create an R2 API token with **Object Read & Write** on that bucket + only. Each object carries its own `Cache-Control`, so no cache rule is + needed. +3. Add the token and account ID: + + ```sh + gh secret set LINUX_REPO_R2_ACCOUNT_ID --repo stevyhacker/lokalbot + gh secret set LINUX_REPO_R2_ACCESS_KEY_ID --repo stevyhacker/lokalbot + gh secret set LINUX_REPO_R2_SECRET_ACCESS_KEY --repo stevyhacker/lokalbot + ``` + +4. Publish the current release and check it is served: + `gh workflow run linux-repos.yml -f tag=v0.11.0`, then + `curl -fsS https://pkg.lokalbot.com/apt/dists/stable/InRelease`. Its + packages don't add the repositories, so nobody uses them yet. +5. Commit `lokalbot-archive-keyring.asc`. The next release's `.deb` and + `.rpm` add the repositories, and every release after it reaches them with + `apt upgrade` and `dnf upgrade`. + +### Key expiry and rotation + +The job warns when the key expires within 180 days. After it expires, apt and +dnf refuse the repositories until the computer has the extended key, and the +only way to deliver it is a package update signed before the expiry. So extend +the key well ahead, with the backup imported into a temporary `GNUPGHOME`: + +```sh +gpg --quick-set-expire "$FPR" 3y +gpg --armor --export-options export-minimal --export "$FPR" > LinuxWindows/packaging/linux/lokalbot-archive-keyring.asc +gpg --armor --export-secret-keys "$FPR" | gh secret set LINUX_REPO_GPG_PRIVATE_KEY --repo stevyhacker/lokalbot +``` + +Commit the key, store the new backup and ship a release; the published +`lokalbot.gpg` updates with it. apt reads the keyring file, so the installed +release is enough. dnf keeps the key it imported in the rpm database +(`gpg-pubkey`); `sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-lokalbot` +updates its expiry there (checked with rpm 6.0 on Fedora 44), which the +release notes should mention. A computer that missed +every update until the old expiry gets the key back with +`curl -fsSL https://pkg.lokalbot.com/lokalbot.gpg | sudo tee /usr/share/keyrings/lokalbot-archive-keyring.gpg > /dev/null` +(apt) or by installing the latest `.rpm` from the release page (dnf). + +To move to a new key, generate it, commit both keys in one armored file +(`gpg --armor --export-options export-minimal --export OLD NEW`) and ship a +release, still signed with the old key. Once most installs have that release, +switch `LINUX_REPO_GPG_PRIVATE_KEY` to the new key; the next release's indexes +are signed with it. Drop the old key from the file a few releases later. + +### If the key leaks + +The key alone can't reach users: an attacker also needs to serve files as +`pkg.lokalbot.com`. Still, act at once: + +1. Delete `LINUX_REPO_GPG_PRIVATE_KEY`, and revoke and replace the R2 token in + Cloudflare. +2. Generate a new key. Import the old key's revocation certificate + (`gpg --import OLD.rev`) and commit the new key and the revoked old key in + one armored file. +3. Ship a release whose repositories are still signed with the old key (set + it as the secret for this one run): installs trust only that key until + they have the new keyring. Then set the new key as the secret and run + `gh workflow run linux-repos.yml -f tag=`, which signs the + same repositories again with it. +4. In the release notes, tell users who hadn't updated yet to install the + latest package from the release page, and dnf users to remove the old key + with `sudo rpm -e gpg-pubkey-` (dnf keeps keys it imported). + +### Taking a version back + +apt and dnf never downgrade, so the fix for a bad release is a newer one. To +stop more computers from upgrading in the meantime, delete the version's +`apt/pool/main/l/lokalbot/lokalbot_X.Y.Z-1_amd64.deb` and +`rpm/lokalbot-X.Y.Z-1.x86_64.rpm` from the bucket in the Cloudflare dashboard, +then run `gh workflow run linux-repos.yml -f tag=vPREVIOUS` with the previous +good tag: it rebuilds the indexes from what is left. + --- ## Sanity checks diff --git a/Scripts/guide_pages.py b/Scripts/guide_pages.py index c2bdc5c7..f4d48748 100644 --- a/Scripts/guide_pages.py +++ b/Scripts/guide_pages.py @@ -332,7 +332,7 @@
  • System: x86_64 with glibc 2.39 or later, such as Ubuntu 24.04, Debian 13, or Fedora 40 or newer.
  • Graphics: a Vulkan or OpenGL driver. Mesa is fine.
  • Audio: PulseAudio or PipeWire. LokalBot records the output monitor, so it captures everything the computer plays, not just the meeting app.
  • -
  • Install: download the Linux archive from the latest release, extract it, and run ./install.sh. It installs LokalBot to ~/.local/bin and adds it to your applications menu. The same release also has a .deb (Debian, Ubuntu), an .rpm (Fedora, openSUSE) and an Arch package; installed that way, LokalBot tells you about updates instead of installing them.
  • +
  • Install: download the Linux archive from the latest release, extract it, and run ./install.sh. It installs LokalBot to ~/.local/bin and adds it to your applications menu. The same release also has a .deb (Debian, Ubuntu), an .rpm (Fedora, openSUSE) and an Arch package. Installing the .deb, or the .rpm on Fedora, also adds the LokalBot repository, so sudo apt upgrade or sudo dnf upgrade brings later versions; with the Arch package, LokalBot tells you about updates instead of installing them.
  • Disk: about 4 GB for the default models, plus your recordings. The built-in llama.cpp runs on a Vulkan GPU or on the CPU.
  • GNOME: the tray icon needs the AppIndicator extension, and day memory needs the Window Calls extension.
  • Library: ~/.local/share/LokalBot.
  • diff --git a/web/system-requirements.html b/web/system-requirements.html index c5616f83..3735d4db 100644 --- a/web/system-requirements.html +++ b/web/system-requirements.html @@ -96,7 +96,7 @@

    Linux (alpha)

  • System: x86_64 with glibc 2.39 or later, such as Ubuntu 24.04, Debian 13, or Fedora 40 or newer.
  • Graphics: a Vulkan or OpenGL driver. Mesa is fine.
  • Audio: PulseAudio or PipeWire. LokalBot records the output monitor, so it captures everything the computer plays, not just the meeting app.
  • -
  • Install: download the Linux archive from the latest release, extract it, and run ./install.sh. It installs LokalBot to ~/.local/bin and adds it to your applications menu. The same release also has a .deb (Debian, Ubuntu), an .rpm (Fedora, openSUSE) and an Arch package; installed that way, LokalBot tells you about updates instead of installing them.
  • +
  • Install: download the Linux archive from the latest release, extract it, and run ./install.sh. It installs LokalBot to ~/.local/bin and adds it to your applications menu. The same release also has a .deb (Debian, Ubuntu), an .rpm (Fedora, openSUSE) and an Arch package. Installing the .deb, or the .rpm on Fedora, also adds the LokalBot repository, so sudo apt upgrade or sudo dnf upgrade brings later versions; with the Arch package, LokalBot tells you about updates instead of installing them.
  • Disk: about 4 GB for the default models, plus your recordings. The built-in llama.cpp runs on a Vulkan GPU or on the CPU.
  • GNOME: the tray icon needs the AppIndicator extension, and day memory needs the Window Calls extension.
  • Library: ~/.local/share/LokalBot.
  • From dba56b90a9d563760466bf3712738927ec1efc7e Mon Sep 17 00:00:00 2001 From: stevyhacker Date: Tue, 20 Oct 2026 15:15:59 +0200 Subject: [PATCH 2/5] Keep the user-facing repository docs for the PR that turns the repositories on The README and website copy says the .deb and .rpm add the LokalBot repository, which is only true once its key is committed. It moves to its own pull request, merged with that key, so master never promises updates the current packages can't give. --- LinuxWindows/README.md | 14 ++------------ README.md | 2 +- README.zh-CN.md | 2 +- Scripts/guide_pages.py | 2 +- web/system-requirements.html | 2 +- 5 files changed, 6 insertions(+), 16 deletions(-) diff --git a/LinuxWindows/README.md b/LinuxWindows/README.md index 57caf657..c084e8fa 100644 --- a/LinuxWindows/README.md +++ b/LinuxWindows/README.md @@ -74,10 +74,8 @@ release has none yet, use a CI run's artifacts (`lokalbot-linux-x86_64`, `lokalb The same release has packages that install into `/usr/bin` with their dependencies: `LokalBot--linux-amd64.deb` (Debian, Ubuntu: `sudo apt install ./LokalBot-…deb`), `LokalBot--linux-x86_64.rpm` (Fedora, openSUSE: `sudo dnf install ./LokalBot-…rpm`) and - `LokalBot--linux-x86_64.pkg.tar.zst` (Arch: `sudo pacman -U ./LokalBot-…zst`). Installing the - `.deb`, or the `.rpm` on Fedora, also adds the LokalBot repository at `pkg.lokalbot.com`, so - `sudo apt upgrade` or `sudo dnf upgrade` brings later versions. With the Arch package, LokalBot tells you about updates and - offers the download instead of replacing its files. + `LokalBot--linux-x86_64.pkg.tar.zst` (Arch: `sudo pacman -U ./LokalBot-…zst`). A package + install tells you about updates and offers the download instead of replacing its files. - **Windows** 10/11 (x64): run `LokalBot--windows-x86_64-setup.exe`. It installs for your user into `%LOCALAPPDATA%\Programs\LokalBot` without administrator rights, adds a Start menu shortcut (which also lets notifications show LokalBot's name) and an Installed apps entry; uninstalling keeps your library. Or @@ -122,14 +120,6 @@ its old file is removed on a later start. Your library is never touched. When Lo files (a package manager's copy, `/usr`, `/opt`, Program Files, a folder you cannot write), it offers the download from GitHub instead. Debug builds never update. -The `.deb` and `.rpm` update with your package manager, from the LokalBot repository they add: -`/etc/apt/sources.list.d/lokalbot.sources` with its key in `/usr/share/keyrings`, or -`/etc/yum.repos.d/lokalbot.repo` with its key in `/etc/pki/rpm-gpg`. apt and dnf check the repository's -signature and each package's. Your edits to those files survive upgrades; add `Enabled: no` or set -`enabled=0` to stop updates from it. The repository has the newest three stable versions, never pre-releases. -openSUSE's zypper doesn't read `/etc/yum.repos.d`, so there the `.rpm` is told about updates like the Arch -package. - On first launch LokalBot asks what to remember and downloads the models you choose: about 4 GB for the default transcription, Think and search models. diff --git a/README.md b/README.md index 5779fdcf..81388a2e 100644 --- a/README.md +++ b/README.md @@ -162,7 +162,7 @@ LokalBot runs on Apple Silicon Macs with macOS 15 or later. Windows and Linux ar | | macOS | Windows (alpha) | Linux (alpha) | | --- | --- | --- | --- | | Runs on | Apple Silicon (M1 or later), macOS 15+ | Windows 10 or 11, x64 | x86_64 with glibc 2.39 or later (Ubuntu 24.04, Debian 13, Fedora 40 or newer), PulseAudio or PipeWire, and a Vulkan or OpenGL driver | -| Install | [LokalBot.dmg](https://github.com/stevyhacker/lokalbot/releases/latest/download/LokalBot.dmg): drag LokalBot to Applications. Or `brew install --cask stevyhacker/tap/lokalbot`. | [`LokalBot--windows-x86_64-setup.exe`](https://github.com/stevyhacker/lokalbot/releases/latest): run it to install LokalBot for your user, with a Start menu shortcut and no administrator rights. It isn't code-signed yet, so SmartScreen asks first: **More info → Run anyway**. The same release has a portable zip. | [`LokalBot--linux-x86_64.tar.gz`](https://github.com/stevyhacker/lokalbot/releases/latest): extract it and run `./install.sh`, which adds LokalBot to your applications menu. Or install the `.deb` (Debian, Ubuntu), `.rpm` (Fedora, openSUSE) or Arch package from the same release. The `.deb`, and the `.rpm` on Fedora, add the LokalBot repository, so `sudo apt upgrade` or `sudo dnf upgrade` brings later versions; with the Arch package, LokalBot tells you about updates instead of installing them. | +| Install | [LokalBot.dmg](https://github.com/stevyhacker/lokalbot/releases/latest/download/LokalBot.dmg): drag LokalBot to Applications. Or `brew install --cask stevyhacker/tap/lokalbot`. | [`LokalBot--windows-x86_64-setup.exe`](https://github.com/stevyhacker/lokalbot/releases/latest): run it to install LokalBot for your user, with a Start menu shortcut and no administrator rights. It isn't code-signed yet, so SmartScreen asks first: **More info → Run anyway**. The same release has a portable zip. | [`LokalBot--linux-x86_64.tar.gz`](https://github.com/stevyhacker/lokalbot/releases/latest): extract it and run `./install.sh`, which adds LokalBot to your applications menu. Or install the `.deb` (Debian, Ubuntu), `.rpm` (Fedora, openSUSE) or Arch package from the same release; those tell you about updates instead of installing them. | | Your library | `~/Library/Application Support/me.dotenv.LokalBot` | `%LOCALAPPDATA%\LokalBot` | `~/.local/share/LokalBot` | A library recorded on one platform opens on the others. Not every Mac feature is on Windows and Linux yet; see [what works where](LinuxWindows/README.md#what-works-where). diff --git a/README.zh-CN.md b/README.zh-CN.md index 2601d8ec..2e4eca5c 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -168,7 +168,7 @@ LokalBot 支持 macOS 15 或更新版本的 Apple Silicon Mac;Windows 和 Linu | | macOS | Windows(alpha) | Linux(alpha) | | --- | --- | --- | --- | | 系统要求 | Apple Silicon(M1 或更新),macOS 15 及以上 | Windows 10 或 11,x64 | x86_64,glibc 2.39 或更新(Ubuntu 24.04、Debian 13、Fedora 40 或更新),PulseAudio 或 PipeWire,以及 Vulkan 或 OpenGL 驱动 | -| 安装 | [LokalBot.dmg](https://github.com/stevyhacker/lokalbot/releases/latest/download/LokalBot.dmg):把 LokalBot 拖进“应用程序”文件夹;或者运行 `brew install --cask stevyhacker/tap/lokalbot`。 | [`LokalBot--windows-x86_64-setup.exe`](https://github.com/stevyhacker/lokalbot/releases/latest):运行它即可为当前用户安装 LokalBot,会添加开始菜单快捷方式,不需要管理员权限。它还没有代码签名,SmartScreen 会先询问:选择**更多信息 → 仍要运行**。同一版本中也有免安装的 zip。 | [`LokalBot--linux-x86_64.tar.gz`](https://github.com/stevyhacker/lokalbot/releases/latest):解压后运行 `./install.sh`,它会把 LokalBot 添加到应用程序菜单。也可以安装同一版本中的 `.deb`(Debian、Ubuntu)、`.rpm`(Fedora、openSUSE)或 Arch 软件包。`.deb` 和 Fedora 上的 `.rpm` 会添加 LokalBot 软件源,之后运行 `sudo apt upgrade` 或 `sudo dnf upgrade` 即可获得新版本;用 Arch 软件包安装时,有新版本会提示你,但不会自动安装。 | +| 安装 | [LokalBot.dmg](https://github.com/stevyhacker/lokalbot/releases/latest/download/LokalBot.dmg):把 LokalBot 拖进“应用程序”文件夹;或者运行 `brew install --cask stevyhacker/tap/lokalbot`。 | [`LokalBot--windows-x86_64-setup.exe`](https://github.com/stevyhacker/lokalbot/releases/latest):运行它即可为当前用户安装 LokalBot,会添加开始菜单快捷方式,不需要管理员权限。它还没有代码签名,SmartScreen 会先询问:选择**更多信息 → 仍要运行**。同一版本中也有免安装的 zip。 | [`LokalBot--linux-x86_64.tar.gz`](https://github.com/stevyhacker/lokalbot/releases/latest):解压后运行 `./install.sh`,它会把 LokalBot 添加到应用程序菜单。也可以安装同一版本中的 `.deb`(Debian、Ubuntu)、`.rpm`(Fedora、openSUSE)或 Arch 软件包;用软件包安装时,有新版本会提示你,但不会自动安装。 | | 资料库位置 | `~/Library/Application Support/me.dotenv.LokalBot` | `%LOCALAPPDATA%\LokalBot` | `~/.local/share/LokalBot` | 在一个平台上录制的资料库可以在其他平台上打开。并非所有 Mac 功能都已移植到 Windows 和 Linux,详见[各平台功能对照(英文)](LinuxWindows/README.md#what-works-where)。 diff --git a/Scripts/guide_pages.py b/Scripts/guide_pages.py index f4d48748..c2bdc5c7 100644 --- a/Scripts/guide_pages.py +++ b/Scripts/guide_pages.py @@ -332,7 +332,7 @@
  • System: x86_64 with glibc 2.39 or later, such as Ubuntu 24.04, Debian 13, or Fedora 40 or newer.
  • Graphics: a Vulkan or OpenGL driver. Mesa is fine.
  • Audio: PulseAudio or PipeWire. LokalBot records the output monitor, so it captures everything the computer plays, not just the meeting app.
  • -
  • Install: download the Linux archive from the latest release, extract it, and run ./install.sh. It installs LokalBot to ~/.local/bin and adds it to your applications menu. The same release also has a .deb (Debian, Ubuntu), an .rpm (Fedora, openSUSE) and an Arch package. Installing the .deb, or the .rpm on Fedora, also adds the LokalBot repository, so sudo apt upgrade or sudo dnf upgrade brings later versions; with the Arch package, LokalBot tells you about updates instead of installing them.
  • +
  • Install: download the Linux archive from the latest release, extract it, and run ./install.sh. It installs LokalBot to ~/.local/bin and adds it to your applications menu. The same release also has a .deb (Debian, Ubuntu), an .rpm (Fedora, openSUSE) and an Arch package; installed that way, LokalBot tells you about updates instead of installing them.
  • Disk: about 4 GB for the default models, plus your recordings. The built-in llama.cpp runs on a Vulkan GPU or on the CPU.
  • GNOME: the tray icon needs the AppIndicator extension, and day memory needs the Window Calls extension.
  • Library: ~/.local/share/LokalBot.
  • diff --git a/web/system-requirements.html b/web/system-requirements.html index 3735d4db..c5616f83 100644 --- a/web/system-requirements.html +++ b/web/system-requirements.html @@ -96,7 +96,7 @@

    Linux (alpha)

  • System: x86_64 with glibc 2.39 or later, such as Ubuntu 24.04, Debian 13, or Fedora 40 or newer.
  • Graphics: a Vulkan or OpenGL driver. Mesa is fine.
  • Audio: PulseAudio or PipeWire. LokalBot records the output monitor, so it captures everything the computer plays, not just the meeting app.
  • -
  • Install: download the Linux archive from the latest release, extract it, and run ./install.sh. It installs LokalBot to ~/.local/bin and adds it to your applications menu. The same release also has a .deb (Debian, Ubuntu), an .rpm (Fedora, openSUSE) and an Arch package. Installing the .deb, or the .rpm on Fedora, also adds the LokalBot repository, so sudo apt upgrade or sudo dnf upgrade brings later versions; with the Arch package, LokalBot tells you about updates instead of installing them.
  • +
  • Install: download the Linux archive from the latest release, extract it, and run ./install.sh. It installs LokalBot to ~/.local/bin and adds it to your applications menu. The same release also has a .deb (Debian, Ubuntu), an .rpm (Fedora, openSUSE) and an Arch package; installed that way, LokalBot tells you about updates instead of installing them.
  • Disk: about 4 GB for the default models, plus your recordings. The built-in llama.cpp runs on a Vulkan GPU or on the CPU.
  • GNOME: the tray icon needs the AppIndicator extension, and day memory needs the Window Calls extension.
  • Library: ~/.local/share/LokalBot.
  • From 5fe37174434e4cc9cdd365d492c14e15dc652edb Mon Sep 17 00:00:00 2001 From: stevyhacker Date: Tue, 20 Oct 2026 15:16:40 +0200 Subject: [PATCH 3/5] Point the repository setup at its docs PR --- RELEASING.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/RELEASING.md b/RELEASING.md index 254c6c7e..453dcffc 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -636,9 +636,10 @@ any Linux machine with Docker: `gh workflow run linux-repos.yml -f tag=v0.11.0`, then `curl -fsS https://pkg.lokalbot.com/apt/dists/stable/InRelease`. Its packages don't add the repositories, so nobody uses them yet. -5. Commit `lokalbot-archive-keyring.asc`. The next release's `.deb` and - `.rpm` add the repositories, and every release after it reaches them with - `apt upgrade` and `dnf upgrade`. +5. Commit `lokalbot-archive-keyring.asc`, with the README and website copy + that tells users about the repositories (#339). The next release's `.deb` + and `.rpm` add the repositories, and every release after it reaches them + with `apt upgrade` and `dnf upgrade`. ### Key expiry and rotation From 508b5528c17e7cc807189369d2fd9560bcf70b62 Mon Sep 17 00:00:00 2001 From: stevyhacker Date: Tue, 20 Oct 2026 15:47:25 +0200 Subject: [PATCH 4/5] Tell .deb and .rpm installs with the LokalBot repository to apt or dnf upgrade The release's own lokalbot package was always told to download and install the next package. Once a .deb or .rpm has added LokalBot's repository (/etc/apt/sources.list.d/lokalbot.sources, /etc/yum.repos.d/lokalbot.repo, not turned off with Enabled: no or enabled=0), package::owner() marks it as having a repository, so it routes to the package manager and is told "sudo apt upgrade" or "sudo dnf upgrade". The Arch package, and a .deb or .rpm from before the repository, keep the install command. The check reads the shipped repository files in its tests and runs off the UI thread with the package lookup. --- .../crates/lokalbot-app/src/updates.rs | 27 ++- .../crates/lokalbot-update/src/package.rs | 177 ++++++++++++++---- LinuxWindows/scripts/package-linux-distros.sh | 3 +- RELEASING.md | 2 +- 4 files changed, 173 insertions(+), 36 deletions(-) diff --git a/LinuxWindows/crates/lokalbot-app/src/updates.rs b/LinuxWindows/crates/lokalbot-app/src/updates.rs index 31c87eec..ed2f5725 100644 --- a/LinuxWindows/crates/lokalbot-app/src/updates.rs +++ b/LinuxWindows/crates/lokalbot-app/src/updates.rs @@ -12,7 +12,9 @@ //! own files (`/opt`, Program Files, a source build), the update is offered //! as a download from GitHub instead. A copy a package manager installed is //! told to update through it: the AUR's `lokalbot-bin` with `yay -Syu`, the -//! release's own .deb, .rpm and Arch packages by installing the new package. +//! release's .deb and .rpm with `apt upgrade` and `dnf upgrade` once they +//! added LokalBot's repository, and otherwise (the Arch package, or a .deb +//! or .rpm from before the repository) by installing the new package. use std::ffi::OsString; use std::sync::Arc; @@ -661,6 +663,7 @@ mod tests { Package { manager, name: name.into(), + repository: false, } } @@ -690,6 +693,28 @@ mod tests { ); } + #[test] + fn release_packages_with_the_repository_are_told_to_upgrade() { + let with_repository = |manager| Package { + repository: true, + ..package(manager, "lokalbot") + }; + assert_eq!( + offered(Some(&with_repository(Manager::Dpkg)), true), + ( + Route::PackageManager, + "Update LokalBot with your package manager, for example sudo apt upgrade.".into() + ) + ); + assert_eq!( + offered(Some(&with_repository(Manager::Rpm)), false), + ( + Route::PackageManager, + "Update LokalBot with your package manager, for example sudo dnf upgrade.".into() + ) + ); + } + #[test] fn release_packages_are_told_to_install_the_new_package() { let deb = package(Manager::Dpkg, "lokalbot"); diff --git a/LinuxWindows/crates/lokalbot-update/src/package.rs b/LinuxWindows/crates/lokalbot-update/src/package.rs index 376bd48f..f8bff51a 100644 --- a/LinuxWindows/crates/lokalbot-update/src/package.rs +++ b/LinuxWindows/crates/lokalbot-update/src/package.rs @@ -13,9 +13,11 @@ use crate::Version; /// The name of the release's own `.deb`, `.rpm` and Arch packages /// (`scripts/package-linux-distros.sh`). They are installed from a -/// downloaded file with no repository behind them, so the package manager -/// cannot fetch the next version. Other names, such as the AUR's -/// `lokalbot-bin`, come from a repository. +/// downloaded file. Once the repository key is committed, the `.deb` and +/// `.rpm` also add LokalBot's apt or dnf repository ([`repository_file`]), +/// and from then on the package manager fetches the next version; the Arch +/// package never has one. Other names, such as the AUR's `lokalbot-bin`, +/// come from a repository. pub const RELEASE_PACKAGE: &str = "lokalbot"; #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -29,12 +31,15 @@ pub enum Manager { pub struct Package { pub manager: Manager, pub name: String, + /// LokalBot's own repository is set up and enabled for this manager. + pub repository: bool, } impl Package { - /// One of the release's own packages, installed from a downloaded file. + /// One of the release's own packages with no repository to update it: + /// installed from a downloaded file, so the next one is too. pub fn from_release(&self) -> bool { - self.name == RELEASE_PACKAGE + self.name == RELEASE_PACKAGE && !self.repository } /// Updates a package from its repository. @@ -60,32 +65,48 @@ impl Package { } } +/// The repository file the release's `.deb` (deb822) or `.rpm` installs +/// for LokalBot's apt or dnf repository (`packaging/linux`). +pub fn repository_file(manager: Manager) -> Option<&'static Path> { + match manager { + Manager::Dpkg => Some(Path::new("/etc/apt/sources.list.d/lokalbot.sources")), + Manager::Rpm => Some(Path::new("/etc/yum.repos.d/lokalbot.repo")), + Manager::Pacman => None, + } +} + /// The package that installed `executable`, or `None` when no package -/// database lists it. Starts the package tools, so call it off the UI thread. +/// database lists it. Starts the package tools and reads the repository +/// file, so call it off the UI thread. pub fn owner(executable: &Path) -> Option { if !cfg!(target_os = "linux") { return None; } - owner_with(executable, &|program, args| { - let output = Command::new(program) - .args(args) - .env("LC_ALL", "C") - .stdin(Stdio::null()) - .stderr(Stdio::null()) - .output() - .ok()?; - output - .status - .success() - .then(|| String::from_utf8_lossy(&output.stdout).into_owned()) - }) + owner_with( + executable, + &|program, args| { + let output = Command::new(program) + .args(args) + .env("LC_ALL", "C") + .stdin(Stdio::null()) + .stderr(Stdio::null()) + .output() + .ok()?; + output + .status + .success() + .then(|| String::from_utf8_lossy(&output.stdout).into_owned()) + }, + &|path| std::fs::read_to_string(path).ok(), + ) } /// [`owner`] with `run` starting a package tool and returning what it -/// printed when it succeeded. +/// printed when it succeeded, and `read` reading a file. fn owner_with( executable: &Path, run: &dyn Fn(&str, &[&OsStr]) -> Option, + read: &dyn Fn(&Path) -> Option, ) -> Option { let path = executable.as_os_str(); // A system can have another distribution's tools (dpkg on Arch, to build @@ -106,7 +127,32 @@ fn owner_with( ]; queries.into_iter().find_map(|(manager, program, args)| { let name = parse(manager, &run(program, args)?)?; - Some(Package { manager, name }) + let repository = repository_file(manager) + .and_then(read) + .is_some_and(|text| enabled(&text)); + Some(Package { + manager, + name, + repository, + }) + }) +} + +/// Whether a repository file leaves its repository on: no `Enabled: no` +/// (deb822) or `enabled=0` (dnf) line. +fn enabled(text: &str) -> bool { + !text.lines().any(|line| { + let line = line.trim(); + if line.starts_with('#') { + return false; + } + let Some((key, value)) = line.split_once([':', '=']) else { + return false; + }; + key.trim().eq_ignore_ascii_case("enabled") + && ["no", "false", "off", "0"] + .iter() + .any(|off| value.trim().eq_ignore_ascii_case(off)) }) } @@ -139,17 +185,34 @@ mod tests { use std::cell::RefCell; fn lookup(answers: &[(&str, &str)]) -> (Option, Vec) { + lookup_with_files(answers, &[]) + } + + /// [`owner_with`] with tool answers and the files that exist. + fn lookup_with_files( + answers: &[(&str, &str)], + files: &[(&str, &str)], + ) -> (Option, Vec) { let asked = RefCell::new(Vec::new()); - let package = owner_with(Path::new("/usr/bin/lokalbot"), &|program, args| { - let args: Vec<_> = args.iter().map(|a| a.to_string_lossy()).collect(); - asked - .borrow_mut() - .push(format!("{program} {}", args.join(" "))); - answers - .iter() - .find(|(tool, _)| *tool == program) - .map(|(_, out)| out.to_string()) - }); + let package = owner_with( + Path::new("/usr/bin/lokalbot"), + &|program, args| { + let args: Vec<_> = args.iter().map(|a| a.to_string_lossy()).collect(); + asked + .borrow_mut() + .push(format!("{program} {}", args.join(" "))); + answers + .iter() + .find(|(tool, _)| *tool == program) + .map(|(_, out)| out.to_string()) + }, + &|path| { + files + .iter() + .find(|(file, _)| Path::new(file) == path) + .map(|(_, text)| text.to_string()) + }, + ); (package, asked.into_inner()) } @@ -160,7 +223,8 @@ mod tests { package, Some(Package { manager: Manager::Pacman, - name: "lokalbot-bin".into() + name: "lokalbot-bin".into(), + repository: false }) ); assert_eq!(asked, ["pacman -Qqo /usr/bin/lokalbot"]); @@ -170,7 +234,8 @@ mod tests { package, Some(Package { manager: Manager::Dpkg, - name: "lokalbot".into() + name: "lokalbot".into(), + repository: false }) ); assert_eq!( @@ -189,6 +254,51 @@ mod tests { ); } + #[test] + fn release_packages_with_the_repository_file_update_from_it() { + const SOURCES: &str = "/etc/apt/sources.list.d/lokalbot.sources"; + const REPO: &str = "/etc/yum.repos.d/lokalbot.repo"; + let sources = include_str!("../../../packaging/linux/lokalbot.sources"); + let repo = include_str!("../../../packaging/linux/lokalbot.repo"); + let deb = [("dpkg-query", "lokalbot: /usr/bin/lokalbot\n")]; + let rpm = [("rpm", "lokalbot\n")]; + + let package = lookup_with_files(&deb, &[(SOURCES, sources)]).0.unwrap(); + assert!(package.repository && !package.from_release()); + let package = lookup_with_files(&rpm, &[(REPO, repo)]).0.unwrap(); + assert!(package.repository && !package.from_release()); + + // Installed before the repository existed, or with it turned off. + assert!(lookup(&deb).0.unwrap().from_release()); + let off = format!("{sources}Enabled: no\n"); + assert!( + lookup_with_files(&deb, &[(SOURCES, &off)]) + .0 + .unwrap() + .from_release() + ); + let off = repo.replace("enabled=1", "enabled=0"); + assert!( + lookup_with_files(&rpm, &[(REPO, &off)]) + .0 + .unwrap() + .from_release() + ); + // The other manager's file says nothing about this package. + assert!( + lookup_with_files(&deb, &[(REPO, repo)]) + .0 + .unwrap() + .from_release() + ); + // The Arch package never has a repository. + let arch = lookup_with_files( + &[("pacman", "lokalbot\n")], + &[(SOURCES, sources), (REPO, repo)], + ); + assert!(arch.0.unwrap().from_release()); + } + #[test] fn no_package_and_odd_answers_are_no_owner() { assert_eq!(lookup(&[]).0, None); @@ -229,6 +339,7 @@ mod tests { let package = |manager, name: &str| Package { manager, name: name.into(), + repository: false, }; assert!(package(Manager::Dpkg, "lokalbot").from_release()); assert!(!package(Manager::Pacman, "lokalbot-bin").from_release()); diff --git a/LinuxWindows/scripts/package-linux-distros.sh b/LinuxWindows/scripts/package-linux-distros.sh index b771251a..a890a3f8 100755 --- a/LinuxWindows/scripts/package-linux-distros.sh +++ b/LinuxWindows/scripts/package-linux-distros.sh @@ -17,7 +17,8 @@ # installs the new package; the file names below are in that command too. # Once the repository key is in packaging/linux, the .deb and .rpm also add # the apt and dnf repositories at pkg.lokalbot.com (scripts/linux-repos.sh) -# with that key, so later versions come with `apt upgrade` and `dnf upgrade`. +# with that key, so later versions come with `apt upgrade` and `dnf upgrade`, +# which LokalBot then suggests (package::repository_file). set -euo pipefail cd "$(dirname "$0")/.." VERSION=${PACKAGE_VERSION:-$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)} diff --git a/RELEASING.md b/RELEASING.md index b64c6e3b..30cc4110 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -443,7 +443,7 @@ work), checks each feed against its archive, and attaches to the release: | `LokalBot--linux-x86_64.tar.gz` (+ `.sha256`) | Linux archive with `install.sh` | | `LokalBot--windows-x86_64.zip` (+ `.sha256`) | Windows archive | | `LokalBot--windows-x86_64-setup.exe` (+ `.sha256`) | per-user Windows installer of the same files (`scripts/package-windows-installer.ps1`, Inno Setup 6): `%LOCALAPPDATA%\Programs\LokalBot`, a Start menu shortcut with the notification app id, and an Installed apps entry, without administrator rights. The release installs and uninstalls it on the Windows runner. Updates still replace the files in place from the zip, so the Installed apps version can lag until the next installer run | -| `LokalBot--linux-amd64.deb`, `LokalBot--linux-x86_64.rpm`, `LokalBot--linux-x86_64.pkg.tar.zst` (+ `.sha256`) | Debian/Ubuntu, Fedora/openSUSE and Arch packages of the same binaries (`scripts/package-linux-distros.sh`, nfpm). The release checks each one installs in a clean Ubuntu 24.04, Fedora 41 and Arch container. They install into `/usr/bin` from no repository, so LokalBot announces updates for them and shows the command that installs the new package instead of replacing its files; it recognizes them by the package name `lokalbot` | +| `LokalBot--linux-amd64.deb`, `LokalBot--linux-x86_64.rpm`, `LokalBot--linux-x86_64.pkg.tar.zst` (+ `.sha256`) | Debian/Ubuntu, Fedora/openSUSE and Arch packages of the same binaries (`scripts/package-linux-distros.sh`, nfpm). The release checks each one installs in a clean Ubuntu 24.04, Fedora 41 and Arch container. They install into `/usr/bin`, so LokalBot announces updates for them instead of replacing its files; it recognizes them by the package name `lokalbot`. A `.deb` or `.rpm` that added the LokalBot repository ([Linux package repositories](#linux-package-repositories)) is told to run `sudo apt upgrade` or `sudo dnf upgrade`; the others get the command that installs the new package | | `linux-x86_64.json`, `windows-x86_64.json` | update feeds | A feed names the version, the minimum installed version that may install it From 04a021d4343a0a902cc81ef204a5bd2a2e59c0db Mon Sep 17 00:00:00 2001 From: stevyhacker Date: Tue, 20 Oct 2026 15:48:41 +0200 Subject: [PATCH 5/5] Run the UI suite on the ready pull request