From d787d8d6c3f1d5c8ad924b3aa77a1f401ad70d42 Mon Sep 17 00:00:00 2001 From: David Hollinger Date: Tue, 6 Oct 2026 19:49:04 -0500 Subject: [PATCH 1/4] docs: improve README documentation --- README.md | 215 ++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 162 insertions(+), 53 deletions(-) diff --git a/README.md b/README.md index 3d3b529..7560d4b 100644 --- a/README.md +++ b/README.md @@ -6,26 +6,70 @@ This is designed to be more streamlined, performant, and easier to ship for user This server is a REST API server designed to accept Webhooks from version control systems, such as GitHub or GitLab, and execute actions based on those webhooks. Specifically, the following tasks: * Trigger r10k environment and module deploys onto Puppet Servers -* Send notifications to ChatOps systems, such as Slack and RocketChat +* Send notifications to ChatOps systems, such as Slack, RocketChat, and Microsoft Teams ## Prerequisites While there are no prerequisites for running the webhook server itself, for it to be useful, you will need the following installed on the same server or another server for this tool to be useful: * Puppet Server -* [r10k](https://github.com/puppetlabs/r10k) >= 3.9.0 -* Puppet Bolt (optional) +* [r10k](https://github.com/puppetlabs/r10k) >= 3.9.0 (or [g10k](https://github.com/xorpaul/g10k)) * Windows or Linux server to run the server on. MacOS is not supported. ## Installation -Download a Pre-release Binary from the [Releases](https://github.com/voxpupuli/webhook-go/releases) page, make it executable, and run the server. +### Packages and binaries + +Released artifacts are published on the [Releases](https://github.com/voxpupuli/webhook-go/releases) page, including: + +* `deb` and `rpm` packages (linux amd64, arm, and arm64) +* Archive files for linux (amd64, arm, arm64), windows (amd64), and freebsd (amd64, arm64) + +The `deb` and `rpm` packages install the `webhook-go` binary to `/usr/bin/webhook-go`, a configuration file to `/etc/voxpupuli/webhook.yml` (marked as `config|noreplace`, so it is not overwritten on upgrades), and a systemd unit (`webhook-go.service`) that enables and starts the service on install. + +For a manual binary installation, download the archive for your platform, extract the `webhook-go` binary, make it executable, and place it somewhere in your `PATH`. + +### Docker + +Versioned container images are published to [ghcr.io/voxpupuli/webhook-go](https://github.com/voxpupuli/webhook-go/pkgs/container/webhook-go): + +```sh +docker run -d -p 4000:4000 ghcr.io/voxpupuli/webhook-go:2.15.1 +``` + +The image is built `FROM scratch` and contains only the `webhook-go` binary and a default configuration file at `/webhook.yml` (which is in the default config search path). To use your own configuration, mount it over the baked-in file, or pass the `--config` flag (arguments after the image name are appended to the `webhook-go server` entrypoint): + +```sh +docker run -d -p 4000:4000 \ + -v /etc/voxpupuli/webhook.yml:/webhook.yml:ro \ + ghcr.io/voxpupuli/webhook-go:2.15.1 +``` + +Note that the server executes `r10k` (or `g10k`) as a local process, and the published image does not contain one. To deploy from a container, derive your own image that includes an `r10k` binary, or mount one in and point `r10k.command_path` at it. + +### From source + +Building from source requires [Go](https://go.dev/dl/) >= 1.25: + +```sh +git clone https://github.com/voxpupuli/webhook-go.git +cd webhook-go +go build -o webhook-go . +./webhook-go server --config ./build/webhook.yml +``` ## Configuration The Webhook API server uses a configuration file called `webhook.yml` to configure the server. Several of the required options have defaults pre-defined so that a configuration file isn't needed for basic function. -`webhook.yaml.example`: +When the `--config` flag is not given, the server looks for a `webhook.yml` in the following locations, in order: + +* `./webhook.yml` (current working directory) +* `/etc/voxpupuli/webhook/webhook.yml` +* `../config/webhook.yml` +* `config/webhook.yml` + +Example `webhook.yml`: ```yaml server: @@ -40,7 +84,7 @@ server: queue: enabled: true max_concurrent_jobs: 10 - max_history_items: 20 + max_history_items: 50 chatops: enabled: false service: slack @@ -50,48 +94,19 @@ chatops: server_uri: "https://rocketchat.local" r10k: config_path: /etc/puppetlabs/r10k/r10k.yaml - default_branch: main + default_branch: master allow_uppercase: false verbose: true mappings: long-repo-name: lrp ``` -### Microsoft Teams notifications - -Create an "Incoming Webhook" connector in Teams at the designated channel as described in the documentation: [Create Incoming Webhooks at learn.microsoft.com](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook). Keep the URL confidential! - -Configure the service in the `webhook.yaml`: -```yaml -chatops: - enabled: true - service: teams - server_uri: "" -``` - -Notifications are colored, according to their status. -green: Success -red: Failure -orange: Warning - -Press the `Details` button to get more information. - -If the queue is enabled in the `server` part of `webhook.yaml`, then two notifications are emitted: First, when the request is added to the queue and second, when the request was processed. - -### Bolt authentication - -Due to the inherent security risk associated with passing plain text passwords to the Bolt CLI tool, all ability to set it within the application have been removed. - -Instead, it is recommended to instead utilize the Bolt [Transport configuration options](https://puppet.com/docs/bolt/latest/bolt_transports_reference.html) and place them within the `bolt-defaults.yaml` file. - -If you want to utilize an `inventory.yaml` and place the targets and auth config within that file, you can. Just be sure to remember to add the target name containing the nodes you need to the `webhook.yml` file - ### Server options #### `protected` Type: bool -Description: Enforces authentication via basic Authentication +Description: Enforces authentication via Basic Authentication on all `/api/v1/*` routes. The `/health` route always stays unauthenticated. Default: `false` #### `user` @@ -108,14 +123,14 @@ Default: `nil` #### `port` -Type: int64 +Type: int Description: Port to run the server on. Optional. Default: `4000` #### `deploy_on_success_only` Type: bool -Description: Only run a deploy if checks pass +Description: Only run a deploy if the pipeline/workflow associated with the webhook (e.g. a GitHub `workflow_run` or GitLab `pipeline` event) completed successfully. Failed pipelines are rejected with `424 Failed Dependency`. Default: `false` #### `tls` @@ -144,7 +159,7 @@ Default: `nil` #### `queue` Type: struct -Description: Struct containing Queue options +Description: Struct containing Queue options. When enabled, deploy requests are queued and processed by a background worker instead of running synchronously; the API responds immediately with the queued job. ##### `enabled` @@ -159,6 +174,7 @@ Description: How many jobs could be stored in queue Default: `10` ##### `max_history_items` + Type: int Description: How many queue items should be stored in the history Default: `50` @@ -201,6 +217,27 @@ Type: string Description: The ChatOps service API URI to send the message to. For MS Teams, this is the Webhook URL created at the channel connectors. Default: nil +### Microsoft Teams notifications + +Create an "Incoming Webhook" connector in Teams at the designated channel as described in the documentation: [Create Incoming Webhooks at learn.microsoft.com](https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook). Keep the URL confidential! + +Configure the service in the `webhook.yaml`: +```yaml +chatops: + enabled: true + service: teams + server_uri: "" +``` + +Notifications are colored, according to their status. +green: Success +red: Failure +yellow: Warning + +Press the `Details` button to get more information. + +If the queue is enabled in the `server` part of `webhook.yaml`, then two notifications are emitted: First, when the request is added to the queue and second, when the request was processed. + ### r10k options #### `config_path` @@ -212,13 +249,13 @@ Default: `/etc/puppetlabs/r10k/r10k.yaml` #### `default_branch` Type: string -Description: Name of the default branch for r10k to pull from. Optional. -Default: `main` +Description: Name of the default branch for r10k to pull from, used when the webhook payload doesn't include a branch. Optional. +Default: `master` #### `prefix` Type: string -Description: An r10k prefix to apply to the module or environment being deployed. Optional. +Description: An r10k prefix to apply to the environment being deployed, producing `prefix_branch` environment names. Optional. Set to the special value `mapping` to look up the prefix per repository in the `mappings` section instead. Default: `nil` #### `allow_uppercase` @@ -254,7 +291,7 @@ Default: `true` #### `env_incremental` Type: bool -Description: Use `--incremental` flag when updating an environment +Description: Use `--incremental` flag when updating an environment (only used with the `--modules` deploy mode, not the legacy `--puppetfile` flag) Default: `false` #### `command_path` @@ -290,20 +327,27 @@ r10k: - dependabot/ ``` +#### `use_g10k_commands` + +Type: `boolean` +Description: Use g10k commands instead of r10k (parameters are a little bit different) +Default: `false` + ### `mappings` Type: `map` -Description: A map of long repository names to short names. This is useful for repositories that have long names that are not suitable for use in the URL. This is useful for multi tenant environments where you also want to use a shorter prefix for the environment. +Description: A map of long repository names to short names. This is useful for repositories that have long names that are not suitable for use in the URL. This is useful for multi tenant environments where you also want to use a shorter prefix for the environment. Used when `r10k.prefix` is set to `mapping`. Default: `{}` +## Usage -### `use_g10k_commands` +Start the server with: -Type: `boolean` -Description: Use g10k commands instead of r10k (parameters are a little bit different) -Default: `false` +```sh +webhook-go server --config /path/to/webhook.yml +``` -## Usage +The binary is a small CLI (run `webhook-go --help` for details) with a single subcommand, `server`, plus the usual `help` and `completion` commands. Webhook API provides following paths @@ -311,9 +355,21 @@ Webhook API provides following paths Get health assessment about the Webhook API server +```sh +curl http://localhost:4000/health +``` + +```json +{"message":"running"} +``` + ### GET /api/v1/queue -Get current queue status of the Webhook API server +Get current queue status of the Webhook API server. Returns the list of queued jobs, including each job's id, name, command, state (`added`, `success`, `failed`), timestamps, and the command output once processed. This endpoint requires authentication when `server.protected` is enabled. + +```sh +curl -u puppet:puppet http://localhost:4000/api/v1/queue +``` ### POST /api/v1/r10k/environment @@ -323,11 +379,64 @@ Available URL arguments (`?argument=value`): * `no_mods=(true|false)` - If set, this will only update an environment with no modules (no flags `--modules` or `--puppetfile`). This option is usefull when you need to update only files (for example [separate hiera data](https://github.com/puppetlabs/r10k/blob/main/doc/dynamic-environments/configuration.mkd#separate-hiera-data)) +Example, sending a GitHub `push` event for the `production` branch: + +```sh +curl -u puppet:puppet -X POST http://localhost:4000/api/v1/r10k/environment \ + -H "X-GitHub-Event: push" \ + -H "Content-Type: application/json" \ + -d '{"ref":"refs/heads/production","deleted":false,"repository":{"name":"webhook-go","full_name":"voxpupuli/webhook-go","owner":{"name":"voxpupuli"}}}' +``` + +On success the server responds with `202 Accepted`: the raw r10k output when the queue is disabled, or the queued job when it is enabled. Deleting a ref or pushing a branch matching an `ignore_branch_prefixes` entry is answered with `200 OK` and nothing is deployed; a `blocked_branches` entry is answered with `403 Forbidden`. + ### POST /api/v1/r10k/module Updates a puppet module, ie. `r10k deploy module`. The default behavior of r10k is to update the module in all environments that have it. Module name defaults to the git repository name. Available URL arguments (`?argument=value`): -* `branch_only=(true|false)` - If set, this will only update the module in an environment set by the branch, as opposed to all environments. This is equivalent to the `--environment` r10k option. DEFAULT: `false` -* `module_name=name` - Sometimes git repository and module name cannot have the same name due to arbitrary naming restrictions. This option forces the module name to be the given value instead of repository name. +* `branch_only=(true|false)` - If set to any non-empty value, this will only update the module in an environment set by the branch, as opposed to all environments. This is equivalent to the `--environment` r10k option. DEFAULT: `false` +* `module_name=name` - Sometimes git repository and module name cannot have the same name due to arbitrary naming restrictions. This option forces the module name to be the given value instead of repository name. Must match `^[a-z][a-z0-9_]*$`. + +Example, deploying the `stdlib` module only in the environment of the pushed branch: + +```sh +curl -u puppet:puppet -X POST "http://localhost:4000/api/v1/r10k/module?branch_only=true&module_name=stdlib" \ + -H "X-GitHub-Event: push" \ + -H "Content-Type: application/json" \ + -d '{"ref":"refs/heads/main","deleted":false,"repository":{"name":"puppetlabs-stdlib","full_name":"puppetlabs/puppetlabs-stdlib","owner":{"name":"puppetlabs"}}}' +``` + +### Supported webhook providers + +The server detects the provider from the event headers each request carries and parses the payload accordingly. Supported providers and events: + +| Provider | Detected via | Supported events | +|---|---|---| +| GitHub | `X-GitHub-Event` | `push`, `workflow_run` | +| GitLab | `X-Gitlab-Event` | `Push Hook`, `Pipeline Hook` | +| Bitbucket Cloud | `X-Event-Key` + `X-Hook-UUID` | `repo:push` | +| Bitbucket Server | `X-Event-Key` + `X-Request-Id` | `repo:refs_changed` | +| Azure DevOps | `X-Azure-DevOps` | `git.push` | +| Gitea | `X-Gitea-Event` | `push`, `delete` | + +Requests from other providers, or with unsupported event types, fail with `500` and an explanatory message. + +## Development + +The [Makefile](Makefile) provides the usual targets; run `make help` to list them: + +* `make run` - run the server from source with `./build/webhook.yml` +* `make test` - run the test suite (`go test ./...`) +* `make binary` - build a local binary into `bin/` with GoReleaser +* `make compile` - build for all supported OS/architectures (snapshot) +* `make snapshot` - build all release artifacts without publishing (snapshot) +* `make release` - build and publish a release (requires a `v*` tag) +* `make clean` - clean up build and dependency artifacts + +Building release artifacts requires [GoReleaser](https://goreleaser.com/). CI runs on every push/PR to `master` and builds and tests with Go 1.25. + +## License + +This project is licensed under the Apache 2.0 license. See [LICENSE](LICENSE) for details. From 189d8dc1d60e2149637589cf4651cb2de3d0eeea Mon Sep 17 00:00:00 2001 From: David Hollinger Date: Tue, 6 Oct 2026 19:51:46 -0500 Subject: [PATCH 2/4] docs: update g10k link to voxpupuli fork --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 7560d4b..60a360f 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ This server is a REST API server designed to accept Webhooks from version contro While there are no prerequisites for running the webhook server itself, for it to be useful, you will need the following installed on the same server or another server for this tool to be useful: * Puppet Server -* [r10k](https://github.com/puppetlabs/r10k) >= 3.9.0 (or [g10k](https://github.com/xorpaul/g10k)) +* [r10k](https://github.com/puppetlabs/r10k) >= 3.9.0 (or [g10k](https://github.com/voxpupuli/g10k)) * Windows or Linux server to run the server on. MacOS is not supported. ## Installation From a97ef38b09923504edf34e32f98e0a08814d6643 Mon Sep 17 00:00:00 2001 From: David Hollinger Date: Tue, 6 Oct 2026 19:56:11 -0500 Subject: [PATCH 3/4] docs: add per-OS development dependencies to README --- README.md | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 60a360f..0ff5539 100644 --- a/README.md +++ b/README.md @@ -425,6 +425,76 @@ Requests from other providers, or with unsupported event types, fail with `500` ## Development +### Development dependencies + +Required on all platforms: + +* [Go](https://go.dev/dl/) >= 1.25 — to build, test, and run the server +* `git` — the version for `make binary` is derived from `git describe --tags` +* `make` — optional; every Makefile target wraps a command that can be run directly +* [GoReleaser](https://goreleaser.com/) — only for the build/release targets (`make binary`, `make compile`, `make snapshot`, `make release`) +* Docker with [Buildx](https://docs.docker.com/build/buildx/) — only for building the container image (`make snapshot`, `make release`) +* [mockery](https://github.com/vektra/mockery) — only needed to regenerate the interface mocks in `lib/*/mocks/` + +#### Linux + +Build tools (`make` and friends): + +```sh +sudo apt install build-essential # Debian/Ubuntu +sudo dnf groupinstall "Development Tools" # RHEL/Fedora +``` + +GoReleaser: + +```sh +# Debian/Ubuntu +echo 'deb [trusted=yes] https://repo.goreleaser.com/apt/ /' | sudo tee /etc/apt/sources.list.d/goreleaser.list +sudo apt update +sudo apt install goreleaser + +# RHEL/Fedora +echo '[goreleaser] +name=GoReleaser +baseurl=https://repo.goreleaser.com/yum/ +enabled=1 +gpgcheck=0 +exclude=goreleaser-pro' | sudo tee /etc/yum.repos.d/goreleaser.repo +sudo yum install goreleaser +``` + +Install Go from your distribution's packages or the official tarballs at [go.dev/dl](https://go.dev/dl/); distribution packages often lag behind the required 1.25. + +#### macOS + +```sh +xcode-select --install # installs make and git +brew install go +brew install goreleaser +``` + +While macOS is not a supported platform for running the service, developing on it works fine — Go cross-compiles for all release targets. + +#### Windows + +```powershell +winget install GoLang.Go +winget install goreleaser +``` + +`make` doesn't ship with Windows. Either run the underlying commands directly (`go test ./...`, `go run main.go --config ./build/webhook.yml`, `goreleaser build --single-target --clean`, ...) or install make with Chocolatey (`choco install make`) and run the targets from an environment that provides the usual Unix tools, such as Git Bash or WSL. + +#### Any OS + +GoReleaser and mockery can also be installed with `go install`: + +```sh +go install github.com/goreleaser/goreleaser/v2@latest # requires Go 1.27+ +go install github.com/vektra/mockery/v2@latest +``` + +### Make targets + The [Makefile](Makefile) provides the usual targets; run `make help` to list them: * `make run` - run the server from source with `./build/webhook.yml` @@ -435,7 +505,7 @@ The [Makefile](Makefile) provides the usual targets; run `make help` to list the * `make release` - build and publish a release (requires a `v*` tag) * `make clean` - clean up build and dependency artifacts -Building release artifacts requires [GoReleaser](https://goreleaser.com/). CI runs on every push/PR to `master` and builds and tests with Go 1.25. +CI runs on every push/PR to `master` and builds and tests with Go 1.25. ## License From fbc791051292ff6c2de300fbb7f246cab292cb3c Mon Sep 17 00:00:00 2001 From: David Hollinger Date: Tue, 6 Oct 2026 19:58:16 -0500 Subject: [PATCH 4/4] docs: break out Linux dev dependencies by distribution --- README.md | 41 ++++++++++++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 0ff5539..c3baf91 100644 --- a/README.md +++ b/README.md @@ -438,32 +438,59 @@ Required on all platforms: #### Linux -Build tools (`make` and friends): +##### Ubuntu/Debian + +Build tools (`make` and friends) and Go: ```sh -sudo apt install build-essential # Debian/Ubuntu -sudo dnf groupinstall "Development Tools" # RHEL/Fedora +sudo apt install build-essential +sudo apt install golang-go ``` GoReleaser: ```sh -# Debian/Ubuntu echo 'deb [trusted=yes] https://repo.goreleaser.com/apt/ /' | sudo tee /etc/apt/sources.list.d/goreleaser.list sudo apt update sudo apt install goreleaser +``` + +##### Arch Linux + +Build tools (`make` and friends) and Go: + +```sh +sudo pacman -S base-devel go +``` + +GoReleaser (from the [AUR](https://aur.archlinux.org/packages/goreleaser-bin), with an AUR helper such as yay): + +```sh +yay -S goreleaser-bin +``` + +##### Fedora-based -# RHEL/Fedora +Build tools (`make` and friends) and Go: + +```sh +sudo dnf groupinstall "Development Tools" +sudo dnf install golang +``` + +GoReleaser (the same repo also works on RHEL/CentOS with `yum`): + +```sh echo '[goreleaser] name=GoReleaser baseurl=https://repo.goreleaser.com/yum/ enabled=1 gpgcheck=0 exclude=goreleaser-pro' | sudo tee /etc/yum.repos.d/goreleaser.repo -sudo yum install goreleaser +sudo dnf install goreleaser ``` -Install Go from your distribution's packages or the official tarballs at [go.dev/dl](https://go.dev/dl/); distribution packages often lag behind the required 1.25. +Distribution Go packages often lag behind the required 1.25; if yours is older, install an official tarball from [go.dev/dl](https://go.dev/dl/). #### macOS