Skip to content

fix(bench): identify docker-compose's engine by asking the engine - #1940

Merged
Jaro-c merged 1 commit into
developfrom
fix/bench-engine-detection
Sep 27, 2026
Merged

Jaro-c merged 1 commit into
developfrom
fix/bench-engine-detection

Conversation

@Jaro-c

@Jaro-c Jaro-c commented Sep 27, 2026

Copy link
Copy Markdown
Member

Summary

  • bench/run.sh decided which engine docker-compose drove with docker info. The docker CLI honours DOCKER_HOST, so with Docker installed and DOCKER_HOST pointed at the Podman socket, the run was labelled a Docker (cross-engine) comparison. The same-engine comparison published in docs/benchmarks.md could no longer be selected on such a host.
  • A Docker run did not pre-pull the pinned images into Docker, so the first timed up would have measured the download.

Changes

  • bench/engine.sh (new, sourced): probes /version at the endpoint docker-compose uses (DOCKER_HOST, or the default Docker socket) and reads the server's answer. Podman lists a Podman Engine component and Docker does not. A DOCKER_CONTEXT, an ssh:// host or an endpoint that does not answer leaves docker-compose unmeasured, with the reason printed.
  • bench/run.sh: uses it, pre-pulls into Docker for a Docker run (and refuses to measure docker-compose on Docker if the docker CLI is missing), and removes the build scenario's image from the engine that built it.
  • bench/README.md: how the engine is chosen.

Test plan

On a host with both Podman 5.7.0 and Docker 29.8.1:

state before after
DOCKER_HOST = Podman socket docker info succeeds, labelled docker podman
DOCKER_HOST unset docker docker
DOCKER_HOST = missing socket skipped skipped, "no engine answered"
DOCKER_CONTEXT=other n/a skipped with the reason
DOCKER_HOST=ssh://x n/a skipped with the reason
  • Sabotage: with the Podman Engine branch deleted, the Podman socket is reported as docker. Restored, it reports podman.
  • bash bench/run.sh --smoke once per engine: results/engine reads podman and docker respectively, and each run measured all three tools.
  • bash -n and shellcheck -x clean on both scripts

The smoke runs also showed restart failing with HTTP 500 (rootless netns: kill network process: permission denied) for podup, and for docker-compose on the Podman socket. That comes from the host's AppArmor pasta profile refusing a signal from the Podman service, and it is unrelated to this change. I am tracking it separately.

run.sh chose the engine with `docker info`. The docker CLI honours
DOCKER_HOST, so on a host with Docker installed and DOCKER_HOST pointed
at the Podman socket, `docker info` answered from Podman and the run was
labelled a Docker (cross-engine) comparison. The same-engine comparison
that docs/benchmarks.md publishes could not be selected at all once
Docker was installed.

bench/engine.sh now probes `/version` at the endpoint docker-compose will
use (DOCKER_HOST, or the default Docker socket) and reads the answer:
Podman lists a "Podman Engine" component and Docker does not. A
DOCKER_CONTEXT, an ssh:// host or a silent endpoint leaves docker-compose
unmeasured with the reason, rather than guessed.

A Docker run also pre-pulls the pinned images into Docker's own store,
since the Podman pre-pull did not reach it and the first timed `up` would
have measured the download, and the build scenario removes its image from
the engine that built it.

Signed-off-by: Jaro-c <75870284+Jaro-c@users.noreply.github.com>
@Jaro-c Jaro-c added type:bug A defect to fix prio:P3 Low priority effort:S Small area:ci Subsystem: ci labels Sep 27, 2026
@Jaro-c
Jaro-c merged commit a4957fe into develop Sep 27, 2026
35 checks passed
@Jaro-c
Jaro-c deleted the fix/bench-engine-detection branch September 27, 2026 16:10
Jaro-c added a commit that referenced this pull request Sep 27, 2026
Bumps `Cargo.toml`, `Cargo.lock` and `debian/changelog` to 5.10.5
together.

Patch release carrying #1942: when stopping a container fails because
the host's AppArmor `pasta` profile refuses Podman's signal, podup
prints
the commands that fix that host. It also carries #1940 and #1943, which
change only the benchmark harness.

Signed-off-by: Jaro-c <75870284+Jaro-c@users.noreply.github.com>

Signed-off-by: Jaro-c <75870284+Jaro-c@users.noreply.github.com>
@Jaro-c Jaro-c mentioned this pull request Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ci Subsystem: ci effort:S Small prio:P3 Low priority type:bug A defect to fix

Development

Successfully merging this pull request may close these issues.

1 participant