Unable to install codspeed in self-hosted runners

[XSAM]

The codspeed action was not able to install codspeed CLI in our self-hosted runners. The action works fine on github runners.

Error

> Run # Validate required inputs
Error: Installation of CodSpeed CLI with version 4.11.0 failed.
Reason: 
Error: Process completed with exit code 1.

Failed action: https://github.com/open-telemetry/opentelemetry-go/actions/runs/22638668316/job/65608368522?pr=7998#logs
PR: open-telemetry/opentelemetry-go#7998

[GuillaumeLagrange]

Hello, we are looking into it and will let you know ASAP, thanks for the report

[GuillaumeLagrange]

@XSAM I have #193 with the fix, can you try with the action at hash 7fa35d2e6d3c02af655afd4561807edbce71fcfd on your self hosted runner ?

The root cause that we were using curl --fail-with-body which was introduced in curl 7.76.0, while the the os you are using, according to the logged kernel version (5.15.0-308.179.6.el8uek.x86_64) is REHL 8.

A very superficial level check showed that it was using curl 7.61.1

$ docker run --rm -it oraclelinux:8 bash -c "curl --version"
curl 7.61.1 (x86_64-redhat-linux-gnu) libcurl/7.61.1 OpenSSL/1.1.1k zlib/1.2.11 brotli/1.0.6 libidn2/2.2.0 libpsl/0.20.2 (+libidn2/2.2.0) libssh/0.9.6/openssl/zlib nghttp2/1.33.0
Release-Date: 2018-09-05
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz brotli TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

Please note that I do not have access to your exact runner, hence my asking for a quick test before merging the PR

[XSAM]

@GuillaumeLagrange it is not working. https://github.com/open-telemetry/opentelemetry-go/actions/runs/22678571756/job/65742235545?pr=7998

> Run # Validate required inputs
Installer hash verified for version 4.11.0
/home/opc/actions-runner/_work/_temp/af22f54c-66de-4880-a7ee-6cd0f739e5a6.sh: line 105: codspeed: command not found
Error: Process completed with exit code 127.

I feel the downloaded codspeed is not in the PATH. Maybe use an absolute path in the script?

[GuillaumeLagrange]

I think I have found the root cause, but just to check: has the codspeed action ever worked on your self hosted runner? Because the root cause seems to be that the installer script does not add the script to the path directly, and was most often relying on ~/.cargo/bin being already in the PATH. That's the case for github actions runners as well as codspeed-macro runners, but it appears not to be on your self hosted runners.

As far as I was able to dig, it has never worked on your self hosted runners since you made the switch here open-telemetry/opentelemetry-go@e413dfc
If that's not the case then what I am talking about is not the actual issue 😬

f5e5fb3fb96b1855449c231a9d69e8c4bcba227a added to #193 should fix the problem, are you able to test again?

Sorry to bother you again 👼

[XSAM]

has the codspeed action ever worked on your self hosted runner?

No, it hasn't worked for once.

f5e5fb3 added to #193 should fix the problem, are you able to test again?

Just tested it, it failed on a new error

Installer hash verified for version 4.11.0
Run # Validate required inputs

   ______            __ _____                         __
  / ____/____   ____/ // ___/ ____   ___   ___   ____/ /
 / /    / __ \ / __  / \__ \ / __ \ / _ \ / _ \ / __  /
/ /___ / /_/ // /_/ / ___/ // /_/ //  __//  __// /_/ /
\____/ \____/ \__,_/ /____// .___/ \___/ \___/ \__,_/
  https://codspeed.io     /_/          runner v4.11.0

Running the benchmarks
  systemd-run: unrecognized option '--same-dir'
  Error: Error: failed to execute the benchmark process: exit status: 1
  Error: Process completed with exit code 1.

https://github.com/open-telemetry/opentelemetry-go/actions/runs/22682530224/job/65756627866

Sorry to bother you again 👼

Thanks for the rapid support! ❤️

[GuillaumeLagrange]

Okay now the whole installation works so we are going a bit out of scope of the action itself and more into requirements for https://github.com/CodSpeedHQ/codspeed

This particular error is because systemd-run relies on the --same-dir which was introduced in systemd 240
This version was released toward the end of 2018, and we currently don't have the workforce to try and circumvent this. Do you have the possibility to use a more recent version of linux on your self-hosted runners?

[XSAM]

@GuillaumeLagrange, we are now using Ubuntu 24.04 as the image, but encountered new errors.

• No make command is found. Is that normal to make sure make is available in a github action?

  I use a quick fix here by just executing sudo apt-get update && sudo apt-get install -y make

• New error when running codspeed

Running the benchmarks
  cd ./bridge/opencensus && go test -run='^$' -bench=. -benchtime=500ms ./...
  warning: flag '-run=^$' is not supported by CodSpeed Go runner, ignoring
  go: downloading go.opencensus.io v0.24.0
  go: downloading github.com/stretchr/testify v1.11.1
  go: downloading github.com/go-logr/logr v1.4.3
  go: downloading github.com/cespare/xxhash/v2 v2.3.0
  go: downloading go.opentelemetry.io/auto/sdk v1.2.1
  go: downloading github.com/go-logr/stdr v1.2.2
  go: downloading github.com/google/uuid v1.6.0
  go: downloading golang.org/x/sys v0.41.0
  go: downloading github.com/davecgh/go-spew v1.1.1
  go: downloading github.com/pmezard/go-difflib v1.0.0
  go: downloading gopkg.in/yaml.v3 v3.0.1
  go: downloading github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8
  WORK=/tmp/go-build367140737
  # testing
  Error: /tmp/.tmpIEK36N/codspeed.go:17:20: undefined: InstrumentHooks
  Error: /tmp/.tmpIEK36N/benchmark.go:142:22: undefined: CurrentTimestamp
  Error: /tmp/.tmpIEK36N/benchmark.go:147:18: undefined: CurrentTimestamp
  Error: /tmp/.tmpIEK36N/benchmark.go:174:22: undefined: CurrentTimestamp
  Error: /tmp/.tmpIEK36N/benchmark.go:464:19: undefined: CurrentTimestamp
  Error: /tmp/.tmpIEK36N/benchmark.go:747:22: undefined: NewInstrumentHooks
  FAIL	go.opentelemetry.io/otel/bridge/opencensus [build failed]
  FAIL	go.opentelemetry.io/otel/bridge/opencensus/internal [build failed]
  FAIL	go.opentelemetry.io/otel/bridge/opencensus/internal/oc2otel [build failed]
  FAIL	go.opentelemetry.io/otel/bridge/opencensus/internal/ocmetric [build failed]
  FAIL	go.opentelemetry.io/otel/bridge/opencensus/internal/otel2oc [build failed]
  FAIL
  [WARN  codspeed_go_runner::runner] Command output: 
  [WARN  codspeed_go_runner::runner] Command error output: 
  Error: Failed to run benchmarks: Failed to run benchmark. Exit status: exit status: 1
  
  Stdout:
  
  
  Stderr:
  
  make: *** [Makefile:188: benchmark/./bridge/opencensus] Error 1
  Error: Error: failed to execute the benchmark process: exit status: 2
  Error: Process completed with exit code 1.

https://github.com/open-telemetry/opentelemetry-go/actions/runs/22695372512/job/65800569749?pr=7998

I think you could pull the docker run --rm -it ubuntu:24.04, and see if this action is runnable under that image.

[GuillaumeLagrange]

@XSAM I've created a dedicated issue for codspeed-go, as it's no longer related to the action.

@not-matthias will be better off to help you out on this

[GuillaumeLagrange]

@XSAM As for the make issue, since the make usage comes from https://github.com/open-telemetry/opentelemetry-go/actions/runs/22695372512/workflow?pr=7998#L55, codspeed itself does not expect make to be available, it's just a matter of your self hosted runner configuration