macos-15-intel slow, always 100% CPU?

[haampie]

Description

The macos-15-intel runners seems to be very slow, and I noticed bad scaling running tests in parallel.

Digging a bit into it, I noticed that the load is consistently >1 even when the runner is idle for a while.

It looks like it's always running one process at 100% CPU:

USER               PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND
root             10834  95.2  0.1 33699288  14884   ??  Rs    1:59PM   2:37.55 /usr/libexec/PerfPowerServices

That is /usr/libexec/PerfPowerServices takes 100% CPU all the time, slowing down parallel tests.

Is this expected?

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 22.04
• Ubuntu 24.04
• Ubuntu Slim
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• macOS 26 Arm64
• Windows Server 2019
• Windows Server 2022
• Windows Server 2025

Image version and build link

Is it regression?

Unknown

Expected behavior

Load of 0 when runner is idle.

Actual behavior

Load of 1 when runner is idle.

Repro steps

Start a job.

[haampie]

It seems that the following workaround fixes the 100% CPU issue:

sudo defaults -currentHost write /Library/Preferences/com.apple.powerlogd SMCMonitorCadence 0
sudo killall PerfPowerServices || true

parallel tests run significantly faster afterwards.

[haampie]

Confirmed in spack/spack#51647, the workaround unlocks another CPU core and reduces runtime by 25%.

[erik-bershel]

Hey @haampie!

Yes, this is a known macOS 15 bug since around May – it also occurs on physical machines. We're investigating the possibility of disabling the service for runners, as it should provide a significant performance boost. The main requirement is to avoid disrupting any user groups or resource balancing between virtual machines.
Stay tuned! ❤️

[haampie]

Thanks for confirming. Curious that it wasn't reported yet, at least as far as I can see.

[erik-bershel]

Yes, that's really interesting. This issue is fairly well-known among those who need maximum CPU performance, but there was mention somewhere of one of the other services that causes problems specifically when running tasks for certain simulators:
#13275 (comment)