Skip to content

Commit fa237db

Browse files
Christian Schmittelchristian-wr
authored andcommitted
feat(build): build the whisper helper natively for Windows on ARM
18x faster than the emulated x64 helper, and the only one that produces a usable transcript at all. Same 60 s of German speech, same ggml-small-q8_0 model, both binaries driven directly over /inference: native arm64 (ggml threads) load 0.9s inference 10.1s 9059 chars, language "de" emulated x64 (Vulkan) load 1.3s inference 183.3s failed, 0 chars The emulated run is not merely slow. It returns nothing after three minutes, and in-app it reported "auto-detected language: th" on German speech — the same shape of failure as the D3D11 preview drawing black on this adapter. Adreno under emulation produces results that look like application bugs. Five things had to change, each a separate x64 assumption: `os_arch_tag` derived the architecture from `uname -m`. Git Bash is an x86_64 build, so on Windows on ARM it reports x86_64 while running emulated, and the script would have built and staged an x64 helper on an ARM64 host without a word. Node is native and knows better. `backend_flag_for_host` had no win32-arm64 case and exited with "Unknown os-arch". ggml refuses MSVC for ARM outright ("MSVC is not supported for ARM, use clang") — its NEON intrinsics are absent from MSVC's ARM64 code generator. The build now uses clang-cl through Ninja rather than the `-T ClangCL` VS toolset, so it reuses the standalone LLVM the compositor addon already requires for bindgen instead of pulling a second ~1 GB copy into Visual Studio. Ninja gets no MSVC environment of its own, so configure and build run inside vcvarsall via a throwaway .cmd — nested quoting through bash, MSYS and cmd does not survive, and the same file needs Windows-native paths because Git Bash stops rewriting them once cmd is the caller. `-DGGML_OPENMP=OFF` is a licence constraint, not a tuning choice. clang-cl links ggml against libomp140.aarch64.dll, and the only copy on a Visual Studio install sits under `VC/Redist/.../debug_nonredist/` — a directory whose name is the licence position. ggml's own thread pool needs no runtime we cannot ship, and at 6x realtime it is plainly fast enough. NOTE for CI: this makes LLVM a build prerequisite for the arm64 branch. build-whisper-stt.yml still has no arm64 matrix entry, so packages keep falling back to the staged x64 helper until it gains one.
1 parent 33e7df2 commit fa237db

1 file changed

Lines changed: 105 additions & 3 deletions

File tree

‎scripts/build-whisper-stt.sh‎

Lines changed: 105 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,14 @@ os_arch_tag() {
6161
Linux:x86_64) os_arch="linux-x64" ;;
6262
Linux:aarch64) os_arch="linux-arm64" ;;
6363
MINGW*|CYGWIN*|MSYS*)
64+
# `uname -m` cannot be trusted here. Git Bash is an x86_64 build, so on Windows
65+
# on ARM it runs emulated and reports x86_64 while the machine is arm64 — the
66+
# script would then build and stage an x64 helper on an ARM64 host without ever
67+
# saying so. Node is a native binary and reports the real architecture, and this
68+
# is a Node project, so it is always at hand. `uname -m` stays as the fallback.
6469
local arch
65-
arch="$(uname -m)"
70+
arch="$(node -p 'process.arch' 2>/dev/null || true)"
71+
[[ -n "${arch}" ]] || arch="$(uname -m)"
6672
os_arch="win32-${arch/x86_64/x64}"
6773
;;
6874
*) echo "Unsupported host: $(uname -s):$(uname -m)" >&2; exit 1 ;;
@@ -84,6 +90,19 @@ backend_flag_for_host() {
8490
darwin-arm64) echo "-DOSC_ENABLE_METAL=ON" ;;
8591
darwin-x64) echo "" ;;
8692
win32-x64|linux-x64|linux-arm64) echo "-DOSC_ENABLE_VULKAN=ON" ;;
93+
# Windows on ARM: CPU (ARM NEON), not Vulkan. ggml's Vulkan backend needs glslc
94+
# from the Vulkan SDK to compile its shaders, and requiring a ~1 GB SDK install
95+
# would make the arm64 helper unbuildable on a stock machine. Adreno itself is a
96+
# capable Vulkan target — the emulated x64 helper already runs inference on it —
97+
# so switching this to Vulkan is a worthwhile follow-up once the SDK is a declared
98+
# build prerequisite. Measure before assuming it wins: native NEON on 12 Oryon
99+
# cores against emulated code driving the GPU is not an obvious comparison.
100+
# `-DGGML_OPENMP=OFF` is not optional here. clang-cl links ggml against
101+
# `libomp140.aarch64.dll`, and the only copy of that file on a Visual Studio
102+
# install lives under `VC/Redist/.../debug_nonredist/` — a directory whose name
103+
# states the licence position: it may not be redistributed. Without OpenMP,
104+
# ggml uses its own thread pool and the helper needs no runtime we cannot ship.
105+
win32-arm64) echo "-DGGML_OPENMP=OFF" ;;
87106
*) echo "Unknown os-arch: ${OS_ARCH}" >&2; exit 1 ;;
88107
esac
89108
}
@@ -141,6 +160,62 @@ relocate_macos_rpaths() {
141160
echo "[whisper-stt] rewrote macOS rpaths to @loader_path in ${out_dir}"
142161
}
143162

163+
# Runs a command inside the MSVC environment for the host/target pair.
164+
#
165+
# Only Windows on ARM needs this. Everywhere else the Visual Studio generator sets the
166+
# environment up itself, and Unix hosts have no such notion — so this is a no-op unless
167+
# a vcvarsall is both needed and found, and a plain `"$@"` otherwise.
168+
run_in_vs_env() {
169+
if [[ "${OS_ARCH}" != "win32-arm64" ]]; then
170+
"$@"
171+
return
172+
fi
173+
local vcvars
174+
vcvars="$(ls "/c/Program Files (x86)/Microsoft Visual Studio/2022"/*/VC/Auxiliary/Build/vcvarsall.bat 2>/dev/null | head -1)"
175+
if [[ -z "${vcvars}" ]]; then
176+
echo "[whisper-stt] WARN: vcvarsall.bat not found; building without an MSVC environment" >&2
177+
"$@"
178+
return
179+
fi
180+
# Host arch decides the vcvars argument: `arm64` when building natively on ARM64,
181+
# `amd64_arm64` when cross-compiling from an x64 host such as a CI runner.
182+
local host vcarg
183+
host="$(node -p 'process.arch' 2>/dev/null || echo x64)"
184+
if [[ "${host}" == "arm64" ]]; then vcarg="arm64"; else vcarg="amd64_arm64"; fi
185+
# A throwaway .cmd rather than nested quoting. `cmd //c "call \"...\" && ..."`
186+
# has to survive bash, MSYS path mangling and cmd in turn, and it did not: the
187+
# escaped quotes reached cmd verbatim and it reported the batch file "not found".
188+
# scripts/build-windows-compositor-addon.mjs solves it the same way.
189+
# Where the standalone LLVM lives. `command -v` first so a PATH install wins;
190+
# the default installer location is the fallback.
191+
local llvm_bin=""
192+
if command -v clang-cl >/dev/null 2>&1; then
193+
llvm_bin="$(dirname "$(command -v clang-cl)")"
194+
elif [[ -x "/c/Program Files/LLVM/bin/clang-cl.exe" ]]; then
195+
llvm_bin="/c/Program Files/LLVM/bin"
196+
fi
197+
local script
198+
script="$(mktemp -t whisper-vsenv-XXXXXX.cmd)"
199+
{
200+
echo "@echo off"
201+
printf 'call "%s" %s || exit /b 1
202+
' "$(cygpath -w "${vcvars}")" "${vcarg}"
203+
# vcvarsall does not add a standalone LLVM to PATH, and CMake then fails to find
204+
# clang-cl at all. Prepend it so the compiler ggml demands for ARM is visible
205+
# without turning its absolute path (which contains spaces) into another quoting
206+
# problem inside the batch file.
207+
if [[ -n "${llvm_bin}" ]]; then
208+
printf 'set "PATH=%s;%%PATH%%"
209+
' "$(cygpath -w "${llvm_bin}")"
210+
fi
211+
echo "$*"
212+
} > "${script}"
213+
local status=0
214+
cmd //c "$(cygpath -w "${script}")" || status=$?
215+
rm -f "${script}"
216+
return ${status}
217+
}
218+
144219
build_variant() {
145220
local variant_name="$1"
146221
shift
@@ -158,14 +233,41 @@ build_variant() {
158233
# variable") even though bash 4+ treats it as zero words. The
159234
# `${arr[@]+"${arr[@]}"}` idiom expands to nothing when the array is empty
160235
# and to the normal quoted expansion otherwise, on both bash versions.
161-
cmake -S "${SRC_DIR}" -B "${build_dir}" \
236+
# Windows on ARM must be built with clang, not MSVC: ggmls CPU backend refuses
237+
# outright ("MSVC is not supported for ARM, use clang" in
238+
# ggml/src/ggml-cpu/CMakeLists.txt), because the NEON intrinsics it relies on are
239+
# absent from MSVCs ARM64 code generator. The Visual Studio generator can host the
240+
# LLVM toolset, so the target platform stays ARM64 and only the compiler changes.
241+
local toolchain_flags=()
242+
if [[ "${OS_ARCH}" == "win32-arm64" ]]; then
243+
# clang-cl from a standalone LLVM, driven by Ninja rather than the Visual Studio
244+
# generator. `-T ClangCL` would need the "C++ Clang tools for Windows" VS
245+
# component, which is a second ~1 GB LLVM next to the one the compositor addon
246+
# already requires for bindgen; Ninja lets both builds share one toolchain.
247+
# Ninja gets no MSVC environment of its own, so configure and build run inside
248+
# vcvarsall (see run_in_vs_env below) for the headers, libs and the linker.
249+
toolchain_flags+=(-G Ninja -DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl)
250+
fi
251+
252+
# Paths must be Windows-native for the arm64 path: cmake runs from inside a .cmd,
253+
# where Git Bash no longer rewrites `/c/...` for the native program it invokes, and
254+
# CMake reports the source directory as missing. Everywhere else the MSYS form is
255+
# what the surrounding tooling expects, so only this branch converts.
256+
local src_arg="${SRC_DIR}" build_arg="${build_dir}"
257+
if [[ "${OS_ARCH}" == "win32-arm64" ]]; then
258+
src_arg="$(cygpath -w "${SRC_DIR}")"
259+
build_arg="$(cygpath -w "${build_dir}")"
260+
fi
261+
262+
run_in_vs_env cmake -S "${src_arg}" -B "${build_arg}" \
162263
-DCMAKE_BUILD_TYPE=Release \
264+
${toolchain_flags[@]+"${toolchain_flags[@]}"} \
163265
${extra_cmake_flags[@]+"${extra_cmake_flags[@]}"}
164266

165267
echo "[whisper-stt] building ${variant_name}"
166268
# ponytail: macOS has no `nproc` (it is GNU coreutils), so this silently built
167269
# with -j4 on an 8-core Mac. `sysctl -n hw.ncpu` is the BSD equivalent.
168-
cmake --build "${build_dir}" --config Release \
270+
run_in_vs_env cmake --build "${build_arg}" --config Release \
169271
-j "$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)"
170272

171273
local bin_name="whisper-stt-server"

0 commit comments

Comments
 (0)