Skip to content

testing.md

Latest commit

 

History

History
63 lines (47 loc) · 2.24 KB

testing.md

File metadata and controls

63 lines (47 loc) · 2.24 KB
Raw

Testing

The open-source snapshot keeps the test programs while limiting the built-in runtime implementation to the reference backend.

Reference unit tests

Unit tests require an existing LibTorch installation. Configure its CMake package directory explicitly:

cmake -S . -B build \
  -DSHL_BUILD_TESTS=ON \
  -DLIBTORCH_PATH=/path/to/libtorch/share/cmake/Torch
cmake --build build --target shl_unit_testcases
ctest --test-dir build --output-on-failure

GoogleTest is downloaded from its public upstream repository unless FETCHCONTENT_SOURCE_DIR_GOOGLETEST points to a local checkout.

RISC-V QEMU unit tests

With a RISC-V LibTorch distribution, the portable GoogleTest suite can be cross-compiled unchanged. QEMU discovery has a 60-second default timeout and registers the same 1824 CTest cases as the x86 build under the riscv label.

cmake -S . -B build-riscv \
  -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain/riscv64-linux-gnu.cmake \
  -DSHL_BUILD_TESTS=ON \
  -DSHL_BUILD_QEMU_TESTS=OFF \
  -DSHL_BUILD_BENCHMARKS=OFF \
  -DLIBTORCH_PATH=/path/to/riscv64/libtorch
cmake --build build-riscv --target shl_qemu_unit_tests -j

shl_qemu_unit_tests runs all 1824 GoogleTest cases in one QEMU process. Individual cases remain available through CTest, for example ctest --test-dir build-riscv -L riscv -R MatMul --output-on-failure. Set SHL_GTEST_DISCOVERY_TIMEOUT if a slower emulator needs more than 60 seconds to enumerate the suite.

The RISC-V-specific NC1HWC0 layout tests are outside the portable reference parity set and remain opt-in through SHL_BUILD_RISCV_LAYOUT_TESTS=ON.

For a lightweight ReLU check without LibTorch, configure SHL_BUILD_TESTS=OFF and SHL_BUILD_QEMU_TESTS=ON, or run make test_riscv_qemu_smoke.

The toolchain accepts SHL_RISCV_ARCH and SHL_RISCV_ABI overrides; their defaults are rv64gc and lp64d.

Performance tests

Enable the benchmark programs with SHL_BUILD_BENCHMARKS=ON. Google Benchmark is downloaded from its public upstream repository unless FETCHCONTENT_SOURCE_DIR_BENCHMARK points to a local checkout.

Backend conformance sources

The smoke and bare-metal test sources are included for external backend SDK integration. They are disabled by default and require the backend headers,