Add hostcalls for device-to-host services

[maleadt]

This PR adds a hostcall foundation that lets CUDA kernels request work from Julia on the host and optionally receive a result.

The first consumer is device exception reporting, replacing limited printf diagnostics with structured exception names, reasons, and stack traces. The same foundation can later support features such as dynamic allocation and I/O without introducing separate device-to-host protocols.

Design

Hostcalls use warp-collective mailboxes in mapped host memory. A dedicated adopted thread services requests independently of Julia’s thread pools, allowing calls to progress while other threads are blocked in CUDA.

The compiler records statically known targets alongside cached kernel images and registers them when linking. Runtime HostFunction handles support closures and dynamically selected operations. Blocking, asynchronous, large-value, graph-replay, and error paths share the same protocol.

Built-in, static, and runtime target identifiers occupy separate ranges, collisions are detected, and runtime handles have an explicit lifetime. Handler and kernel failures are attributed to their CUDA context and reported at the next stream, event, or device synchronization.

Platform support

Hostcalls are enabled by default on Windows, including WDDM. Command submission is flushed before polling, and the server uses a portable idle backoff. Kernels remain subject to the platform’s normal display watchdog, so handlers should avoid long or unbounded waits.

Multiple CUDA contexts and devices share one hostcall server. Requests are serviced in their originating context, exceptions are reported to the correct device, and context teardown releases associated hostcall and exception resources.

The device protocol supports pre-Volta GPUs using volatile mailbox accesses and legacy memory barriers, while Volta and newer use scoped memory operations. Both paths are covered by architecture-specific code-generation tests.

Scope and constraints

Hostcalls are intended for uncommon control paths rather than bulk data transfer. Handlers must not wait on the calling kernel, perform nested hostcalls, load new kernels, or depend on Julia task scheduling or libuv I/O. Print-family calls are queued and flushed safely during synchronization.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.65%. Comparing base (f1c2dcb) to head (5319ac3).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3243      +/-   ##
==========================================
- Coverage   92.66%   92.65%   -0.02%     
==========================================
  Files         184      184              
  Lines       16176    16176              
==========================================
- Hits        14990    14988       -2     
- Misses       1186     1188       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

CUDA.jl Benchmarks

Details

Benchmark suite	Current: 7942a64	Previous: f1c2dcb	Ratio
array/accumulate/Float32/1d	98680 ns	97757 ns	1.01
array/accumulate/Float32/dims=1	72315 ns	72251 ns	1.00
array/accumulate/Float32/dims=1L	1599935 ns	1599352 ns	1.00
array/accumulate/Float32/dims=2	137456 ns	137934 ns	1.00
array/accumulate/Float32/dims=2L	660282 ns	659774 ns	1.00
array/accumulate/Int64/1d	118373 ns	117850 ns	1.00
array/accumulate/Int64/dims=1	75741 ns	76396 ns	0.99
array/accumulate/Int64/dims=1L	1718311 ns	1715658 ns	1.00
array/accumulate/Int64/dims=2	150892 ns	148676 ns	1.01
array/accumulate/Int64/dims=2L	986329 ns	987161 ns	1.00
array/broadcast	17788 ns	17712 ns	1.00
array/broadcast launch	9000 ns	8751.333333333334 ns	1.03
array/construct	900.3809523809524 ns	874.6607142857143 ns	1.03
array/copy	17124 ns	16385 ns	1.05
array/copyto!/cpu_to_gpu	208834 ns	208209 ns	1.00
array/copyto!/gpu_to_cpu	241474 ns	240797 ns	1.00
array/copyto!/gpu_to_gpu	10329 ns	8958.666666666666 ns	1.15
array/iteration/findall/bool	133181 ns	132638 ns	1.00
array/iteration/findall/int	146452 ns	146498 ns	1.00
array/iteration/findfirst/bool	69935 ns	68518 ns	1.02
array/iteration/findfirst/int	70526 ns	70195 ns	1.00
array/iteration/findmin/1d	66088 ns	64408 ns	1.03
array/iteration/findmin/2d	131692 ns	99376 ns	1.33
array/iteration/logical	192358 ns	187866 ns	1.02
array/iteration/scalar	63465 ns	62527 ns	1.02
array/permutedims/2d	48870 ns	48266 ns	1.01
array/permutedims/3d	49877 ns	50042 ns	1.00
array/permutedims/4d	50036 ns	49777 ns	1.01
array/random/rand/Float32	11113 ns	11710 ns	0.95
array/random/rand/Int64	20631 ns	22484 ns	0.92
array/random/rand!/Float32	7774 ns	7826.75 ns	0.99
array/random/rand!/Int64	20591 ns	19938 ns	1.03
array/random/randn/Float32	33830 ns	32948 ns	1.03
array/random/randn!/Float32	23060 ns	23411 ns	0.99
array/reductions/mapreduce/Float32/1d	32934 ns	32707 ns	1.01
array/reductions/mapreduce/Float32/dims=1	57263 ns	37678 ns	1.52
array/reductions/mapreduce/Float32/dims=1L	50572 ns	50368 ns	1.00
array/reductions/mapreduce/Float32/dims=2	56748 ns	54900 ns	1.03
array/reductions/mapreduce/Float32/dims=2L	75046 ns	67068 ns	1.12
array/reductions/mapreduce/Int64/1d	39466 ns	39790 ns	0.99
array/reductions/mapreduce/Int64/dims=1	62807 ns	40639 ns	1.55
array/reductions/mapreduce/Int64/dims=1L	88674 ns	88392 ns	1.00
array/reductions/mapreduce/Int64/dims=2	60581 ns	57524 ns	1.05
array/reductions/mapreduce/Int64/dims=2L	90304 ns	82933 ns	1.09
array/reductions/reduce/Float32/1d	32952 ns	32599 ns	1.01
array/reductions/reduce/Float32/dims=1	56591 ns	37807 ns	1.50
array/reductions/reduce/Float32/dims=1L	50559 ns	50239 ns	1.01
array/reductions/reduce/Float32/dims=2	58552 ns	55233 ns	1.06
array/reductions/reduce/Float32/dims=2L	75139 ns	68967 ns	1.09
array/reductions/reduce/Int64/1d	38837 ns	39454 ns	0.98
array/reductions/reduce/Int64/dims=1	62922 ns	40134 ns	1.57
array/reductions/reduce/Int64/dims=1L	88536 ns	88225 ns	1.00
array/reductions/reduce/Int64/dims=2	60581 ns	57175 ns	1.06
array/reductions/reduce/Int64/dims=2L	90651 ns	82630 ns	1.10
array/reverse/1d	17065 ns	16492 ns	1.03
array/reverse/1dL	69786 ns	69275 ns	1.01
array/reverse/1dL_inplace	67574 ns	67321 ns	1.00
array/reverse/1d_inplace	8629 ns	10129.333333333334 ns	0.85
array/reverse/2d	19375 ns	20001 ns	0.97
array/reverse/2dL	73495 ns	73440 ns	1.00
array/reverse/2dL_inplace	67222 ns	67081 ns	1.00
array/reverse/2d_inplace	9900 ns	10270 ns	0.96
array/sorting/1d	2665596 ns	2656268 ns	1.00
array/sorting/2d	1033553 ns	1029271 ns	1.00
array/sorting/by	3216393 ns	3181708 ns	1.01
cuda/synchronization/context/auto	1096 ns	1011.2 ns	1.08
cuda/synchronization/context/blocking	861.1267605633802 ns	775.875 ns	1.11
cuda/synchronization/context/nonblocking	6922.5 ns	5740.5 ns	1.21
cuda/synchronization/stream/auto	926.0625 ns	856.6833333333333 ns	1.08
cuda/synchronization/stream/blocking	719.2121212121212 ns	655.3548387096774 ns	1.10
cuda/synchronization/stream/nonblocking	6833.4 ns	5544 ns	1.23
hostcall/async_1warp	18519 ns
hostcall/async_512warps	3883013 ns
hostcall/blocking_1warp	40154 ns
hostcall/blocking_512warps	4290689 ns
hostcall/launch_armed	3933.8571428571427 ns
integration/byval/reference	147890 ns	147726 ns	1.00
integration/byval/slices=1	149231 ns	148600 ns	1.00
integration/byval/slices=2	292016 ns	291478 ns	1.00
integration/byval/slices=3	435230 ns	434349 ns	1.00
integration/cudadevrt	104933 ns	104975 ns	1.00
integration/volumerhs	9140280 ns	9136526 ns	1.00
kernel/indexing	12964 ns	12801 ns	1.01
kernel/indexing_checked	13861 ns	13502 ns	1.03
kernel/launch	2086.777777777778 ns	2195.5555555555557 ns	0.95
kernel/occupancy	701.5179856115108 ns	820.3666666666667 ns	0.86
kernel/rand	13934 ns	13730 ns	1.01
latency/import	4098694430 ns	4095440314 ns	1.00
latency/precompile	4866413076 ns	4856791188 ns	1.00
latency/ttfp	4575935079 ns	4540682922 ns	1.01

This comment was automatically generated by workflow using github-action-benchmark.