use device libs from AMDGPU_LLVM_Backend_jll (2)

[simeonschaub]

We downgrade these into a scratch space to be compatible with Julia's LLVM version

ref #919 (comment)

Reopened to run MI300 CI

[github-actions]

AMDGPU.jl Benchmarks

Details

Benchmark suite	Current: 115cd50	Previous: 9c34f3b	Ratio
amdgpu/synchronization/context/device	562.5 ns	550 ns	1.02
amdgpu/synchronization/stream/blocking	235 ns	232.5 ns	1.01
amdgpu/synchronization/stream/nonblocking	315 ns	310 ns	1.02
array/accumulate/Float32/1d	78078.75 ns	77941 ns	1.00
array/accumulate/Float32/dims=1	283249 ns	284706 ns	0.99
array/accumulate/Float32/dims=1L	88236.25 ns	80633.5 ns	1.09
array/accumulate/Float32/dims=2	81883.5 ns	81153.5 ns	1.01
array/accumulate/Float32/dims=2L	2756813.75 ns	2612203.75 ns	1.06
array/accumulate/Int64/1d	80173.75 ns	80043.5 ns	1.00
array/accumulate/Int64/dims=1	244663.5 ns	244500.5 ns	1.00
array/accumulate/Int64/dims=1L	84083.75 ns	84651 ns	0.99
array/accumulate/Int64/dims=2	86461 ns	86998.75 ns	0.99
array/accumulate/Int64/dims=2L	2893023 ns	2892407.5 ns	1.00
array/broadcast	72958.5 ns	73931 ns	0.99
array/construct	2165 ns	2442.75 ns	0.89
array/copy	36785.5 ns	37490.5 ns	0.98
array/copyto!/cpu_to_gpu	111294.25 ns	111364 ns	1.00
array/copyto!/gpu_to_cpu	111399.25 ns	111931.5 ns	1.00
array/copyto!/gpu_to_gpu	58768.25 ns	59278.25 ns	0.99
array/iteration/findall/bool	138536.75 ns	139074.25 ns	1.00
array/iteration/findall/int	150217.25 ns	147846.75 ns	1.02
array/iteration/findfirst/bool	147159.75 ns	146122 ns	1.01
array/iteration/findfirst/int	145212 ns	145421.75 ns	1.00
array/iteration/findmin/1d	124307 ns	122144 ns	1.02
array/iteration/findmin/2d	114374.25 ns	112986.5 ns	1.01
array/iteration/logical	255541 ns	249995.5 ns	1.02
array/iteration/scalar	294824.25 ns	307011.25 ns	0.96
array/permutedims/2d	71676 ns	72293.25 ns	0.99
array/permutedims/3d	71146 ns	71771 ns	0.99
array/permutedims/4d	73718.5 ns	74601 ns	0.99
array/random/rand/Float32	45298.25 ns	45865.5 ns	0.99
array/random/rand/Int64	54170.75 ns	54655.5 ns	0.99
array/random/rand!/Float32	64990.75 ns	65905.75 ns	0.99
array/random/rand!/Int64	73331 ns	73723.25 ns	0.99
array/random/randn/Float32	80443.75 ns	80023.5 ns	1.01
array/random/randn!/Float32	59398.5 ns	81618.5 ns	0.73
array/reductions/mapreduce/Float32/1d	103296.5 ns	103161.25 ns	1.00
array/reductions/mapreduce/Float32/dims=1	94573.75 ns	89753.5 ns	1.05
array/reductions/mapreduce/Float32/dims=1L	831839.25 ns	832490 ns	1.00
array/reductions/mapreduce/Float32/dims=2	88488.75 ns	88768.75 ns	1.00
array/reductions/mapreduce/Float32/dims=2L	144047 ns	145439.25 ns	0.99
array/reductions/mapreduce/Int64/1d	102239 ns	101651 ns	1.01
array/reductions/mapreduce/Int64/dims=1	90081.25 ns	91316.25 ns	0.99
array/reductions/mapreduce/Int64/dims=1L	829916.75 ns	832435 ns	1.00
array/reductions/mapreduce/Int64/dims=2	88343.75 ns	88383.75 ns	1.00
array/reductions/mapreduce/Int64/dims=2L	145024.75 ns	145499.25 ns	1.00
array/reductions/reduce/Float32/1d	102609 ns	102053.75 ns	1.01
array/reductions/reduce/Float32/dims=1	92676.25 ns	91916 ns	1.01
array/reductions/reduce/Float32/dims=1L	834781.75 ns	829267.5 ns	1.01
array/reductions/reduce/Float32/dims=2	88593.75 ns	88813.5 ns	1.00
array/reductions/reduce/Float32/dims=2L	144014.5 ns	144801.75 ns	0.99
array/reductions/reduce/Int64/1d	103261.5 ns	98088.75 ns	1.05
array/reductions/reduce/Int64/dims=1	91291.5 ns	91691 ns	1.00
array/reductions/reduce/Int64/dims=1L	830606.75 ns	835007.5 ns	0.99
array/reductions/reduce/Int64/dims=2	90051.25 ns	86968.75 ns	1.04
array/reductions/reduce/Int64/dims=2L	145154.5 ns	144809.25 ns	1.00
array/reverse/1d	45768 ns	46398 ns	0.99
array/reverse/1dL	75783.75 ns	76588.5 ns	0.99
array/reverse/1dL_inplace	80543.75 ns	81233.5 ns	0.99
array/reverse/1d_inplace	61116 ns	54388 ns	1.12
array/reverse/2d	49120.75 ns	51085.75 ns	0.96
array/reverse/2dL	86218.75 ns	85028.5 ns	1.01
array/reverse/2dL_inplace	92133.75 ns	92708.5 ns	0.99
array/reverse/2d_inplace	63556 ns	64085.75 ns	0.99
array/sorting/1d	334317 ns	334726.75 ns	1.00
integration/byval/reference	39420 ns	39790 ns	0.99
integration/byval/slices=1	40120 ns	40660 ns	0.99
integration/byval/slices=2	146622 ns	154532 ns	0.95
integration/byval/slices=3	240104 ns	242793 ns	0.99
integration/volumerhs	5013470 ns	4999260 ns	1.00
kernel/indexing	54615.75 ns	40125.5 ns	1.36
kernel/indexing_checked	59621 ns	59693.25 ns	1.00
kernel/launch	1470 ns	1390 ns	1.06
kernel/rand	101656.25 ns	82931 ns	1.23
latency/import	1703196175 ns	1686756300 ns	1.01
latency/precompile	39123387667 ns	38963838277 ns	1.00
latency/ttfp	2339594037 ns	2310415827 ns	1.01

This comment was automatically generated by workflow using github-action-benchmark.

[simeonschaub]

OK, I think this should be good to go now

[luraess]

Running downgrade_device_libs against the real JLLs on 1.10/1.11 (target 14) and 1.12/1.13 (target 18) — all 70 .bc files convert and re-parse, ~0.4 s one-time. A few things that may still need attention (robot helped):

1. The LLVMDowngrader_jll = "0.9.1" floor is undocumented. With 0.9.0, asanrtl.bc fails at target 14:

LLVM ERROR: pointers in constant aggregates are not yet supported by the IR downgrader

run() throws, which aborts the whole loop, so 0.9.0 silently drops the entire artifact path on 1.10/1.11. Worth a comment so the bound doesn't get relaxed later?

2. AMDGPU only links hc/hip/irif/ockl/opencl/ocml plus the oclc_* control libs. asanrtl.bc is in neither ondemand_libs() nor control_libs(). Skipping files that fail (or only converting the ones actually used) makes this robust against the next downgrader gap instead of dependent on a compat floor.

3. isdefined(AMDGPU_LLVM_Backend_jll, :bitcode_path) is doing more work than it looks. 22.1.8**+0** exports only llc, lld; bitcode_path first appears in +1. Since compat = "22.1.8" admits +0, that check is the only thing between a +0 resolve and a silent fallback. Is a (comment or) a tighter bound needed?

4. Quiet fallback? from_artifact is unconditionally true, so on failure the user gets a warning and then libdevice_libs == "" on any host without a system ROCm, surfacing much later as unsupported call to __ocml_*. Also: @get_scratch! writes to the depot from __init__, so a read-only shared depot (common on HPC, relevant to the CSCS MI300 target) fails on every load. Maybe have the warning name the consequence when find_device_libs also comes back empty.

Additional: v"5"/v"7" are unreachable (floor is 1.10 = LLVM 15) and fail anyway on llvm.amdgcn.implicitarg.ptr; maximum would throw on an empty iterator if libllvm_version < 5. And the marker + per-file mv leaves a small window on 1.10/1.11, where mv(force=true) does rm(dst) then rename rather than renaming straight over (fine on >= 1.12). A Pidfile.mkpidlock could close it.

[luraess]

Seems mostly good. I left a few comments while checking things also with the robot's help.

[simeonschaub]

Addressed points 2 and 4 as well as the one regarding the LLVM 5 and 7 paths being unreachable.

Re 1, I think just the compat entry is enough, I don't know why we would ever accidentally relax it. I think 3 is fine too, since we just fall back to the old behavior if the user for some reason is still on the old backend jll. I don't think Pkg understands compat entries depending on the build number

[luraess]

Agree. Thanks!