Remove some allocations from unsafe_wrap(::Type{CuArray{T,N,M}}, p::P…

[ameligrana]

…tr{T}, dims::NTuple{N,Int})

This is better e.g.

julia> cpu = [1,2,3,4,5,6,7,8,9,10,11,12,13];

julia> @time gpu = unsafe_wrap(CuArray, cpu);
  0.000018 seconds (13 allocations: 624 bytes) # this pr
  0.000027 seconds (15 allocations: 688 bytes) # main

[ameligrana]

mmh, the generated function violates indeed some assumptions

[github-actions]

CUDA.jl Benchmarks

Details

Benchmark suite	Current: 4a81f11	Previous: 2e985c1	Ratio
array/accumulate/Float32/1d	97891 ns	97770 ns	1.00
array/accumulate/Float32/dims=1	74816 ns	74735 ns	1.00
array/accumulate/Float32/dims=1L	1600792 ns	1600838 ns	1.00
array/accumulate/Float32/dims=2	139885 ns	140530 ns	1.00
array/accumulate/Float32/dims=2L	659919 ns	659369 ns	1.00
array/accumulate/Int64/1d	117734 ns	117404 ns	1.00
array/accumulate/Int64/dims=1	79001 ns	78778 ns	1.00
array/accumulate/Int64/dims=1L	1717027 ns	1717025 ns	1.00
array/accumulate/Int64/dims=2	151799 ns	153139 ns	0.99
array/accumulate/Int64/dims=2L	986727 ns	986329 ns	1.00
array/broadcast	18321 ns	18263 ns	1.00
array/construct	880.5428571428571 ns	844.3846153846154 ns	1.04
array/copy	15860 ns	15874 ns	1.00
array/copyto!/cpu_to_gpu	208491 ns	206828 ns	1.01
array/copyto!/gpu_to_cpu	240339 ns	239403 ns	1.00
array/copyto!/gpu_to_gpu	8601 ns	8625.666666666666 ns	1.00
array/iteration/findall/bool	131152 ns	131761 ns	1.00
array/iteration/findall/int	145761 ns	145259 ns	1.00
array/iteration/findfirst/bool	67687 ns	67218 ns	1.01
array/iteration/findfirst/int	69827 ns	68333 ns	1.02
array/iteration/findmin/1d	63206 ns	60163 ns	1.05
array/iteration/findmin/2d	99329 ns	98991 ns	1.00
array/iteration/logical	185876 ns	185496 ns	1.00
array/iteration/scalar	61324 ns	60471 ns	1.01
array/permutedims/2d	48200 ns	48292 ns	1.00
array/permutedims/3d	49576 ns	49911 ns	0.99
array/permutedims/4d	49516 ns	49543 ns	1.00
array/random/rand/Float32	11109 ns	11466 ns	0.97
array/random/rand/Int64	20862 ns	22897 ns	0.91
array/random/rand!/Float32	7810.25 ns	7653.25 ns	1.02
array/random/rand!/Int64	17629 ns	20231 ns	0.87
array/random/randn/Float32	32748 ns	32579 ns	1.01
array/random/randn!/Float32	23505 ns	24044 ns	0.98
array/reductions/mapreduce/Float32/1d	31283 ns	30812 ns	1.02
array/reductions/mapreduce/Float32/dims=1	37226 ns	37313 ns	1.00
array/reductions/mapreduce/Float32/dims=1L	49996 ns	49883 ns	1.00
array/reductions/mapreduce/Float32/dims=2	54542 ns	54345 ns	1.00
array/reductions/mapreduce/Float32/dims=2L	66094 ns	65717 ns	1.01
array/reductions/mapreduce/Int64/1d	38665 ns	37981 ns	1.02
array/reductions/mapreduce/Int64/dims=1	39977 ns	39838 ns	1.00
array/reductions/mapreduce/Int64/dims=1L	87882 ns	87598 ns	1.00
array/reductions/mapreduce/Int64/dims=2	56939 ns	56771 ns	1.00
array/reductions/mapreduce/Int64/dims=2L	82588 ns	82005 ns	1.01
array/reductions/reduce/Float32/1d	31404 ns	30827 ns	1.02
array/reductions/reduce/Float32/dims=1	37200 ns	37131 ns	1.00
array/reductions/reduce/Float32/dims=1L	49899 ns	49720 ns	1.00
array/reductions/reduce/Float32/dims=2	54446 ns	54318 ns	1.00
array/reductions/reduce/Float32/dims=2L	67629 ns	67378 ns	1.00
array/reductions/reduce/Int64/1d	38824 ns	38272 ns	1.01
array/reductions/reduce/Int64/dims=1	39992 ns	39599 ns	1.01
array/reductions/reduce/Int64/dims=1L	87984 ns	87650 ns	1.00
array/reductions/reduce/Int64/dims=2	56866 ns	56456 ns	1.01
array/reductions/reduce/Int64/dims=2L	82524 ns	81846 ns	1.01
array/reverse/1d	16393 ns	15888 ns	1.03
array/reverse/1dL	69036 ns	68698 ns	1.00
array/reverse/1dL_inplace	67014 ns	66960 ns	1.00
array/reverse/1d_inplace	9820.333333333334 ns	8217.333333333334 ns	1.20
array/reverse/2d	19244 ns	18875 ns	1.02
array/reverse/2dL	72681 ns	72306 ns	1.01
array/reverse/2dL_inplace	66813 ns	66810 ns	1.00
array/reverse/2d_inplace	9908 ns	9452 ns	1.05
array/sorting/1d	2650032 ns	2648089 ns	1.00
array/sorting/2d	1037940 ns	1037771 ns	1.00
array/sorting/by	3192760 ns	3191059 ns	1.00
cuda/synchronization/context/auto	1043.7 ns	1001.3 ns	1.04
cuda/synchronization/context/blocking	803.019801980198 ns	790.3979591836735 ns	1.02
cuda/synchronization/context/nonblocking	5737.5 ns	5793.833333333333 ns	0.99
cuda/synchronization/stream/auto	877.4909090909091 ns	871.8965517241379 ns	1.01
cuda/synchronization/stream/blocking	674.8354430379746 ns	677.3525641025641 ns	1.00
cuda/synchronization/stream/nonblocking	5554 ns	5626.666666666667 ns	0.99
integration/byval/reference	147397 ns	147313 ns	1.00
integration/byval/slices=1	149334 ns	149365 ns	1.00
integration/byval/slices=2	291920 ns	292154 ns	1.00
integration/byval/slices=3	434729 ns	434710 ns	1.00
integration/cudadevrt	104279 ns	104410 ns	1.00
integration/volumerhs	9306364 ns	9306370 ns	1.00
kernel/indexing	12450 ns	12391 ns	1.00
kernel/indexing_checked	13216 ns	13196 ns	1.00
kernel/launch	2030.111111111111 ns	1998.3 ns	1.02
kernel/occupancy	672.0188679245283 ns	649.280487804878 ns	1.04
kernel/rand	13622 ns	17152 ns	0.79
latency/import	4103713308 ns	4111005294 ns	1.00
latency/precompile	4816872437 ns	4803991823 ns	1.00
latency/ttfp	5534449284 ns	5543883283 ns	1.00

This comment was automatically generated by workflow using github-action-benchmark.

[ameligrana]

ok, fixed it, the fix is now much simpler. This is now concrete, while before it was abstract:

julia> @code_warntype unsafe_wrap(CuArray{Int,1,CUDACore.UnifiedMemory}, cpu)
MethodInstance for unsafe_wrap(::Type{CuArray{Int64, 1, CUDACore.UnifiedMemory}}, ::Vector{Int64})
  from unsafe_wrap(::Type{CuArray{T, N, M}}, a::Array{T, N}) where {T, N, M} @ CUDACore ~/.julia/dev/CUDA/CUDACore/src/array.jl:349
Static Parameters
  T = Int64
  N = 1
  M = CUDACore.UnifiedMemory
Arguments
  #self#::Core.Const(unsafe_wrap)
  _::Core.Const(CuArray{Int64, 1, CUDACore.UnifiedMemory})
  a::Vector{Int64}
Body::CuArray{Int64, 1, CUDACore.UnifiedMemory}
1 ─ %1 = Core.apply_type(CUDACore.CuArray, $(Expr(:static_parameter, 1)), $(Expr(:static_parameter, 2)), $(Expr(:static_parameter, 3)))::Core.Const(CuArray{Int64, 1, CUDACore.UnifiedMemory})
│   %2 = CUDACore.pointer(a)::Ptr{Int64}
│   %3 = CUDACore.size(a)::Tuple{Int64}
│   %4 = CUDACore.unsafe_wrap(%1, %2, %3)::CuArray{Int64, 1, CUDACore.UnifiedMemory}
└──      return %4