Hello!
When I run the quick demo:
cd SeqDex/dexteroushand
python train_rlgames.py --task BlockAssemblyGUI --num_envs=1 --play
there has an error:
num_object_shapes: 958 num_object_bodies: 138 num_object_shapes: 959 num_object_bodies: 139
num_object_shapes: 960 num_object_bodies: 140 num_object_shapes: 961 num_object_bodies: 141
num_object_shapes: 962 num_object_bodies: 142 num_object_shapes: 963 num_object_bodies: 143
num_object_shapes: 964 num_object_bodies: 144 num_object_shapes: 965 num_object_bodies: 145
num_object_shapes: 966 num_object_bodies: 146 Started convex decomposition for mesh '../assets/urdf/blender/assets_for_insertion/urdf/../origin_obj/8x8x1_real/8x8x1_real.stl' Finished convex decomposition for mesh '../assets/urdf/blender/assets_for_insertion/urdf/../origin_obj/8x8x1_real/8x8x1_real.stl': 183 hulls num_extra_lego_asset_shapes: 1149 num_extra_lego_asset_bodies: 147 [Warning] [carb.gym.plugin] User aggregate size exceeded [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on rgbImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on depthImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on segmentationImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on optical flow buffer with error 101 *** Can't create empty tensor *** Can't create empty tensor mount_rigid_body_index: 7 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on rgbImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on depthImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on segmentationImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on optical flow buffer with error 101 *** Can't create empty tensor [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on rgbImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on depthImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on segmentationImage buffer with error 101 [Error] [carb.gym.plugin] cudaExternamMemoryGetMappedBuffer failed on optical flow buffer with error 101 *** Can't create empty tensor *** Can't create empty tensor Traceback (most recent call last): File "train_rlgames.py", line 75, in <module> task, env = parse_task(args, cfg, cfg_train, sim_params, agent_index) File "/home/yuhai/SeqDex/dexteroushandenvs/utils/parse_task.py", line 166, in parse_task task = eval(args.task)( File "/home/yuhai/SeqDex/dexteroushandenvs/tasks/block_assembly/allegro_hand_block_assembly_gui.py", line 187, in __init__ super().__init__(cfg=self.cfg, enable_camera_sensors=self.enable_camera_sensors) File
"/home/yuhai/SeqDex/dexteroushandenvs/tasks/hand_base/base_task.py", line 83, in __init__ self.create_sim() File
"/home/yuhai/SeqDex/dexteroushandenvs/tasks/block_assembly/allegro_hand_block_assembly_gui.py", line 498, in create_sim self._create_envs(self.num_envs, self.cfg["env"]['envSpacing'], int(np.sqrt(self.num_envs))) File
"/home/yuhai/SeqDex/dexteroushandenvs/tasks/block_assembly/allegro_hand_block_assembly_gui.py", line 1301, in _create_envs self.camera_rgbd_image_tensors = torch.stack(self.camera_tensors,
dim=0).view(self.num_envs, -1) TypeError: expected Tensor as element 0 in argument 0, but got NoneType
My machine info is there:
I asked ChatGPT,
and here is the answer
User aggregate size exceeded: This warning suggests that the simulation is attempting to allocate more resources than allowed or available. This might be a configuration issue or a limitation of the system.
Can't create empty tensor: This error is likely a result of the previous CUDA errors. Since the GPU memory buffers could not be created, the tensors that are supposed to hold this data cannot be formed, hence the "empty tensor" error.
TypeError: The final error in the traceback shows that there was an attempt to stack a list of tensors, but at least one element in the list was None instead of a tensor object. This is consistent with the previous errors because if the GPU buffers failed to initialize, then the expected tensors would not be created, leading to this error when attempting to stack them.```
To address these errors, consider the following steps:
```Check System Resources: Ensure that your system has enough resources (RAM, GPU memory) to allocate for the simulation.
Review Configuration: Look into the configuration of the simulation (in this case, the cfg object) to ensure that the resource limits are set correctly.
Update GPU Drivers: Make sure your GPU drivers are up to date and compatible with the version of CUDA you're using.
CUDA Compatibility: Verify that the GPU is compatible with the CUDA version installed and that PyTorch (or whichever framework you're using) supports that CUDA version.
Restart the System: Sometimes, simply restarting the system can clear up any issues with GPU memory allocation.
Error Handling: Modify the code to handle the case where tensors could not be created, potentially by adding checks before trying to stack them.```
Hello!
When I run the quick demo:
cd SeqDex/dexteroushandpython train_rlgames.py --task BlockAssemblyGUI --num_envs=1 --playthere has an error:
My machine info is there:
I asked ChatGPT,
and here is the answer
User aggregate size exceeded: This warning suggests that the simulation is attempting to allocate more resources than allowed or available. This might be a configuration issue or a limitation of the system.