Depth Renderer

Python Depth Map Renderer for 3D Bounding Boxes with C++/CUDA Backend

How to Install

You can install Depth Renderer via pip:

pip install git+ssh://git@github.com/Tomoya-Matsubara/depth-renderer

Alternatively, if you use uv as your package manager, you can install it with:

uv add git+ssh://git@github.com/Tomoya-Matsubara/depth-renderer

In some cases, the installation may fail due to a mismatch in CUDA versions between your (virtual) environment and the temporary build environment used during the installation. In such cases, you can try to install with the --no-build-isolation option:

pip install --no-build-isolation \
  git+ssh://git@github.com/Tomoya-Matsubara/depth-renderer

For uv, you can use the following command:

uv add --no-build-isolation git+ssh://git@github.com/Tomoya-Matsubara/depth-renderer

How to Use

You can use Depth Renderer to render depth maps from 3D bounding boxes as follows:

import logging

import torch
from torch import cuda

import depth_renderer

if not cuda.is_available():
    message = (
        "CUDA is not available. "
        "Please run this script on a machine with a CUDA-capable GPU."
    )
    raise RuntimeError(message)


width = 640
height = 480
fx = 500
fy = 500

device = torch.device("cuda")
center = torch.tensor([0.0, 0.0, 3.0], device=device, dtype=torch.float32)
extent = torch.tensor([1.0, 1.0, 1.0], device=device, dtype=torch.float32)

transform_from_world_to_camera = torch.eye(4, device=device, dtype=torch.float32)
transform_from_world_to_ndc = torch.eye(4, device=device, dtype=torch.float32)

foreground_depths, background_depths = depth_renderer.get_bounding_box_depth(
    width=width,
    height=height,
    fx=fx,
    fy=fy,
    center=center,
    extent=extent,
    # NOTE: depth_renderer expects the column-major matrices (transposed matrices)
    # If you have row-major matrices, transpose them before passing
    transform_from_world_to_ndc=transform_from_world_to_ndc.T,
    transform_from_world_to_camera=transform_from_world_to_camera.T,
    log_level=logging.INFO,
)

print(foreground_depths.shape)
# Expected output: torch.Size([480, 640])
print(background_depths.shape)
# Expected output: torch.Size([480, 640])

Notes:

• Depth Renderer requires a CUDA-capable GPU and the appropriate CUDA toolkit installed.
• The input transformation matrices must be in column-major order (transposed).
• The camera's principal point is assumed to be at the center of the image (i.e., cx = width / 2 and cy = height / 2).

Build with CMake

For some reason, you might want to build Depth Renderer manually. Depth Renderer supports building with CMake.

You can build the C++/CUDA extension to the Python package by running the following commands from the root directory:

cmake -S ./cpp \
      -B ./cpp/build \
      -G Ninja
cmake --build ./cpp/build

This will create the _C.so file in the src/depth_renderer directory, and the depth_renderer package will be ready to use.