Skip to content

README.md

Latest commit

 

History

History
executable file
·
111 lines (84 loc) · 3.75 KB

README.md

File metadata and controls

executable file
·
111 lines (84 loc) · 3.75 KB
Raw

Enhancing Monocular 3D Hand Reconstruction with Learned Texture Priors

Official repository for the paper:

Enhancing Monocular 3D Hand Reconstruction with Learned Texture Priors
Giorgos Karvounas, Nikolaos Kyriazis, Iason Oikonomidis, Georgios Pavlakos, Antonis A. Argyros
Paper Page

Teaser Figure

📌 Overview

This work revisits the role of texture in monocular 3D hand reconstruction, treating it not only as a tool for photorealism but as a dense, spatially grounded supervisory signal that enhances pose and shape estimation.

We propose a lightweight, transformer-based texture module that consolidates sparse UV–RGB observations into a full texture prior. Integrated into existing pipelines such as HaMeR, our approach delivers measurable accuracy and realism gains, particularly in occluded and egocentric scenarios, without introducing any test-time overhead.

🚀 Key Contributions

  • Introduces the first unified framework for learning texture priors from sparse, monocular observations.
  • A transformer-based module with pixel-level attention for coherent texture reconstruction.
  • Weakly supervised training through differentiable rendering, without manual texture annotations.
  • Improves state-of-the-art monocular hand reconstruction benchmarks, including gains on occluded hands.

🐳 Installation (Docker Only)

This project builds upon the HaMeR framework and follows its Docker-based setup.

1. Clone the repository

git clone --recursive https://github.com/gkarv/Hand-Texture-Module.git
cd Hand-Texture-Module

2. Build and launch the container

docker compose -f ./docker/docker-compose.yml up -d

3. Enter the container

docker compose -f ./docker/docker-compose.yml exec hamer-dev /bin/bash

4. Install PyTorch3D

pip install "git+https://github.com/facebookresearch/pytorch3d.git"

5. Download demo data

bash fetch_demo_data.sh

6. Download MANO model

Register at the MANO website and download the right-hand model.

Place the file:

_DATA/data/mano/MANO_RIGHT.pkl

Resolve NumPy / pyrender / OSMesa issues

If you encounter compatibility issues related to NumPy, pyrender, or OSMesa, run:

pip install --upgrade --force-reinstall numpy==1.26.4
conda install -n base -c conda-forge "libstdcxx-ng>=12" "libgcc-ng>=12"

🔧 Setup Texture-Supervised Weights

  1. Download the checkpoint
    👉 texture_supervised_hamer_weights

  2. Place it under:

    _DATA/hamer_ckpts/checkpoints/

  3. Run HaMeR with the texture-supervised checkpoint

    python demo.py \
    --checkpoint _DATA/hamer_ckpts/checkpoints/texture_supervised_hamer_weights.ckpt \
    --img_folder example_data \
    --out_folder demo_out \
    --batch_size 48 \
    --side_view \
    --save_mesh \
    --full_frame

▶️ Notes

  • The installation procedure follows the official HaMeR Docker workflow.
  • This repository currently provides the texture-supervised weights and project description; the full training and release utilities are being prepared.
  • Using --checkpoint avoids modifying HaMeR source files manually.

📢 Updates

  • Release code
  • Add preparation instructions
  • Provide demo notebooks
  • Release new weights for HaMeR