FlexSplat: Flexible Feed-Forward 3D Gaussian Splatting without Point Cloud Correspondence

       

Project Webpage . Project Video

Official research code for FlexSplat: Flexible Feed-Forward 3D Gaussian Splatting without Point Cloud Correspondence, accepted to the British Machine Vision Conference (BMVC) 2026.

FlexSplat reconstructs object-centric scenes from uncalibrated multi-view images. It predicts cameras and depth with a jointly trained VGGT geometry front-end, then decodes a compact set of 3D Gaussian queries with multi-view deformable attention. The Gaussian budget is independent of image resolution and is not tied to point-cloud or pixel-level correspondence.

FlexSplat workflow

Uncalibrated input views pass through learned camera/depth estimation and Gaussian decoding to produce a compact object-centered 3D representation.

FlexSplat architecture

The architecture fuses depth-aware image features and decodes a controlled set of Gaussian queries for feed-forward novel view synthesis.

Highlights

• Feed-forward novel view synthesis without input camera poses or ground-truth depth.
• Query-based, correspondence-free Gaussian representation with 10K-20K primitives.
• Joint geometry learning with VGGT camera/depth prediction in the training path.
• Multi-view deformable cross-attention over fused appearance and depth-aware features.
• Reproduction configs for ShapeNet-SRN, Objaverse-LVIS, GSO, timing, pose accuracy, and ablations.

Results

Setting	Main result
ShapeNet-SRN single-view	25.48 PSNR on Chairs and 25.09 PSNR on Cars
GSO four-view	29.73 PSNR, 0.949 SSIM, 0.041 LPIPS without input poses
View scaling on GSO	21.93 -> 27.10 -> 29.73 PSNR for 1, 2, and 4 input views
Inference time	0.706 s single-view and 0.992 s four-view end to end

Paper figures are kept as PDFs in artifacts/; README-friendly PNG copies are in docs/figures/.

Installation

conda create -n flexsplat python=3.10 -y
conda activate flexsplat

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt

The renderer uses gsplat, so use a CUDA-enabled environment for training, testing, and demo export. The first install may compile CUDA extensions.

Data

Place datasets under:

datasets/
├── shapenet_srn/
├── objaverse_lvis/
└── gso/

For the default ShapeNet-like layout, each scene should contain:

scene_id/
├── rgb/          # images
├── pose/         # 4x4 world/camera matrices as .txt
└── intrinsics/   # 3x3 intrinsics as .txt

GSO and Objaverse also support a Nerfstudio-style transforms.json layout through the dataset config.

Training

Train the main multi-view Objaverse-LVIS model:

python -m src.main +experiment=objaverse

Train the single-view ShapeNet-SRN model:

python -m src.main +experiment=shapenet

For local runs without Weights & Biases:

python -m src.main +experiment=objaverse wandb.mode=disabled

Evaluation and paper tables

Set CKPT to a trained checkpoint path before running the test commands.

export CKPT=outputs/objaverse/checkpoints/last.ckpt

Single-view NVS on ShapeNet-SRN:

python -m src.main +experiment=shapenet mode=test \
  checkpointing.load=outputs/shapenet/checkpoints/last.ckpt

Four-view NVS on GSO:

python -m src.main +experiment=gso checkpointing.load=$CKPT

Single-view GSO:

python -m src.main +experiment=gso checkpointing.load=$CKPT \
  dataset.gso.view_sampler.num_context_views=1 \
  model.encoder.num_gaussians=10000

Effect of input view count:

for v in 1:10000 2:15000 4:20000; do
  python -m src.main +experiment=gso checkpointing.load=$CKPT \
    dataset.gso.view_sampler.num_context_views=${v%%:*} \
    model.encoder.num_gaussians=${v##*:} \
    test.output_path=outputs/views_${v%%:*}
done

Inference time:

python -m src.benchmark +experiment=gso checkpointing.load=$CKPT

Ablations:

for a in frozen_geometry random_init no_depth_features no_depth_loss framewise_gaussians; do
  python -m src.main +experiment=ablation/$a
done

Robustness to camera-pose perturbation:

for s in 0 1 2 5 10 20; do
  python -m src.main +experiment=gso checkpointing.load=$CKPT \
    model.encoder.pose_noise_deg=$s \
    test.output_path=outputs/noise_$s
done

Camera-pose accuracy:

python -m src.eval_pose +experiment=gso checkpointing.load=$CKPT

Demo on custom images

FlexSplat can reconstruct from a folder of RGB images without camera or depth input:

python -m src.demo \
  --images path/to/images \
  --checkpoint $CKPT \
  --export-ply

The demo writes rendered views and, when requested, a Gaussian PLY file to outputs/demo/.

Tests

pytest

The tests cover projection geometry, Gaussian parameterization, quaternion/covariance conventions, multi-view deformable attention, pose perturbation, pose metrics, and the main loss terms.

Repository structure

config/       Hydra experiments, datasets, losses, encoder and decoder settings
src/          training, evaluation, datasets, model, geometry, rendering, visualization
tests/        geometry, decoder, pose, and loss tests
artifacts/    source PDF figures from the paper
docs/         README figures and collaboration logos

Citation

@inproceedings{sabbaghziarani2026flexsplat,
  title={FlexSplat: Flexible Feed-Forward 3D Gaussian Splatting without Point Cloud Correspondence},
  author={Sabbaghziarani, Amir and Ye, Hanting and Gorlatova, Maria and Ding, Yi},
  booktitle={British Machine Vision Conference},
  year={2026},
  url={https://arxiv.org/abs/2608.07937},
  eprint={2608.07937},
  archivePrefix={arXiv}
}