Performance when using fisheye images / recommendations for distortion?

[duanshuangda]

Hi team,

First of all, thank you for releasing Fast-Foundation-Stereo!

I am currently testing the model's performance on stereo images captured by fisheye cameras. In my preliminary tests, I noticed that the inference results (disparity/depth estimation) are quite poor compared to standard pinhole camera images.

I would love to know:

Has the model been trained or tested on fisheye/highly-distorted images before?

Do you have any recommended preprocessing steps or configurations to improve performance in this scenario?

What I have tried / observed:

Standard rectification doesn't seem to fully resolve the massive radial distortion on the edges, leading to severe artifacts in the depth map.

The model seems to struggle significantly with the non-linear pixel displacement inherent to fisheye lenses.

Any insights, advice, or recommended pipelines (e.g., undistorting to pinhole models first vs. adapting the intrinsic parameters) would be greatly appreciated!

[K.txt](https://github.com/user-attachments/files/29406214/K.txt)

mower_3_calibration.yaml

Thanks in advance!

[kvnloo]

For fisheye input I think the useful comparison is not simply "undistort or don't."

The model's stereo correspondence is still based on rectified epipolar geometry, so I would benchmark two controlled preprocessing paths:

1. fisheye -> rectified pinhole with conservative FOV/cropping;
2. fisheye -> rectified pinhole retaining more edge FOV with stronger resampling.

Then report disparity error separately in:

• central image region;
• outer radial bands.

That would show whether the remaining degradation is caused mainly by:

• residual vertical disparity;
• severe resampling near the edges;
• or a true domain gap after rectification.

It would also help to record the post-rectification intrinsics actually supplied to the depth conversion rather than the original fisheye intrinsics.

If useful, I can put together a small calibration/rectification evaluation script around one stereo pair before proposing model-side changes.