COLMAP stuck if increase image_width

[oscarqht]

Hi. Thanks for this wonderful work. I have a question regarding the usual processing & training time. I’m trying to work on my own image set on Colab. I found that when I increase the image_width to something bigger than 400 (for example 720), then the processing time will be drastically increased, specifically it will stuck at this step for a very long time, resulting in Colab to timeout and disconnected:

==============================================================================
Exhaustive feature matching
==============================================================================

Matching block [1/1, 1/1]

May I know what is the expected processing and training time if I want to render high resolution output, and what is the recommended hardware I’ll need? Thank you :)

[pureexe]

Feature matching takes a long time because there has a lot of features. So, I recommend you decrease the number of features.

There is two approach.

1.limit the feature
You can add the argument --SiftExtraction.max_num_features to the COLMAP feature_extractor

2.Manipulate the calibration Run feature matching on low-resolution (Eg, width 400), and you can use this calibration in any resolution you want. However, you have to change the principal point and focal length. This requires you to understand how the camera's intrinsic and extrinsic work. Maybe check this article

I recommend you try the first approach first, but it may degrade the camera calibration, leading to a poor result. If the first approach does not satisfy you, you have to take time to understand how camera calibration work.

For COLMAP's part, it takes about an hour for 50 images with width 800 on Threadripper 3960x.

For the expected processing time, the neural network part depends on how many iterations to train. A bigger image is likely to take more iteration to converse. I expected a day of training for width 800 on Nvidia V100.

However, I would recommend you to check Nvidia's Instant-NGP, which targets a similar view synthesis as our work and can be trained in 5 minutes.

[oscarqht]

Hi @pureexe , thanks for the reply. I've installed the latest version of COLMAP 3.8 (if you install via apt install colmap, you'll get 3.6 instead) on colab and enabled GPU (removed --SiftExtraction.use_gpu=false and --SiftMatching.use_gpu=false options). Now the Run SFM step is a lot faster, but I'm getting this error when running the load_colmap_data function:

/content/nex-code
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
[<ipython-input-17-4c90f959bb10>](https://localhost:8080/#) in <module>
      2 get_ipython().run_line_magic('cd', '/content/nex-code')
      3 from utils.colmap_runner import load_colmap_data, save_poses
----> 4 poses, pts3d, perm, hwf_cxcy = load_colmap_data(PROJECT)
      5 save_poses(PROJECT, poses, pts3d, perm, hwf_cxcy)

[/content/nex-code/utils/colmap_runner.py](https://localhost:8080/#) in load_colmap_data(realdir)
     83   cam = camdata[list_of_keys[0]]
     84 
---> 85   h, w, fx, fy, cx, cy = cam.height, cam.width, cam.params[0], cam.params[1], cam.params[2], cam.params[3]
     86 
     87   hwf_cxcy = np.array([h, w, fx, fy, cx, cy]).reshape([6,1])

IndexError: index 3 is out of bounds for axis 0 with size 3

For the full output, you can see this colab notebook:
https://colab.research.google.com/drive/1eK22cev7AD1nOdNHZixKvRiObQxwuaMq#scrollTo=RjhVd5ME0vAM

Do you know how shall I fix this issue? Thanks :)

[pureexe]

The shape of camera model is incorrect.
Please add --ImageReader.camera_model="PINHOLE" when running the feature extractor.