Noticeable difference in brightness between ground truth images in shiny dataset, and those in evaluation folders

[benattal]

Hi! While poking around in the data for the cd shiny sequence, I noticed that there is a difference in brightness between the ground truth heldout images provided in the evaluation folder, and those provided in the dataset folder (it's a lot more obvious if you overlay the images and cycle between them). The same brightness discrepancy appears for most of the other sequences as well. I'm wondering if you can tell me why this is the case. Additionally, after running NeRF on your data and writing out the ground truth images, I get images that look identical to those in the dataset folder, but not the evaluation folder:

(CD Evaluation Folder)

(CD Dataset Folder)

(NeRF Ground Truth)

Thanks in advance!

[pureexe]

Why color is different on evaluation and dataset?

The difference of brightness comes from PNG metadata

The left image is from images_distort folder while the right comes from images. You can see a lot of metadata is gone missing.

images folder comes straight out from COLMAP image undistorter which NeX is requires an undistorted image to use as training data.

How to handle this problem?

I deeply apologize that I forgot the exact method that we handle this problem but here is the remaining clue from my hazy memory.

From my hazy memory

For NeX

i think that we train NeX with the png from the images folder that doesn't have metadata and uses some way to do the gamma correction to make the output image look dark as same as ground truth.

For NeRF

NeRF dataset contain lens distortion as mention in the README.md

So, we trained NeRF with images in the folder images_distort. After finishing training, we undistort the image with COLMAP using the COLMAP's model that stored in a sparse folder.

[benattal]

I see, this makes sense -- thank you! Is there any way that you can share the code / parameters for gamma correction (perhaps white balancing as well, as it seems that the white balance metadata is also missing), or point to functions within a relevant python image processing library? I suspect that this has an impact on evaluation. In trying to reproduce the NeRF results you published for the shiny dataset, I trained NeRF directly on the images subdirectory and observed worse quantitative metrics than those published in the paper.