Repository navigation
On evaluation on ReasonSeg testing set #3
Description
Congrats and thanks for the great work!
One question: it seems that for testing on ReasonSeg dataset, you are testing on "val" split of the ReasonSeg dataset? Is that right?
READ/dataloaders/test_dataset.py
Line 140 in 2cee311
I would appreciate it very much if you could reply
Activity
rui-qian commentedon Mar 19, 2025
Thank you for your interest. You can refer to val_read.sh, Line 33:
--val_split="test", and train_read.py, Line 274:train_test_split=args.val_split.Additionally, if you are only concerned with the ReasonSeg test set results, you can use the following code:
https://huggingface.co/datasets/rui-qian/misc/blob/main/READ-7B_scratch_test.tar.gz
Along with its corresponding model parameters:
https://huggingface.co/rui-qian/READ-LLaVA-v1.5-7B-for-ReasonSeg-testset.
Please read the information in the News section on GitHub. We are confident that the results in the paper are reproducible. If you have any questions, feel free to contact us again.
wyzjack commentedon Mar 19, 2025
Great, got it :), thanks so much for your information and reply!
Congratulations on the nice work again! Appreciate your transparent and open-source efforts very much!
wyzjack commentedon Mar 22, 2025
Hi authors, I am now trying to train the 13b model, I got bug below
/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/torch/_dynamo/eval_frame.py:600: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.
return fn(*args, **kwargs)
/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/torch/utils/checkpoint.py:295: FutureWarning:
torch.cpu.amp.autocast(args...)is deprecated. Please usetorch.amp.autocast('cpu', args...)instead.with torch.enable_grad(), device_autocast_ctx, torch.cpu.amp.autocast(**ctx.cpu_autocast_kwargs): # type: ignore[attr-defined]
[rank0]: Traceback (most recent call last):
[rank0]: File "/sensei-fs/users/yizhouw/projects/reasoning_seg/READ/train_read.py", line 510, in
[rank0]: main(sys.argv[1:])
[rank0]: File "/sensei-fs/users/yizhouw/projects/reasoning_seg/READ/train_read.py", line 328, in main
[rank0]: train_iter, global_iters = train(
[rank0]: File "/sensei-fs/users/yizhouw/projects/reasoning_seg/READ/train_read.py", line 406, in train
[rank0]: output_dict = model(**input_dict)
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
[rank0]: return self._call_impl(*args, **kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
[rank0]: return forward_call(*args, **kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/deepspeed/utils/nvtx.py", line 18, in wrapped_fn
[rank0]: ret_val = func(*args, **kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/deepspeed/runtime/engine.py", line 1987, in forward
[rank0]: loss = self.module(*inputs, **kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
[rank0]: return self._call_impl(*args, **kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
[rank0]: return forward_call(*args, **kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/peft/peft_model.py", line 922, in forward
[rank0]: return self.base_model(
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
[rank0]: return self._call_impl(*args, **kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/anaconda3/envs/ml/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
[rank0]: return forward_call(*args, **kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/projects/reasoning_seg/READ/model/READ.py", line 216, in forward
[rank0]: return self.model_forward(**kwargs)
[rank0]: File "/sensei-fs/users/yizhouw/projects/reasoning_seg/READ/model/READ.py", line 517, in model_forward
[rank0]: points_list, labels_list = self.similarity_as_points(
[rank0]: File "/sensei-fs/users/yizhouw/projects/reasoning_seg/READ/model/READ.py", line 678, in similarity_as_points
[rank0]: points1, labels1 = self.similarity_map_to_points(similarity[0, :, 0], images_size_list[bs], t=0.8)
[rank0]: File "/sensei-fs/users/yizhouw/projects/reasoning_seg/READ/model/READ.py", line 902, in similarity_map_to_points
[rank0]: points = Discrete_to_Continuous(origin_sm[0, ..., 0], points)
[rank0]: File "/sensei-fs/users/yizhouw/projects/reasoning_seg/READ/model/READ.py", line 798, in Discrete_to_Continuous
[rank0]: selected_x = selected_points[:, 0]
[rank0]: IndexError: index 0 is out of bounds for dimension 1 with size 0
[2025-03-21 21:11:18,881] [INFO] [launch.py:319:sigkill_handler] Killing subprocess 57288
[2025-03-21 21:11:18,882] [ERROR] [launch.py:325:sigkill_handler] ['/sensei-fs/users/yizhouw/anaconda3/envs/ml/bin/python', '-u', 'train_read.py', '--local_rank=0', '--version=../../reasonseg_dataset/dataset/llava-v1.5-13b', '--model_max_length=2048', '--vision_tower=../../reasonseg_dataset/dataset/clip-vit-large-patch14-336', '--dataset_dir=../../reasonseg_dataset/dataset', '--vision_pretrained=../../reasonseg_dataset/dataset/sam_vit_h_4b8939.pth', '--exp_name=reasoning_train_13b', '--dataset=sem_seg||refer_seg||correct_refer_seg||vqa||neg_refer_seg||reason_seg', '--sample_rates=3,10,2,3,1,10', '--batch_size=2', '--grad_accumulation_steps', '10', '--num_classes_per_sample=3', '--auto_resume'] exits with return code = 1
Any advice on how to solve this bug? Would appreciate it very much if you could help, thanks
rui-qian commentedon Mar 22, 2025
It looks like the current sample does not contain a seg token, causing the data to be empty when computing DtoC. From a debugging perspective, you can first comment out the DtoC module (
model/READ.py: Line 897) and setLine 307as follows:Then observe whether the program runs normally. If you have any further questions, feel free to reach out anytime.
wyzjack commentedon Mar 23, 2025
Thanks for your suggestions and guidance!