Empty tensor? and Reproducibility?

[sieumap43]

Hi. I have two questions.
The first is that I encounter an error training (log below) when running python run/train_track_nf.py

Traceback (most recent call last):
  File "run/train_track_nf.py", line 187, in <module>
    main()
  File "run/train_track_nf.py", line 183, in main
    meta=meta)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmtrack/apis/train.py", line 175, in train_model
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
    self.run_iter(data_batch, train_mode=True, **kwargs)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
    **kwargs)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 75, in train_step
    return self.module.train_step(*inputs[0], **kwargs[0])
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmtrack/models/mot/mo3tr.py", line 375, in train_step
    losses = self(**data)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmtrack/models/mot/mo3tr.py", line 370, in forward
    return self.forward_train(**kwargs)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmtrack/models/mot/mo3tr.py", line 387, in forward_train
    track_prev = self.forward_prev_nf(all_frames, last_frame)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmtrack/models/mot/mo3tr.py", line 460, in forward_prev_nf
    prev_hs, prev_bboxes = self.temporal_model(prev_hs, prev_bboxes)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/lhome/MO3TR/moter_venv/lib/python3.7/site-packages/mmtrack/models/mot/mo3tr.py", line 490, in forward
    prev_hs = torch.cat(([torch.cat([torch.zeros((self.seq_len - len(hs), 1, hs.shape[-1]), device=hs.device), hs]) for hs in prev_hs]), dim=1)
RuntimeError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat.  This usually means that this function requires a non-empty list of Tensors.  Available functions are [CPU, CUDA, QuantizedCPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched, VmapMode].

I looked into the code and found that in this section of mot3tr_tracker.py

            # Select with threshold
            det_scores = det_cls.sigmoid()[:, 0]
            valid_det_idx = det_scores > self.init_track_thr
            det_labels = torch.zeros_like(det_scores, dtype=torch.int32)

            # Generate Valid Dets
            valid_det_bboxes = det_bboxes[valid_det_idx]
            ids = torch.arange(len(valid_det_bboxes), device=valid_det_bboxes.device)
            valid_det_labels = det_labels[valid_det_idx]
            valid_det_hs = det_hs[valid_det_idx]
            valid_det_scores = det_scores[valid_det_idx]
            valid_det_cls = det_cls[valid_det_idx]
            valid_det_bboxes = torch.cat([valid_det_bboxes, valid_det_scores.unsqueeze(-1)], dim=-1)
            self.max_track_id = len(valid_det_bboxes) - 1
            self.update(ids=ids, bboxes=valid_det_bboxes, labels=valid_det_labels, frame_ids=frame_id, hs=valid_det_hs)

the valid_det_idx is all false. Hence the emtpy prev_hs. Can you have a look into this?

My second question is that is your code intended to reproduce the result? In your paper you trained in two stages, first 300 epochs on 3 datasets then 100 epochs on MOT. But in the config file cfg_mo3tr_nf_git.py the epoch number is 20 and trained only on MOT.

Many thanks.