Running Inference with pre-trained model

[rohansaw]

Is it possible to use the provided codebase solely for inference? The documentation only containes examples for training and not also for just simply running inference on unlabelled datasets for pose keypoint detection for a specific species.

Thank you!

[MitchFuchs]

Hi @rohansaw,

You can indeed use the GUI to infer keypoint locations and probability on videos by selecting POSEC3D > create_dataset

which will call the dlc.analyze_videos() method from DLC (https://github.com/DeepLabCut/DeepLabCut/blob/main/deeplabcut/pose_estimation_tensorflow/predict_videos.py)

If you want to visualize the results on your videos you can convert the predictions using dlc methods such as dlc.create_video_with_all_detections() or dlc.create_labeled_video()

Hope this helps, let me know if you need more assistance.

Regards,
Mitch

[Lasklu]

Hi @MitchFuchs , thanks for your answer. I am working with Rohan and I am currently trying to set it up. The command I am executing looks like this:
python main.py --model POSEC3D --task create_dataset --dataset omc --project dlc-215009-Mitch-2024-09-09 --iteration NEW --snapshot best_test_error --gpu 0 --shuffle all

Does this look promising? It is currently looking for models and returns the error: No such file or directory: '/home/lukas/tests/im/asbar/models/deeplabcut/dlc-215009-Mitch-2024-09-09/dlc-models/iteration-0'. Can I put here any of the pretrained models you provide? Which file could I take?

Thanks and best wishes
Lukas

[Lasklu]

Hey, @MitchFuchs, I think I got some steps closer to the goal (which is executing the pretrained models on gorilla videos for pose estimation). I now get some errors, such as (1) NOT_FOUND: Key decoder_filters/decoder_parallel_1/BatchNorm/beta not found in checkpoint . However, I had to assume a lot of configs in, for example, the pose_cfg.yaml. Could you maybe share these files with us, so that we can use them to reproduce your pre-trained model using deeplabcut?

Thank you in advance
Lukas

[MitchFuchs]

Hi Lukas,

Sorry for my late reply. I am attaching the config files to this post. configs.zip

Knowing that you have not trained dlc you might miss a few folders. So, I believe you should first create the necessary folders by running:

cd /home/lukas/tests/im/asbar/models/deeplabcut/dlc-215009-Mitch-2024-09-09/dlc-models/
mkdir -p iteration-0/dlc-215009Sep09-trainset100shuffle1/train
mkdir -p iteration-0/dlc-215009Sep09-trainset100shuffle1/test

then copy the config files in the right location by
cd <full path of the unzipped folder containing the config files>
cp config.yaml /home/lukas/tests/im/asbar/models/deeplabcut/dlc-215009-Mitch-2024-09-09/config.yaml
cp train_pose_cfg.yaml /home/lukas/tests/im/asbar/models/deeplabcut/dlc-215009-Mitch-2024-09-09/dlc-models/iteration-0/dlc-215009Sep09-trainset100shuffle1/train/pose_cfg.yaml
cp test_pose_cfg.yaml /home/lukas/tests/im/asbar/models/deeplabcut/dlc-215009-Mitch-2024-09-09/dlc-models/iteration-0/dlc-215009Sep09-trainset100shuffle1/test/pose_cfg.yaml
cp inference.yaml /home/lukas/tests/im/asbar/models/deeplabcut/dlc-215009-Mitch-2024-09-09/dlc-models/iteration-0/dlc-215009Sep09-trainset100shuffle1/test/inference.yaml

you should also make sure that the pretrained dlc model is saved in
/home/lukas/tests/im/asbar/models/deeplabcut/dlc-215009-Mitch-2024-09-09/dlc-models/iteration-0/dlc-215009Sep09-trainset100shuffle1/train/

Feel free to share your entire terminal output so that I can see best where the bug is. You can also fork the repo and add your configs so that I can run it on my end if necessary.

I hope this helps.
Cheers,
Mitch