How to reproduce the results on MuJoCo？

[MsterDC]

Thank you for your excellent work.

I'm currently trying to reproduce your results on CortexBench-MuJoCo (Adroit/Metaworld/DMC).

I cloned the CortexBench repository and followed its configuration instructions step by step, while also attempting to integrate the Theia-trained model into the CortexBench project.
However, I'm encountering a lot of trouble, and I'm unsure if my process aligns with yours. I'd appreciate any tips or clarification.

Thank you in advance for your help!

[MsterDC]

hi @jshang-bdai ,

I have been able to train and test theia models on Cortex, but the code running is very slow. Many modifications are needed to the original Cortex code before the Theia-trained model can be integrated, which increases the risk of reproduction failure.

According to your previous reply on this issue, you seem to use a GPU to evaluate. How did you achieve that?

Could you provide the training code for Cortex-MuJoCo (cortexbench/mujoco_vc/visual_imitation/train_loop.py) so that I can quickly reproduce it? Alternatively, offer some guidance on how to expedite the evaluation process?

TIA

[MsterDC]

In the current reproduction, I have not been able to fully reproduce the results of Theia reported in the paper, as the success rate or score on some tasks is lower than the values in the paper.

Due to the lack of necessary evaluation scripts, I still don’t know where the problem lies.

[HanfengLiao]

In the current reproduction, I have not been able to fully reproduce the results of Theia reported in the paper, as the success rate or score on some tasks is lower than the values in the paper.

Due to the lack of necessary evaluation scripts, I still don’t know where the problem lies.

Could you show the results that you reproduced?