Will human video data be open-sourced?

[Wuli10]

A wonderful work! The same request: could the human video data from the first phase be made available? Thanks!

[PuzhenYuan]

Same question. Could you describe the filter method for selecting a subset from RH20T?

[jiaming-zhou]

Thank you for your interest in our work. Regarding the human data, we utilized the open-source RH20T dataset, which already includes human video sequences.

Our data filtering process was straightforward and did not involve complex custom rules. Specifically, we filtered out human videos that lacked corresponding robot video pairs from the same camera views (since RH20T is a multi-view dataset, some pairs are incomplete). This resulted in a clean set of multi-view human-robot video pairs. Regarding the view selection, we may have selected a single view for each pair, though I would recommend verifying the final data size for confirmation. Finally, we trained our model on a randomly selected subset of this filtered data.