Muskie: Multi-view Masked Image Modeling for 3D Vision Pre-training

Wenyu Li+, Sidun Liu+, Peng Qiao, Yong Dou, Tongrui Hu
National University of Defense Technology
+co-first author
Paper arXiv Code


Interpolate start reference image.




Abstract

We present Muskie, a native multi-view vision backbone designed for 3D vision tasks.

Unlike existing models, which are frame-wise and exhibit limited multi-view consistency, Muskie is designed to process multiple views simultaneously and introduce multi-view consistency in pre-training stage. Muskie is trained to reconstruct heavily masked content in one view by finding and utilizing geometric correspondences from other views. Through this pretext task and our proposed aggressive masking strategy, the model implicitly to learn view-invariant features and develop strong geometric understanding without any 3D supervision.

Compared with state-of-the-art frame-wise backbones such as DINO, Muskie achieves higher multi-view correspondence accuracy. Furthermore, we demonstrate that using Muskie as a backbone consistently enhances performance on downstream 3D tasks, including camera pose estimation and pointmap reconstruction.



Video







Method



Interpolate start reference image.

Unlike existing frame-wise backbones, Muskie is designed to process multiple views simultaneously and can be used as a backbone for 3D tasks such as camera pose estimation and pointmap reconstruction. Muskie is trained to reconstruct heavily masked content in one view by finding and utilizing geometric correspondences from other views.

BibTeX

@misc{li2025muskiemultiviewmaskedimage,
        title={Muskie: Multi-view Masked Image Modeling for 3D Vision Pre-training}, 
        author={Wenyu Li and Sidun Liu and Peng Qiao and Yong Dou and Tongrui Hu},
        year={2025},
        eprint={2511.18115},
        archivePrefix={arXiv},
        primaryClass={cs.CV},
        url={https://arxiv.org/abs/2511.18115}}