Collaborative Learning of Local 3D Occupancy Prediction and Versatile Global Occupancy Mapping

Shanshuai Yuan¹, Julong Wei¹, Muer Tie¹, Xiangyun Ren², Zhongxue Gan¹, Wenchao Ding¹

¹Fudan University, ²Chongqing Changan Automobile CO., Ltd

Accepted by ICRA 2026

Abstract

Vision-based 3D semantic occupancy prediction is vital for autonomous driving, enabling unified modeling of static infrastructure and dynamic agents. Global occupancy maps serve as long-term memory priors, providing valuable historical context that enhances local perception. This is particularly important in challenging scenarios such as occlusion or poor illumination, where current and nearby observations may be unreliable or incomplete. Priors aggregated from previous traversals under better conditions help fill gaps and enhance the robustness of local 3D occupancy prediction. In this paper, we propose Long-term Memory Prior Occupancy (LMPOcc), a plug-and-play framework that incorporates global occupancy priors to boost local prediction and simultaneously updates global maps with new observations. To realize the information gain from global priors, we design an efficient and lightweight Current-Prior Fusion module that adaptively integrates prior and current features. Meanwhile, we introduce a model-agnostic prior format to enable continual updating of global occupancy and ensure compatibility across diverse prediction baselines. LMPOcc achieves state-of-the-art local occupancy prediction performance validated on the Occ3D-nuScenes benchmark, especially on static semantic categories. Furthermore, we verify LMPOcc’s capability to build largescale global occupancy maps through multi-vehicle crowdsourcing, and utilize occupancy-derived dense depth to support the construction of 3D open-vocabulary maps. Our method opens up a new paradigm for continuous global information updating and storage, paving the way towards more comprehensive and scalable scene understanding in large outdoor environments.

Framework

LMPOcc fristly generates Current Latent Features from surround-view images. Then it extracts spatiallyaligned Prior Features from global occupancy and integrates them via the Current-Prior Fusion Module to generate Refined Latent Features. The refined latent features decode current occupancy logits, which are stored into corresponding locations in the global occupancy after visibility masking. Existing occupancy priors at these locations are replaced by the updated logits. Finally, the occupancy logits are converted into local current occupancy prediction results.

BibTeX

@article{yuan2025lmpocc, title={LMPOcc: 3D Semantic Occupancy Prediction Utilizing Long-Term Memory Prior from Historical Traversals}, author={Yuan, Shanshuai and Wei, Julong and Tie, Muer and Ren, Xiangyun and Gan, Zhongxue and Ding, Wenchao}, journal={arXiv preprint arXiv:2504.13596}, year={2025} }

Collaborative Learning of Local 3D Occupancy Prediction and Versatile Global Occupancy Mapping

Abstract

Framework

Visualization

The left, center, and right columns demonstrate the performance under sunny, rainy, and night-time conditions respectively.

Visualization results of global occupancy construction via crowdsourcing methodologies. Three collaborative agents construct the global occupancy map through crowdsourcing.

Quantitative Experiments

3D occupancy prediction performance on the Occ3D-nuScenes validation set. Both the small version and large version of LMPOcc outperform the models that have similar settings.

The performance of occupancy prediction methods and their LMOP versions on the Occ3D-nuScenes validation set. By adding long-term memory knowledge, LMOP consistently improves these methods.

Video

BibTeX