HilDA: Hierarchical Distillation with Diffusion for Advancing Self-Supervised LiDAR Pre-trainin

HilDA is a self-supervised pretraining framework for LiDAR backbones that enhances knowledge distillation from Vision Foundation Models (VFMs) by employing hierarchical distillation and a temporal occupancy diffusion objective. This approach allows for multi-layer semantic alignment and global context integration, resulting in superior performance on cross-modal distillation benchmarks for tasks such as 3D object detection and semantic occupancy prediction. The advancements in capturing both semantic and geometric information are significant for practitioners aiming to improve autonomous driving applications with limited annotated data.