Vanishing Depth: Training Generalized Depth Adapters with Sinusoidal Depth Preprocessing for Pretrained RGB Encoders

The article presents a self-supervised training method that enhances pretrained RGB encoders with a depth adapter, enabling them to integrate metric depth into a shared latent space while maintaining RGB feature extraction integrity. The proposed approach employs sinusoidal depth encoding, resulting in improved performance across various RGBD tasks, achieving a mean Intersection over Union (mIoU) of 56.05 on the SUN-RGBD segmentation benchmark, surpassing existing depth-aware models. This advancement is significant for practitioners as it allows for robust depth feature extraction in vision-guided robotics without the need for finetuning, even when traditional depth data is unavailable.