TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations

The paper presents TerraTransfer, a novel approach for training end-to-end driving policies without the need for expert demonstrations, leveraging self-play in vectorized simulators to generate a rich state distribution. The method involves pretraining a single policy and aligning its latent space with a pretrained vision backbone using action KL divergence and a low-rank structural loss, significantly reducing the reliance on costly labeled datasets. This advancement is significant for practitioners as it offers a more efficient training paradigm that can match or exceed the performance of existing methods while minimizing data collection costs.