Contrastive and Adaptive Multi-modal Masked Autoencoder for Spatial Transcriptomics

The study introduces a Contrastive and Adaptive Multi-modal Masked Autoencoder (CAMMST) for spatial transcriptomics, addressing the challenge of predicting gene expression from histology images by treating it as a spatial imputation problem. The model employs a cross-modal joint encoder that integrates visual and genetic modalities, utilizing a bio-saliency score and learning-to-rank strategy to identify informative tissue spots, achieving superior accuracy in gene expression prediction with as little as 10% transcriptomic coverage. This advancement is significant for practitioners as it enhances the predictive capabilities of spatial transcriptomics while reducing reliance on extensive genetic data.