Task-guided cross-subject latent alignment: a multi-encoder-decoder VAE

The article presents the Multi-Encoder-Decoder Variational Autoencoder (MED-VAE), a novel approach for cross-subject neural activity alignment that does not require shared stimuli. By leveraging a pretrained artificial neural network, MED-VAE establishes common latent spaces that exhibit improved semantic organization and cross-subject alignment, outperforming traditional methods, particularly in scenarios with held-out stimuli. This advancement is significant for practitioners as it facilitates more effective cross-subject neural predictions and enhances the generalizability of models in naturalistic settings.