Semantic search for 100M+ galaxy images using AI-generated captions

The article presents AION-Search, a semantic search engine designed to analyze over 100 million unlabeled galaxy images by generating AI-driven captions using Vision-Language Models (VLMs). This system contrasts a pre-trained astronomy foundation model with generated descriptions to create searchable embeddings, achieving superior zero-shot performance compared to traditional image similarity searches, and improving recall for challenging targets through a VLM-based re-ranking method. This advancement allows for efficient exploration of vast scientific image datasets, facilitating the discovery of new astronomical phenomena and enhancing data accessibility across various scientific fields.