ActiveSAM: Image-Conditional Class Pruning for Fast and Accurate Open-Vocabulary Segmentation

ActiveSAM is a new framework that enhances the Segment Anything Model 3 (SAM 3) for open-vocabulary semantic segmentation (OVSS) by enabling active-vocabulary segmentation without requiring training or weight updates. It operates by estimating an image-conditioned active class set from a low-resolution preview, allowing only relevant classes to be decoded at full resolution, resulting in a speed improvement of up to 5.5x and an average increase of 1.4 mIoU over the previous state-of-the-art, SegEarth-OV3. This method is particularly advantageous for applications in noisy environments, such as autonomous driving, due to its robustness against image corruption.