Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models

The article introduces a novel approach to membership inference in Large Language Models (LLMs) by focusing on entity-level information rather than individual samples. The proposed method includes five interrogation strategies that utilize limited entity clues to prompt LLMs, achieving an area under the curve (AUC) of up to 0.97 and improving balanced accuracy by 6.0% to 17.5% over existing sample-level methods. This work is significant for practitioners as it enhances understanding of privacy risks associated with LLMs and provides tools for assessing the exposure of training data related to specific entities.