Detecting Malicious Agent Skills in the Wild using Attention

The article presents "Locate-and-Judge," a two-stage detection system for identifying malicious skills in the marketplace for LLM agents, which can execute third-party instructions with user privileges. The system employs a lightweight locator that scores instruction spans based on attention metrics, followed by a detailed examination of the top-K spans, significantly enhancing scalability and precision while reducing costs compared to traditional LLM-based scanning methods. This approach not only flags numerous previously undetected malicious skills but also provides a labeled dataset for further research, addressing a critical security vulnerability in agentic systems.