SkillAudit: From Fixed-Suite Benchmarking to Skill-Centered Assessment

SkillAudit is a newly introduced framework designed for skill-centered assessment of agent skills in large language models, addressing the limitations of fixed task suite evaluations. It generates multi-dimensional evaluation reports based on the skill artifact, utilizing isolated sandbox environments for task execution and LLM-based judging for automated checks. This framework is significant for practitioners as it provides a more accurate measure of skill utility, efficiency, and safety, revealing that over 7% of analyzed skills may pose safety risks, thus guiding deployment decisions in skill marketplaces.