FFinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming

FinRED is a newly introduced expert-guided red-teaming framework specifically designed for evaluating the safety of financial LLMs, addressing unique risks such as regulatory compliance violations and fraud. It features a two-level taxonomy that maps global standards to various threats and employs a scalable pipeline for generating context-rich Behavioral Prompts from real financial documents, validated by experts to ensure realism. This framework aims to enhance the evaluation of financial models by reducing false negatives in safety assessments and is currently deployed within South Korea's Financial Security Institute for practical applications in generative AI security.