HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation

The article introduces HardSecBench, a benchmark designed to evaluate the security awareness of large language models (LLMs) in hardware code generation, consisting of 924 tasks across Verilog and C, targeting 76 hardware-related Common Weakness Enumeration (CWE) entries. The benchmark features a structured specification, secure reference implementations, and executable tests, utilizing a multi-agent pipeline for artifact synthesis and verification. The findings indicate that while LLMs can meet functional requirements, they often overlook security vulnerabilities, underscoring the need for improved security assessments in LLM-assisted hardware design.