Text2DSL: LLM-Based Code Generation for Domain-Specific Languages

The paper introduces Text2DSL, a novel approach for automatic code generation in domain-specific languages (DSLs) from natural language descriptions, distinct from existing paradigms like Text-to-SQL. It presents the PolkitBench dataset, containing 4,204 validated natural-language-to-Polkit-rule pairs, and evaluates two mixture-of-experts models: GigaChat-10B-A1.8B and Nemotron-3-Nano-30B. Key findings reveal that incorporating structured context significantly enhances syntactic and structural validity, achieving up to 99.4% syntactic validity and a 95% increase in CodeBLEU scores, highlighting the importance of formal specifications in improving LLM performance for DSL code generation.