Decodable but Not Faithful: Coupling Natural-Language Rationales to Programmatic Verifiers

The paper introduces a verifier-coupled reasoning framework that enhances the decodability of language model rationales by integrating inline claims and training a consistency head to predict outputs from programmatic verifiers. Key findings reveal a disconnect between decodability and faithfulness; while consistency training improves the extraction of verifier information, it does not ensure that generated explanations accurately reflect the model's reasoning. This research is significant for AI practitioners as it highlights the limitations of current approaches in generating trustworthy explanations and suggests that while consistency losses can improve model diagnostics, they are insufficient for ensuring faithful reasoning in AI systems.