Latent Thought Flow: Efficient Latent Reasoning in Large Language Models

The paper introduces Latent Thought Flow (LTF), a novel approach for efficient latent reasoning in Large Language Models (LLMs) that addresses the limitations of explicit Chain-of-Thought (CoT) by utilizing variable-length continuous trajectories for reasoning. LTF employs a continuous GFlowNet with stochastic latent transitions and incorporates an Entropy-Weighted Subtrajectory Balance objective to improve reward allocation, achieving a 9.5% accuracy increase and a 27.2% reduction in reasoning length compared to existing methods. This advancement is significant for practitioners as it enhances the efficiency and effectiveness of reasoning processes in LLMs, potentially reducing computational overhead in real-world applications.