ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling

ScaleToT is a new model designed to enhance user modeling for billions of low-activity users by leveraging structured reasoning techniques. It employs a bounded entropy-guided Tree-of-Thought (ToT) refinement to create typed user-state chains from a small LLM-processed subset, which are then used to train a lightweight profile encoder via supervised fine-tuning and Outcome-Driven Segment-Aware Implicit Reward Policy Optimization (OSIPO). This approach significantly reduces computational costs associated with LLM inference while improving lifetime value (LTV) prediction, as demonstrated by a 6.738% increase in LT30 during a billion-scale advertising deployment.