Improving Topic Modeling by Distilling Soft Labels from Language Models

The paper presents a novel framework for topic modeling called Distilling Soft Labels (DSL), which leverages language models to enhance the training of topic models by utilizing contextual information from next token probabilities. DSL demonstrates significant improvements in topic coherence and assignment accuracy compared to traditional methods, as well as superior performance in a new retrieval-based metric for identifying semantically similar documents. This advancement is crucial for practitioners looking to integrate improved topic modeling techniques into applications that require enhanced semantic understanding and document retrieval.