Popular approaches to topic modeling often invoke the use of probabilistic generative models, such as Latent Dirichlet Allocation (LDA). While such models have enjoyed widespread use and proven fruitful, these models or generalizing them to incorporate human input requires detailed and often unrealistic assumptions about the data generating process. We introduce a new approach to topic modeling via Correlation Explanation (CorEx), which leverages an information-theoretic framework to bypass typical topic modeling assumptions. Using two challenging, real-world datasets, we demonstrate that CorEx yields results that are comparable to LDA in terms of semantic coherence and document classification. We then devise a flexible methodology for incorporating word-level domain knowledge into CorEx by introducing anchor words in a manner reminiscent of the information bottleneck. Augmenting CorEx with anchor words allows the topic model to be guided with minimal human intervention towards topics that do not naturally emerge. Furthermore, we show that these new topics are often highly coherent and act as better predictors in document classification.
['Ryan J. Gallagher', 'Kyle Reing', 'David C. Kale', 'Greg Ver Steeg']
Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge