| Challenge: | Existing methods have achieved two out of three criteria simultaneously, but never all three at once. |
| Approach: | They propose an online inference algorithm which leverages stochasticity to scale well in the number of documents and sparsity to achieve accurate inference. |
| Outcome: | The proposed algorithm scales well in the number of documents and topics while achieving accurate inference. |
Similar Papers
Explicit Bayesian Inference to Uncover the Latent Themes of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive generative capabilities, yet their inner mechanisms remain largely opaque. |
| Approach: | They propose a variational autoencoder-based neural topic model to interpret LLMs generation process through an explicit Bayesian framework by inferring latent topic variables via variational inference. |
| Outcome: | The proposed model outperforms state-of-the-art topic models on intrinsic measures of coherence and diversity on multiple datasets and shows significant gains on classification and summarization tasks. |
Auto-Encoding Variational Bayes for Inferring Topics and Visualization (2020.coling-main)
Copied to clipboard
| Challenge: | Visualization and topic modeling are widely used for text analysis, but there is a lack of semantic interpretation as they do not extract topics. |
| Approach: | They propose a fast Auto-Encoding Variational Bayes based inference method for jointly inferring topics and visualization using a single generative model. |
| Outcome: | The proposed method can handle model changes efficiently with little rederivation effort. |
Scale-Invariant Infinite Hierarchical Topic Model (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing hierarchical topic models yield fragmented topics with overlapping themes whose expected probability becomes exponentially smaller along the depth of the tree. |
| Approach: | They propose a hierarchical infinite hierarchic topic model that adapts to topic creation to make expected topic probability decay considerably slower than existing models. |
| Outcome: | The proposed model has better topic uniqueness and hierarchical diversity than existing approaches. |
Sharpness-Aware Minimization for Topic Models with High-Quality Document Representations (2025.naacl-long)
Copied to clipboard
Tung Nguyen, Tue Le, Hoang Tran Vuong, Quang Duc Nguyen, Duc Anh Nguyen, Linh Ngo Van, Sang Dinh, Thien Huu Nguyen
| Challenge: | Recent advances in topic models have significantly enhanced the performance compared to conventional probabilistic approaches. |
| Approach: | They propose to use a topic model optimization method to improve generalization and performance. |
| Outcome: | The proposed method improves topic models on popular benchmark datasets. |
Generative Text Modeling through Short Run Inference (2021.eacl-main)
Copied to clipboard
| Challenge: | Latent variable models for text capture global semantic and syntactic features when trained correctly. |
| Approach: | They propose a short run dynamics for inference that initializes from the prior distribution of the latent variable and runs a small number of Langevin dynamics steps guided by its posterior distribution. |
| Outcome: | The proposed model is able to generate coherent sentences with smooth transition and shows no sign of posterior collapse. |
Meta-Learning Online Adaptation of Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models encode impressively broad world knowledge in their parameters, but the knowledge in static models becomes stale. |
| Approach: | They propose to use a small, autoregressive model to reweight the language modeling loss for each token during online fine-tuning to maximize the out-of-date model’s ability to answer questions about a document. |
| Outcome: | The proposed approach improves the model's ability to answer questions on streams of thousands of documents compared with fine-tuning and baseline heuristics. |
A Bayesian Topic Model for Human-Evaluated Interpretability (2022.lrec-1)
Copied to clipboard
| Challenge: | Topic modeling is an effective way to analyze unstructured textual data. |
| Approach: | They propose to combine nonparametric and weakly-supervised topic models to produce interpretable topics. |
| Outcome: | The proposed model outperforms weakly-supervised models in the field of topic modeling. |
Enhancing Short-Text Topic Modeling with LLM-Driven Context Expansion and Prefix-Tuned VAEs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing topic models often lack sufficient word co-occurrence in short texts, resulting in incoherent topics. |
| Approach: | They propose to use large language models to extend short texts into more detailed sequences before applying topic modeling to solve semantic inconsistency problem. |
| Outcome: | The proposed approach significantly outperforms current state-of-the-art topic models on real-world datasets with extreme data sparsity. |
TopicGPT: A Prompt-based Topic Modeling Framework (2024.naacl-long)
Copied to clipboard
| Challenge: | TopicGPT uses large language models to uncover latent topics in text . topic models represent topics as bags of words that require "reading the tea leaves" topic models also offer limited control over formatting and specificity of topics . |
| Approach: | TopicGPT uses large language models to uncover latent topics in text . authors propose a prompt-based framework that produces topics that align better with human categorizations . |
| Outcome: | TopicGPT produces topics that align better with human categorizations compared to competing methods. |
Topic Modeling for Short Texts with Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can be used to solve topic modeling challenges for short texts by contextually learning the meanings of words. |
| Approach: | They propose two approaches to using Large Language Models (LLMs) for topic modeling: parallel prompting and sequential prompting. |
| Outcome: | The proposed methods identify more coherent topics than existing ones while maintaining the diversity of the induced topics. |