Papers with LDA
Hey Siri. Ok Google. Alexa: A topic modeling of user reviews for smart speakers (D19-55)
Copied to clipboard
| Challenge: | Using coherence scores to choose topics, we test whether the results help us to understand user interests and concerns. |
| Approach: | They analyze user reviews from Best Buy US website for smart speakers to determine whether they provide useful information for product analysis. |
| Outcome: | The proposed models capture brand performance and differences and differentiate the market into two distinct groups with different properties. |
Visualization of the Topic Space of Argument Search Results in args.me (D18-2)
Copied to clipboard
Yamen Ajjour, Henning Wachsmuth, Dora Kiesel, Patrick Riehmann, Fan Fan, Giuliano Castiglia, Rosemary Adejoh, Bernd Fröhlich, Benno Stein
| Challenge: | args.me is the first search engine for controversial topics . it ranks pro and con arguments by their relevance to a topic . |
| Approach: | They propose a visualization interface for result exploration that provides an overview of main aspects in a barycentric coordinate system. |
| Outcome: | The proposed search engine is the first dedicated argument search engine on the web. |
Improving Privacy Guarantee and Efficiency of Latent Dirichlet Allocation Model Training Under Differential Privacy (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Latent Dirichlet allocation (LDA) is a widely used topic model to discover the latent semantic of text data. |
| Approach: | They propose to combine a subsampling method with CGS to improve efficiency while amplifying privacy by using a novel metric, the efficiency–privacy function. |
| Outcome: | The proposed algorithm improves efficiency while amplifying privacy while subsampling in CGS increases efficiency while preserving privacy. |
SentenceLDA: Discriminative and Robust Document Representation with Sentence Level Topic Model (2024.eacl-long)
Copied to clipboard
| Challenge: | SentenceLDA is a sentence-level topic model that can be used to discriminate between different contexts. |
| Approach: | They propose a sentence-level topic model that extends the semantic unit from word to sentence and a corpus-level key opinion mining model that uses a lexical property to discriminate between different contexts. |
| Outcome: | The proposed model returns more discriminative document representation than other topic models while maintaining LDA’s elegant probabilistic interpretability. |
Neural Topic Modeling with Bidirectional Adversarial Training (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that neural topic models for automatic topic extraction avoid complicated mathematical derivations for model inference. |
| Approach: | They propose a bidirectional adversarial topic model which uses a generator and an encoder to infer topic distribution. |
| Outcome: | The proposed model outperforms baselines and competitive models in three benchmark corpora. |
Deep Dirichlet Multinomial Regression (N18-1)
Copied to clipboard
| Challenge: | supervised topic models can incorporate arbitrary document-level features to inform topic priors, but their ability to model corpora is limited by the representation and selection of these features. |
| Approach: | They propose a generative topic model that simultaneously learns document feature representations and topics. |
| Outcome: | The proposed model outperforms DMR and LDA on three datasets and human subjects judge it more representative of associated document features. |
Improving the TENOR of Labeling: Re-evaluating Topic Models for Content Analysis (2024.eacl-long)
Copied to clipboard
Zongxia Li, Andrew Mao, Daniel Stephens, Pranav Goel, Emily Walpole, Alden Dima, Juan Fung, Jordan Boyd-Graber
| Challenge: | Existing evaluation metrics such as coherence and coherency are inadequate for neural topic models. |
| Approach: | They conduct the first evaluation of neural, supervised and classical topic models in an interactive task-based setting. |
| Outcome: | The proposed model performs better on cluster evaluation metrics and human evaluations than classical models on real-world tasks. |
Learning VAE-LDA Models with Rounded Reparameterization Trick (2020.emnlp-main)
Copied to clipboard
| Challenge: | No reparameterization form of Dirichlet distributions is known to date for topic models . |
| Approach: | They propose a method to reparameterize Dirichlet distributions for the learning of VAE-LDA models by using a latent Dirichlets prior. |
| Outcome: | The proposed method outperforms existing neural topic models on benchmark datasets and on a synthetic dataset. |
A Multilingual Topic Model for Learning Weighted Topic Links Across Corpora with Low Comparability (D19-1)
Copied to clipboard
| Challenge: | Existing models implicitly assume that documents in different languages are highly comparable, a false assumption. |
| Approach: | They propose a multilingual topic model that learns weighted topic links and connects cross-lingual topics only when the dominant words defining them are similar. |
| Outcome: | The proposed model outperforms existing models in low-resource language tasks and outperformed LDA and previous models in classification tasks using documents’ topic posteriors as features. |
Detecting Denial-of-Service Attacks from Social Media Text: Applying NLP to Computer Security (N18-1)
Copied to clipboard
| Challenge: | Distributed Denial of Service (DDoS) attacks are becoming more frequent and more severe in their impact. |
| Approach: | They propose a feed-forward neural network and a partially labeled LDA model that use social media as an indirect measure of network service status. |
| Outcome: | The proposed model outperforms previous work by significant margins and provides the first fine-grained analysis of how the public reacts to ongoing network attacks. |
TopicGPT: A Prompt-based Topic Modeling Framework (2024.naacl-long)
Copied to clipboard
| Challenge: | TopicGPT uses large language models to uncover latent topics in text . topic models represent topics as bags of words that require "reading the tea leaves" topic models also offer limited control over formatting and specificity of topics . |
| Approach: | TopicGPT uses large language models to uncover latent topics in text . authors propose a prompt-based framework that produces topics that align better with human categorizations . |
| Outcome: | TopicGPT produces topics that align better with human categorizations compared to competing methods. |
RollingLDA: An Update Algorithm of Latent Dirichlet Allocation to Construct Consistent Time Series from Textual Data (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating time series on textual data are not efficient . |
| Approach: | They propose a rolling version of the Latent Dirichlet Allocation, called RollingLDA . they compute similarity of sequentially obtained topic and word distributions over consecutive time periods . |
| Outcome: | The proposed method is based on the popular model Latent Dirichlet Allocation . it is able to build time series consistent with previous states of the model . |
More Than Words: Collocation Retokenization for Latent Dirichlet Allocation Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Latent Dirichlet Allocation models ingest words to discover their latent topics . but it is unclear how to achieve the best results for languages without marked word boundaries . |
| Approach: | They propose to use retokenization to merge frequent token ngrams into collocations in input to a Latent Dirichlet Allocation model. |
| Outcome: | The proposed model improves topic coherence and coherency in Chinese and Thai . the proposed model is more coherent and clearer than unmerged models . |
Real-time Change Point Detection using On-line Topic Models (C18-1)
Copied to clipboard
| Challenge: | Existing methods for detecting events from publicly available data streams such as twitter have been used to model topics from large corpora. |
| Approach: | They propose to use on-line Latent Dirichlet Allocation to model topic shifts and on-lines change point detection algorithms to detect when significant changes occur. |
| Outcome: | The proposed algorithm yields F-scores up to 52% on the detection of real-life changes from social media data streams. |
IndoCL: Benchmarking Indonesian Language Development Assessment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent interest has surged in applying natural language processing (NLP) and machine learning (ML) to evaluate language development in both first (L1) and second (L2) language acquisition. |
| Approach: | They propose to use an Indonesian corpus as a benchmark for LDA tasks and to use existing large-scale language models to improve performance. |
| Outcome: | The proposed model extracts language-independent features, relieving laborious computation and reliance on specific language. |
Scalable Collapsed Inference for High-Dimensional Topic Models (N19-1)
Copied to clipboard
| Challenge: | Existing methods have achieved two out of three criteria simultaneously, but never all three at once. |
| Approach: | They propose an online inference algorithm which leverages stochasticity to scale well in the number of documents and sparsity to achieve accurate inference. |
| Outcome: | The proposed algorithm scales well in the number of documents and topics while achieving accurate inference. |
The Effects of Unimodal Representation Choices on Multimodal Learning (L18-1)
Copied to clipboard
| Challenge: | In the real world, multiple modes of information are gathered to create knowledge in a way humans can understand. |
| Approach: | They propose to combine unimodal representations to map multiple modes of information to a single space . they argue that the way they are combined can affect performance and classification metrics . |
| Outcome: | The proposed model can be used to correlate words in a textual description of an object with multimodal representations. |
Semantic Relatedness Based Re-ranker for Text Spotting (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to text spotting are limited by semantic similarity, but they can be useful for other tasks. |
| Approach: | They propose a neural approach to learn semantic relatedness from existing sentences. |
| Outcome: | The proposed approach outperforms existing approaches when applied to a text spotting task. |
Optimizing Annotation Effort Using Active Learning Strategies: A Sentiment Analysis Case Study in Persian (2020.lrec-1)
Copied to clipboard
Seyed Arad Ashrafi Asli, Behnam Sabeti, Zahra Majdabadi, Preni Golazizian, Reza Fahmi, Omid Momenzadeh
| Challenge: | Existing deep learning approaches require huge amounts of data to be trained properly. |
| Approach: | They propose to use Persian as a model to choose the samples for annotation instead of labeling the whole dataset. |
| Outcome: | The proposed models achieve the baseline performance with a significantly lower amount of labeled data. |
Evaluating Topic Quality with Posterior Variability (D19-1)
Copied to clipboard
| Challenge: | Probabilistic topic models such as latent Dirichlet allocation (LDA) are widely used for NLP tasks which require the extraction of latent themes. |
| Approach: | They propose to measure topic quality using the variability of posterior distributions of probabilistic topic models. |
| Outcome: | The proposed metric achieves state-of-the-art correlations with human judgments of topic quality in experiments on three corpora. |
Detecting Multiple Transitions in Literary Texts (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing systems that can detect multiple transitions in texts are ineffective due to the large amount of texts available. |
| Approach: | They propose a system that can detect multiple transitions in literary texts . they extend existing system so it can detect transitions, and introduce multiple transition topics . |
| Outcome: | The proposed system outperforms the existing system on texts with known transitions and on single boundary texts. |
Hierarchical Dirichlet Gaussian Marked Hawkes Process for Narrative Reconstruction in Continuous Time Domain (D18-1)
Copied to clipboard
| Challenge: | Existing methods for reconstructing narratives and thread structures of news articles and discussions are lacking . temporal characteristics, triggering event relations, and meta information are used to solve the problem . |
| Approach: | They propose a Hierarchical Dirichlet Gaussian Marked Hawkes process for reconstructing narratives and thread structures of news articles and discussion posts. |
| Outcome: | The proposed model outperforms baseline models on real-world datasets and Wikipedia conversations. |
Large Language Models Struggle to Describe the Haystack without Human Help: A Social Science-Inspired Evaluation of Topic Models (2025.acl-long)
Copied to clipboard
Zongxia Li, Lorena Calvo-Bartolomé, Alexander Miserlis Hoyle, Paiheng Xu, Daniel Kofi Stephens, Juan Francisco Fung, Alden Dima, Jordan Lee Boyd-Graber
| Challenge: | a common use of NLP is to facilitate the understanding of large document collections. |
| Approach: | They propose to use large language models to replace probabilistic topic models in real-world applications. |
| Outcome: | The proposed model generates more human-readable topics and shows higher average win probabilities than traditional models for data exploration. |
GraphBTM: Graph Enhanced Autoencoded Variational Inference for Biterm Topic Model (D18-1)
Copied to clipboard
| Challenge: | Conventional topic models suffer different problems in different settings. |
| Approach: | They propose a novel way to model word-pairs named biterms in the whole corpus and a Graph Convolutional Networks (GCNs) with residual connections to extract transitive features from biterm. |
| Outcome: | The proposed model generates more coherent topics compared with previous methods. |
Large Language Models Offer an Alternative to the Traditional Approach of Topic Modelling (2024.lrec-main)
Copied to clipboard
| Challenge: | Topic modelling has found extensive use in automatically detecting significant topics within a corpus of documents, but there are certain drawbacks. |
| Approach: | They propose a framework that prompts large language models to generate topics from a given set of documents and establish evaluation protocols to assess the clustering efficacy of LLMs. |
| Outcome: | The proposed model generates relevant topic titles and adheres to human guidelines to refine and merge topics. |
An Empirical Analysis of the Writing Styles of Persona-Assigned LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent efforts to "personalize" large language models by assigning them specific personas are limited by current knowledge of how well they perform. |
| Approach: | They use a style embedding model to analyze writing styles of persona-assigned LLMs . they find significant style differences between personas using Kullback-Leibler divergence . |
| Outcome: | The proposed model shows significant differences in writing styles among personas across socio-demographic groups. |
Tell, Don’t Show: Leveraging Language Models’ Abstractive Retellings to Model Literary Themes (2025.findings-acl)
Copied to clipboard
| Challenge: | Literature challenges traditional bag-of-words approaches for topic modeling because narrative language focuses on immersive sensory details instead of abstractive description or exposition. |
| Approach: | They propose a topic modeling approach that prompts generative language models to *tell* what passages *show*, thereby translating narratives’ surface forms into higher-level concepts and themes. |
| Outcome: | The proposed model can translate narratives’ surface forms into higher-level concepts and themes than by running LDA alone or directly asking LMs to list topics. |