Sparse Parallel Training of Hierarchical Dirichlet Process Topic Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | To scale non-parametric extensions of probabilistic topic models, practitioners rely increasingly on parallel and distributed systems. |
| Approach: | They propose a data-parallel sampler that utilizes all available sources of sparsity found in natural language to control memory requirements and computational complexity. |
| Outcome: | The proposed sampler is able to train a hierarchical Dirichlet process topic model on a well-known corpus (PubMed) with 8m documents and 768m tokens, using a single multi-core machine in under four days. |
Similar Papers
Deep Dirichlet Multinomial Regression (N18-1)
Copied to clipboard
| Challenge: | supervised topic models can incorporate arbitrary document-level features to inform topic priors, but their ability to model corpora is limited by the representation and selection of these features. |
| Approach: | They propose a generative topic model that simultaneously learns document feature representations and topics. |
| Outcome: | The proposed model outperforms DMR and LDA on three datasets and human subjects judge it more representative of associated document features. |
Integration of Knowledge Graph Embedding Into Topic Modeling with Hierarchical Dirichlet Process (N19-1)
Copied to clipboard
| Challenge: | Topic models are used to extract topical structures from document-word frequency representations of the text corpus without supervision. |
| Approach: | They propose a Bayesian nonparametric topic modeling with knowledge graph embedding to employ knowledge graphs to extract more coherent topics. |
| Outcome: | The proposed model performs better on three public datasets than state-of-the-art models on topic coherence and document classification accuracy. |
HyHTM: Hyperbolic Geometry-based Hierarchical Topic Model (2023.findings-acl)
Copied to clipboard
| Challenge: | Hierarchical Topic Models (HTMs) often produce hierarchies where lower-level topics are unrelated and not specific enough to their higher-level subjects. |
| Approach: | They propose a Hyperbolic geometry-based Hierarchical Topic Model that incorporates hierarchical information from hyperbolic geometrics to explicitly model hierarchies in topic models. |
| Outcome: | The proposed model is significantly faster and leaves a much smaller memory footprint than the best-performing baseline. |
Nonlinear Structural Equation Model Guided Gaussian Mixture Hierarchical Topic Modeling (2023.acl-long)
Copied to clipboard
| Challenge: | Existing topic models assume that topics are independent and that they are not a tree structure, which complicates the analysis. |
| Approach: | They propose a neural topic model with a Gaussian mixture prior distribution to improve the model’s ability to adapt to sparse data. |
| Outcome: | The proposed model outperforms baseline models on sparse data on a set of widely used datasets and generates more coherent topics and rational topic structures. |
HiCOT: Improving Neural Topic Models via Optimal Transport and Contrastive Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in neural topic models (NTMs) have improved topic quality but still face challenges: weak document-topic alignment, high inference costs due to large pretrained language models, and limited modeling of hierarchical topic structures. |
| Approach: | They propose a framework that integrates hierarchical clustering and contrastive learning to refine document-topic relationships using compact PLM-based embeddings. |
| Outcome: | The proposed framework improves topic coherence, topic performance, representation quality and computational efficiency over existing NTMs. |
Scale-Invariant Infinite Hierarchical Topic Model (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing hierarchical topic models yield fragmented topics with overlapping themes whose expected probability becomes exponentially smaller along the depth of the tree. |
| Approach: | They propose a hierarchical infinite hierarchic topic model that adapts to topic creation to make expected topic probability decay considerably slower than existing models. |
| Outcome: | The proposed model has better topic uniqueness and hierarchical diversity than existing approaches. |
CluHTM - Semantic Hierarchical Topic Modeling based on CluWords (2020.acl-main)
Copied to clipboard
| Challenge: | Hierarchical Topic modeling (HTM) exploits latent topics and relationships among them as a powerful tool for data analysis and exploration. |
| Approach: | They propose a hierarchical matrix factorization that exploits latent topics and relationships among them to create a powerful tool for data analysis and exploration. |
| Outcome: | The proposed method outperforms baselines and datasets in the vast majority of cases. |
Hierarchical Topic Modeling via Contrastive Learning and Hyperbolic Embedding (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing hierarchical topic models are based on Euclidean space, which cannot retain the hierarchically semantic information in the corpus, leading to irrational structure of the generated topics. |
| Approach: | They propose a novel hierarchical topic model that uses contrastive learning to capture information from documents. |
| Outcome: | The proposed model performs on topic coherence and topic diversity, and on the rationality of the topic hierarchy. |
DocHieNet: A Large and Diverse Dataset for Document Hierarchy Parsing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document hierarchy parsing are limited due to the small scale and inconsistency of datasets. |
| Approach: | They propose a document hierarchy parsing dataset to compensate for the data scarcity problem and propose 'dHP' framework to grasp fine-grained text content and coarse-grounded pattern at layout element level. |
| Outcome: | The proposed framework grasps both fine-grained text content and coarse-grounded pattern at layout element level, enhancing the capacity of pre-trained text-layout models in handling multi-page and multi-level challenges. |
Topic Modeling for Short Texts with Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can be used to solve topic modeling challenges for short texts by contextually learning the meanings of words. |
| Approach: | They propose two approaches to using Large Language Models (LLMs) for topic modeling: parallel prompting and sequential prompting. |
| Outcome: | The proposed methods identify more coherent topics than existing ones while maintaining the diversity of the induced topics. |