| Challenge: | Existing topic models do not organize topics into coherent groups or hierarchies. |
| Approach: | They propose a tree-structured neural topic model with an infinite number of branches and a topic distribution over a forest. |
| Outcome: | The proposed model improves data scalability and competitive performance when inducing latent topics and tree structures. |
Similar Papers
Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for topic modeling learn topics with a flat structure . however, such methods have data scalability issues . |
| Approach: | They propose to use nonparametric neural variational inference to extract a tree-structured topic model with reasonable structure, low redundancy, and adaptable widths. |
| Outcome: | The proposed model extracts a tree-structured topic hierarchy with reasonable structure, low redundancy, and adaptable widths. |
Nonparametric Forest-Structured Neural Topic Modeling (2022.coling-1)
Copied to clipboard
| Challenge: | Existing hierarchical neural topic models can only extract topics at the same level. |
| Approach: | They propose to use self-attention mechanism to capture parent-child topic relationships and build a sparse directed acyclic graph to form a topic forest. |
| Outcome: | The proposed model outperforms baseline models on topic hierarchical rationality and affinity. |
Scale-Invariant Infinite Hierarchical Topic Model (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing hierarchical topic models yield fragmented topics with overlapping themes whose expected probability becomes exponentially smaller along the depth of the tree. |
| Approach: | They propose a hierarchical infinite hierarchic topic model that adapts to topic creation to make expected topic probability decay considerably slower than existing models. |
| Outcome: | The proposed model has better topic uniqueness and hierarchical diversity than existing approaches. |
Dynamic Structured Neural Topic Model with Self-Attention Mechanism (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent topic models that capture the time-series evolution of topics assume that topics evolve independently without interaction. |
| Approach: | They propose a dynamic structured neural topic model which captures topic dependencies while capturing their dependencies. |
| Outcome: | The proposed model outperforms a prior dynamic embedded topic model regarding perplexity and coherence while maintaining sufficient diversity across topics. |
Neural Topic Modeling based on Cycle Adversarial Training and Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural topic models have been widely used to extract common topics across documents. |
| Approach: | They propose a framework to apply contrastive learning directly to the decoder . they propose 'self-supervised' contrastive loss to make the generator capture similar topic information . |
| Outcome: | The proposed framework outperforms baselines on four benchmark datasets. |
Benchmarking Neural Topic Models: An Empirical Study (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural topic modeling has been attracting much attention recently due to its ability to leverage the advantages of both neural networks and probabilistic topic models. |
| Approach: | They propose to evaluate neural topic models in three tasks using large datasets and a set of metrics to compare them. |
| Outcome: | The proposed models perform better in the first and third tasks than the traditional probabilistic models and are better in many cases. |
Neural Topic Modeling with Cycle-Consistent Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances on deep generative models have attracted significant interest in neural topic modeling. |
| Approach: | They propose an adversarial-neural topic model which uses Dirichlet prior to capture the semantic patterns in latent topics. |
| Outcome: | The proposed models outperform competing models on unsupervised/supervised topic modeling and text classification. |
Towards Reinterpreting Neural Topic Models via Composite Activations (2022.emnlp-main)
Copied to clipboard
| Challenge: | Most Neural Topic Models (NTMs) use a variational auto-encoder framework producing K topics limited to the size of the encoder’s output. |
| Approach: | They propose a model-free two-stage process to reinterpret NTM and derive further insights on the state of the trained model. |
| Outcome: | The proposed model-free process decouples the strict interpretation of topics from the original NTM and evaluates them on a large external corpus. |
TAN-NTM: Topic Attention Networks for Neural Topic Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Topic models have been widely used to learn text representations and gain insight into document corpora. |
| Approach: | They propose a framework which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. |
| Outcome: | The proposed model improves on two downstream tasks: document classification and topic guided keyphrase generation. |
Topic Modeling: Contextual Token Embeddings Are All You Need (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current neural approaches to topic modeling have not been able to solve all of the problems. |
| Approach: | They propose a topic modeling approach that uses document contextual token embeddings to find topics and find topic spans within documents. |
| Outcome: | The proposed model outperforms the current state-of-the-art models on a comprehensive set of topic model evaluation metrics. |