Dynamic Structured Neural Topic Model with Self-Attention Mechanism (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent topic models that capture the time-series evolution of topics assume that topics evolve independently without interaction. |
| Approach: | They propose a dynamic structured neural topic model which captures topic dependencies while capturing their dependencies. |
| Outcome: | The proposed model outperforms a prior dynamic embedded topic model regarding perplexity and coherence while maintaining sufficient diversity across topics. |
Similar Papers
TAN-NTM: Topic Attention Networks for Neural Topic Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Topic models have been widely used to learn text representations and gain insight into document corpora. |
| Approach: | They propose a framework which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. |
| Outcome: | The proposed model improves on two downstream tasks: document classification and topic guided keyphrase generation. |
Dynamic Topic Modeling by Clustering Embeddings from Pretrained Language Models: A Research Proposal (2022.aacl-srw)
Copied to clipboard
| Challenge: | Neural Topic Models (NTMs) are topic models that are created with the help of a pretrained language model. |
| Approach: | They propose to do Neural Topic Modeling by Clustering document Embeddings (NTM-CE) with a pretrained language model to create dynamic topic models. |
| Outcome: | The proposed model can be evaluated theoretically and practically using quantitative measurements of coherence and human evaluation to evaluate the model. |
Nonparametric Forest-Structured Neural Topic Modeling (2022.coling-1)
Copied to clipboard
| Challenge: | Existing hierarchical neural topic models can only extract topics at the same level. |
| Approach: | They propose to use self-attention mechanism to capture parent-child topic relationships and build a sparse directed acyclic graph to form a topic forest. |
| Outcome: | The proposed model outperforms baseline models on topic hierarchical rationality and affinity. |
Tree-Structured Neural Topic Model (2020.acl-main)
Copied to clipboard
| Challenge: | Existing topic models do not organize topics into coherent groups or hierarchies. |
| Approach: | They propose a tree-structured neural topic model with an infinite number of branches and a topic distribution over a forest. |
| Outcome: | The proposed model improves data scalability and competitive performance when inducing latent topics and tree structures. |
Coherence-Aware Neural Topic Modeling (D18-1)
Copied to clipboard
| Challenge: | Topic models are evaluated for their ability to describe documents well (i.e. low perplexity) topic coherence is not optimized for and is only evaluated after training. |
| Approach: | They propose to incorporate a topic coherence objective into the training process by incorporating a coherency objective into a model. |
| Outcome: | The proposed model exhibits similar level of perplexity as baseline models but significantly higher topic coherence. |
Beyond Coherence: Improving Temporal Consistency and Interpretability in Dynamic Topic Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing topic models capture bag-of-words statistics but lack semantic priors . interpretability remains shallow, relying on noisy top-word lists that obscure thematic clarity. |
| Approach: | They propose a variational framework to capture more faithful temporal trajectories . they propose to use entropy-regularized optimal transport to align entire topic constellations . |
| Outcome: | The proposed framework captures more faithful temporal trajectories and improves interpretability. |
Self-Supervised Neural Topic Modeling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Topic models are useful tools for analyzing and interpreting the main underlying themes of large corpora of text. |
| Approach: | They propose a self-supervised neural topic model that learns a topic representation jointly from three co-occurring words and a document that the triple originates from. |
| Outcome: | The proposed model outperforms existing topic models in coherence metrics and document clustering accuracy. |
Dynamic and Static Topic Model for Analyzing Time-Series Document Collections (P18-2)
Copied to clipboard
| Challenge: | a collection of documents often has dynamic structures, i.e., topics evolve along time depending on multiple topics in the past. |
| Approach: | They propose a dynamic and static topic model that considers dynamic and dynamic structures of topic evolution and static structures of the topic hierarchy at each time. |
| Outcome: | The proposed model outperforms conventional models on scientific papers . it shows that extracted topic structures are useful for analyzing research activities . |
Towards Reinterpreting Neural Topic Models via Composite Activations (2022.emnlp-main)
Copied to clipboard
| Challenge: | Most Neural Topic Models (NTMs) use a variational auto-encoder framework producing K topics limited to the size of the encoder’s output. |
| Approach: | They propose a model-free two-stage process to reinterpret NTM and derive further insights on the state of the trained model. |
| Outcome: | The proposed model-free process decouples the strict interpretation of topics from the original NTM and evaluates them on a large external corpus. |
Benchmarking Neural Topic Models: An Empirical Study (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural topic modeling has been attracting much attention recently due to its ability to leverage the advantages of both neural networks and probabilistic topic models. |
| Approach: | They propose to evaluate neural topic models in three tasks using large datasets and a set of metrics to compare them. |
| Outcome: | The proposed models perform better in the first and third tasks than the traditional probabilistic models and are better in many cases. |