| Challenge: | Existing systems struggle to have consistent long term conversations with the users and fail to build rapport. |
| Approach: | They propose a hierarchical model with self attention for topic spotting . they compare it to previous proposed techniques for topic detection . |
| Outcome: | The proposed model outperforms existing models for topic spotting and deep models for text classification in an online setting. |
Similar Papers
Dialog-Post: Multi-Level Self-Supervised Objectives and Hierarchical Model for Dialogue Post-Training (2023.acl-long)
Copied to clipboard
| Challenge: | a new method for dialogue representation and understanding is proposed . pre-trained language models (PLMs) are inappropriate for dialogue understanding tasks . |
| Approach: | They propose a method that trains pre-trained language models to fit dialogues . they use a hierarchical segment-wise self-attention network to model dialogues more comprehensively . |
| Outcome: | The proposed method outperforms existing models and achieves a 3.3% improvement on average. |
A Hierarchical Neural Attention-based Text Classifier (D18-1)
Copied to clipboard
| Challenge: | Existing hierarchical classification models are unable to handle large corpora and the number of categories increases with increasing corpus. |
| Approach: | They propose to use external knowledge to introduce a hierarchical neural attention-based classifier to help with the classification of documents. |
| Outcome: | The proposed model performs better than or comparable to state-of-the-art hierarchical models at significantly lower computational cost while maintaining high interpretability. |
Dialogue Act Classification with Context-Aware Self-Attention (N19-1)
Copied to clipboard
| Challenge: | Recent work in Dialogue Act classification has treated the task as a sequence labeling problem using hierarchical deep neural networks. |
| Approach: | They propose a hierarchical deep neural network to model different levels of utterance and dialogue act semantics and use contextual dependencies to improve performance. |
| Outcome: | The proposed model improves on the Switchboard Dialogue Act Corpus while maintaining high accuracy. |
Phrase-level Self-Attention Networks for Universal Sentence Encoding (D18-1)
Copied to clipboard
| Challenge: | Phrase-level self-attention networks (PSAN) can capture context dependencies at the phrase level instead of the sentence level. |
| Approach: | They propose to perform self-attention across words inside a phrase to capture context dependencies at the phrase level and use the gated memory updating mechanism to refine each word’s representation hierarchically with longer-term context dependency captured in a larger phrase. |
| Outcome: | The proposed model can achieve state-of-the-art performance across a plethora of NLP tasks including binary and multi-class classification, natural language inference and sentence similarity. |
Neural Attention-Aware Hierarchical Topic Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural topic models (NTMs) use deep neural networks to learn topic information. |
| Approach: | They propose a variational autoencoder model that reconstructs sentence and document word counts using bag-of-words embeddings and pre-trained semantic embedders. |
| Outcome: | The proposed model lowers reconstruction errors at sentence and document levels and finds more coherent topics from real-world datasets. |
Hierarchical Topic Modeling via Contrastive Learning and Hyperbolic Embedding (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing hierarchical topic models are based on Euclidean space, which cannot retain the hierarchically semantic information in the corpus, leading to irrational structure of the generated topics. |
| Approach: | They propose a novel hierarchical topic model that uses contrastive learning to capture information from documents. |
| Outcome: | The proposed model performs on topic coherence and topic diversity, and on the rationality of the topic hierarchy. |
TAN-NTM: Topic Attention Networks for Neural Topic Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Topic models have been widely used to learn text representations and gain insight into document corpora. |
| Approach: | They propose a framework which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. |
| Outcome: | The proposed model improves on two downstream tasks: document classification and topic guided keyphrase generation. |
Hierarchical Attention Graph for Scientific Document Summarization in Global and Local Level (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for document summarization focus on one type of relation, neglecting the simultaneous effective modeling of both relations. |
| Approach: | They propose a graph neural network-based approach to local and global document summarization using hierarchical discourses. |
| Outcome: | The proposed approach improves on two benchmark datasets and shows that hierarchical structures are important for document summarization. |
Self-Attention Networks Can Process Bounded Hierarchical Languages (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models that can process formal languages with hierarchical structure are limited in their performance. |
| Approach: | They propose to use a subset of Dyck-k with depth bounded by D to train self-attention networks. |
| Outcome: | The proposed model can process Dyck-(k, D) with depth bounded by D, which better captures the hierarchical structure of natural language. |
Improving Context Modeling in Neural Topic Segmentation (2020.aacl-main)
Copied to clipboard
| Challenge: | Recent work favors highly effective neural supervised approaches for topic segmentation but current neural solutions are limited in how they model context. |
| Approach: | They propose to enhance a hierarchical attention biLSTM network-based topic segmenter to better model context by adding a coherence-related auxiliary task and restricted self-attention. |
| Outcome: | The proposed model outperforms SOTA approaches on three datasets and on four real-world benchmarks. |