Decay-Function-Free Time-Aware Attention to Context and Speaker Indicator for Spoken Language Understanding (N19-1)
Copied to clipboard
| Challenge: | Existing models that use contextual information of dialogues to improve spoken language understanding (SLU) select the wrong history when the histories are similar in content. |
| Approach: | They propose time-aware models that automatically learn the latent time-decay function of the history without a manual time- decay. |
| Outcome: | The proposed models achieve higher F1 scores than state-of-the-art models on a benchmark dataset . |
Similar Papers
How Time Matters: Learning Time-Decay Attention for Contextual Spoken Language Understanding in Dialogues (N18-1)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) is an essential component in conversational systems. |
| Approach: | They propose a universal time-decay attention mechanism that can be used to decay utterances on the sentence-level and speaker-level. |
| Outcome: | The proposed model significantly improves the state-of-the-art model for contextual understanding performance on the benchmark Dialogue State Tracking Challenge (DSTC4) dataset. |
Temporal Generalization for Spoken Language Understanding (2022.naacl-industry)
Copied to clipboard
| Challenge: | Spoken Language Understanding models are usually trained offline on historical data, but must perform well on incoming user requests after deployment. |
| Approach: | They propose different strategies for achieving good temporal generalization . they focus on temporal drift, where the distribution of utterances may change . |
| Outcome: | The proposed model can perform well on unseen domains, e.g., upcoming data. |
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks (2023.acl-long)
Copied to clipboard
Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin, Ankita Pasad, Felix Wu, Roshan S Sharma, Wei-Lun Wu, Hung-yi Lee, Karen Livescu, Shinji Watanabe
| Challenge: | Spoken language understanding (SLU) tasks have received little attention and resources compared to lower-level tasks like speech and speaker recognition. |
| Approach: | They propose annotated SLU benchmark tasks based on freely available speech data to complement existing benchmarks and address gaps in the evaluation landscape. |
| Outcome: | The proposed benchmarks complement existing benchmarks and address gaps in the evaluation landscape. |
Contextual Domain Classification with Temporal Representations (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing studies that incorporate context in SLU have focused on domains where context is limited to a few minutes. |
| Approach: | They propose temporal representations that combine wall-clock second difference and turn order offset information to utilize both recent and distant context in a novel large-scale setup. |
| Outcome: | The proposed model reduces 13.04% of classification errors compared to baseline . previous studies have focused on domains where context is limited to a few minutes . |
Syntax Matters: Towards Spoken Language Understanding via Syntax-Aware Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on SLU systems have focused on integrating syntactic information into language models. |
| Approach: | They propose a model where attention scopes are constrained based on syntactic relationships. |
| Outcome: | The proposed model improves on three datasets and can be integrated into other language models to further boost their performance. |
PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding (2024.findings-emnlp)
Copied to clipboard
Trang Le, Daniel Lazar, Suyoun Kim, Shan Jiang, Duc Le, Adithya Sagar, Aleksandr Livshits, Ahmed Aly, Akshat Shrivastava
| Challenge: | End-to-end models for Spoken Language Understanding have been autoregressive, resulting in higher latencies. |
| Approach: | They propose a method that uses Connectionist Temporal Classification to train robust non-autoregressive deliberation models. |
| Outcome: | The proposed method achieves 10x latency reduction over autoregressive models while preserving ability to correct ASR mistranscriptions. |
MoE-SLU: Towards ASR-Robust Spoken Language Understanding via Mixture-of-Experts (2024.findings-acl)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) is a crucial task in task-oriented dialogue systems. |
| Approach: | They propose an ASR-Robust SLU framework based on the mixture-of-experts technique to generate additional transcripts from clean transcripts and use it to weigh the representations of the generated transcripts, ASR transcripts . |
| Outcome: | The proposed framework achieves state-of-the-art on three benchmark SLU datasets. |
CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots (D19-1)
Copied to clipboard
| Challenge: | Prior work on contextual NLU has been limited in terms of the types of contextual signals used and the understanding of their impact on the model. |
| Approach: | They propose a context-aware self-attentive NLU model that uses multiple signals over a variable context window, such as previous intents, slots, dialog acts and utterances, in addition to the current user uttered. |
| Outcome: | The proposed model outperforms a baseline model on two conversational datasets yielding a gain of up to 7% on the IC task. |
Time-Aware Language Models as Temporal Knowledge Bases (2022.tacl-1)
Copied to clipboard
Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, William W. Cohen
| Challenge: | Existing language models are trained on snapshots of data collected at a specific moment in time. |
| Approach: | They propose a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time. |
| Outcome: | The proposed method improves memorization of seen facts and calibration on unseen facts from future time periods. |
Incremental processing of noisy user utterances in the spoken language understanding task (D19-55)
Copied to clipboard
| Challenge: | triggered actions with high executions times can cause dialog systems to react slowly due to high latency and high latex. |
| Approach: | They propose a model-agnostic method to achieve high quality in processing incrementally produced partial utterances. |
| Outcome: | The proposed method improves the metric F1-score by 47.91 percentage points . the proposed method can be used to create low-latency natural language understanding components on ATIS datasets. |