Incremental processing of noisy user utterances in the spoken language understanding task (D19-55)
Copied to clipboard
| Challenge: | triggered actions with high executions times can cause dialog systems to react slowly due to high latency and high latex. |
| Approach: | They propose a model-agnostic method to achieve high quality in processing incrementally produced partial utterances. |
| Outcome: | The proposed method improves the metric F1-score by 47.91 percentage points . the proposed method can be used to create low-latency natural language understanding components on ATIS datasets. |
Similar Papers
Incremental Natural Language Processing: Challenges, Strategies, and Evaluation (C18-1)
Copied to clipboard
| Challenge: | In this survey, I consolidate and categorize the approaches, identifying similarities and differences in computation and data, and show trade-offs that have to be considered. |
| Approach: | They consolidate and categorize approaches to incremental processing and show trade-offs that have to be considered. |
| Outcome: | The proposed approaches show that they have similarities and differences in computation and data and that they are not trivial. |
Re-framing Incremental Deep Language Models for Dialogue Processing with Multi-task Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Using a multi-task learning framework, we train a universal incremental dialogue processing model with four tasks of disfluency detection, language modelling, part-of-speech tagging and utterance segmentation in a simple deep recurrent setting. |
| Approach: | They propose a multi-task learning framework to train a universal incremental dialogue processing model with four tasks of disfluency detection, language modelling, part-of-speech tagging and utterance segmentation in a simple deep recurrent setting. |
| Outcome: | The proposed model outperforms individual tasks and delivers competitive performance. |
A Targeted Assessment of Incremental Processing in Neural Language Models and Humans (2021.acl-long)
Copied to clipboard
| Challenge: | Using by-word reaction time data, we compare incremental processing in humans and neural language models across a range of structural phenomena. |
| Approach: | They propose to scale up incremental processing in humans and language models by collecting by-word reaction time data for 16 different syntactic test suites. |
| Outcome: | The proposed model outputs match human and model accuracy scores, but underpredict the difference in magnitude of incremental processing difficulty between grammatical and ungrammatically-spaced sentences. |
A Comprehensive Evaluation of Incremental Speech Recognition and Diarization for Conversational AI (2020.coling-main)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems are increasingly powerful and more numerous with several options existing as a service. |
| Approach: | They evaluate the most popular automatic speech recognition systems with metrics and experiments designed with these standards in mind. |
| Outcome: | The most popular ASR systems are Microsoft and IBM, and none are suitable for natural spontaneous conversations in real-time. |
Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work attempts to apply incremental processing to NLUs but this is computationally expensive and does not scale efficiently for long sequences. |
| Approach: | They propose to apply Transformers incrementally via restart-incrementality by repeatedly feeding, to an unchanged model, increasingly longer input prefixes to produce partial outputs. |
| Outcome: | The proposed model has better incremental performance and faster inference speed compared to the standard Transformer and LT with restart-incrementality, at the cost of part of the non-incremental quality. |
Best of Both Worlds: Making High Accuracy Non-incremental Transformer-based Disfluency Detection Incremental (2021.acl-long)
Copied to clipboard
| Challenge: | Currently, Transformer-based text classifiers are not suitable for live incremental processing, operating only on the level of complete sentence inputs. |
| Approach: | They propose to introduce a method for word-by-word left-to-right incremental processing to Transformers such as BERT, models without an intrinsic sense of linear order. |
| Outcome: | The proposed method maintains high non-incremental performance while operating strictly incrementally. |
Online Semantic Parsing for Latency Reduction in Task-Oriented Dialogue (2022.acl-long)
Copied to clipboard
| Challenge: | Standard conversational semantic parsing maps a user's intent into an executable program, but execution is slow when expensive function calls are included. |
| Approach: | They propose a task of online semantic parsing to predict and execute function calls while the user is still speaking. |
| Outcome: | The proposed approach reduces latency with good parsing quality and execution cost. |
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks (2023.acl-long)
Copied to clipboard
Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin, Ankita Pasad, Felix Wu, Roshan S Sharma, Wei-Lun Wu, Hung-yi Lee, Karen Livescu, Shinji Watanabe
| Challenge: | Spoken language understanding (SLU) tasks have received little attention and resources compared to lower-level tasks like speech and speaker recognition. |
| Approach: | They propose annotated SLU benchmark tasks based on freely available speech data to complement existing benchmarks and address gaps in the evaluation landscape. |
| Outcome: | The proposed benchmarks complement existing benchmarks and address gaps in the evaluation landscape. |
Adversarial NLI: A New Benchmark for Natural Language Understanding (2020.acl-main)
Copied to clipboard
| Challenge: | a new large-scale NLI benchmark dataset is presented to test models on a variety of popular NLIs. |
| Approach: | They propose a large-scale NLI benchmark dataset that is iteratively compared with a human-and-model-in-the-loop procedure. |
| Outcome: | The proposed method can be applied in a never-ending learning scenario, becoming a moving target for NLU, rather than a static benchmark that will quickly saturate. |
MoE-SLU: Towards ASR-Robust Spoken Language Understanding via Mixture-of-Experts (2024.findings-acl)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) is a crucial task in task-oriented dialogue systems. |
| Approach: | They propose an ASR-Robust SLU framework based on the mixture-of-experts technique to generate additional transcripts from clean transcripts and use it to weigh the representations of the generated transcripts, ASR transcripts . |
| Outcome: | The proposed framework achieves state-of-the-art on three benchmark SLU datasets. |