UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructions (2024.naacl-long)
Copied to clipboard
Siddhant Arora, Hayato Futami, Jee-weon Jung, Yifan Peng, Roshan Sharma, Yosuke Kashiwagi, Emiru Tsunoo, Karen Livescu, Shinji Watanabe
| Challenge: | Recent studies leverage large language models with multi-tasking capabilities, using natural language prompts to guide the model’s behavior and surpassing performance of task-specific models. |
| Approach: | They adapt a pre-trained automatic speech recognition model to additional tasks using single-token task specifiers. |
| Outcome: | The proposed model can generalize to new datasets and languages for seen task types. |
Similar Papers
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks (2023.acl-long)
Copied to clipboard
Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin, Ankita Pasad, Felix Wu, Roshan S Sharma, Wei-Lun Wu, Hung-yi Lee, Karen Livescu, Shinji Watanabe
| Challenge: | Spoken language understanding (SLU) tasks have received little attention and resources compared to lower-level tasks like speech and speaker recognition. |
| Approach: | They propose annotated SLU benchmark tasks based on freely available speech data to complement existing benchmarks and address gaps in the evaluation landscape. |
| Outcome: | The proposed benchmarks complement existing benchmarks and address gaps in the evaluation landscape. |
Zero-Shot Spoken Language Understanding via Large Language Models: A Preliminary Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown promising results in zero-shot settings, which motivates us to explore prompt-based methods. |
| Approach: | They propose a two-stage framework which transforms the SLU task into a question-answering problem by directly prompting LLMs. |
| Outcome: | The proposed framework can be built by directly prompting LLMs to understand user needs without training data. |
On the Evaluation of Speech Foundation Models for Spoken Language Understanding (2024.findings-acl)
Copied to clipboard
Siddhant Arora, Ankita Pasad, Chung-Ming Chien, Jionghao Han, Roshan Sharma, Jee-weon Jung, Hira Dhamyal, William Chen, Suwon Shon, Hung-yi Lee, Karen Livescu, Shinji Watanabe
| Challenge: | Spoken language understanding evaluation (SLUE) benchmarks are used to benchmark complex spoken language understanding tasks on natural speech. |
| Approach: | They propose a set of benchmark tasks to evaluate spoken language understanding on natural speech . they use pre-trained speech foundation models to evaluate the utility of different SFMs . |
| Outcome: | The proposed framework outperforms pre-trained speech foundation models on natural speech . the proposed framework also outperformed self-supervised SFMs on the sequence generation tasks . |
OpenSLU: A Unified, Modularized, and Extensible Toolkit for Spoken Language Understanding (2023.acl-demo)
Copied to clipboard
| Challenge: | Spoken Language Understanding (SLU) is a task-oriented dialogue system . open-source toolkit provides a unified, modularized, and extensible toolkit for SLU . |
| Approach: | They introduce an open-source toolkit to provide a unified toolkit for spoken language understanding. |
| Outcome: | The proposed toolkit unifies 10 models for both single-intent and multi-intention scenarios. |
75 Languages, 1 Model: Parsing Universal Dependencies Universally (D19-1)
Copied to clipboard
| Challenge: | UDify is a multilingual multi-task model that can predict universal part-of-speech, morphological features, lemmas, and dependency trees. |
| Approach: | They evaluate UDify, a multilingual multi-task model capable of predicting universal part-of-speech, morphological features, lemmas, and dependency trees simultaneously for all 124 Universal Dependencies treebanks across 75 languages. |
| Outcome: | The proposed model can predict universal part-of-speech, morphological features, lemmas, and dependency trees for all 124 treebanks across 75 languages. |
NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialogue (2022.findings-naacl)
Copied to clipboard
| Challenge: | NLU++ provides a more challenging evaluation environment for dialogue NLU models . Typical ToD systems still rely on a modular design . |
| Approach: | They propose to use NLU++ to provide a more challenging evaluation environment for dialogue NLU models. |
| Outcome: | The proposed dataset improves existing datasets and provides a much more challenging evaluation environment for dialogue NLU models. |
A Learnable Skill Combination Strategy for Multi-task Learning in Natural Language Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | a novel multi-task learning framework for domain-specific natural language understanding tasks addresses these limitations by combing multiple tasks into a single framework. |
| Approach: | They propose a multi-task learning framework that decomposes the language model into modular skill components and employs a dynamic, learnable skill-combination mechanism to adaptively handle diverse tasks. |
| Outcome: | The proposed framework surpasses conventional multi-task learning approaches in performance. |
The Interpreter Understands Your Meaning: End-to-end Spoken Language Understanding Aided by Speech Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Modern artificial intelligence is characterized by large pretrained language models with strong language capabilities to be adapted to various downstream tasks. |
| Approach: | They propose to use the task of speech translation (ST) to pretrain speech models for end-to-end SLU on intra- and cross-lingual scenarios. |
| Outcome: | The proposed model achieves higher performance over baselines on monolingual and multilingual intent classification as well as spoken question answering using SLURP, MINDS-14, and NMSQA benchmarks. |
Multi-task Learning of Spoken Language Understanding by Integrating N-Best Hypotheses with Hierarchical Attention (2020.coling-industry)
Copied to clipboard
| Challenge: | Existing methods to integrate hypotheses into speech recognition systems are noisy and can cause information loss. |
| Approach: | They propose to integrate hypotheses into multi-task learning and transfer learning to improve performance. |
| Outcome: | The proposed model improves domain and intent classification by 19% and 37% compared to current methods . the proposed model could recover transcription and rewrite the query for a better understanding . |
MoE-SLU: Towards ASR-Robust Spoken Language Understanding via Mixture-of-Experts (2024.findings-acl)
Copied to clipboard
| Challenge: | Spoken language understanding (SLU) is a crucial task in task-oriented dialogue systems. |
| Approach: | They propose an ASR-Robust SLU framework based on the mixture-of-experts technique to generate additional transcripts from clean transcripts and use it to weigh the representations of the generated transcripts, ASR transcripts . |
| Outcome: | The proposed framework achieves state-of-the-art on three benchmark SLU datasets. |