A Label-Aware BERT Attention Network for Zero-Shot Multi-Intent Detection in Spoken Language Understanding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for speech-based dialogs have found the inadequacy of relying on simple classification techniques to accomplish the automation task. |
| Approach: | They propose a Label-Aware BERT Attention Network (LABAN) for zero-shot multi-intent detection by encoding input utterances with BERT and building a label embedded space by considering embedded semantics in intent labels. |
| Outcome: | The proposed approach can detect many unseen intent labels correctly on a few/zero-shot setting, and achieves state-of-the-art performance on five multi-intent datasets in normal cases. |
Similar Papers
Zero-Shot-BERT-Adapters: a Zero-Shot Pipeline for Unknown Intent Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Intent discovery remains a crucial task in natural language processing . identifying novel, unseen intents remains one of the biggest challenges in this field . |
| Approach: | They propose a multi-language approach to intent discovery using Adapters and a Transformer architecture. |
| Outcome: | The proposed pipeline outperforms baselines in two zero-shot settings for intent classification and unseen intent discovery. |
Zero-shot User Intent Detection via Capsule Neural Networks (D18-1)
Copied to clipboard
| Challenge: | Existing methods to classify intents are labor-intensive and time-consuming as intents will be diverse and new intents may be involved. |
| Approach: | They propose a zero-shot intent detection problem which aims to detect emerging user intents where no labeled utterances are currently available. |
| Outcome: | The proposed model can discriminate emerging intents when no labeled utterances are available in training data. |
Dual Class Knowledge Propagation Network for Multi-label Few-shot Intent Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on multi-label intent detection are confused by the identical representation of the utterance with multiple labels and overlook the intrinsic intra-class and inter-class relations. |
| Approach: | They propose a dual class knowledge propagation network to learn well-separated representations for utterances with multiple intents. |
| Outcome: | The proposed method outperforms baselines on two multi-label intent datasets by a large margin. |
Effectiveness of Pre-training for Few-shot Intent Classification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing paradigms further pre-train language models such as BERT on vast amount of unlabeled corpus, but we find it highly effective and efficient to simply fine-tune BERT with roughly 1,000 labeled utterances from public datasets. |
| Approach: | They propose to fine-tune BERT with a small set of labeled utterances from public datasets to achieve a pre-trained model based on a set of 1,000 labeles. |
| Outcome: | The proposed model can outperform existing models on domains with very different semantics on novel domains. |
Reconstructing Capsule Networks for Zero-shot Intent Classification (D19-1)
Copied to clipboard
| Challenge: | Existing methods for intent classification are limited due to fast-emerging intents . a recent study shows that existing methods are not effective in recognizing unseen intents. |
| Approach: | They propose to reconstruct capsule networks for zero-shot intent classification by using latent information from labeled utterances. |
| Outcome: | The proposed method outperforms existing methods on two task-oriented dialogue datasets in different languages. |
LCAN: A Label-Aware Contrastive Attention Network for Multi-Intent Recognition and Slot Filling in Task-Oriented Dialogue Systems (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-intent utterances processing remains a persistent challenge due to intricate intent-slot dependencies and semantic ambiguities. |
| Approach: | They propose a label-aware contrastive attention network (LCAN) that integrates label-based attention and contrastive learning strategies to improve semantic understanding and generalization in multi-intent scenarios. |
| Outcome: | The proposed model improves intent recognition and slot filling performance in multi-intent dialogue systems. |
ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to relation extraction require labeled data, but labeling is difficult . Existing models cannot recognize rare instances that are never covered by training data . |
| Approach: | They propose a multi-task learning model that directly predicts unseen relations without hand-crafted attribute labeling and multiple pairwise classifications. |
| Outcome: | The proposed model outperforms existing methods by 13.54% on two well-known datasets. |
Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification (2023.emnlp-main)
Copied to clipboard
Mujeen Sung, James Gung, Elman Mansimov, Nikolaos Pappas, Raphael Shu, Salvatore Romeo, Yi Zhang, Vittorio Castelli
| Challenge: | Existing methods for IC training do not provide sufficient examples for each intent . a novel pre-training method is proposed to provide a better understanding of intents . |
| Approach: | They propose a method that uses contrastive learning with intent psuedo-labels to produce embeddings that are well-suited for IC tasks. |
| Outcome: | The proposed method achieves 5.4% and 4.0% higher accuracy than the current state-of-the-art method on four IC datasets. |
Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that multi-level matching is difficult due to the scarcity of available annotated utterances. |
| Approach: | They propose a method where semantic components are distilled from utterances via multi-head self-attention with additional dynamic regularization constraints. |
| Outcome: | The proposed method improves representations of labeled and unlabeled instances while retaining high-level information. |
Joint Multiple Intent Detection and Slot Labeling for Goal-Oriented Dialog (N19-1)
Copied to clipboard
| Challenge: | Neural network models have gained traction for sentence-level intent classification and token-based slot-label identification. |
| Approach: | They propose a neural network model that performs multi-label classification for identifying multiple intents and produces token-based slot-l labels at the token-level. |
| Outcome: | The proposed model provides a small but statistically significant improvement on the ATIS dataset and 55% accuracy improvement on an internal multi-intent dataset. |