Pre-training Intent-Aware Encoders for Zero- and Few-Shot Intent Classification (2023.emnlp-main)
Copied to clipboard
Mujeen Sung, James Gung, Elman Mansimov, Nikolaos Pappas, Raphael Shu, Salvatore Romeo, Yi Zhang, Vittorio Castelli
| Challenge: | Existing methods for IC training do not provide sufficient examples for each intent . a novel pre-training method is proposed to provide a better understanding of intents . |
| Approach: | They propose a method that uses contrastive learning with intent psuedo-labels to produce embeddings that are well-suited for IC tasks. |
| Outcome: | The proposed method achieves 5.4% and 4.0% higher accuracy than the current state-of-the-art method on four IC datasets. |
Similar Papers
Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that fine-tuning pre-trained language models with a small set of labeled utterances in a supervised manner is helpful, but it yields an anisotropic feature space, which may suppress the expressive power of the semantic representations. |
| Approach: | They propose to regularize supervised pre-training towards isotropy by contrastive learning and correlation matrix regularizers. |
| Outcome: | The proposed methods improve supervised pre-training by regularizing the feature space towards isotropy. |
Effectiveness of Pre-training for Few-shot Intent Classification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing paradigms further pre-train language models such as BERT on vast amount of unlabeled corpus, but we find it highly effective and efficient to simply fine-tune BERT with roughly 1,000 labeled utterances from public datasets. |
| Approach: | They propose to fine-tune BERT with a small set of labeled utterances from public datasets to achieve a pre-trained model based on a set of 1,000 labeles. |
| Outcome: | The proposed model can outperform existing models on domains with very different semantics on novel domains. |
Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning (2021.emnlp-main)
Copied to clipboard
Jianguo Zhang, Trung Bui, Seunghyun Yoon, Xiang Chen, Zhiwei Liu, Congying Xia, Quan Hung Tran, Walter Chang, Philip Yu
| Challenge: | Existing methods address few-shot intent detection tasks from two perspectives: data augmentation and task-adaptive training with pre-trained models. |
| Approach: | They propose a few-shot intent detection schema using contrastive pre-training and fine-tuning. |
| Outcome: | The proposed method achieves state-of-the-art performance on three challenging intent detection datasets under 5-shot and 10-shot settings. |
IntenDD: A Unified Contrastive Learning Approach for Intent Detection and Discovery (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Intent-related tasks are typically modeled as separate tasks, but a unified approach is proposed . INTENDD uses an entirely unsupervised contrastive learning strategy for representation learning . |
| Approach: | They propose a unified approach to identifying intents from dialogue utterances . they propose an unsupervised contrastive learning strategy for representation learning . |
| Outcome: | The proposed approach outperforms baselines on three intent-related tasks on multiple datasets. |
An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Intent classification and slot filling are key building blocks in task-oriented dialogue systems. |
| Approach: | They propose an explicit-joint and supervised-contrastive learning framework for few-shot intent classification and slot filling. |
| Outcome: | The proposed model extracts intent and slot representations via bidirectional interactions and extends prototypical network to achieve explicit-joint learning. |
New Intent Discovery with Pre-training and Contrastive Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for identifying intents from unlabeled utterances are label-intensive, inefficient, and inaccurate. |
| Approach: | They propose a multi-task strategy to leverage unlabeled data and external labeled data for representation learning. |
| Outcome: | The proposed method outperforms state-of-the-art methods on three intent recognition benchmarks. |
Label Semantic Aware Pre-training for Few-shot Text Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models for text classification use label semantics but few studies have attempted to give models access to informative representations of labels. |
| Approach: | They propose to use label semantics to train generative models by performing secondary pre-training on labeled sentences from a variety of domains. |
| Outcome: | The proposed approach improves generalization and data efficiency of text classification systems while maintaining comparable performance to state-of-the-art models. |
Label Agnostic Pre-training for Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to text classification assume a fixed set of labels . however, in real-world applications, there exists an infinite label space for describing a given text . |
| Approach: | They propose two new methods that inject aspect-level understanding into pre-trained models at train time to improve zero-shot generalization. |
| Outcome: | The proposed methods improve zero-shot generalization on a set of challenging datasets. |
Zero-Shot-BERT-Adapters: a Zero-Shot Pipeline for Unknown Intent Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Intent discovery remains a crucial task in natural language processing . identifying novel, unseen intents remains one of the biggest challenges in this field . |
| Approach: | They propose a multi-language approach to intent discovery using Adapters and a Transformer architecture. |
| Outcome: | The proposed pipeline outperforms baselines in two zero-shot settings for intent classification and unseen intent discovery. |
Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classification (2025.acl-short)
Copied to clipboard
| Challenge: | Current intent classification systems face significant challenges due to the vast number of possible intents and significant semantic overlap among similar intent classes. |
| Approach: | They propose a dynamic label refinement method that retrieves relevant examples for a test input and leverages a large language model to dynamically refine intent labels based on semantic understanding. |
| Outcome: | The proposed method resolves confusion between semantically similar intents and generates more interpretable intent labels. |