Papers with out-of-domain
Robust ASR Error Correction with Conservative Data Filtering (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Error correction (EC) based on large language models is an emerging technology to enhance the performance of automatic speech recognition systems. |
| Approach: | They propose to pair large set of ASR hypotheses with gold references to improve linguistic acceptability over sources and be inferable from available context. |
| Outcome: | The proposed approach significantly reduces overcorrection and improves quality in out-of-domain (OOD) settings. |
A Comprehensive Evaluation of Inductive Reasoning Capabilities and Problem Solving in Large Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Inductive reasoning is fundamental to both human and artificial intelligence. |
| Approach: | They evaluated the inductive reasoning abilities of current Large Language Models (LLMs) and their performance on symbolic tasks. |
| Outcome: | The proposed models fail on symbolic tasks and show that chain-of-thought prompts help them by decomposing the problem-solving process, but the LLMs learn limitedly. |
Efficient Out-of-Domain Detection for Sequence to Sequence Models (2023.findings-acl)
Copied to clipboard
Artem Vazhentsev, Akim Tsvigun, Roman Vashurin, Sergey Petrakov, Daniil Vasilev, Maxim Panov, Alexander Panchenko, Artem Shelmanov
| Challenge: | Sequence-to-sequence (seq2sequ) models are a ubiquitous tool for text generation but they are not suitable for many other tasks. |
| Approach: | They propose to use UE techniques to identify out-of-domain (OOD) inputs where the model is susceptible to errors. |
| Outcome: | The proposed methods outperform heavyweight ensembles on the task of OOD detection. |
Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for finding out-of-domain intents suffer from in-domain overfitting problem . previous methods fail to transfer prior knowledge to downstream clustering . |
| Approach: | They propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents . they propose IND pre-training objective to learn discriminative features while maintaining intra-class diversity . |
| Outcome: | The proposed framework improves on three benchmark datasets. |
Memory-Based Invariance Learning for Out-of-Domain Text Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learning invariant representations rely on the assumption that training and test sets come from the same domain. |
| Approach: | They propose to extend a classification model trained on multiple source domains to an unseen target domain by using key-value memory. |
| Outcome: | The proposed method improves on sentiment analysis and natural language inference tasks. |
CATfOOD: Counterfactual Augmented Training for Improving Out-of-Domain Performance and Calibration (2024.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable generalization capabilities, performing well on various tasks such as question answering (QA), complex reasoning, and code generation. |
| Approach: | They propose to augment training data of smaller language models with automatically generated counterfactuals (CF) instances to improve out-of-domain (OOD) performance of SLMs in extractive question answering setup. |
| Outcome: | The proposed approach improves out-of-domain (OOD) performance of small language models in extractive question answering setup. |
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness (2022.findings-acl)
Copied to clipboard
| Challenge: | Data modification has been proposed as an effective solution for generalizing to out-of-domain (OOD) inputs. |
| Approach: | They propose to use data modification to generalize to out-of-domain inputs . they also analyze their adversarial robustness using a synthetic dataset . |
| Outcome: | The proposed data modification strategies improve OOD accuracy and AR, but data filtering hurts OOD on other tasks. |
Unsupervised Domain Adaptation for Joint Information Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current JIE methods focus on standard supervised learning setting where training and test data come from the same domain. |
| Approach: | They propose a method to induce domain-invariant representations for the tasks in JIE by a generalized version of domain-adversarial learning. |
| Outcome: | The proposed method improves out-of-domain performance for current pipeline approaches for all IE tasks. |
Explanation Regularisation through the Lens of Attributions (2025.coling-main)
Copied to clipboard
| Challenge: | Explanation regularisation (ER) is a method to guide text classifiers to form their predictions relying on tokens that humans consider plausible. |
| Approach: | They introduce an auxiliary explanation loss to measure how well an input attribution technique's output agrees with human-annotated rationales. |
| Outcome: | The proposed model improves classification performance in out-of-domain (OOD) settings by relying on tokens humans consider plausible. |
Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | MultiUAT dynamically adjusts training data usage based on model’s uncertainty on a small set of trusted clean data for multi-corpus machine translation. |
| Approach: | They propose an approach that dynamically adjusts the training data usage based on the model’s uncertainty on a small set of trusted clean data for multi-corpus machine translation. |
| Outcome: | The proposed approach outperforms baselines on 16 languages and 2 domains on English-German translation. |
Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT (2023.emnlp-main)
Copied to clipboard
Xiaoshuai Song, Keqing He, Pei Wang, Guanting Dong, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu
| Challenge: | Existing methods to fine-tune discriminative models address these challenges by focusing on in-domain intents. |
| Approach: | They evaluate ChatGPT on OOD intent discovery and generalized intent discovery tasks . they outline the strengths and weaknesses of ChatGPt and outline their results . |
| Outcome: | The proposed task aims to extend a closed intent classifier to open-world intent sets. |
To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Recent advances in open-domain question answering have demonstrated impressive accuracy on general-purpose domains like Wikipedia. |
| Approach: | They propose a more realistic end-to-end domain shift evaluation setting covering five diverse domains to assess model adaption. |
| Outcome: | The proposed model improves by 24 points when adapted to unsupervised datasets. |