Papers with ID
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
Ivan Vulić, Pei-Hao Su, Samuel Coope, Daniela Gerz, Paweł Budzianowski, Iñigo Casanueva, Nikola Mrkšić, Tsung-Hsien Wen
| Challenge: | Existing Transformer-based language models (LMs) are not effective as sentence encoders when used off-the-shelf. |
| Approach: | They propose a method which turns a pretrained LM into a universal conversational encoder and task-specialised sentence encoder. |
| Outcome: | The proposed framework achieves state-of-the-art ID performance across the board with particular gains in the most challenging, few-shot setups. |
ER-Test: Evaluating Explanation Regularization Methods for Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Explanation regularization (ER) aims to improve NLM generalization by pushing the NLM’s machine rationales to align with human rationale. |
| Approach: | They propose a framework for evaluating ER models’ OOD generalization along three dimensions: unseen datasets, contrast set tests, and functional tests. |
| Outcome: | The proposed framework evaluates ER models’ OOD generalization across unseen datasets, contrast set tests, and functional tests. |
Debias NLU Datasets via Training-free Perturbations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to debiase NLU models capture biased features that are independent of the task but spuriously correlated to labels. |
| Approach: | They propose a framework that conducts training-free perturbations on samples containing biased features to Debias NLU Datasets. |
| Outcome: | The proposed framework shows competitive performance with previous state-of-the-art debiasing strategies. |
Linear Steerability in Language Models: When It Emerges and How It Evolves (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new framework for steering language models reveals how concepts become linearly separable as training progresses . |
| Approach: | They propose a framework to analyze steerability in language models by using hidden state and representation analysis. |
| Outcome: | The proposed framework reveals how steerability evolves over training . concepts become linearly separable as training progresses, the framework shows . |