Papers with zero-shot
Contrastive Data and Learning for Natural Language Processing (2022.naacl-tutorials)
Copied to clipboard
| Challenge: | Current NLP models heavily rely on effective representation learning algorithms. |
| Approach: | This tutorial introduces contrastive learning and provides an introduction to the techniques. |
| Outcome: | This tutorial provides an introduction to the fundamentals of contrastive learning approaches and the theory behind them. |
Primacy Effect of ChatGPT (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing machine learning models may lead to poor performance in discriminative natural language understanding tasks. |
| Approach: | They propose to use ChatGPT to query large amounts of human-written text to find the answer to a question. |
| Outcome: | The proposed model has a high chance to select labels at earlier positions as the answer. |
MizanQA: A Benchmark for Multi-Answer Moroccan Legal QA (2026.eacl-industry)
Copied to clipboard
| Challenge: | Using a benchmark, large language models can be evaluated on Moroccan legal MCQs . despite their ability to comprehend and process Arabic, the language is still a challenge . |
| Approach: | They propose a benchmark for assessing LLMs on Moroccan legal MCQs . they use Arabic-based questions enriched with Moroccan idioms to assess their accuracy . |
| Outcome: | The proposed benchmark covers 1,776 expert-verified questions in Arabic enriched with Moroccan idioms . it measures accuracy, precision-penalized F1-like score, and calibration errors . |
Zero-Resource Neural Machine Translation with Monolingual Pivot Data (D19-56)
Copied to clipboard
| Challenge: | Neural machine translation systems have relied on large amounts of parallel training data between the source and target language. |
| Approach: | They propose methods for generating pseudo-parallel corpora using pivot-language data . they use English as the pivot language to train the zero-shot system . |
| Outcome: | The proposed methods improve the zero-shot neural machine translation system for a high-resource language pair using English as the pivot language. |
Turkish Universal Conceptual Cognitive Annotation (2022.lrec-1)
Copied to clipboard
| Challenge: | UCCA-annotated datasets have been released in English, French, and German . a semi-automatic annotation approach is used to annotate the datasets . |
| Approach: | They propose to use an external semantic parser to annotate Turkish sentences . they use the same parsers for evaluation purposes and conducted experiments . |
| Outcome: | The proposed dataset is the first UCCA-annotated Turkish dataset . the results show that the parser can improve on the initial annotations . |
How to Translate Your Samples and Choose Your Shots? Analyzing Translate-train & Few-shot Cross-lingual Transfer (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have focused on zero-shot cross-lingual transfer of pretrained languages. |
| Approach: | They propose to use few-shot cross-lingual transfer to improve zero-shot performance of multilingual pretrained language models. |
| Outcome: | The proposed model can be scaled to high-quality samples and improves on zero-shot performance. |
The Gutenberg Dialogue Dataset (2021.eacl-main)
Copied to clipboard
| Challenge: | Current open-domain dialogue datasets offer a trade-off between quality and size . we build a dataset of 14.8M utterances in English and smaller datasets in german, Dutch, Spanish, Portuguese, Italian, and Hungarian . |
| Approach: | They build a high-quality dialogue corpus of 14.8M utterances in English using public-domain books from Project Gutenberg. |
| Outcome: | The proposed datasets show that the extracted dialogues are more accurate and more accurate than the larger Opensubtitles dataset. |
Multi-Document Summarization with Centroid-Based Pretraining (2023.acl-short)
Copied to clipboard
| Challenge: | In Multi-Document Summarization, the input is a set of documents, and the output is its summary. |
| Approach: | They propose a novel pretraining objective that uses the ROUGE-based centroid of each document cluster as a proxy for its summary. |
| Outcome: | The proposed model is better or comparable to state-of-the-art models. |
Can Large Language Models Serve as Effective Classifiers for Hierarchical Multi-Label Classification of Scientific Documents at Industrial Scale? (2025.coling-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated great potential in complex tasks such as multi-label classification, but the vast number of labels can exceed LLMs’ input limits. |
| Approach: | They propose a method that integrates large language models with dense retrieval techniques to overcome these challenges. |
| Outcome: | The proposed methods avoid frequent retraining by leveraging zero-shot and few-shot learning for real-time label assignment. |
Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue systems perform poorly on unseen topics due to limited topics covered in training data. |
| Approach: | They propose a language model that homogenizes different knowledge sources to a unified knowledge representation for knowledge-grounded dialogue generation tasks. |
| Outcome: | The proposed language model generalizes well across knowledge-grounded dialogue tasks. |
Detecting and Mitigating Challenges in Zero-Shot Video Summarization with Video LLMs (2025.findings-acl)
Copied to clipboard
Luca Cagliero, Lorenzo Vaiani, Eliana Pastor, Alkis Koudounas, Elena Baralis, Vittorio Mazzia, Sandro Pollastrini, Thomas Gueudre, Manuel Giollo, Daniele Amberti, Yue Wu
| Challenge: | Video Large Language Models (VLLMs) exhibit impressive zero-shot capabilities in video analysis, but their performance varies significantly depending on the LLM prompt, the characteristics of the video, and the properties of the training data and LLM architecture. |
| Approach: | They propose to use Chain-of-Thought prompting to inject knowledge extracted by external, lightweight models into video summarization benchmarks to evaluate their performance. |
| Outcome: | The proposed solutions improve summarization performance by injecting knowledge extracted by external, lightweight models. |
Cultural Concept Adaptation on Multimodal Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Past methods focused on multilingual and multimodal capabilities, and the improvement of multicultural competence is still an unexplored problem. |
| Approach: | They propose an annotation-free method for cultural-concept adaptation and construct a concept mapping set to facilitate model's comprehension of cultural-consensual mappings. |
| Outcome: | The proposed method outperforms baseline models on zero-shot and few-shot settings on five languages and cultures. |
Can Large Language Models Identify Authorship? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional capacity for reasoning and problem-solving, but their potential in authorship analysis remains under-explored. |
| Approach: | They propose to integrate explicit linguistic features into LLMs to provide explanations into their reasoning processes. |
| Outcome: | The proposed models demonstrate their ability to perform zero-shot, end-to-end authorship verification effectively and provide explainability through explicit linguistic features. |
PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents (2024.eacl-long)
Copied to clipboard
| Challenge: | Using chain-of-thought prompting, large language models perform better on complex reasoning tasks. |
| Approach: | They propose a prompting framework that decomposes a question into a sequence of actions and executes them over the document to obtain the answer. |
| Outcome: | The proposed framework outperforms zero-shot and chain-of-thought prompting on a QuALITY dataset . it proposes a plan based on actions mined from a training set and executes it step by step . |
Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition (2020.aacl-main)
Copied to clipboard
| Challenge: | despite recent advances in multimodal emotion recognition, two problems still exist: sub-optimal performance and low-resource emotions. |
| Approach: | They propose a modality-transferable model with emotion embeddings to solve these problems . they use pre-trained word embedders to represent emotion categories for textual data . |
| Outcome: | The proposed model outperforms baselines in zero-shot and few-shot scenarios for unseen emotions. |
Few-shot fine-tuning SOTA summarization models for medical dialogues (2022.naacl-srw)
Copied to clipboard
| Challenge: | Abstractive summarization of medical dialogues is a challenge for standard training approaches due to the paucity of suitable datasets. |
| Approach: | They propose to use medical dialogues to generate abstractive summaries using transformer-based models with zero-shot and few-shot learning strategies. |
| Outcome: | The proposed models were compared with a medical dialogue dataset with 143 snippets and a general domain and dialogue-specific text to assess their performance. |
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Instruction tuning is emerging in NLP, but has not been explored for dialogue-related tasks. |
| Approach: | They propose an instruction tuning framework for dialogue that leverages natural language instructions with language models to induce zero-shot generalization on unseen tasks. |
| Outcome: | The proposed framework enables good zero-shot performance on unseen datasets and tasks such as dialogue evaluation and intent detection. |
EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning (2024.naacl-short)
Copied to clipboard
| Challenge: | Language models are adopting inference-time prompting techniques such as zero-shot and few-shot prompting. |
| Approach: | They propose a prompting technique that prompts the model to rephrase its queries before answering them. |
| Outcome: | The proposed prompt improves zero-shot-CoT performance of code-davinci-002 by 5% . the proposed prompt is tailored for four scenarios in both zero- and few-shot settings . |
MultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for detecting social-media texts are limited to the English language and longer texts are not easily recognisable by humans. |
| Approach: | They propose to use a multilingual and multi-platform dataset to compare machine-generated text detection methods in the social-media domain to compare them to human-written texts. |
| Outcome: | The proposed dataset contains 472,097 texts, of which about 58k are human-written and approximately the same amount is generated by each of 7 multilingual LLMs. |
Familarity: Better Evaluation of Zero-Shot Named Entity Recognition by Quantifying Label Shifts in Synthetic Training Data (2025.naacl-long)
Copied to clipboard
| Challenge: | Current research relies on large synthetic datasets to train zero-shot named entity recognition models. |
| Approach: | They propose a metric that captures the semantic similarity between entity types in training and evaluation to estimate label shift. |
| Outcome: | The proposed metric captures semantic similarity between entity types in training and evaluation, and their frequency in training data to provide an estimate of label shift. |
ELIOT: Zero-Shot Video-Text Retrieval through Relevance-Boosted Captioning and Structural Information Extraction (2025.naacl-srw)
Copied to clipboard
| Challenge: | Recent advances in video-text retrieval (VTR) have relied on supervised learning and fine-tuning. |
| Approach: | They propose a zero-shot video-text retrieval framework that leverages off-the-shelf captioners, large language models, and text retrieval methods without additional training or annotated data. |
| Outcome: | The proposed framework outperforms existing methods on video-text retrieval benchmarks without data. |
Semantic Span Annotation: An Exploratory Study of LLM Annotation (2026.acl-srw)
Copied to clipboard
| Challenge: | Structured span extraction research is siloed by context length, annotation task, and domain . Identifying a span within a natural language text and affixing it with a semantic label has been considered a core task in NLP . |
| Approach: | They propose a framework for structured span annotation that integrates five datasets under a common JSONL format with character-level offsets. |
| Outcome: | The proposed framework can generalize across four domains under three prompting configurations. |
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022.emnlp-main)
Copied to clipboard
Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu
| Challenge: | Structured knowledge grounding (SKG) uses structured knowledge to complete user requests . since inputs and outputs of SKG tasks are heterogeneous, they have been studied separately . |
| Approach: | They propose a framework that unifies 21 SKG tasks into a text-to-text format . they use unifiedSKG to benchmark T5 with different sizes . |
| Outcome: | The proposed framework unifies 21 SKG tasks into a text-to-text format . it achieves state-of-the-art performance on almost all of the 21 tasks, the authors show . |
Slot Dependency Modeling for Zero-Shot Cross-Domain Dialogue State Tracking (2022.coling-1)
Copied to clipboard
| Challenge: | Existing zero-shot learning methods ignore slot dependencies in a multidomain dialogue . experimental results show the effectiveness of our proposed method over existing state-of-art generation methods . |
| Approach: | They propose to use slot prompts combination, slot values demonstration and slot constraint object to model slot-slot dependency, slot-value dependency and slot-context dependency respectively. |
| Outcome: | The proposed method outperforms state-of-the-art methods under zero-shot/few-shot settings. |
MIND: A Multi-agent Framework for Zero-shot Harmful Meme Detection (2025.acl-long)
Copied to clipboard
| Challenge: | a rapid expansion of memes on social media highlights the need for effective methods to detect harmful content. |
| Approach: | They propose a multi-agent framework for zero-shot harmful meme detection that does not rely on annotated data. |
| Outcome: | The proposed framework outperforms existing zero-shot approaches on three meme datasets. |
Synthesizing Parallel Data of User-Generated Texts with Zero-Shot Neural Machine Translation (2020.tacl-1)
Copied to clipboard
| Challenge: | Neural machine translation systems are usually trained on clean parallel data, but the quality of translations is poor when translating noisy texts. |
| Approach: | They synthesize parallel data of UGT and exploit monolingual data to generate translations . they propose to use monolingual parallel data to train or adapt NMT systems . |
| Outcome: | The proposed approach improves the translation quality of noisy texts while making them more robust. |
Kompetencer: Fine-grained Skill Classification in Danish Job Postings via Distant Supervision and Transfer Learning (2022.lrec-1)
Copied to clipboard
| Challenge: | Several studies focus on Skill Identification, but there is little work in further categorizing the identified skills. |
| Approach: | They propose a Danish job posting dataset annotated for nested spans of competences . they use the European Skills, Competences, Qualifications and Occupations (ESCO) taxonomy API to obtain fine-grained labels via distant supervision. |
| Outcome: | The proposed dataset outperforms existing models in the Danish job postings. |
Towards LLM-driven Dialogue State Tracking (2023.emnlp-main)
Copied to clipboard
| Challenge: | emergence of large language models (LLMs) such as GPT3 and ChatGPT has sparked considerable interest in assessing their efficacy across diverse applications. |
| Approach: | They present a framework for a domain-slot instruction tuning method that allows LDST to achieve performance on par with ChatGPT. |
| Outcome: | The proposed framework performs better in zero-shot and few-shot settings than previous SOTA methods. |
Dialetto, ma Quanto Dialetto? Transcribing and Evaluating Dialects on a Continuum (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent work on dialect variation in NLP treats dialects as discrete categories . dialect variation is a focus of increasing interest in the field . |
| Approach: | They examine performance differences between Italian dialects by incorporating performance data from different regions of the world. |
| Outcome: | The results show that performance disparities are due to dialects that are more similar to the standard variety. |
Opponent Modeling in Negotiation Dialogues by Related Data Adaptation (2022.findings-naacl)
Copied to clipboard
| Challenge: | In a multi-issue negotiation, it involves inferring the relative importance that the opponent assigns to each issue under discussion, which is crucial for finding high-value deals. |
| Approach: | They propose a ranker for inferring the priority order of the opponent from partial dialogues without needing additional annotations for training. |
| Outcome: | The proposed model performs better than baselines while accessing fewer utterances from the opponent. |
EfficientOCR: An Extensible, Open-Source Package for Efficiently Digitizing World Knowledge (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing OCR engines fail to provide accurate, cost-effective and sample-efficient character recognition for public domain documents. |
| Approach: | EffOCR is an open-source optical character recognition package that is accurate, cheap to deploy and sample efficient to customize to novel collections, languages, and character sets. |
| Outcome: | EffOCR model trains character retrieval problem and scales to novel collections, languages, and character sets. |
Improving Cross-lingual Transfer through Subtree-aware Word Reordering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that multilingual language models are not effective when dealing with less-represented languages. |
| Approach: | They propose a powerful reordering method that learns word-order patterns conditioned on the syntactic context from a small amount of annotated data. |
| Outcome: | The proposed method outperforms baselines on a variety of tasks and is effective in both zero-shot and few-shot scenarios. |
An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers (2021.acl-short)
Copied to clipboard
| Challenge: | Existing word-level quality estimation models require labelled data for each language pair and expensive maintenance. |
| Approach: | They propose to use multilingual QE models to generalise across languages . they propose to train models on other language pairs to predict word-level quality . |
| Outcome: | The proposed models generalise well across languages, making them more useful in real-world scenarios. |
Does GPT-3 Generate Empathetic Dialogues? A Novel In-Context Example Selection Method and Automatic Evaluation Metric for Empathetic Dialogue Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Empathy is a multi-dimensional concept consisting of cognitive and affective aspects. |
| Approach: | They propose two new in-context example selection methods that utilize emotion and situational information. |
| Outcome: | The proposed method is effective in measuring the degree of human empathy. |
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation (2021.naacl-main)
Copied to clipboard
Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, Yashar Mehdad
| Challenge: | Recent advances in summarization are driven by the availability of large datasets such as the CNN-DailyMail corpus and the New York Times corpus. |
| Approach: | They propose a method for fine-tuning pretrained models for summarization in unsupervised manner . they use Wikipedia data to produce pseudo-summaries which contain characteristics of target dataset . |
| Outcome: | The proposed method achieves state-of-the-art, zero-shot abstractive summarization performance on CNN-DailyMail dataset and compares with other methods on other datasets. |
Question Answering Infused Pre-training of General-Purpose Contextualized Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing pretraining objectives for question answering (QA) are not optimized for being immediately useful without fine-tuning. |
| Approach: | They propose a pre-training objective based on question answering (QA) that is based more directly on context. |
| Outcome: | The proposed model matches predictions of a more accurate cross-encoder model on 80 million synthesized QA pairs and achieves large improvements over previous state-of-the-art models on paraphrase detection and fewshot named entity recognition. |
Self-Training Pre-Trained Language Models for Zero- and Few-Shot Multi-Dialectal Arabic Sequence Labeling (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to fine-tune pre-trained language models for downstream tasks require labeled data. |
| Approach: | They propose to self-train pre-trained language models to improve performance on data-scarce varieties by as large as 10% F1 and 2% accuracy. |
| Outcome: | The proposed model improves zero-shot MSA-to-DA transfer by as large as 10% F1 (NER) and 2% accuracy (POS tagging). |
CodeT5+: Open Code Large Language Models for Code Understanding and Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing code LLMs adopt a specific architecture or rely on a unified encoder-decoder network for downstream tasks, lacking flexibility to operate in the optimal architecture for a particular task. |
| Approach: | They propose to initialize code LLMs with frozen off-the-shelf LLM and explore instruction-tuning to align with natural language instructions. |
| Outcome: | The proposed model outperforms open-source LLMs on 20 code-related benchmarks. |
LLMs Are Zero-Shot Context-Aware Simultaneous Translators (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing SiMT systems operate on a sentence level, disregarding the context established by previous sentences or the broader context implied by previous words. |
| Approach: | They show that open-source LLMs perform on par with or better than some state-of-the-art baselines in simultaneous machine translation tasks, zero-shot. |
| Outcome: | The proposed models perform on par with or better than state-of-the-art baselines in simultaneous machine translation tasks, zero-shot. |
Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification (2026.acl-srw)
Copied to clipboard
| Challenge: | idiomatic light verb constructions (LVCs) are challenging for multiword expression processing . they share the same surface form as fully literal verb–object combinations . |
| Approach: | They frame Turkish LVC detection as a binary classification task . they compare a supervised Turkish encoder baseline to three instruction-tuned LLMs . |
| Outcome: | The proposed method improves Turkish LVC detection on a controlled set with matched negatives and positives. |
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models suffer from the problem of hallucinations, where a summary contains facts or entities not present in the original document. |
| Approach: | They propose an abstractive summarization model that addresses the problem of factuality during pre-training and fine-tuning. |
| Outcome: | Experiments on three downstream tasks show that FactPEGASUS significantly improves factuality compared to the original pre-training objective in zero-shot and few-shot settings. |
Read Before Grounding: Scene Knowledge Visual Grounding via Multi-step Parsing (2025.coling-main)
Copied to clipboard
| Challenge: | Existing VG datasets use simple textual descriptions with limited attribute and spatial information between images and text. |
| Approach: | They propose a method that transforms visual knowledge into concise, information-dense visual descriptions. |
| Outcome: | The proposed method significantly improves performance of multimodal grounding models. |
DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for summarizing dialogues lack in taking into account the structure of dialogues and rely heavily on labeled data. |
| Approach: | They propose a pre-trained encoder-decoder model for summarizing dialogues in any new domain. |
| Outcome: | The proposed model outperforms existing methods on six datasets and shows ROUGE scores in zero-shot and few-shot settings. |
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to dialogue state tracking are dependent on domain ontology and lack of sharing knowledge across domains. |
| Approach: | They propose a transferable dialogue state generator that generates dialogue states from utterances using copy mechanism. |
| Outcome: | Empirical results show that TRADE achieves state-of-the-art 48.62% joint goal accuracy for the five domains of MultiWOZ. |
Tonal Salience in Cognitive Decline: In-Context MCI Detection with Multimodal LLMs (2026.acl-srw)
Copied to clipboard
| Challenge: | a recent study shows that tonal languages like Chinese have a higher classification performance than non-tonal languages like English. |
| Approach: | a new study examines the differences between tonal and non-tonal language classifications . they hypothesize that the difference is rooted in language typology . early cognitive decline is notoriously difficult to detect . |
| Outcome: | The proposed method compared to TAUKADIAL audio shows that Chinese and English perform better on Chinese . the findings suggest that language typology should inform the design of audio-based cognitive screening tools . |
From Schema to State: Zero-Shot Scheme-Only Dialogue State Tracking via Diverse Synthetic Dialogue and Step-by-Step Distillation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing research classifies zero-shot, scheme-only DST into two main types: the cross-domain scenario and the zero-schemaonly setting. |
| Approach: | They propose a zero-shot, scheme-only approach that generates synthetic dialogues that balance diversity with schema alignment and distills knowledge from a large language model into a smaller model. |
| Outcome: | The proposed approach achieves state-of-the-art performance under zero-shot, scheme-only situation and generalizes effectively to few-shot scenarios. |
El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsing (2021.eacl-main)
Copied to clipboard
| Challenge: | Code-switching (CS) is the alternation of languages within an utterance or conversation. |
| Approach: | They propose to use translation-and-align and augment with a generation model followed by match-and filter to improve CS generalizability of cross-lingual models when data for only one language is available. |
| Outcome: | The proposed models improve when only English data is available alongside zero or a few CS training instances. |
Context-Aware Abbreviation Expansion Using Large Language Models (2022.naacl-main)
Copied to clipboard
Shanqing Cai, Subhashini Venugopalan, Katrin Tomanek, Ajit Narayanan, Meredith Morris, Michael Brenner
| Challenge: | Existing paradigms for text entry in augmentative and alternative communication (AAC) for people with severe motor impairments require 3-5 predictions to save keystrokes. |
| Approach: | They propose a paradigm in which phrases are abbreviated aggressively as word-initial letters. |
| Outcome: | The proposed paradigm can save up to 77% on expansions on conversation turn . the proposed paradigm could be used in augmentative and alternative communication (AAC) |
Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection (2021.acl-short)
Copied to clipboard
| Challenge: | a lack of labeled, non-English resources for hate speech detection limits research on hate speech . a recent study shows that zero-shot, cross-lingual learning models cannot be used as they are . lack of consistency limits research, and lack of models for non-english languages limits learning . |
| Approach: | They propose a zero-shot, cross-lingual transfer learning framework for hate speech detection . they use benchmark data sets in English, Italian, and Spanish to detect hate speech . |
| Outcome: | The proposed framework can't be used as it is, but needs to be carefully designed, the authors say . they find that non-hateful, language-specific taboo interjections are misinterpreted as signals of hate speech . |
A synthetic data approach for domain generalization of NLI models (2024.acl-long)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) datasets are important benchmark tasks for LLMs . however, their realistic performance on out-of-distribution/domain data is less well-understood . a T5-small model trained with our data improves around 7% on average compared to the best alternative dataset . |
| Approach: | They propose a new approach for generating NLI data in diverse domains and lengths . they show that models trained on this data have the best generalization to completely new downstream test settings . |
| Outcome: | The proposed model can be trained on datasets with high-quality examples with meaningful premises and high accuracy. |
Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to train a multilingual NMT model for low-resource languages are lacking in terms of number of supervised examples. |
| Approach: | They propose to use decoder pre-training and back-translation to solve the degeneracy problem by analyzing spurious correlations between source and decoded sentences. |
| Outcome: | The proposed methods show significant improvement over the pivot-based approach on three challenging multilingual datasets. |
A Language-First Approach for Procedure Planning (2023.findings-acl)
Copied to clipboard
| Challenge: | Developing intelligent agents requires the ability to produce plans on the fly based on visual observations. |
| Approach: | They propose a language-first procedure planning framework with a modularized design . they first align current and goal observations with corresponding steps and then use a pre-trained LM to predict intermediate steps. |
| Outcome: | The proposed framework matches state-of-the-art procedures on COIN and CrossTask benchmarks. |
Exploring the Potential of Large Language Models in Computational Argumentation (2024.acl-long)
Copied to clipboard
| Challenge: | Argumentation is an essential tool in various domains, including law, public policy, and artificial intelligence. |
| Approach: | They propose to evaluate LLMs on various computational argumentation tasks . they organize existing tasks into six main categories and standardize the format of 14 datasets . |
| Outcome: | The proposed model performs well on argument mining and argument generation tasks. |
Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot slot filling ignore constraints in the latent space and lack robustness. |
| Approach: | They propose a Contrastive Zero-Shot Learning with Adversarial Attack method for slot filling . they propose to map slot value contextual representations to slot description representations . |
| Outcome: | The proposed method outperforms state-of-the-art models under zero-shot and few-shot settings. |
RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation (2023.acl-short)
Copied to clipboard
| Challenge: | Attribute-controlled translation (ACT) is a subtask of machine translation that involves controlling stylistic or linguistic attributes (like formality and gender) of outputs. |
| Approach: | They propose a new approach to attribute-controlled translation that leverages multilingual language models to perform ACT in few-shot and zero-shot settings. |
| Outcome: | The proposed approach improves generation accuracy over the standard prompting approach in both zero-shot and few-shot settings. |
MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning (2026.findings-eacl)
Copied to clipboard
Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Tasnim Mohiuddin, Md Mofijul Islam, Swakkhar Shatabda
| Challenge: | Existing benchmarks primarily focus on English or a narrow subset of high-resource languages, leaving significant gaps in assessing multilingual and cross-lingual mathematical reasoning. |
| Approach: | They propose a parallel multilingual benchmark for mathematical problem solving and reasoning that encompasses 2,890 parallel Bangla-English gold standard artifacts. |
| Outcome: | The proposed model encompasses 2,890 parallel Bangla-English gold standard artifacts, totaling 30K aligned question–answer pairs across thirteen languages, representing high-, medium-, and low-resource linguistic settings. |
SchemaGraphSQL: Efficient Schema Linking with Pathfinding Graph Algorithms for Text-to-SQL on Large-Scale Databases (2026.findings-eacl)
Copied to clipboard
AmirHossein Safdarian, Milad Mohammadi, Ehsan Jahanbakhsh Bashirloo, Mona Shahamat Naderi, Heshaam Faili
| Challenge: | Text-to-SQL systems translate natural language questions into executable SQL queries. |
| Approach: | They propose a schema linking approach that first constructs a graph based on foreign key relations and then uses a single prompt to a lightweight LLM to extract source and destination tables from the user query. |
| Outcome: | The proposed method outperforms specialized, fine-tuned, and complex multi-step approaches on BIRD and Spider 2.0 benchmarks. |
Read to Hear: A Zero-Shot Pronunciation Assessment Using Textual Descriptions and LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Automatic pronunciation assessment is typically performed by acoustic models trained on audio-score pairs. |
| Approach: | They propose a zero-shot, textual description-based Pronunciation Assessment approach that utilizes human-readable representations of speech signals fed into an LLM to assess pronunciation accuracy and fluency. |
| Outcome: | The proposed approach is cost-efficient and competitive in performance . it significantly improves the performance of conventional audio-score-trained models on out-of-domain data . |
MolXPT: Wrapping Molecules with Text for Generative Pre-training (2023.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that Generative pre-trained Transformers (GPT) have great success in natural language processing. |
| Approach: | They propose a unified language model of text and molecules pre-trained on SMILES wrapped by text. |
| Outcome: | The proposed model outperforms strong baselines of molecular property prediction on MoleculeNet and performs comparably to the best model in text-molecule translation while using less than half of its parameters. |
IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models (2025.naacl-long)
Copied to clipboard
David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba Oluwadara Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En-Shiun Annie Lee, Chiamaka Ijeoma Chukwuneke, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Foutse Yuehgoh, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Salomey Osei, Shamsuddeen Hassan Muhammad, Sokhar Samb, Tadesse Kebede Guge, Tombekai Vangoni Sherman, Pontus Stenetorp
| Challenge: | Large language models (LLMs) are limited to a few high-resource languages . many low-resourced languages are evaluated only on basic text classification tasks . |
| Approach: | They propose to use IrokoBench to evaluate 17 low-resource African languages . they use human-translated benchmark datasets to evaluate zero-shot, few-shot and translate-test settings . |
| Outcome: | The proposed model performs well in English and French, but the highest performing model perform poorly in proprietary models. |
HUE: Pretrained Model and Dataset for Understanding Hanja Documents of Ancient Korea (2022.findings-naacl)
Copied to clipboard
| Challenge: | Historical records in Korea before the 20th century were primarily written in Hanja, an extinct language based on Chinese characters. |
| Approach: | They present a dataset that includes tasks for attribution, topic classification, named entity recognition and summary retrieval for classical Hanja documents. |
| Outcome: | The proposed models improve on the Annals of the Joseon Dynasty and Diaries of the Royal Secretariats datasets. |
Evaluating Zero-Shot Event Structures: Recommendations for Automatic Content Extraction (ACE) Annotations (2023.acl-short)
Copied to clipboard
| Challenge: | Zero-shot event extraction (EE) methods infer richly structured event records from unstructured text data, based on a user-supplied natural language specification and no training examples. |
| Approach: | They propose recommendations for future evaluations so the research community can better utilize ACE as an event evaluation resource. |
| Outcome: | The proposed methods can be used to evaluate zero-shot and other low-supervision EE methods, considering up to 32% of correctly identified arguments and 25% of correctly ignored event mentions as false negatives. |
AccentFold: A Journey through African Accents for Zero-Shot ASR Adaptation to Target Accents (2024.findings-eacl)
Copied to clipboard
| Challenge: | AccentFold uses spatial relationships to improve speech recognition for accented speech . existing methods for accent recognition have been limited due to data scarcity and budget constraints . |
| Approach: | They propose a method that exploits spatial relationships between learned accent embeddings to improve downstream automatic speech recognition. |
| Outcome: | The proposed method outperforms baseline methods in accented speech training. |
Advancing Precise Outline-Conditioned Text Generation with Task Duality and Explicit Outline Control (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies on outline-conditioned text generation focus on generating text using provided outlines as rough sketches, but lack of clarity and rationality of the rough outlines hampers quality of the generated text. |
| Approach: | They propose a novel task that requires generating stories based on specific, sentence-level outlines. |
| Outcome: | The proposed framework improves the quality of precise outline-conditioned text generation. |
CRAFT: Training-Free Cascaded Retrieval for Tabular QA (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for open-domain table question answering require retraining or fine-tuning on new datasets. |
| Approach: | They propose a zero-shot, cascaded retrieval approach that uses a sparse retrieval model to filter a subset of candidates before applying more expensive dense models as re-rankers. |
| Outcome: | The proposed method outperforms state-of-the-art retrieval models on the NQ-Tables dataset. |
Task-Level Thinking Steps Help Large Language Models for Challenging Classification Task (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results prove the superiority of our proposed method on challenging classification tasks. |
| Approach: | They propose a task-level thinking step that eliminates bias introduced by demonstrations . they propose 'progressive revision framework' which can improve the thinking steps by correcting hard demonstrations. |
| Outcome: | The proposed method achieves best performance on three kinds of classification tasks in zero-shot and few-shot settings. |
The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing instruction tuned large language models (LLMs) struggle to understand cross-lingual sociopragmatic meaning (SM) lack of comprehensive investigation into their ability to understand SM is partly due to SM not being adequately represented in any of the existing benchmarks. |
| Approach: | They evaluate the performance of instruction tuned large language models (LLMs) on a multilingual benchmark specifically designed for SM understanding. |
| Outcome: | The proposed benchmark outperforms instruction tuned large language models on a wide range of tasks but falls behind task-specific finetuned models. |
Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing methods for multilingual transfer are limited by their dynamic nature. |
| Approach: | They propose a method that utilizes deep contextual embeddings, pretrained in an unsupervised fashion. |
| Outcome: | The proposed method outperforms the state-of-the-art on 6 languages, yielding an improvement of 6.8 LAS points on average. |
Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for zero-shot CoT are limited to a single language, making it difficult to generalize to other languages and hindering global development. |
| Approach: | They introduce cross-lingual prompting (CLP) to improve zero-shot CoT reasoning across languages. |
| Outcome: | The proposed method outperforms existing prompting methods on several benchmarks. |
Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches for NLU use CL to improve in-distribution data performance via heuristic-oriented or task-agnostic difficulties. |
| Approach: | They propose to use CL to improve in-distribution data performance by taking advantage of training dynamics as difficulty metrics instead of heuristic-oriented or task-agnostic difficulties. |
| Outcome: | The proposed model schedulers improve on in-distribution, out-of-distortion and zero-shot cross-lingual transfer datasets while being 20% faster on average. |
Do Prompt-Based Models Really Understand the Meaning of Their Prompts? (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that prompts help models to learn faster in the same way that humans learn faster when provided with task instructions expressed in natural language. |
| Approach: | They experiment with 30 prompts manually written for natural language inference (NLI) they find that models can learn just as fast with many irrelevant or pathologically misleading prompts . |
| Outcome: | The proposed model can learn as fast with irrelevant or pathologically misleading prompts as with instructively “good” prompts. |
StoryWars: A Dataset and Instruction Tuning Baselines for Collaborative Story Understanding and Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Understanding and generating collaborative stories remains an underexplored area due to the lack of open-domain corpora. |
| Approach: | They propose to use a dataset of 40,000 collaborative stories written by 9,400 different authors from an online platform to generate a multi-task benchmark. |
| Outcome: | The proposed model achieves the best performance on fully-supervised, few-shot, and zero-shot scenarios while achieving the best results on the fully-supervised tasks. |
A Representation Sharpening Framework for Zero Shot Dense Retrieval (2026.eacl-long)
Copied to clipboard
| Challenge: | Zero-shot dense retrieval requires generic, pretrained DRs, which struggle to represent semantic differences between similar documents. |
| Approach: | They propose a training-free representation sharpening framework that augments a document’s representation with information that helps differentiate it from similar documents in the corpus. |
| Outcome: | The proposed framework is compatible with prior approaches to zero-shot dense retrieval and consistently improves their performance. |
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to answer open domain questions rely on unlabeled text or synthetically generated question-answer pairs. |
| Approach: | They propose a large-scale open-domain question-answering dataset based on the Common Crawl project that can be used to in-domain pre-train popular language models. |
| Outcome: | The proposed dataset achieves promising results in zero-shot, low resource and fine-tuned settings across multiple tasks, models and benchmarks. |
ScalingFilter: Assessing Data Quality through Inverse Utilization of Scaling Laws (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing quality filtering methods rely on a high-quality dataset as reference . Existing methods introduce potential biases and compromise diversity . |
| Approach: | They propose a method that evaluates text quality based on the perplexity difference between two language models trained on the same data. |
| Outcome: | The proposed approach improves performance of pre-trained models without increasing training costs. |
Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent work shows that Relation Extraction tasks can be recasted as Textual Entailment tasks using verbalizations. |
| Approach: | They propose to recasted RE tasks as Textual Entailment tasks using verbalizations . they show that entailment reduces the need for manual annotation to 50% and 20% . |
| Outcome: | The proposed method reduces the need for manual annotation to 50% and 20% in event argument extraction tasks while achieving the same performance as with full training. |
Prompt Consistency for Zero-Shot Task Generalization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that pre-trained language models can perform zero-shot generalization to new tasks without annotated examples. |
| Approach: | They propose to regularize prompt consistency to encourage consistent predictions over a diverse set of prompts. |
| Outcome: | The proposed approach outperforms the state-of-the-art zero-shot learner, T0, on 9 out of 11 datasets across 4 NLP tasks by 10.6 absolute points in terms of accuracy. |
In-Context Learning for Few-Shot Dialogue State Tracking (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot and few-shot learning dialogue state tracking are hard and expensive. |
| Approach: | They propose an in-context learning framework for zero-shot and few-shot learning dialogue state tracking (DST) a large pretrained language model takes a test instance and a few exemplars as input and directly decodes the dialogue state . |
| Outcome: | The proposed framework outperforms state-of-the-art models in few-shot settings . it is flexible and scalable, and requires less data to adapt to new domains and scenarios . |
Conformal Predictor for Improving Zero-Shot Text Classification Efficiency (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have been shown effective for zero-shot (0shot) text classification. |
| Approach: | They propose to limit the number of likely labels using a fast base classifier-based conformal predictor calibrated on samples labeled by the 0shot model. |
| Outcome: | The proposed models reduce the average inference time for NLI- and NSP-based models by 25.6% and 22.2% without dropping performance below the predefined error rate of 1%. |
DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that multilingual models are less robust for semantic parsing compared to other tasks. |
| Approach: | They propose a constrained optimization technique to optimize multilingual parsing systems for multilingual use. |
| Outcome: | The proposed technique outperforms XLM-R and mT5-Large on three benchmarks and significantly outperformed other models. |
Measuring Immediate Adaptation Performance for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Incremental domain adaptation improves interactive machine translation performance . users of interactive systems are sensitive to the speed of adaptation . |
| Approach: | They propose to measure the speed of lexical acquisition for in-domain vocabulary . they propose to use this to choose the most suitable adaptation method for neural machine translation . |
| Outcome: | The proposed measures measure the speed of lexical acquisition for in-domain vocabulary . they show that the most suitable adaptation method is chosen from a range of different techniques . |
AVA: Attentive VLM Agent for Mastering StarCraft II (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing StarCraft II benchmarks rely on abstract state representations that deviate from human perception . Existing systems rely only on abstract representations, creating an artificial gap between how humans process battlefield information and limiting ecological validity of learned behaviors. |
| Approach: | They introduce AVACraft, the first multimodal benchmark environment for complex decision-making in StarCraft II. |
| Outcome: | The AVACraft benchmark supports both traditional and modern multi-agent reinforcement learning paradigms. |
What do Large Language Models Need for Machine Translation Evaluation? (2024.emnlp-main)
Copied to clipboard
Shenbin Qian, Archchana Sindhujan, Minnie Kabra, Diptesh Kanojia, Constantin Orasan, Tharindu Ranasinghe, Fred Blain
| Challenge: | Existing research shows that large language models can perform better in machine translation tasks. |
| Approach: | They propose to use large language models for machine translation evaluations . authors explore what translation information is needed for LLMs to evaluate MT quality . |
| Outcome: | The proposed model performs comparable to fine-tuned multilingual pre-trained models. |
Cross-Lingual Leveled Reading Based on Language-Invariant Features (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Leveled reading (LR) aims to automatically classify texts by the cognitive levels of readers. |
| Approach: | They propose to use adversarial training and cross-lingual pre-training methods to transfer LR knowledge from annotated data in resource-rich English to Chinese. |
| Outcome: | The proposed method captures language-invariant features between English and Chinese. |
CLoSE: Contrastive Learning of Subframe Embeddings for Political Bias Classification of News Media (2022.coling-1)
Copied to clipboard
| Challenge: | Framing is a political strategy in which journalists and politicians emphasize certain aspects of an issue to influence and sway public opinion. |
| Approach: | They propose a BERT-based model which embeds indicators of frames from news articles in order to predict political bias. |
| Outcome: | The proposed model performs on subframes and political bias classification tasks and is able to detect political bias on both zero-shot and few-shot learning tasks. |
Stepwise Reasoning Disruption Attack of LLMs (2025.acl-long)
Copied to clipboard
Jingyu Peng, Maolin Wang, Xiangyu Zhao, Kai Zhang, Wanyu Wang, Pengyue Jia, Qidong Liu, Ruocheng Guo, Qi Liu
| Challenge: | Existing attacks on LLM reasoning are constrained by specific settings or lack of imperceptibility, limiting their feasibility and generalizability. |
| Approach: | They propose a stepwise rEasoning error disruption attack that subtly injects errors into prior reasoning steps to mislead the model into producing incorrect subsequent reasoning and final answers. |
| Outcome: | The proposed attack is compatible with zero-shot and few-shot settings, maintains the natural reasoning flow, and ensures covert execution without modification of the instruction. |
Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation (2023.acl-long)
Copied to clipboard
| Challenge: | Parameter-Efficient Transfer Learning (PETL) has the potential to address this problem, but it has yet to be applied to the zero-shot domain adaptation. |
| Approach: | They propose to use descriptions of target domain slots to generate dynamic prefixes that are concatenated to the key and values at each layer’s self-attention mechanism. |
| Outcome: | The proposed method outperforms previous methods on the MultiWOZ and SGD benchmarks. |
Leveraging Knowledge in Multilingual Commonsense Reasoning (2022.findings-acl)
Copied to clipboard
| Challenge: | Commonsense reasoning is a language-agnostic process, but most comprehensive knowledge sources are limited to a small number of languages, especially English. |
| Approach: | They propose to use English as a pivot language to integrate commonsense reasoning into models using a translate-retrieve-translate strategy. |
| Outcome: | The proposed model outperforms the state-of-the-art on the XCSR benchmarks. |
Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks typically need large labeled data for training and are not easily interpretable. |
| Approach: | They propose a type of recurrent neural networks that combine neural networks and regular expression rules. |
| Outcome: | The proposed recurrent neural networks outperform previous neural approaches in low- and zero-shot scenarios and remain very competitive in rich-resource settings. |
ECHo: A Visio-Linguistic Dataset for Event Causality Inference via Human-Centric Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | ECHo is a diagnostic dataset of event causality inference grounded in visio-linguistic social scenarios. |
| Approach: | They propose a diagnostic dataset of event causality inference grounded in visio-linguistic social scenarios. |
| Outcome: | The proposed framework examines the reasoning capability of current AI systems on three human-centric tasks. |
OASum: Large-Scale Open Domain Aspect-based Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing generic summarization methods generate only one summary for all different requests which is not optimal for diverse demands. |
| Approach: | They use crowd-sourced knowledge on Wikipedia to create a large-scale open-domain aspect-based summarization dataset with 1 million different aspects on 2 million Wikipedia pages. |
| Outcome: | The proposed model can generate diverse aspect-based summarizations on Wikipedia with zero/few-shot and fine-tuning on seven downstream datasets. |
Enhancing Zero-shot and Few-shot Stance Detection with Commonsense Knowledge Graph (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for stance detection are not applicable to zero-shot and few-shot scenarios. |
| Approach: | They propose a model that integrates commonsense knowledge into a stance detection model. |
| Outcome: | The proposed model outperforms state-of-the-art methods on zero-shot and few-shot stance detection tasks. |
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization (2023.acl-long)
Copied to clipboard
Pengcheng He, Baolin Peng, Song Wang, Yang Liu, Ruochen Xu, Hany Hassan, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang
| Challenge: | Z-Code++ is a pre-trained language model optimized for abstractive text summarization. |
| Approach: | They propose a pre-trained language model optimized for abstractive text summarization that uses a two-phase pre-training technique to improve model's performance. |
| Outcome: | The proposed model outperforms the competing models on low-resource summarization tasks in zero-shot and few-shot settings. |
BnTTS: Few-Shot Speaker Adaptation in Low-Resource Setting (2025.findings-naacl)
Copied to clipboard
Mohammad Jahid Ibna Basher, Md Kowsher, Md Saiful Islam, Rabindra Nath Nandi, Nusrat Jahan Prottasha, Mehadi Hasan Menon, Tareq Al Muntasir, Shammur Absar Chowdhury, Firoj Alam, Niloofar Yousefi, Ozlem Garibay
| Challenge: | Empirical evaluations in few-shot settings show that BnTTS significantly improves the naturalness, intelligibility, and speaker fidelity of synthesized Bangla speech. |
| Approach: | They propose to integrate Bangla into a multilingual TTS pipeline with modifications to account for the phonetic and linguistic characteristics of the language. |
| Outcome: | The proposed framework improves the naturalness, intelligibility, and speaker fidelity of synthesized Bangla speech compared to state-of-the-art systems. |
NSP-BERT: A Prompt-based Few-Shot Learner through an Original Pre-training Task —— Next Sentence Prediction (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies have shown that using prompts to utilize language models to perform downstream tasks is more effective than using token-level methods such as PET. |
| Approach: | They propose to use a BERT original pre-training task abandoned by RoBERTa and other models to construct a sentence-level prompt-based method that does not need to fix the length of the prompt or the position to be predicted. |
| Outcome: | The proposed method performs better than PET and EFL on a BERT pre-training task and is comparable to other prompt-based methods. |
Towards Zero-shot Language Modeling (D19-1)
Copied to clipboard
| Challenge: | a number of natural questions have been asked about the inductive biases of neural networks on core NLP tasks. |
| Approach: | They construct an informative prior for held-out languages on a task of character-level, open-vocabulary language modelling. |
| Outcome: | The proposed model outperforms baseline models with an uninformative prior in both zero-shot and few-shot settings, showing that it is imbued with universal linguistic knowledge. |
MURAL: Multimodal, Multitask Representations Across Languages (2021.findings-emnlp)
Copied to clipboard
Aashi Jain, Mandy Guo, Krishna Srinivasan, Ting Chen, Sneha Kudugunta, Chao Jia, Yinfei Yang, Jason Baldridge
| Challenge: | Image-caption pairs and translation pairs provide the means to learn deep representations of and connections between languages. |
| Approach: | They propose a dual encoder that integrates image-text matching and translation pairs to solve two tasks by learning from billions of pairs. |
| Outcome: | The proposed encoder outperforms ALIGN's cross-modal retrieval performance on well-resourced languages and significantly improves on under-resource languages. |
LLM-Driven Implicit Target Augmentation and Fine-Grained Contextual Modeling for Zero-Shot and Few-Shot Stance Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on zero-shot and few-shot stance detection neglect implicit yet semantically important targets. |
| Approach: | They propose a framework that uses Large Language Models to annotate implicit targets . they also propose 'DyMCA' to dynamically adjust text-target contributions based on context . |
| Outcome: | The proposed framework achieves state-of-the-art on a benchmark dataset. |
QUENCH: Measuring the gap between Indic and Non-Indic Contextual General Reasoning in LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | QUENCH is a text-based English quizzing benchmarking system for large language models (LLMs). |
| Approach: | They propose a text-based English Quizzing Benchmark manually curated from YouTube quiz videos. |
| Outcome: | The proposed system assesses the world knowledge and deduction capabilities of large language models via a zero-shot, open-domain quizzing setup. |
When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications? (2024.findings-emnlp)
Copied to clipboard
Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Dligach, Timothy Miller, Danielle Bitterman, Matthew Churpek, Majid Afshar
| Challenge: | Numerical data is pivotal for medical questions and answers, but tabular data is not fully integrated into LLMs. |
| Approach: | They examine the effectiveness of vector representations from last hidden states of LLMs for medical diagnostics and prognostics using electronic health record data. |
| Outcome: | The proposed representations outperform those using raw numerical EHR data in medical diagnostics and prognostics. |
Label Words as Local Task Vectors in In-Context Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable abilities, one of the most important being in-context learning (ICL). |
| Approach: | They hypothesized that the network creates a task vector in specific positions during ICL, which can be computed by averaging across the dataset. |
| Outcome: | The proposed model can achieve zero-shot performance with dummy inputs comparable to few-shot learning by patching the global task vector. |
Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averaging (2023.acl-long)
Copied to clipboard
| Challenge: | Massively multilingual language models have shown strong performance in zero-shot (ZS-XLT) and few-shot cross-lingual transfer setups where models are fine-tuned on task data in a source language are transferred without any or with only a few annotated instances to the target language(s). |
| Approach: | They propose a method that averages different checkpoints during task fine-tuning to improve model robustness. |
| Outcome: | The proposed method overestimates model performance in cross-lingual transfer setups where models are evaluated at checkpoints that generalize best to validation instances in the target languages. |
Preserving Zero-shot Capability in Supervised Fine-tuning for Multi-label Text Classification (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods that assume label descriptions ensure zero-shot capability lose their zero-shot capability during training. |
| Approach: | They propose a method that preserves the zero-shot capabilities of powerful dual encoders and label-wise attention networks by freezing the label encoder. |
| Outcome: | The proposed methods preserve the zero-shot capabilities of powerful dual encoder and label-wise attention network architectures by freezing the label encoder. |
FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models (2025.naacl-long)
Copied to clipboard
Xin Guo, Haotian Xia, Zhaowei Liu, Hanyang Cao, Zhi Yang, Zhiqiang Liu, Sizhe Wang, Jinyi Niu, Chuqi Wang, Yanhui Wang, Xiaolong Liang, Xiaoming Huang, Bing Zhu, Zhongyu Wei, Yun Chen, Weining Shen, Liwen Zhang
| Challenge: | Large language models have demonstrated outstanding performance in various natural language processing tasks, but their security capabilities in the financial domain have not been explored. |
| Approach: | They propose to use a benchmark to evaluate large language models' financial domain knowledge and practical abilities. |
| Outcome: | The proposed benchmark evaluates large language models' financial domain knowledge and practical abilities. |
Merely Judging Metaphor is Not Enough: Research on Reasonable Metaphor Detection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current metaphor detection tasks only provide labels without interpreting how to understand them. |
| Approach: | They propose to improve the current metaphor detection task by using mainstream Large Language Models. |
| Outcome: | The proposed model is based on the original sentence, target word, and usage . the model is then evaluated using manual evaluation . |
HateCOT: An Explanation-Enhanced Dataset for Generalizable Offensive Speech Detection via Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Social media has amplified the propagation of hateful sentiments, highlighting the contested nature of "offensive content" research shows that "of offensive content" is still a contested construct due to varying definitions and labeling. |
| Approach: | They propose a dataset that features human-curated explanations for offensive content in English . they show that HateCOT pretraining improves performance of open-source LLMs . |
| Outcome: | The proposed model improves on three benchmark datasets for offensive content detection . the model improve the quality of its explanations, as confirmed by the human evaluation . |
CrisiText: A dataset of warning messages for LLM training in emergency communication (2026.findings-eacl)
Copied to clipboard
| Challenge: | Identifying threats and mitigating their potential damage during crisis situations is paramount for safeguarding endangered individuals. |
| Approach: | They present a large-scale dataset for the generation of warning messages across 13 different types of crisis scenarios. |
| Outcome: | The proposed dataset contains more than 400,000 warning messages (spanning almost 18,000 crisis situations) aimed at assisting civilians during and after such events. |
A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank (2022.naacl-main)
Copied to clipboard
| Challenge: | Pretraining languages improve cross-lingual transfer for BERT-based models . Interestingly, PLMs exhibit zero-shot cross-linguistic abilities on downstream examples in languages seen only during pretraining. |
| Approach: | They develop a quadratic time complexity method to estimate pretraining languages' relations between linguistic features and two downstream tasks. |
| Outcome: | The proposed method is effective on a diverse set of languages spanning different linguistic features and two downstream tasks. |
LexGen: Domain-aware Multilingual Lexicon Generation (2025.acl-long)
Copied to clipboard
Ayush Maheshwari, Atul Kumar Singh, N J Karthika, Krishnakant Bhatt, Preethi Jyothi, Ganesh Ramakrishnan
| Challenge: | Lexicon generation is a key task in specialized domains due to infrequent usage of terms . a new model is proposed to generate dictionary words for 6 Indian languages . |
| Approach: | They propose a model to generate dictionary words for 6 Indian languages in the multi-domain setting. |
| Outcome: | The proposed model generalizes to unseen domains and unsealed languages. |
Zero-Shot Cross-Lingual Transfer with Meta Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | There are more than 7,000 languages spoken in the world, over 90 of which have more than 10 million native speakers each. |
| Approach: | They propose to use meta-learning to train a model on multiple languages at the same time . they use standard supervised, zero-shot cross-lingual, and few-shot crosses-lingual settings for different natural language understanding tasks. |
| Outcome: | The proposed setup improves on the state-of-the-art for a total of 15 languages. |
Cross-Lingual Learning vs. Low-Resource Fine-Tuning: A Case Study with Fact-Checking in Turkish (2024.lrec-main)
Copied to clipboard
| Challenge: | Currently, most of the research on misinformation is focused on the English language . however, there is a scarcity of datasets for other languages, including Turkish . |
| Approach: | They propose a dataset that spans multiple domains and incorporates evidence from three Turkish fact-checking organizations. |
| Outcome: | The proposed dataset has the potential to advance research in the Turkish language. |
Leveraging Machine-Generated Rationales to Facilitate Social Meaning Detection in Conversations (2024.acl-long)
Copied to clipboard
| Challenge: | Existing models for language from a social perspective are gaining popularity . we present a generalizable classification approach that leverages Large Language Models . |
| Approach: | They propose a generalizable classification approach that leverages Large Language Models to detect social meaning in conversations. |
| Outcome: | The proposed approach improves on two social meaning detection tasks over 2,340 settings. |
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models (2022.acl-long)
Copied to clipboard
| Challenge: | Massively Multilingual Transformer based Language Models have been shown to be effective on zero-shot transfer across languages, though performance varies from language to language depending on pivot language(s) used for fine-tuning. |
| Approach: | They propose to combine multi-task learning problems with multi-lingual Transformers to model zero-shot transfer across languages. |
| Outcome: | The proposed model can predict zero-shot transfer across languages with a multi-task learning problem with pretraining data in very few languages. |
CSSWiki: A Chinese Sentence Simplification Dataset with Linguistic and Content Operations (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for sentence simplification focus on English, but limited in Chinese . SS tasks are aimed at improving readability and making sentences more accessible for readers . |
| Approach: | They propose an open-source dataset for Chinese sentence simplification based on Wikipedia . they analyze differences in annotation scheme and data statistics between datasets . |
| Outcome: | The proposed dataset contains 1.6k source sentences paired with their simplified versions. |
Aligning Black-box Language Models with Human Judgments (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used as automated judges to evaluate recommendation systems, search engines, and other subjective tasks. |
| Approach: | They propose a framework to align LLM judgments with individual human evaluators or their aggregated judgments without retraining or fine-tuning the LLM. |
| Outcome: | The proposed framework achieves 142% improvement in agreement across 29 tasks and exceeds inter-human agreement on four out of six tasks. |
A Zero-Shot Open-Vocabulary Pipeline for Dialogue Understanding (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to DST are limited by their computational resources or lack flexibility to adapt to new slots. |
| Approach: | They propose a system that integrates domain classification and DST in a single pipeline and uses self-refining prompts to adapt dynamically. |
| Outcome: | The proposed system improves on existing methods on multiWOZ datasets and provides 20% better Joint Goal Accuracy (JGA) over existing methods with 90% fewer requests to the LLM API. |
On the Relation between Syntactic Divergence and Zero-Shot Performance (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in cross-lingual transfer methods have enabled significant advances in grammatical processing tasks. |
| Approach: | They examine the extent to which syntactic relations are preserved in translation and parsability in a zero-shot setting. |
| Outcome: | The proposed model is based on a translation task in English and a subset of a standard English RE benchmark translated to Russian and Korean. |
Pre-training to Match for Unified Low-shot Relation Extraction (2022.acl-long)
Copied to clipboard
| Challenge: | Low-shot relation extraction (RE) aims to recognize novel relations with very few or even no samples. |
| Approach: | They propose a method that leverages triplet paraphrase to pre-train zero-shot label matching ability and uses meta-learning paradigm to learn few-shot instance summarizing ability. |
| Outcome: | The proposed method outperforms strong baselines and achieves the best performance on few-shot RE leaderboard. |
NewsClaims: A New Benchmark for Claim Detection from News with Attribute Knowledge (2022.emnlp-main)
Copied to clipboard
Revanth Gangi Reddy, Sai Chetan Chinthakindi, Zhenhailong Wang, Yi Fung, Kathryn Conger, Ahmed ELsayed, Martha Palmer, Preslav Nakov, Eduard Hovy, Kevin Small, Heng Ji
| Challenge: | Current claims detection methods focus on sentence analysis, ignoring other attributes . a key element of identifying misinformation is detecting the claims and the arguments that have been presented. |
| Approach: | They propose a benchmark for attribute-aware claim detection in the news domain . they extend the problem to include extraction of additional attributes related to each claim . |
| Outcome: | The proposed system performs well on the test, but human performance is still poor. |
Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives (2021.acl-long)
Copied to clipboard
| Challenge: | Recent Word Sense Disambiguation systems have approached the upper bound of the task on standard evaluation benchmarks. |
| Approach: | They propose to convert the nearly isolated decisions into interrelated ones by exposing senses in context when learning sense embeddings in a similarity-based Sense Aware Context Exploitation architecture. |
| Outcome: | The proposed approach surpasses state-of-the-art on English and multilingual datasets by large margins. |
Parameter-Efficient Instruction Tuning of Large Language Models For Extreme Financial Numeral Labelling (2024.naacl-long)
Copied to clipboard
Subhendu Khatuya, Rajdeep Mukherjee, Akash Ghosh, Manjunath Hegde, Koustuv Dasgupta, Niloy Ganguly, Saptarshi Ghosh, Pawan Goyal
| Challenge: | Existing methods to automatically annotate relevant numerals (GAAP metrics) occurring in financial documents are not cost-effective nor scalable. |
| Approach: | They propose a generative paradigm for annotating GAAP metrics with XBRL tags using metric metadata and a parameter efficient model using LoRA. |
| Outcome: | The proposed model outperforms baseline models on two financial numeric labeling datasets and outperformed several strong baseline models. |
DUQGen: Effective Unsupervised Domain Adaptation of Neural Rankers by Diversifying Synthetic Query Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | State-of-the-art rankers pre-trained on large task-specific training data such as MS-MARCO exhibit strong performance on various ranking tasks without domain adaptation, also called zero-shot. |
| Approach: | They propose a method to generate unsupervised domain adaptation for ranking using large-scale task-specific training data such as MS-MARCO and Wikipedia retrieval. |
| Outcome: | The proposed method outperforms all zero-shot baselines and significantly outperfies the SOTA baselines on 16 out of 18 datasets, for an average of 4% relative improvement across all datasets. |
TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multiple choice question answering benchmarks employ automatic translation for multilingual evaluation, but this approach is error-prone and potentially introduces culturally biased questions. |
| Approach: | They introduce the first multitask, multiple-choice Turkish QA benchmark, TurkishMMLU . they evaluate over 20 LLMs including open-source, closed-source and Turkish-adapted models . |
| Outcome: | The proposed benchmarks evaluate the reasoning, comprehension, and mathematical abilities of large language models. |
Towards Zero-Shot Multilingual Transfer for Code-Switched Responses (2023.acl-long)
Copied to clipboard
| Challenge: | Recent task-oriented dialog systems have had great success building English-based personal assistants, but extending these systems to a global audience may take tremendous efforts. |
| Approach: | They propose a framework that allows for efficient transfer by learning task-specific representations and encapsulating source and target language representations. |
| Outcome: | The proposed framework is able to successfully transfer language knowledge even when the target language corpus is limited. |
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)
Copied to clipboard
Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy. |
| Approach: | They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses . |
| Outcome: | The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples. |
Just Read the Codebook! Make Use of Quality Codebooks in Zero-Shot Classification of Multilabel Frame Datasets (2025.coling-main)
Copied to clipboard
| Challenge: | Recent development of Large Language Models has led to more scrutiny of their performance on complex datasets. |
| Approach: | They propose to use large language models to provide concise instructions on how to code text with a multitude of complex labels on two datasets with varying topics. |
| Outcome: | The proposed approach outperforms few-shot In-Context-Learning setups on two complex datasets and is token-efficient and requires less hands-on engineering. |
Analysis of LLM as a grammatical feature tagger for African American English (2025.findings-naacl)
Copied to clipboard
| Challenge: | African American English (AAE) presents unique challenges in natural language processing (NLP). |
| Approach: | They evaluate the ability of different NLP systems to recognize distinctive AAE grammatical features by using sentence-level binary classification tasks using both zero-shot and fewshot strategies. |
| Outcome: | The evaluation involved sentence-level binary classification tasks, using both zero-shot and few-shot strategies. |
Zero- and Few-Shot Event Detection via Prompt-Based Meta Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for event detection often fail to detect unseen or rare events due to the lack of domain knowledge. |
| Approach: | They propose a meta learning-based framework for zero-shot event detection that uses a prompt-based prompt and a trigger-aware soft verbalizer to efficiently project output to unseen tasks. |
| Outcome: | The proposed framework performs state-of-the-art in zero-shot and few-shot scenarios on benchmark datasets FewEvent and MAVEN. |
AdaPrompt: Adaptive Model Training for Prompt-based NLP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt-based learning can tackle zero-shot and few-shot NLP tasks . authors propose a method that makes use of pre-trained language models . |
| Approach: | They propose to map NLP tasks into natural language prompts, which are then filled by pre-trained language models. |
| Outcome: | The proposed method outperforms standard prompt-based methods in few-shot settings. |
Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit remarkable capabilities in zero-shot and few-shot settings, but they struggle with extending to few- shot and zero- shot settings due to their architectural design. |
| Approach: | They propose a technique that models discriminative tasks as a set of finite statements and trains an encoder model to discriminate between the potential statements to determine the label. |
| Outcome: | The proposed method achieves competitive performance compared to state-of-the-art LLMs with significantly fewer parameters. |
ScreenQA: Large-Scale Question-Answer Pairs Over Mobile App Screenshots (2025.naacl-long)
Copied to clipboard
Yu-Chung Hsiao, Fedir Zubach, Gilles Baechler, Srinivas Sunkara, Victor Carbune, Jason Lin, Maria Wang, Yun Zhu, Jindong Chen
| Challenge: | Existing screen datasets focus on low-level structural and component understanding or on a much higher-level composite task such as navigation and task completion for autonomous agents. |
| Approach: | They propose to annotate 86k question-answer pairs over the RICO dataset to benchmark screen content understanding. |
| Outcome: | The proposed dataset covers full answers, short answer phrases, and corresponding UI contents with bounding boxes, enabling four subtasks to address various application scenarios. |
Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Search (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for text anonymization and de-identification struggle to balance privacy preservation with text naturalness and utility. |
| Approach: | They propose a tree-search-based iterative sentence rewriting algorithm that obfuscates or deletes private information while preserving coherence, relevance, and naturalness. |
| Outcome: | The proposed algorithm outperforms existing baselines on privacy-sensitive datasets. |
Visual Prompt Tuning for Few-Shot Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work on pretraining models for text classification uses image encoders instead of visual prompts. |
| Approach: | They propose a method to deploy large-scale pre-trained models in the prompt-tuning paradigm in few-shot learning. |
| Outcome: | The proposed method outperforms the most recent prompt-tuning methods on five public text classification datasets. |
Flight of the PEGASUS? Comparing Transformers on Few-shot and Zero-shot Multi-document Abstractive Summarization (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work has shown that pre-trained transformers obtain remarkable performance on many natural language processing tasks including automatic summarization. |
| Approach: | They propose to use transformers to generate multi-document summarization where the summary is explicitly conditioned on a user-given topic statement or question. |
| Outcome: | The proposed models perform well on four challenging summarization datasets from the general domain and one from consumer health. |
Responsibility Perspective Transfer for Italian Femicide News (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work has shown that different descriptions of gender-based violence influence the reader’s perception of who is to blame for the violence. |
| Approach: | They propose to automatically rewrite GBV descriptions to alter the perceived level of blame on the perpetrator. |
| Outcome: | The proposed task alters perceived responsibility levels for perpetrators by using unsupervised, zero-shot and few-shot methods. |
Are Language Models Worse than Humans at Following Prompts? It’s Complicated (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that language models perform surprisingly well when given intentionally irrelevant or misleading prompts. |
| Approach: | They challenge an assumption that humans would perform badly when given pathological instructions by ignoring irrelevant prompts and following them faithfully when given misleading instructions. |
| Outcome: | The proposed model performs well when given intentionally irrelevant or misleading prompts, whereas models do not. |
Multi-lingual and Multi-cultural Figurative Language Understanding (2023.findings-acl)
Copied to clipboard
Anubha Kabra, Emmy Liu, Simran Khanuja, Alham Fikri Aji, Genta Winata, Samuel Cahyawijaya, Anuoluwapo Aremu, Perez Ogayo, Graham Neubig
| Challenge: | Figures permeate human communication, but are understudied in NLP. |
| Approach: | They create a figurative language inference dataset for seven languages associated with a variety of cultures, using cultural and regional concepts for figurativ expressions. |
| Outcome: | The results show that the most common figurative expressions are found in Hindi, Indonesian, Javanese, Kannada, Sundanese, Swahili and Yoruba. |
Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to name entity recognition use autoregressive decoding, hallucinated entities, and formatting errors. |
| Approach: | They propose a method that allows causal LLMs to perform discriminative token classification with full bidirectional context. |
| Outcome: | The proposed method surpasses the previous best method on zero-shot NER benchmarks by +7.9 F1 on average across CrossNER and MIT benchmarks. |
Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual pretrained language models (MPLMs) perform strongly in cross-lingual transfer. |
| Approach: | They propose to augment context with similar sentences retrieved from a high-resource language (HRL) they find a significant correlation between cross-lingual transfer performance and similarity between high- and low-resourced languages . |
| Outcome: | The proposed model outperforms finetuning by 3.7% on three downstream tasks with multilingual parallel test sets across 10 LRLs covering 6 language families in unlabeled and labeled settings. |
MultiVerse: Efficient and Expressive Zero-Shot Multi-Task Text-to-Speech (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Text-to-speech systems that scale up the amount of training data have certain limitations: they require a large amount of data, which increases costs, and overlook prosody similarity. |
| Approach: | They propose a zero-shot multi-task TTS system that can perform TTS or speech style transfer in zero- shot and cross-lingual conditions. |
| Outcome: | The proposed system outperforms other TTS systems trained with the same small amount of data and achieves zero-shot performance comparable to data-driven systems. |
To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Medical open-domain question answering requires substantial access to specialized knowledge. |
| Approach: | They propose a framework that generates multiple-choice questions from a set of open-book parameters and a small-scale reader that can outcompete closed-book questions by 706x using fewer parameters. |
| Outcome: | The proposed framework outperforms closed-book models on MedQA-USMLE, MedMCQA, and MMLU while using up to 706x fewer parameters. |
FactEval: Evaluating the Robustness of Fact Verification Systems in the Era of Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have made significant advances in every natural language processing task, but they are vulnerable to small perturbations in the inputs, raising concerns about their robustness in the real world. |
| Approach: | They propose a large-scale benchmark for extensive evaluation of LLMs in the fact verification domain covering 17 realistic word-level and character-level perturbations and 4 types of subpopulations. |
| Outcome: | The proposed model is brittle to small input changes and exhibits performance variations across different subpopulations. |
MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing medical datasets require high quality domain-specific datasets. |
| Approach: | They propose a multi-level, multi-task, and multi-domain medical benchmark to facilitate the development of language models for healthcare. |
| Outcome: | The proposed model provides granular potential usage and supports a wide range of tasks. |
Continuous Entailment Patterns for Lexical Inference in Context (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models can be used to perform lexical inference in context tasks with relatively small training data. |
| Approach: | They propose to combine a pretrained language model with textual patterns to improve performance in both zero-shot and few-shot settings. |
| Outcome: | The proposed method compares pre-trained models with textual patterns on two established benchmarks for lexical inference in context (LIiC) the results show that the proposed patterns improve performance on LIiC, setting a new state of the art. |
A LLM-based Ranking Method for the Evaluation of Automatic Counter-Narrative Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for evaluating CNs are expensive, time-consuming, and subjective, but lack a universal truth and the lack of a 'universal truth' . |
| Approach: | They propose a model ranking pipeline based on pairwise comparisons of generated CNs from different models organized in a tournament-style format to improve the evaluation process. |
| Outcome: | The proposed method achieves a high correlation with human preference, with a score of 0.88, and compares chat, instruct, and base models, exploring their strengths and limitations. |
Detecting Temporal Ambiguity in Questions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Ambiguous questions have different answers depending on their interpretation and can take diverse forms. |
| Approach: | They propose a manually annotated temporally ambiguous QA dataset that captures temporal ambiguity and propose different search strategies based on disambiguate versions of the questions. |
| Outcome: | The proposed approach captures temporal ambiguity and provides non-search, competitive baselines for detecting temporal and few-shot ambiguities. |
Evaluating Prompting Strategies for Grammatical Error Correction Based on Language Proficiency (2024.lrec-main)
Copied to clipboard
| Challenge: | generative LLMs have been known for overcorrection where results obtain higher recall measures than precision measures. |
| Approach: | They propose to use generative LLMs to prompt grammatical error correction using a model based on language proficiency to examine the interaction between LLM's performance and L2 language proficiency. |
| Outcome: | The proposed model improves on zero-shot and few-shot prompting and fine-tuning models for grammatical error correction for learners of English as a foreign language based on the different proficiency levels. |
IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show pre-trained LMs store linguistic and relational knowledge . pre-training LM models can answer "fill-in-the-blank" questions based on pre-defined relations . |
| Approach: | They propose an open information extraction benchmark for pre-trained language models . they turn pre-trained LMs into zero-shot OIE systems to examine open relational information . |
| Outcome: | The proposed benchmark outperforms state-of-the-art methods on factual OIE datasets without training sets. |
Language Agnostic Multilingual Information Retrieval with Contrastive Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Annotated training data is costly to obtain in many languages . |
| Approach: | They propose a semantic contrastive loss to align parallel sentences that share the same semantics in different languages and a language contrastive gain to leverage parallel sentence pairs to remove language-specific information from non-parallel corpora. |
| Outcome: | The proposed model improves retrieval performance while requiring less computational effort. |
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that large language models (LLMs) rank documents based on the probability of generating the query given the content of a document. |
| Approach: | They propose a ranking system that integrates LLMs with a hybrid zero-shot retriever. |
| Outcome: | The proposed system shows exceptional ranking in both zero-shot and few-shot scenarios. |
MoR: Better Handling Diverse Queries with a Mixture of Sparse, Dense, and Human Retrievers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Different retrievers offer distinct, often complementary signals, but they are not optimal for all queries. |
| Approach: | They propose a zero-shot, weighted combination of heterogeneous retrievers . they validate this intuition by incorporating specialized non-oracle human information sources . |
| Outcome: | Experiments show that a mixture of heterogeneous retrievers outperforms each retriever and larger models by +10.8% and +3.9% on average. |
A Zero-Shot Claim Detection Framework Using Question Answering (2022.coling-1)
Copied to clipboard
| Challenge: | Existing claims detection frameworks are portability to emerging events and low-resource training data settings. |
| Approach: | They propose a claim detection framework that leverages zero-shot Question Answering to solve sub-tasks such as topic filtering, claim object detection, and claimer detection. |
| Outcome: | The proposed framework outperforms baselines on the NewsClaims benchmark. |
A Pilot Study on Dialogue-Level Dependency Parsing for Chinese (2023.findings-acl)
Copied to clipboard
| Challenge: | Dialogue-level dependency parsing has received insufficient attention, especially for Chinese. |
| Approach: | They propose a signal-based method to transform seen syntactic dependencies into unseen ones between elementary discourse units (EDUs) they apply single-view and multi-view data selection to access reliable pseudo-labeled instances. |
| Outcome: | The proposed method transforms seen syntactic dependencies into unseen ones between elementary discourse units (EDUs) the proposed method also provides reliable pseudo-labeled instances. |
Extracting Biomedical Entities from Noisy Audio Transcripts (2024.lrec-main)
Copied to clipboard
Nima Ebadi, Kellen Morgan, Adrian Tan, Billy Linares, Sheri Osborn, Emma Majors, Jeremy Davis, Anthony Rios
| Challenge: | Named Entity Recognition (NER) is particularly affected by noise, often termed the ASR-NLP gap. |
| Approach: | They propose a dataset to bridge the ASR-NLP gap in the biomedical domain by extracting adverse drug reactions and mentions of entities from the Brief Test of Adult Cognition by Telephone (BTACT) exam. |
| Outcome: | The proposed method can clean 2,000 clean and noisy recordings and eliminate errors using zero-shot and few-shot methods. |
Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have transformed NLP with their remarkable In-context Learning capabilities. |
| Approach: | They propose to use large language models to generalize from labeled examples of predefined tasks to novel tasks . they use biological neurons and the Transformer architecture to study the potential for information sharing across tasks. |
| Outcome: | The proposed model can generalize from labeled examples of predefined tasks to novel tasks despite no examples from the target task in the context. |
Zero-Shot Dialogue State Tracking via Cross-Task Transfer (2021.emnlp-main)
Copied to clipboard
Zhaojiang Lin, Bing Liu, Andrea Madotto, Seungwhan Moon, Zhenpeng Zhou, Paul Crook, Zhiguang Wang, Zhou Yu, Eunjoon Cho, Rajen Subba, Pascale Fung
| Challenge: | Existing approaches to training a dialogue state tracking model require extensive annotated dialogue data. |
| Approach: | They propose to transfer cross-task knowledge from general question answering corpora to QA model that can handle zero-shot DST. |
| Outcome: | The proposed model improves existing zero-shot and few-shot results on MultiWoz and shows better generalization ability in unseen domains. |
Split and Rephrase with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Split and Rephrase (SPRP) tasks require modelling complex grammatical aspects to provide optimal splits and appropriate rephrasing. |
| Approach: | They evaluate large language models on the Split and Rephrase task . they show they can provide large improvements over the state of the art on main metrics . |
| Outcome: | The proposed model outperforms the state-of-the-art model on the Split and Rephrase task on the main metric, but still lacks in splitting compliance. |
TaskWeb: Selecting Better Source Tasks for Multi-task NLP (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in NLP has shown that knowing task relationships via pairwise task transfer improves choosing one or more source tasks that help to learn a new target task. |
| Approach: | They propose a method to quantify task relationships via pairwise task transfer and build smaller training sets that improve zero-shot performances across 11 different target tasks. |
| Outcome: | The proposed method improves overall rankings and top-k precision of source tasks by 10% and 38%, respectively. |
Low-resource Neural Machine Translation with Cross-modal Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural machine translation techniques rely on large monolingual corpus, which is costly for some low-resource languages. |
| Approach: | They propose a cross-modal contrastive learning method to learn a shared space for all languages by additional visual modality. |
| Outcome: | The proposed method can learn cross-modal and cross-lingual alignment with small amount of image-text pairs and achieves significant improvements over the text-only baseline. |
Can Large Language Models Understand Argument Schemes? (2025.findings-acl)
Copied to clipboard
| Challenge: | Argument schemes are stereotypical forms of reasoning that occur in everyday arguments. |
| Approach: | They propose to use large language models (LLMs) to classify argument schemes based on Walton’s taxonomy to employ formal definitions and LLM-generated descriptions to enhance task instructions. |
| Outcome: | The proposed models perform well on annotated and automatically generated arguments, and provide insights for advancing reasoning capabilities in computational argumentation. |
SafeLawBench: Towards Safe Alignment of Large Language Models (2025.findings-acl)
Copied to clipboard
Chuxue Cao, Han Zhu, Jiaming Ji, Qichao Sun, Zhenghao Zhu, Wu Yinyu, Josef Dai, Yaodong Yang, Sirui Han, Yike Guo
| Challenge: | Recent studies indicate that large language models (LLMs) may exhibit risks, including threats to the protection of private data and the generation of hallucinations. |
| Approach: | They propose to evaluate LLMs from a legal perspective using the SafeLawBench benchmark. |
| Outcome: | The proposed framework categorizes safety risks into three levels based on legal standards and includes 24,860 multi-choice questions and 1,106 open-domain question-answering tasks. |
mCLIP: Multilingual CLIP via Cross-lingual Transfer (2023.acl-long)
Copied to clipboard
Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, Wenping Wang
| Challenge: | Existing multilingual vision-language pretrained models are biased towards English due to the lack of sufficient non-English image-text pairs. |
| Approach: | They propose to train a retrieval-efficient dual-stream multilingual VLP model by aligning CLIP model and a multilingual text encoder through a novel Triangle Cross-modal Knowledge Distillation method. |
| Outcome: | Empirical results show that mCLIP achieves new state-of-the-art performance for both zero-shot and finetuned multilingual image-text retrieval tasks. |
Legally Enforceable Hate Speech Detection for Public Forums (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work does not align systems with enforceable hate speech definitions, which can make outputs inconsistent with the goals of regulators. |
| Approach: | They propose a task for enforceable hate speech detection centred around legal definitions and an annotated dataset of violations by legal experts. |
| Outcome: | The proposed method can be used to detect hate speech in public forums on a large scale. |
Diverse and Effective Synthetic Data Generation for Adaptable Zero-Shot Dialogue State Tracking (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing zero-shot dialogue state tracking datasets are limited in the number of domains and slot types they cover due to the high costs of data collection. |
| Approach: | They propose a fully automatic approach that generates synthetic zero-shot dialogue state tracking datasets. |
| Outcome: | The proposed approach can generate dialogues across 1,000+ domains with silver-standard dialogue state annotations and slot descriptions. |
Large Language Models are Few-Shot Training Example Generators: A Case Study in Fallacy Recognition (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on fallacy recognition is still in its early stages, with limited datasets available. |
| Approach: | They propose to use GPT3.5 to generate synthetic examples and explore prompt settings to improve the representation of the infrequent classes. |
| Outcome: | The proposed model improves on existing models and generates synthetic examples with GPT3.5. |
Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Tracking (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot Dialogue State Tracking have focused on domaintransfers and have not yielded satisfactory results. |
| Approach: | They propose a new In-Context Learning method to introduce additional updating strategies in zero-shot DST by leveraging powerful Large Language Models and translating the original dialogue to JSON through semantic parsing as an intermediate state. |
| Outcome: | The proposed method outperforms existing zero-shot DST methods on MultiWOZ, showing significant improvements in JGA and slot accuracy compared to existing methods. |
CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is popular in reasoning tasks, but its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored. |
| Approach: | They propose a Coarse-to-Fine Chain-of-Thought approach that breaks down NLU tasks into multiple reasoning steps where LLMs can learn to acquire essential concepts. |
| Outcome: | The proposed approach is effective in assisting the LLMs adapt to multi-grained NLU tasks under zero-shot and few-shot multi-domain settings. |
Neuralizing Regular Expressions for Slot Filling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to integrate neural networks and symbolic rules have their merits and weaknesses. |
| Approach: | They propose to integrate regular expressions into neural networks for a slot filling task . they use finite-state transducers to convert regular expression into a neural network . their model has superior zero-shot and few-shot performance . |
| Outcome: | The proposed model outperforms rules in zero-shot and few-shot scenarios and is competitive when training data is available. |
Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT - A Text-to-SQL Parsing Comparison (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a number of open-source large language models claim to be performing better than commercial ones . however, these models fall short of the performance achieved by closed-source models like GPT-3.5 . |
| Approach: | They evaluate six popular large language models against each other to evaluate their performance . authors say open-source models are not as effective as those built by commercial models . |
| Outcome: | a new set of models claim to match or surpass the language understanding abilities of commercial models . the results show that the models performed far below the performance of closed-source models compared to open-source ones . |
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David Clifton
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
Let’s Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models’ Understanding of Sports (2025.emnlp-main)
Copied to clipboard
Punit Kumar Singh, Nishant Kumar, Akash Ghosh, Kunal Pasad, Khushi Soni, Manisha Jaishwal, Sriparna Saha, Syukron Abu Ishaq Alfarozi, Asres Temam Abagissa, Kitsuchart Pasupa, Haiqin Yang, Jose G Moreno
| Challenge: | Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. |
| Approach: | They propose to use multiple-choice questions (MCQs) to assess LMs' understanding of traditional sports across 60 countries and 6 continents. |
| Outcome: | The new benchmark will be publicly available, fostering research in culturally aware AI systems. |
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning (2023.findings-emnlp)
Copied to clipboard
Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Korthikanti, Weili Nie, De-An Huang, Linxi Fan, Zhiding Yu, Shiyi Lan, Bo Li, Mohammad Shoeybi, Ming-Yu Liu, Yuke Zhu, Bryan Catanzaro, Chaowei Xiao, Anima Anandkumar
| Challenge: | Existing methods for image-to-text generation store all knowledge within parameters, thus requiring computational-expensive fine-tuning. |
| Approach: | They propose a Retrieval-augmented Visual Language Model that stores all the knowledge within parameters and can be used to retrieve it from the external database. |
| Outcome: | The proposed model significantly boosts performance for image-to-text generation tasks with 4x less parameters compared with baseline methods. |
DiZiNER: Disagreement-guided Instruction Refinement via Simulating Pilot Annotation for Zero-shot Named Entity Recognition (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models have advanced information extraction (IE) by enabling zero-shot and few-shot named entity recognition (NER) but their outputs still show persistent and systematic errors. |
| Approach: | They propose a framework that simulates the pilot annotation process and employs LLMs as both annotators and supervisors to refine model disagreements. |
| Outcome: | Using a pilot annotation process, the proposed framework outperforms its supervisor model on 18 benchmarks. |
Referral Augmentation for Zero-Shot Information Retrieval (2024.findings-acl)
Copied to clipboard
| Challenge: | Referral-augmented retrieval improves zero-shot document retrieval in a variety of tasks . prior work shows sparse models struggle to reconcile with dense models . |
| Approach: | They propose a technique that concatenates document indices with referrals from other documents that cite or link to the given document. |
| Outcome: | The proposed technique outperforms generative text expansion techniques on structured tasks and improves on ACL paper retrieval. |
Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLMs generate responses based on the dialogue context, overlooking the underlying linguistic cues about the user status exhibited in the context. |
| Approach: | They propose a linguistic cue-based chain-of-thoughts method which enhances the LLMs inference with an intermediate reasoning step to find cues exhibited in the dialogue. |
| Outcome: | The proposed method outperforms standard prompting methods on in-depth dialogue questions and linguistic cues exhibited in the context. |
Structured Confidence–Guided Online Adaptation for LLM-based Multi-Label Classification (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) enable zero-shot and few-shot multi-label text classification . but most approaches perform static inference and degrade under streaming test data . |
| Approach: | They propose a structured confidence-guided online adaptation framework for LLM-based multi-label generation without parameter updates. |
| Outcome: | The proposed framework improves Micro-F1 and Macro-F1, with the largest gains on long-tail labels. |
Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Models (2025.findings-acl)
Copied to clipboard
Ahmed Elshabrawy, Thanh-Nhi Nguyen, Yeeun Kang, Lihan Feng, Annant Jain, Faadil Abdullah Shaikh, Jonibek Mansurov, Mohamed Fazli Mohamed Imam, Jesus-German Ortiz-Barajas, Rendi Chevi, Alham Fikri Aji
| Challenge: | Large Language Models excel in zero-shot and few-shot tasks, but their architecture makes them difficult to use. |
| Approach: | They adapt Large Language Models (LLMs) for zero-shot generalization using Statement Tuning . they find encoders can achieve zero- shot cross-lingual generalization . |
| Outcome: | The proposed model generalizes well across languages while being more efficient. |
LLMs to Replace Crowdsourcing For Parallel Data Creation? The Case of Text Detoxification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Manual annotation methods, such as crowdsourcing, are costly and require intricate task design skills. |
| Approach: | They propose to use open source LLMs to annotate parallel data for text detoxification . they generate a pseudo-parallel detoxification dataset using activation patching . |
| Outcome: | The proposed model performs comparable to the original dataset in automatic detoxification evaluation metrics and superior quality in manual evaluation and side-by-side comparisons. |
Towards Zero-shot Learning for End-to-end Cross-modal Translation Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | End-to-end zero-shot speech translation model is based on a zero-shot approach, but it is less competitive because of the limited amount of data available for multiple modalities. |
| Approach: | They propose an end-to-end zero-shot speech translation model that connects two pre-trained uni-modality modules via word rotator’s distance. |
| Outcome: | The proposed model performs better than or as well as those of the CTC-based models and can be trained in an end-to-end style to avoid error propagation. |
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning (2023.acl-long)
Copied to clipboard
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Yang Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
| Challenge: | Existing methods for data-to-text generation focus on specific types of structured data. |
| Approach: | They propose a method that provides a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. |
| Outcome: | The proposed method improves zero-shot and few-shot scenarios and can adapt to new structured data. |
FormosanBench: Benchmarking Low-Resource Austronesian Languages in the Era of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLMs consistently underperform across all tasks, with 10-shot learning and fine-tuning offering only limited improvements. |
| Approach: | They introduce FormosanBench, a benchmark for evaluating LLMs on low-resource Austronesian languages. |
| Outcome: | The proposed benchmark covers three endangered Formosan languages: Atayal, Amis, and Paiwan . existing LLMs consistently underperform across all tasks, with 10-shot learning and fine-tuning offering only limited improvements. |
CorrSynth - A Correlated Sampling Method for Diverse Dataset Generation from LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance in diverse tasks using zero-shot and few-shot prompting. |
| Approach: | They propose a method which generates data that is more diverse and faithful to the input prompt using a correlated sampling strategy. |
| Outcome: | The proposed method overcomes the complexity drawbacks of other guidance-based techniques and improves student metrics and intrinsic metrics upon competitive baselines across four datasets. |
Learning to Describe for Predicting Zero-shot Drug-Drug Interactions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing computational methods for DDI prediction fail to capture interactions for new drugs due to the lack of knowledge. |
| Approach: | They propose a problem setup as zero-shot DDI prediction that deals with the case of new drugs by using textual information from online databases. |
| Outcome: | The proposed method improves on several settings including zero-shot and few-shot DDI prediction and the selected texts are semantically relevant. |
Investigating the Impact of Conceptual Metaphors on LLM-based NLI through Shapley Interactions (2025.findings-emnlp)
Copied to clipboard
Meghdut Sengupta, Maximilian Muschalik, Fabian Fumagalli, Barbara Hammer, Eyke Hüllermeier, Debanjan Ghosh, Henning Wachsmuth
| Challenge: | Metaphorical language is prevalent in everyday communication, often used unconsciously, as in "rising crime" incorporating conceptual metaphors improves performance on downstream tasks that implicitly require correct metaphor interpretation, such as NLI. |
| Approach: | They extend metaphorical texts in an existing NLI dataset by source and target domains and conduct an ablation study using Shapley values and interactions to assess the extent to which LLMs interpret metaphorical language correctly in NLI. |
| Outcome: | The proposed method improves performance on natural language inference tasks with metaphoric input. |
Transfer-Aware Data Selection for Domain Adaptation in Text Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to improve domain adaptation do not guarantee improved adaptability, but may negatively impact model performance. |
| Approach: | They propose a framework that can effectively improve model adaptability by selecting beneficial data without evaluating all source data. |
| Outcome: | The proposed framework improves model adaptability by selecting beneficial data without evaluating all source data. |
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown striking ability to adapt to target tasks with a few input-output demonstrations. |
| Approach: | They propose a framework which bootstraps LMs’ intrinsic capabilities to perform zero-shot ICL. |
| Outcome: | The proposed framework outperforms baselines on 23 BIG-Bench Hard tasks on average accuracy and head-to-head comparison. |
Can Large Language Models Enhance Predictions of Disease Progression? Investigating Through Disease Network Link Prediction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant strides in various tasks, yet their effectiveness in predicting disease progression remains relatively unexplored. |
| Approach: | They propose a large language model with graph prompting and Retrieval-augmented generation to enhance the prediction performance of disease comorbidity within disease networks. |
| Outcome: | The proposed model outperforms Graph Neural Networks and Graph Prompts and Retrieval-Augmented Generation models in disease progression prediction tasks. |
LMGQS: A Large-scale Dataset for Query-focused Summarization (2023.findings-emnlp)
Copied to clipboard
Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Pengcheng He, Chenguang Zhu, Michael Zeng
| Challenge: | Lack of large-scale datasets for query-focused summarization hinders model development . lack of data limits the ability of QFS models to train robust neural models . |
| Approach: | They propose to generate a query for each summary sentence in a generic summarization annotation using a pretrained language model. |
| Outcome: | The proposed model achieves state-of-the-art zero-shot and supervised performance on multiple existing QFS benchmarks. |
Consistent Discourse-level Temporal Relation Extraction Using Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have spurred research on temporal relation extraction tasks. |
| Approach: | They propose a framework to improve LLMs’ temporal relation extraction capabilities using context selection, prompts inspired by Allen’s interval algebra and reflection-based consistency learning. |
| Outcome: | The proposed framework improves LLMs’ extraction capabilities by focusing on context selection, prompts inspired by Allen’s interval algebra and reflection-based consistency learning. |
MoPE: Mixture of Prefix Experts for Zero-Shot Dialogue State Tracking (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing zero-shot dialogue state tracking models suffer from domain transferring and partial prediction problems. |
| Approach: | They propose to establish connections between similar slots in different domains to improve model transfer performance in unseen domains. |
| Outcome: | Empirical results show that the proposed model achieves the goal accuracy of 57.13% on MultiWOZ2.1 and 55.4. |
AraReasoner: Evaluating Reasoning-Based LLMs for Arabic NLP (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have shown remarkable progress in reasoning abilities and general natural language processing tasks, yet their performance on Arabic data remains underexplored. |
| Approach: | They compare reasoning-focused LLMs with deepSeek models across 15 Arabic NLP tasks . they use zero-shot, few-shot and fine-tuning to evaluate their capacity for linguistic reasoning . |
| Outcome: | The proposed models outperform strong models on Arabic datasets and are compared with other models. |
Calibrated Seq2seq Models for Efficient and Generalizable Ultra-fine Entity Typing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | CASENT predicts ultra-fine entities mentioned in text into types with calibrated confidence scores. |
| Approach: | They propose a model that predicts ultra-fine entities with calibrated confidence scores for entity typing. |
| Outcome: | The proposed model outperforms existing models in terms of F1 score and calibration error while achieving 50 times faster inference speed. |
A Thorough Examination on Zero-shot Dense Retrieval (2023.findings-emnlp)
Copied to clipboard
Ruiyang Ren, Yingqi Qu, Jing Liu, Xin Zhao, Qifei Wu, Yuchen Ding, Hua Wu, Haifeng Wang, Ji-Rong Wen
| Challenge: | Recent advances in dense retrieval (DR) models have been shown to be not as competitive as traditional sparse retrieval models in a zero-shot retrieval setting. |
| Approach: | They propose to examine the zero-shot capability of DR models by analyzing key factors related to source training set and potential bias from target dataset. |
| Outcome: | The proposed model is not as competitive as sparse retrieval models in a zero-shot retrieval setting. |
Towards Low-Resource Alignment to Diverse Perspectives with Sparse Feedback (2025.findings-emnlp)
Copied to clipboard
| Challenge: | popular training paradigms for language models often assume there is one optimal answer for every query. |
| Approach: | They propose to enhance pluralistic alignment of language models using pluralistic decoding and model steering methods. |
| Outcome: | The proposed methods improve pluralistic alignment of language models in a low-resource setting . the proposed methods decrease false positives in several high-stakes tasks . |
TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models typically come with a fixed architecture, but not all layers contribute equally to every downstream task. |
| Approach: | They propose an inference-time method that selectively removes irrelevant or detrimental layers . the method is hardware-agnostic, requires no retraining, and operates entirely at inference time . |
| Outcome: | The proposed method matches or surpasses baseline performance while reducing computational costs. |
Language is Scary when Over-Analyzed: Unpacking Implied Misogynistic Reasoning with Argumentation Theory-Driven Prompts (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new study aims to understand the implicit reasoning used to convey misogynistic comments in Italian and English. |
| Approach: | They propose misogyny detection as an Argumentative Reasoning task and use argumentation theory to build large language models to understand the implicit reasoning used to convey misogany in Italian and English. |
| Outcome: | The proposed task is an argumentative reasoning task in Italian and English. |
Easy as PIE? Identifying Multi-Word Expressions with LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multiword expressions (MWEs) are a semantically non-compositional subclass of multiword expression . authors show that prompt-based LLMs can perform competitively with supervised models . |
| Approach: | They propose a prompt-based approach to identify idiomatic expressions in running text . they find prompt-driven LLMs can perform competitively with supervised models . |
| Outcome: | The proposed approach can perform well with supervised models on annotated data. |
USDC: A Dataset of ̲User ̲Stance and ̲Dogmatism in Long ̲Conversations (2025.findings-acl)
Copied to clipboard
| Challenge: | Previously, studies on stance and dogmatism in user conversations have focused on training models using annotated datasets at the post level, treating each post as independent and randomly sampling posts from conversation threads. |
| Approach: | They build a dataset for studying user opinion fluctuations in 764 long multi-user Reddit conversation threads, called USDC. |
| Outcome: | The proposed dataset analyzes user opinion fluctuations in 764 long multi-user Reddit conversation threads. |
AMANDA: Agentic Medical Knowledge Augmentation for Data-Efficient Medical Visual Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing Med-MLLMs fail when deployed in low-resource settings where abundant labeled data is unavailable. |
| Approach: | They propose a training-free agentic framework that performs medical knowledge augmentation via LLM agents. |
| Outcome: | The proposed framework performs medical knowledge augmentation via LLM agents. |
VLURes: Benchmarking Long-Text Grounding and Cross-Lingual Robustness in Vision Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | ***VLURes** provides a practical testbed for long-text grounding and multilingual robustness in web-realistic agent settings. |
| Approach: | They propose a multilingual benchmark for evaluating vision-language models under long-text grounding. |
| Outcome: | ***VLURes** provides a testbed for long-text grounding and multilingual robustness in web-realistic agent settings. |
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs . |
| Approach: | They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals . |
| Outcome: | The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks. |
Hidden in Plain Sight: Evaluation of the Deception Detection Capabilities of LLMs in Multimodal Settings (2025.acl-long)
Copied to clipboard
| Challenge: | Detecting deception in an increasingly digital world is a critical and challenging task. |
| Approach: | They evaluate the performance of both open-source and proprietary LLMs on three datasets . they find that fine-tuned LLM achieve state-of-the-art performance on textual deception detection . |
| Outcome: | The proposed models achieve state-of-the-art on textual deception detection, whereas LMMs struggle to fully leverage multimodal cues. |
Expectation Preference Optimization: Reliable Preference Estimation for Improving the Reasoning Capability of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Pairwise preference optimization is used to improve supervised fine-tuning performance of large language models. |
| Approach: | They propose an algorithm that takes pairs of sample groups instead of single samples for preference learning. |
| Outcome: | The proposed algorithm outperforms baseline methods on reasoning benchmarks. |
Split-Merge: Scalable and Memory-Efficient Merging of Expert LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a zero-shot merging framework for large language models consolidates specialized domain experts into a single model without any further training. |
| Approach: | They propose a zero-shot merging framework that consolidates specialized domain experts into a single model without further training. |
| Outcome: | Experiments on code generation, mathematical reasoning, medical question answering, and instruction-following benchmarks confirm the versatility and effectiveness of the proposed framework. |
Zero-Shot Spoken Language Understanding via Large Language Models: A Preliminary Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown promising results in zero-shot settings, which motivates us to explore prompt-based methods. |
| Approach: | They propose a two-stage framework which transforms the SLU task into a question-answering problem by directly prompting LLMs. |
| Outcome: | The proposed framework can be built by directly prompting LLMs to understand user needs without training data. |
Label Set Optimization via Activation Distribution Kurtosis for Zero-Shot Classification with Generative Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning (ICL) performance is highly sensitive to prompt design, yet the impact of class label options (e.g. lexicon or order) in zero-shot classification remains underexplored. |
| Approach: | They propose a post-hoc method for selecting optimal label sets in zero-shot ICL with large language models. |
| Outcome: | The proposed method consistently achieves performance gains of 0.54 to 0.76 compared to the conventional method. |
InsAT: Instance-aware Semantic Alignment and Transfer from Human–Object Keypoints for Zero-to-Few-shot Action Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing zero-shot (ZS) approaches emphasize human motion while underutilizing contextual information, particularly human–object interactions. |
| Approach: | They propose a framework for ZS recognition and zero-to-few-shot adaptation that leverages instance-level language descriptions. |
| Outcome: | The proposed framework outperforms keypoint-based ZS methods while remaining data-efficient and robust. |
Activation Reward Models for Few-Shot Model Alignment (2026.findings-acl)
Copied to clipboard
Tianning Chai, Chancharik Mitra, Brandon Huang, Gautam Rajendrakumar Gare, Zhiqiu Lin, Assaf Arbelle, Leonid Karlinsky, Rogerio Feris, Trevor Darrell, Deva Ramanan, Roei Herzig
| Challenge: | A common approach is to use reward models that enable reinforcement-learning post-training. |
| Approach: | They propose a method that steers LLM activations to align with few-shot preference data without finetuning. |
| Outcome: | The proposed method surpasses zero-shot, few-shot and voting-based benchmarks on reward hacking and noise signals. |
False Friends or Cognates? A Cross-lingual Semantic Ambiguity Evaluation for Galician, Portuguese and Spanish (2026.acl-long)
Copied to clipboard
| Challenge: | Closely related languages exhibit a high degree of lexical and orthographic similarity, which can facilitate cross-lingual understanding but also give rise to systematic semantic ambiguity. |
| Approach: | They introduce six cross-lingual datasets that are manually or semi-automatically generated and are able to identify and process false friends among these languages. |
| Outcome: | The proposed models can identify and process false friends among Galician, Portuguese, and Spanish. |
Evaluating Large Vision Language Models on Bangla Medical Visual Question Answering (2026.findings-acl)
Copied to clipboard
Rafid Ahmed, Intesar Tahmid, Mir Sazzat Hossain, Tasnimul Hossain Tomal, Md Mahir Jawad, Anam Borhan Uddin, Md Fahim, Md Farhad Alam Bhuiyan
| Challenge: | Recent advances in Large Language Models and Large Vision Language Model (LVLMs) have demonstrated promising capabilities in complex reasoning tasks, but low-resource contexts like Bangla are underexplored. |
| Approach: | They propose a multilingual medical visual question answering dataset using Bangla. |
| Outcome: | The proposed model performs well on generalized visual tasks but struggles with fine-grained diagnostic reasoning, achieving low accuracy in specialized categories. |