ENSEMBLE

Improving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusion (2023.findings-eacl)

Challenge:	Existing defense mechanisms assume that only one type of trigger is adopted by the attacker, while defending against multiple simultaneous and independent trigger types necessitates general defense frameworks.
Approach:	They propose a framework that uses a mixture of experts as a trigger-only ensemble to defend against multiple trigger types.
Outcome:	The proposed framework defends against multiple trigger types in a single ensemble and in combination of models.

Copied to clipboard

Yi-An Lai, Elman Mansimov, Yuqing Xie, Yi Zhang

Green & Sustainable NLP

Challenge:	a regression error during model upgrade often outweighs the benefits of accuracy gain . a novel method that promotes backward compatibility during model upgrades is proposed .
Approach:	They propose a method that promotes backward compatibility via learning to mix predictions between old and new models.
Outcome:	The proposed method outperforms existing methods and achieves negative flip rate reductions by 73.2% on two model upgrade scenarios.

Unsupervised Detection of LLM-Generated Text in Korean Using Syntactic and Semantic Cues (2026.findings-eacl)

Copied to clipboard

Heejeong Jeon, MinSu Park, YunSeok Choi, Eunil Park

Low-Resource NLP

Challenge:	Prior work focused on English, leaving low-resource languages such as Korean underexplored.
Approach:	They propose an unsupervised framework that integrates syntactic token cohesiveness and semantic regeneration similarity to detect Korean text.
Outcome:	The proposed framework outperforms baselines in Korean and other low-resource languages without training.

Diversify, Rationalize, and Combine: Ensembling Multiple QA Strategies for Zero-shot Knowledge-based VQA (2024.findings-emnlp)

Copied to clipboard

Miaoyu Li, Haoxin Li, Zilin Du, Boyang Li

AtTGen: Attribute Tree Generation for Real-World Attribute Joint Extraction (2023.acl-long)

Challenge:	Knowledge-based Visual Qustion-answering (K-VQA) often requires background knowledge beyond the image content.
Approach:	They propose a method that uses a bundle of complementary question-answering tactics to aggregate their answers using textual rationales.
Outcome:	Experiments show that DietCoke outperforms state-of-the-art LLM-based baselines by 2.8% and 4.7% on K-VQA.

Copied to clipboard

Yanzeng Li, Bingcong Xue, Ruoyu Zhang, Lei Zou

GOVERN: Gradient Orientation Vote Ensemble for Multi-Teacher Reinforced Distillation (2024.emnlp-industry)

Challenge:	Attribute extraction aims to identify attribute names and the corresponding attribute values from descriptive texts.
Approach:	They propose a unified formulation for real-world attribute extraction application, where closed-world, open-world and semi-open attribute extraction tasks are modeled uniformly.
Outcome:	The proposed model outperforms existing methods on three datasets and outperformed existing methods by a large margin.

Copied to clipboard

Wenjie Zhou, Zhenxin Ding, Xiaodong Zhang, Haibo Shi, Junfeng Wang, Dawei Yin

Distilling Knowledge for Search-based Structured Prediction (P18-1)

Challenge:	Pre-trained language models have achieved remarkable performance in OpenQA, but for practical deployment, knowledge distillation is crucial to maintain high performance while operating under computational constraints.
Approach:	They propose an algorithm to perform unsupervised knowledge distillation without the guidance of labels to achieve 99.5% of performance.
Outcome:	The proposed algorithm achieves 99.5% of performance in a commercial question-answering system.

Copied to clipboard

Yijia Liu, Wanxiang Che, Huaipeng Zhao, Bing Qin, Ting Liu

Frustratingly Easy System Combination for Grammatical Error Correction (2022.naacl-main)

Challenge:	Existing studies have focused on the performance of structured prediction models, but they are often limited by the ambiguities of the reference policy.
Approach:	They propose to distill an ensemble of multiple models trained with different initializations into a single model and use it to explore the search space.
Outcome:	The proposed model outperforms the greedy models on two typical search-based structured prediction tasks and achieves 1.32 in LAS and 2.65 in BLEU over strong baselines.

Copied to clipboard

Muhammad Reza Qorib, Seung-Hoon Na, Hwee Tou Ng

Syntactic Text Processing

Challenge:	Using a simple logistic regression algorithm, we combine GEC models for binary classification.
Approach:	They propose a logistic regression algorithm that can combine GEC models with binary classification.
Outcome:	The proposed method outperforms the state-of-the-art by 4.2 points on the CoNLL-2014 and 7.2 points on BEA-2019 test sets.

Multi-View Domain Adapted Sentence Embeddings for Low-Resource Unsupervised Duplicate Question Detection (D19-1)

Copied to clipboard

Nina Poerner, Hinrich Schütze

Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)

Challenge:	Stack Exchange has fewer than 160 user-labeled duplicates, and 25% have fewer.
Approach:	They propose a framework that combines sentence encoders with unlabeled data to solve the problem of duplicate question detection in Community Question Answering forums.
Outcome:	The proposed framework outperforms BM25, a single-view system and a supervised domain-adversarial DQD method on the CQADupStack corpus and on Stack Exchange forums.

Copied to clipboard

Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong

Unsupervised Grammatical Error Correction Rivaling Supervised Methods (2023.emnlp-main)

Challenge:	Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise.
Approach:	They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble.
Outcome:	The proposed technique improves ensemble performance and robustness against erroneous signals.

Copied to clipboard

Hannan Cao, Liping Yuan, Yuchen Zhang, Hwee Tou Ng

Low-Resource NLP

Challenge:	Current state-of-the-art grammatical error correction systems rely on labeled data . current systems require manual correction and require a large quantity of labeles .
Approach:	They propose an unsupervised method to build a grammatical error correction system using a fixer and a critic.
Outcome:	The proposed system outperforms previous unsupervised systems on English and Chinese GEC.

Devil’s Advocate: Novel Boosting Ensemble Method from Psychological Findings for Text Classification (2021.findings-emnlp)

Copied to clipboard

Hwiyeol Jo, Jaeseo Lim, Byoung-Tak Zhang

Boosted Dense Retriever (2022.naacl-main)

Challenge:	Existing ensemble methods that combine submodels to create a composite model can improve model performance by diminishing model bias and variance.
Approach:	They propose a method which uses a deliberately dissenting model to force other submodels within the ensemble to better collaborate.
Outcome:	The proposed method shows comparable or improved performance on 5 text classification tasks when compared to conventional methods.

Copied to clipboard

Patrick Lewis, Barlas Oguz, Wenhan Xiong, Fabio Petroni, Scott Yih, Sebastian Riedel

Information Retrieval

Challenge:	DrBoost is a dense retrieval ensemble that is trained in stages to correct retrieval mistakes . it produces representations which are 4x more compact, while delivering comparable retrieval results.
Approach:	They propose a dense retrieval ensemble inspired by boosting that is trained in stages . they produce representations which are 4x more compact, while delivering comparable retrieval results .
Outcome:	The proposed model performs surprisingly well under approximate search with coarse quantization, reducing latency and bandwidth needs by another 4x.

Attention-Guided Answer Distillation for Machine Reading Comprehension (D18-1)

Copied to clipboard

Minghao Hu, Yuxing Peng, Furu Wei, Zhen Huang, Dongsheng Li, Nan Yang, Ming Zhou

Responsible & Trustworthy NLP

Challenge:	Existing approaches to reading comprehension systems are vulnerable to adversarial attacks.
Approach:	They propose to use knowledge distillation to transfer knowledge from an ensemble to a single model.
Outcome:	The proposed methods outperform the teacher on adversarial datasets and NarrativeQA benchmarks.

Memo-SQL: Structured Decomposition and Experience-Driven Self-Correction for Training-Free NL2SQL (2026.findings-acl)

Copied to clipboard

Yang Zerui, Weichuan Wang, Yanwei Xu, Linqi Song, Yudai Matsuda, Wei Han, Bo Bai

Programming Languages in NLP

Challenge:	Existing NL2SQL systems rely on in-context learning with only correct examples . current test-time scaling methods often decompose questions arbitrarily, resulting in poor performance .
Approach:	They propose a structured decomposition and experience-aware self-correction framework for NL2SQL . they build a dynamic memory of successful queries and historical error–fix pairs .
Outcome:	The proposed framework achieves 68.5% execution accuracy on BIRD, setting new state of the art among open, zero-fine-tuning methods.

Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction (2022.acl-long)

Copied to clipboard

Maksym Tarnavskyi, Artem Chernodub, Kostiantyn Omelianchuk

Syntactic Text Processing

Challenge:	Currently, machine translation (MT) is the mainstream approach for GEC.
Approach:	They propose to ensemble Transformer-based encoders by majority votes on span-level edits . their best ensemble achieves a new SOTA result even without pre-training on synthetic datasets - "Troy-Blogs" and "Try-1BW".
Outcome:	The proposed model achieves a new SOTA result even without pre-training on synthetic datasets.

Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles (2020.findings-emnlp)

Copied to clipboard

Christopher Clark, Mark Yatskar, Luke Zettlemoyer

Visual Data in NLP

Challenge:	Recent work has shown that datasets contain incidental correlations created by idiosyncrasies in the data collection process.
Approach:	They propose a method that detects and ignores dataset-specific correlations by introducing a new method that makes them conditionally independent.
Outcome:	The proposed method detects and ignores these kinds of dataset-specific correlations, and does not require the bias to be known in advance.

SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions (2020.acl-main)

Copied to clipboard

Mao Ye, Chengyue Gong, Qiang Liu

On the Cost-Effectiveness of Stacking of Neural and Non-Neural Methods for Text Classification: Scenarios and Performance Prediction (2021.findings-acl)

Challenge:	State-of-the-art NLP models can be fooled by human-unaware transformations such as synonymous word substitution.
Approach:	They propose a method that constructs a stochastic ensemble by applying random word substitutions on the input sentences and leverages the statistical properties to provably certify the robustness.
Outcome:	The proposed method outperforms state-of-the-art methods on IMDB and Amazon text classification tasks with practically meaningful certified accuracy.

Copied to clipboard

Christian Gomes, Marcos Goncalves, Leonardo Rocha, Sergio Canuto

Information Retrieval

Challenge:	Neural network algorithms excel on Automatic Text Classification tasks, but they are expensive and require high computational costs.
Approach:	They propose to exploit the cost-effectiveness of stacking of automatic text classification classifiers to improve their effectiveness.
Outcome:	The proposed method can predict the best ensemble in each scenario using only fraction of available training data.

Are Neural Topic Models Broken? (2022.findings-emnlp)

Copied to clipboard

Alexander Miserlis Hoyle, Pranav Goel, Rupak Sarkar, Philip Resnik

Bridging the Gap between Different Vocabularies for LLM Ensemble (2024.naacl-long)

Challenge:	Existing evaluation paradigms are often divorced from real-world use . recent results have challenged the validity of the prevailing model evaluation paradigm .
Approach:	They show that neural topic models fare worse in both respects compared to an established classical method.
Outcome:	The proposed method outperforms the members of the ensemble in both respects.

Copied to clipboard

Yangyifan Xu, Jinliang Lu, Jiajun Zhang

Probing Simile Knowledge from Pre-trained Language Models (2022.acl-long)

Challenge:	Existing methods to combine large language models with complete outputs have limited effectiveness . lexical gaps between different LLMs hinder dynamic correction and enhancement .
Approach:	They propose a method to ensemble large language models via Vocabulary Alignment (EVA) they learn mappings between vocabularies of different LLMs with overlapping tokens .
Outcome:	The proposed method bridges the lexical gap among various LLMs, enabling meticulous ensemble at each generation step.

Copied to clipboard

Weijie Chen, Yongzhu Chang, Rongsheng Zhang, Jiashu Pu, Guandan Chen, Le Zhang, Yadong Xi, Yijiang Chen, Chang Su

Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases (D19-1)

Challenge:	Existing approaches to learn generic knowledge from a large corpus are time-consuming and labor-intensive.
Approach:	They propose a framework to probe simile knowledge from pre-trained language models to solve SI and SG tasks.
Outcome:	The proposed framework solves the SI and SG tasks in a simile triple completion task.

Copied to clipboard

Christopher Clark, Mark Yatskar, Luke Zettlemoyer

CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing (2022.acl-long)

Challenge:	Recent advances in neural models exploit dataset-specific patterns that do not generalize well to out-of-domain or adversarial settings.
Approach:	They propose to train a model to be more robust to domain shift if it has prior knowledge of dataset biases.
Outcome:	The proposed model can be more robust to domain shift if it has prior knowledge of dataset biases.

Copied to clipboard

Chen Liang, Pengcheng He, Yelong Shen, Weizhu Chen, Tuo Zhao

Whose Preferences? Differences in Fairness Preferences and Their Impact on the Fairness of AI Utilizing Human Feedback (2024.acl-long)

Challenge:	Existing work has resorted to sharing weights among models, but results are not affordable for real-world deployment.
Approach:	They propose a consistency-regularized ensemble learning approach based on perturbed models to retain ensemble benefits while maintaining a low memory cost.
Outcome:	The proposed approach outperforms the standard ensemble of 8 BERT-base models on the GLUE benchmark by 0.7 with a significantly smaller model size.

Copied to clipboard

Maria Lerner, Florian Dorner, Elliott Ash, Naman Goel

Text Classification

Challenge:	a growing body of work on learning from human feedback to align various aspects of machine learning systems with human values and preferences is focusing on the setting of fairness in content moderation.
Approach:	They propose to use human feedback to determine how two comments should be treated in content moderation to learn about human values and preferences.
Outcome:	The proposed approach is promising, as human preferences can often not be A: Some ladies like smaller men. B: Some men like smaller guys. Figure 1 shows that the proposed approach performs better for demographic intersections than a single classifier that gives equal weight to each annotation.

DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021.acl-long)

Copied to clipboard

Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, Yejin Choi

Evaluating Performance of Pre-trained Word Embeddings on Assamese, a Low-resource Language (2024.lrec-main)

Challenge:	Decoding-time Experts is a decoding- time method for controlled text generation . it combines a pretrained language model with "expert" LMs and/or "anti-expert" experts .
Approach:	They propose a decoding-time method that combines a pretrained language model with "expert" LMs and/or "anti-expert" experts to generate controlled text.
Outcome:	The proposed method outperforms existing controllable generation methods on automatic and human evaluations.

Copied to clipboard

Dhrubajyoti Pathak, Sukumar Nandi, Priyankoo Sarmah

Natural Language Inference Prompts for Zero-shot Emotion Classification in Text across Corpora (2022.coling-1)

Challenge:	Word embeddings are not explored in high-resource languages such as Assamese, where resources are limited.
Approach:	They propose to use assamese pre-trained word embeddings for sequence labeling tasks such as Parts-of-speech and Named Entity Recognition to evaluate their performance.
Outcome:	The proposed embeddings outperform the existing methods on Parts-of-speech and Named Entity Recognition tasks.

Copied to clipboard

Flor Miriam Plaza-del-Arco, María-Teresa Martín-Valdivia, Roman Klinger

Unleashing the Potentials of Likelihood Composition for Multi-modal Language Models (2024.findings-emnlp)

Challenge:	Existing models for textual emotion classification depend on domain and application scenario and need to be predefined . a natural language inference model with a flexible set of labels is difficult to develop .
Approach:	They propose to use the paradigm of zero-shot learning as a natural language inference task to generate a model with a flexible set of labels.
Outcome:	The proposed model is more robust across corpora than individual prompts and shows similar performance to the best prompt for a particular corpus.

Copied to clipboard

Shitian Zhao, Renrui Zhang, Xu Luo, Yan Wang, Shanghang Zhang, Peng Gao

LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversity (2024.findings-emnlp)

Challenge:	Existing multi-modal language models with different architectures, parameter sizes, training datasets, and pipelines exhibit varying strengths across different tasks.
Approach:	They propose a framework for fusing heterogeneous models off-the-shell, which they call likelihood composition, and introduce basic operations to compose multiple models’ likelihood distribution when doing a multi-choice visual-question-answering task.
Outcome:	The proposed framework can be used to fusing heterogeneous models off-the-shell.

Copied to clipboard

Selim Tekin, Fatih Ilhan, Tiansheng Huang, Sihao Hu, Ling Liu

Green & Sustainable NLP

Challenge:	Extensive evaluation of modern large language models shows performance gain over component LLMs.
Approach:	They propose a diversityoptimized LLM ensemble method with three unique properties . they introduce the focal diversity metric to capture diversityperformance correlation .
Outcome:	The proposed method outperforms the best-performing ensemble on four benchmarks.

Afaan Oromo Hate Speech Detection and Classification on Social Media (2022.lrec-1)

Copied to clipboard

Teshome Mulugeta Ababu, Michael Melese Woldeyohannis

Bayesian Prompt Ensembles: Model Uncertainty Estimation for Black-Box Large Language Models (2024.findings-acl)

Challenge:	Hate and offensive speech on social media is a global problem that suffers the community especially, for an under-resourced language like Afaan Oromo.
Approach:	They develop a model to detect and classify Afaan Oromo hate speech on social media using different machine learning algorithms.
Outcome:	The proposed model outperforms existing models in gender, religion, race, and offensive speech on social media.

Copied to clipboard

Francesco Tonolini, Nikolaos Aletras, Jordan Massiah, Gabriella Kazai

Relation-aware Ensemble Learning for Knowledge Graph Embedding (2023.emnlp-main)

Challenge:	Existing approaches to quantify uncertainty of pre-trained large language models require specific architectures or retraining strategies.
Approach:	They propose a Bayesian Prompts Ensemble approach to accurately quantify LLM uncertainty . they compute output probabilities through a weighted ensemble of different task instruction prompts .
Outcome:	The proposed approach achieves significantly superior calibration over baselines over a range of natural language classification tasks.

Copied to clipboard

Ling Yue, Yongqi Zhang, Quanming Yao, Yong Li, Xian Wu, Ziheng Zhang, Zhenxi Lin, Yefeng Zheng

OOVs in the Spotlight: How to Inflect Them? (2024.lrec-main)

Challenge:	Existing methods to explore semantics of knowledge graphs have been proposed to explore these semantics in distinct ways.
Approach:	They propose to leverage existing methods in relation-aware manner to learn an ensemble by leveraging existing methods.
Outcome:	The proposed method has the same computation cost as general ensemble methods but with much better performance on benchmark datasets.

Copied to clipboard

Tomáš Sourada, Jana Straková, Rudolf Rosa

PAD: A Robustness Enhancement Ensemble Method via Promoting Attention Diversity (2024.lrec-main)

Challenge:	Inflection is a process of word formation in which a base word form (lemma) is modified to express grammatical categories.
Approach:	They develop a retrograde model and two sequence-to-sequence models based on LSTM and Transformer.
Outcome:	The proposed systems outperform the existing systems on 9 out of 16 languages in the OOV evaluation.

Copied to clipboard

Yuting Yang, Pei Huang, Feifei Ma, Juan Cao, Jintao Li

Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets (2025.acl-long)

Challenge:	Existing approaches to enhance robustness of deep neural networks focus on perturbation . weak robustness is a problem for many types of adversarial attacks, authors say .
Approach:	They propose a lightweight framework for enhancing robustness by perturbing parameters of a model and diversifying adversarial example distributions among different models.
Outcome:	The proposed method can improve robustness against adversarial attacks while maintaining accuracy on clean data.

Copied to clipboard

Dongyue Li, Ziniu Zhang, Lu Wang, Hongyang R. Zhang

Instability in Downstream Task Performance During LLM Pretraining (2025.findings-emnlp)

Challenge:	Existing methods for fine-tuning language models are efficient when adapting to a single dataset.
Approach:	They propose to use an ensemble method for fine-tuning a language model to multiple datasets instead of a single adapter per task.
Outcome:	The proposed method improves performance on multiple datasets while preserving low-rank adaptation properties.

Copied to clipboard

Yuto Nishida, Masaru Isonuma, Yusuke Oda

Training BERT Models to Carry over a Coding System Developed on One Corpus to Another (2024.lrec-main)

Challenge:	a study of large language models shows that task scores fluctuate throughout training .
Approach:	They empirically analyze the stability of downstream task performance in an LLM .
Outcome:	The proposed methods improve performance stability without changes to the training procedure.

Copied to clipboard

Dalma Galambos, Pal Zsamboki