NQ

ARES: A Reading Comprehension Ensembling Service (2020.emnlp-demos)

Challenge:	Existing studies to solve QA tasks in an integrated manner are not available in other languages because of the lack of QA datasets.
Approach:	They build a Japanese version of Natural Questions using natural questions from query logs of a search engine and crowdsource it using crowdsourcing.
Outcome:	The proposed datasets are based on natural questions from Japanese search engines and crowdsourced.

Copied to clipboard

Anthony Ferritto, Lin Pan, Rishav Chakravarti, Salim Roukos, Radu Florian, J. William Murdock, Avi Sil

Towards building a Robust Industry-scale Question Answering System (2020.coling-industry)

Challenge:	ARES is a machine reading comprehension (MRC) demonstration system which utilizes an ensemble of models to increase F1 by 2.3 points.
Approach:	They propose a machine reading comprehension (MRC) demonstration system which utilizes an ensemble of models to increase F1 by 2.3 points.
Outcome:	The proposed system increases F1 by 2.3 points on a short answer task using an ensemble of models.

Copied to clipboard

Rishav Chakravarti, Anthony Ferritto, Bhavani Iyer, Lin Pan, Radu Florian, Salim Roukos, Avi Sil

Responsible & Trustworthy NLP

Challenge:	Existing systems that use “zero-shot transfer learning” (ZSTL) are difficult to train and have observation biases.
Approach:	They propose a production model called GAAMA which has two characteristics . it is robust and efficient, and trains on the recently introduced Natural Questions dataset .
Outcome:	The proposed model performs on two benchmarks: BioASQ and CovidQA.

Multi-Task Dense Retrieval via Model Uncertainty Fusion for Open-Domain Question Answering (2021.findings-emnlp)

Copied to clipboard

Minghan Li, Ming Li, Kun Xiong, Jimmy Lin

Low-Resource NLP

Challenge:	Existing approaches to multitask dense retrieval are not effective due to corpus inconsistency.
Approach:	They propose to train individual dense passage retrievers for different open-domain question-answering tasks and aggregate their predictions during test time.
Outcome:	The proposed method achieves state-of-the-art performance on 5 benchmark QA datasets, with up to 10% improvement in top-100 accuracy compared to a joint-training multi-task DPR on SQuAD.

Revisiting Evaluation of Question Answering Systems in Low-Resource Indic Languages: Bridging Human and Metric Alignment (2026.acl-short)

Copied to clipboard

Anuj Kumar, Satyadev Ahlawat, Yamuna Prasad, Virendra Singh

Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation (2025.emnlp-main)

Challenge:	Evaluating Question Answering systems in low-resource Indic languages remains challenging due to the scarcity of annotated data and the lack of reliable evaluation metrics.
Approach:	They propose a language-based multi-aspect evaluation framework for question answering systems . the framework integrates semantic similarity, factual completeness, numerical accuracy and contextual relevance .
Outcome:	The proposed metric is evaluated across eight Indic-language QA tasks using multiple LLMs . Across all settings, it shows stronger agreement with human evaluation .

Copied to clipboard

Hengran Zhang, Minghao Tang, Keping Bi, Jiafeng Guo, Shihao Liu, Daiting Shi, Dawei Yin, Xueqi Cheng

Chain-of-Skills: A Configurable Model for Open-Domain Question Answering (2023.acl-long)

Challenge:	Existing studies on large language models for document utility annotations have shown that they improve retrieval performance and RAG outcomes compared to models trained on human annotations.
Approach:	They propose a model that maximizes their summed marginal likelihood to annotate document utility on multiple positive samples per query.
Outcome:	The proposed model maximizes the marginal likelihood of multiple positive samples per query.

Copied to clipboard

Kaixin Ma, Hao Cheng, Yu Zhang, Xiaodong Liu, Eric Nyberg, Jianfeng Gao

Purging the Gray Zone: Latent-Geometric Denoising for Precise Knowledge Boundary Awareness (2026.findings-acl)

Challenge:	Using customized retrieval models, model transferability and scalability are limited.
Approach:	They propose a modular retrieval model where individual modules correspond to key skills that can be reused across datasets.
Outcome:	The proposed model outperforms self-supervised retrievers in zero-shot evaluations and achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQA and OTT-QA.

Copied to clipboard

Hao An, Yibin Lou, Jiayi Guo, Yang Xu

VAULT: VAriable Unified Long Text Representation for Machine Reading Comprehension (2021.acl-short)

Challenge:	Existing abstention fine-tuning methods cause models to suffer from label noise near the decision boundaries.
Approach:	They propose a latent space representation perspective for abstention fine-tuning . they propose 'geometric denoising' framework that constructs a truth hyperplane .
Outcome:	The proposed framework significantly enhances model truthfulness and demonstrates strong generalization in out-of-distribution scenarios.

Copied to clipboard

Haoyang Wen, Anthony Ferritto, Heng Ji, Radu Florian, Avi Sil

Representation Learning

Challenge:	Existing models on Machine Reading Comprehension (MRC) require complex model architecture for effectively modeling long texts with paragraph representation and classification, making inference computationally inefficient for production use.
Approach:	They propose a novel Gaussian distribution-based paragraph representation for Machine Reading Comprehension (MRC) that is light-weight and parallel-efficient.
Outcome:	The proposed model can achieve comparable performance on Wikipedia-based (NQ) and TechNotes (TechQA) with a state-of-the-art (SOTA) complex document modeling approach while being 16 times faster, demonstrating the efficiency of the proposed model.

Exploiting Abstract Meaning Representation for Open-Domain Question Answering (2023.findings-acl)

Copied to clipboard

Cunxiang Wang, Zhikun Xu, Qipeng Guo, Xiangkun Hu, Xuefeng Bai, Zheng Zhang, Yue Zhang

Semantic Text Processing

Challenge:	Existing work attempts to address these challenges using Pretrained Language Models (PLMs) but the diversity of surface form expressions can hinder the model’s ability to capture accurate correlations, especially when the context is lengthy and complex.
Approach:	They propose a method known as Graph-as-Token (GST) to incorporate AMRs into PLMs to assist the model in understanding complex semantic information.
Outcome:	The proposed method outperforms existing methods and significantly improves performance on both Natural Questions and TriviaQA.

Multi-Granularity Guided Fusion-in-Decoder (2024.findings-naacl)

Copied to clipboard

Eunseong Choi, Hyeri Lee, Jongwuk Lee

RFiD: Towards Rational Fusion-in-Decoder for Open-Domain Question Answering (2023.findings-acl)

Challenge:	Open-domain question answering requires deriving factual responses without explicit evidence . recent approaches combine retrieval of relevant information with response generation .
Approach:	They propose a model that concatenates multiple contexts in the decoding phase . they propose MGFiD, which harmonizes passage re-ranking with sentence classification .
Outcome:	The proposed model outperforms existing models on Natural Questions and TriviaQA datasets . it aggregates evident sentences into an anchor vector that instructs the decoder .

Copied to clipboard

Cunxiang Wang, Haofei Yu, Yue Zhang

ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval (2025.findings-naacl)

Challenge:	Open-domain Question Answering (ODQA) systems rely on spurious features instead of genuine causal relationships to generate answers.
Approach:	They propose a model that leverages the encoders of FiD to distinguish between causal relationships and spurious features and guides the decoder to generate answers informed by this discernment.
Outcome:	The proposed model improves on two ODQA datasets and shows that it can identify causal relationships and identify spurious features.

Copied to clipboard

Abdelrahman Abdallah, Jamshid Mozafari, Bhawna Piryani, Adam Jatowt

ARL2: Aligning Retrievers with Black-box Large Language Models via Self-guided Adaptive Relevance Labeling (2024.acl-long)

Challenge:	Retrieval-Augmented Generation models fail to rank the most relevant documents at the top . conventional retrieval methods fail to find the most important documents .
Approach:	They propose a new method for scoring retrieved documents using zero-shot answer scent based on a pre-trained large language model to compute the likelihood of document-derived answers aligning with the answer scent.
Outcome:	The proposed method improves top-1 retrieval accuracy on NQ, TriviaQA, WebQA, ArchivalQA, HotpotQA, and Entity Questions.

Copied to clipboard

LingXi Zhang, Yue Yu, Kuan Wang, Chao Zhang

Semantic Text Processing

Challenge:	Existing retrievers are misaligned with large language models due to separate training processes and inherent black-box nature of LLMs.
Approach:	They propose a retriever learning technique that harnesses LLMs as labelers to annotate and score adaptive relevance evidence.
Outcome:	Extensive experiments show that ARL2 improves accuracy and reduces the cost of API calls.

Corpus Poisoning via Approximate Greedy Gradient Descent (2025.findings-acl)

Copied to clipboard

Jinyan Su, Preslav Nakov, Claire Cardie

Robustness in NLP

Challenge:	Dense retrievers have been shown to be vulnerable to corpus poisoning attacks . a malicious user injects a small fraction of adversarial passages into the retrieval corpus to trick the system into returning these passages among the top-ranked results for a broad set of user queries.
Approach:	They propose an attack on dense retrieval systems based on the widely used HotFlip method for efficiently generating adversarial passages.
Outcome:	The proposed method achieves a high success rate on several datasets and using several retrievers, and can generalize to unseen queries and new domains.

Allies: Prompting Large Language Model with Beam Search (2023.findings-emnlp)

Copied to clipboard

Hao Sun, Xiao Liu, Yeyun Gong, Yan Zhang, Daxin Jiang, Linjun Yang, Nan Duan

DVD: Dynamic Contrastive Decoding for Knowledge Amplification in Multi-Document Question Answering (2024.emnlp-main)

Challenge:	Existing methods to build LLMs with stacking are limited by their information coverage and low fault tolerance.
Approach:	They propose a method that leverages large language models to iteratively generate new queries from an input query.
Outcome:	The proposed method outperforms baselines on open-domain question answering benchmarks.

Copied to clipboard

Jing Jin, Houfeng Wang, Hao Zhang, Xiaoguang Li, Zhijiang Guo

HPE: Answering Complex Questions over Text by Hybrid Question Parsing and Execution (2023.findings-emnlp)

Challenge:	Large language models (LLMs) generate information with hallucinations due to uneven retrieval quality and irrelevant contents.
Approach:	They propose a decoding strategy which dynamically amplifies knowledge from selected documents during the generation phase.
Outcome:	The proposed method outperforms other decoding strategies on ALCE-ASQA, NQ, TQA and PopQA benchmarks.

Copied to clipboard

Ye Liu, Semih Yavuz, Rui Meng, Dragomir Radev, Caiming Xiong, Shafiq Joty, Yingbo Zhou

An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks (2022.emnlp-main)

Challenge:	End-to-end neural networks excel at answering natural language questions but fail on complex ones . a proposed framework for question parsing and execution on textual QA is designed to combine the strengths of neural and symbolic methods.
Approach:	They propose a framework for question parsing and execution on textual QA . they parse questions into an intermediate representation and use deterministic rules to translate them .
Outcome:	The proposed framework outperforms existing methods in supervised, few-shot, and zero-shot settings while preserving its underlying reasoning process.

Copied to clipboard

Yuxiang Wu, Yu Zhao, Baotian Hu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension (2020.findings-emnlp)

Challenge:	Existing methods rely on parametric models that store knowledge in parameters or retrieval-augmented models that have access to external knowledge sources.
Approach:	They propose a parametric parametric model that stores knowledge in its parameters or a retrieval-augmented model that has access to external knowledge sources.
Outcome:	The proposed method runs substantially faster across the board and produces more accurate results on WoW and ELI5.

Copied to clipboard

Xuguang Wang, Linjun Shou, Ming Gong, Nan Duan, Daxin Jiang

Reasoning

Challenge:	Natural Questions (NQ) benchmark sets new challenges for machine reading comprehension.
Approach:	They propose a novel approach to handle all answer types systematically using a two-step training procedure.
Outcome:	The proposed approach achieved the top 1 on both long and short answer leaderboards with F1 scores of 77.2 and 64.1.

Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge (2022.findings-emnlp)

Copied to clipboard

Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao

Reasoning

Challenge:	Existing open-domain question answering methods rely on the retriever to gather all evidence in isolation, but our approach uses an intermediary module to perform a chain of reasoning over the retrieved set.
Approach:	They propose a new open-domain question answering framework that integrates an intermediary module into the current retriever-reader pipeline and integrates it into the model.
Outcome:	The proposed framework outperforms the state-of-the-art on two OTT-QA datasets with an exact match score of 47.3 (45% relative gain).

Distilling Large Embeddings via Hyperspherical Householder Quantization (2026.acl-long)

Copied to clipboard

Yihang Wang, Bin Wu, Yueyang Su, Tianfu Zhang, Yiqi Du, Lei Yu, Jiafeng Guo, Xueqi Cheng

Semantic Text Processing

Challenge:	Existing methods for quantizing large embeddings rely on Euclidean quantization, which is poorly aligned with the angular geometry induced by contrastive embeddment training.
Approach:	They propose a geometry-aware distillation method that compresses large embeddings into short discrete representations via iterative Householder transformations on the unit hypersphere.
Outcome:	The proposed method reduces decoding cost and maintains strong semantic retrieval accuracy.

Question Difficulty Estimation for Large Language Models via Answer Plausibility Scoring (2026.acl-long)

Copied to clipboard

Jamshid Mozafari, Bhawna Piryani, Adam Jatowt

Robustness in NLP

Challenge:	Existing approaches to estimate question difficulty rely on readability formulas, retrieval-based signals, or popularity statistics.
Approach:	They propose a method that estimates question difficulty by computing the entropy of plausibility scores over candidate answers.
Outcome:	The proposed method outperforms baselines across four QA datasets and shows strong robustness across hyperparameter variations and question types.

RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020.acl-main)

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Nan Duan

Synthetic QA Corpora Generation with Roundtrip Consistency (P19-1)

Challenge:	Using Wikipedia pages to answer open-domain questions remains challenging in natural language understanding.
Approach:	They propose a model which reads Wikipedia pages for natural question answering . it uses a dynamic paragraph dual-attention reader and a cascaded answer predictor .
Outcome:	The proposed model outperforms the human model on the Natural Questions dataset . it achieves 74.3 F1 and 57.9 F1 on long-answer and short-answer tasks .

Copied to clipboard

Chris Alberti, Daniel Andor, Emily Pitler, Jacob Devlin, Michael Collins

KazQAD: Kazakh Open-Domain Question Answering Dataset (2024.lrec-main)

Challenge:	Existing methods for generating synthetic question answering corpora are not suitable for QA, but can be constructed from widely available natural text.
Approach:	They propose a method for generating synthetic question answering corpora by combining question generation and answer extraction models and filtering the results to ensure roundtrip consistency.
Outcome:	The proposed model achieves exact match and F1 at less than 0.1% and 0.4% from human performance on SQuAD2 and NQ.

Copied to clipboard

Rustem Yeshpanov, Pavel Efimov, Leonid Boytsov, Ardak Shalkarbayuli, Pavel Braslavski

Multilinguality

Challenge:	KazQAD contains just under 6,000 unique questions with extracted short answers and nearly 12,000 passage-level relevance judgements.
Approach:	They introduce a Kazakh open-domain question answering dataset that can be used in reading comprehension and full ODQA settings.
Outcome:	The proposed dataset can be used in reading comprehension and full ODQA settings, as well as for information retrieval experiments.

You Make me Feel like a Natural Question: Training QA Systems on Transformed Trivia Questions (2024.emnlp-main)

Copied to clipboard

Tasnim Kabir, Yoo Yeon Sung, Saptarashmi Bandyopadhyay, Hao Zou, Abhranil Chandra, Jordan Boyd-Graber

Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception (2025.acl-long)

Challenge:	Existing question-answering datasets are expensive and difficult to annotate and time-consuming to gather.
Approach:	They propose to transform Manchester questions into web queries using the same question datasets.
Outcome:	The proposed questions can be trained on a Manchester QA dataset using the Quiz Bowl (QB) sample.

Copied to clipboard

Shiyu Ni, Keping Bi, Jiafeng Guo, Lulu Yu, Baolong Bi, Xueqi Cheng

Responsible & Trustworthy NLP

Challenge:	Large language models (LLMs) exhibit impressive performance across diverse tasks but struggle to accurately gauge their knowledge boundaries.
Approach:	They propose Consistency-based Confidence Calibration (C3) which assesses confidence consistency through question reformulation to improve LLMs’ ability to recognize their knowledge gaps.
Outcome:	The proposed method improves the unknown perception rate by 5.6% on NQ and 4.9% on HotpotQA.

LLMs are Biased Evaluators But Not Biased for Fact-Centric Retrieval Augmented Generation (2025.findings-acl)

Copied to clipboard

Yen-Shan Chen, Jing Jin, Peng-Ting Kuo, Chao-Wei Huang, Yun-Nung Chen

HASH-RAG: Bridging Deep Hashing with Retriever for Efficient, Fine Retrieval and Augmented Generation (2025.findings-acl)

Challenge:	Recent studies have shown that large language models (LLMs) exhibit significant biases in evaluation tasks, especially in preferentially rating and favoring self-generated content.
Approach:	They propose to simulate two critical phases of retrieval-augmented generation (RAG) frameworks where keyword extraction and factual accuracy take precedence over stylistic elements.
Outcome:	The proposed model emulates two critical phases of the retrieval-augmented generation framework.

Copied to clipboard

Jinyu Guo, Xunlei Chen, Qiyang Xia, Zhaokun Wang, Jie Ou, Libo Qin, Shunyu Yao, Wenhong Tian

Responsible & Trustworthy NLP

Challenge:	Experimental evaluations on NQ, TriviaQA, and HotpotQA datasets demonstrate that our approach achieves a 90% reduction in retrieval time compared to conventional methods while maintaining considerate recall performance.
Approach:	They propose a framework that integrates deep hashing techniques with systematic optimizations to address these limitations.
Outcome:	The proposed framework outperforms retrieval/non-retrieval baselines by 1.4-4.3% in EM scores on NQ, TriviaQA, and HotpotQA datasets.

An Iterative Utility Judgment Framework Inspired by Philosophical Relevance via LLMs (2026.findings-acl)

Copied to clipboard

Hengran Zhang, Keping Bi, Jiafeng Guo, Xueqi Cheng

SURE or Not? Investigating Semantic Understanding in Dense Retrieval Models (2026.acl-long)

Challenge:	Relevance emphasizes the aboutness of a result to a query, while utility refers to the result’s usefulness or value to an information seeker.
Approach:	They propose an Iterative utiliTy judgmEnt fraMework to promote each step in Retrieval-Augmented Generation (RAG) they propose to use relevance ranking, utility judgments, and answer generation to prioritize high-utility results over low-utilitity results.
Outcome:	The proposed framework improves relevance, ranking, and answer generation on retrieval (TREC DL, WebAP), utility judgment task (GTI-NQ), and factoid question-answering (NQ) datasets.

Copied to clipboard

Lingdi Kong, Xuanang Chen, Ben He, Le Sun