MRC

Have my arguments been replied to? Argument Pair Extraction as Machine Reading Comprehension (2022.acl-short)

Challenge:	Experimental results show that unified model outperforms other models that treat encoding and matching separately.
Approach:	They evaluate a unified model with Transformer layers for machine reading comprehension . they find that the model learns different modeling strategies compared with previous models .
Outcome:	The unified model outperforms models with Transformer layers on the machine reading comprehension task.

Copied to clipboard

Jianzhu Bao, Jingyi Sun, Qinglin Zhu, Ruifeng Xu

Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data (2021.findings-emnlp)

Challenge:	Existing studies identify argument pairs indirectly by predicting sentence-level relations between two documents, neglecting the holistic argument-level interactions.
Approach:	They propose to use machine reading comprehension to extract argument pairs from two documents . they propose to employ an AM query to identify all arguments in two documents, then an APE query to extract its paired arguments from another document.
Outcome:	The proposed method outperforms the state-of-the-art method by 7.11% in F1 score.

Copied to clipboard

Dian Yu, Kai Sun, Dong Yu, Claire Cardie

Improving the Robustness of Deep Reading Comprehension Models by Leveraging Syntax Prior (D19-58)

Challenge:	despite considerable progress, most machine reading comprehension tasks lack sufficient training data to fully exploit powerful deep neural network models.
Approach:	They propose to use QA data to generate more training data for machine reading comprehension tasks by crowdsourcing . they first collect a large-scale multiple-choice QA dataset for Chinese, ExamQA, and then use incomplete, yet relevant snippets returned by a web search engine as the context for each QA instance.
Outcome:	The proposed model improves a Chinese MRC task with +5.1% accuracy and +3.8% exact match.

Copied to clipboard

Bowen Wu, Haoyang Huang, Zongsheng Wang, Qihang Feng, Jingsong Yu, Baoxun Wang

A Multilingual Reading Comprehension System for more than 100 Languages (2020.coling-demos)

Challenge:	Recent studies indicate that the current machine reading comprehension systems suffer from weak robustness against adversarial samples.
Approach:	They propose to take sentence syntax as the leverage in the answer predicting process and exploit the syntactic elements of a question to improve the generalization and robustness of MRC models.
Outcome:	The proposed method improves generalization and robustness against adversarial samples, with performance well-maintained.

Copied to clipboard

Anthony Ferritto, Sara Rosenthal, Mihaela Bornea, Kazi Hasan, Rishav Chakravarti, Salim Roukos, Radu Florian, Avi Sil

Cross-Task Knowledge Transfer for Query-Based Text Summarization (D19-58)

Challenge:	Recent advances in open domain question answering (QA) have focused on machine reading comprehension (MRC)
Approach:	They propose a multilingual machine reading comprehension (MRC) demo which can answer questions in over 100 languages.
Outcome:	The proposed system can answer questions in over 100 languages and integrates with IBM Watson's machine translation widget to improve language accessibility.

Copied to clipboard

Elozino Egonmwan, Vittorio Castelli, Md Arafat Sultan

Text Generation

Challenge:	Existing methods for summarization data corpora are limited to extractive and abstractive summarizing.
Approach:	They propose to use machine reading comprehension (MRC) and query-based text summarization to produce extractive and abstractive summaries from pre-trained MRC and MT models.
Outcome:	The proposed model outperforms existing methods on CNN/Daily Mail and Debatepedia datasets and can be used as a baseline for future systems.

REPT: Bridging Language Models and Machine Reading Comprehension via Retrieval-Based Pre-training (2021.findings-acl)

Copied to clipboard

Fangkai Jiao, Yangyang Guo, Yilin Niu, Feng Ji, Feng-Lin Li, Liqiang Nie

MRC Examples Answerable by BERT without a Question Are Less Effective in MRC Model Training (2020.aacl-srw)

Challenge:	Pre-trained language models have achieved great success on Machine Reading Comprehension (MRC) however, the poor support in evidence extraction hinders them from further advancing MRC.
Approach:	They propose a REtrieval-based pre-training approach that strengthens evidence extraction during pre-training by inherited downstream MRC tasks.
Outcome:	The proposed approach strengthens evidence extraction during pre-training, which is further inherited by downstream tasks.

Copied to clipboard

Hongyu Li, Tengyang Chen, Shuting Bai, Takehito Utsuro, Yasuhide Kawada

NER-MQMRC: Formulating Named Entity Recognition as Multi Question Machine Reading Comprehension (2022.naacl-industry)

Challenge:	Existing models for Machine Reading Comprehension (MRC) are unable to predict answers from a question and its related context.
Approach:	They propose a method that splits the training examples into those that are “easy to answer” or “hard to answer”.
Outcome:	The proposed model outperforms the previous models on a large-scale English MRC dataset.

Copied to clipboard

Anubhav Shrimal, Avi Jain, Kartik Mehta, Promod Yenigalla

D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension (D19-58)

Challenge:	Named Entity Recognition (NER) is a task of locating and classifying entities mentioned in unstructured text into predefined categories.
Approach:	They propose to use a BERT-based multi-question MRC task where multiple questions (one question per entity) are considered at the same time for a single text.
Outcome:	The proposed architecture leads to 2.5 times faster training and 2.3 times faster inference on three NER datasets.

Copied to clipboard

Hongyu Li, Xiyuan Zhang, Yibing Liu, Yiming Zhang, Quan Wang, Xiangyang Zhou, Jing Liu, Hua Wu, Haifeng Wang

Adversarial Training for Machine Reading Comprehension with Virtual Embeddings (2021.starsem-1)

Challenge:	MRC requires machines to understand text and answer questions about the text.
Approach:	They propose a simple system Baidu submitted for MRQA 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models.
Outcome:	The proposed system is ranked at top 1 of all participants in terms of averaged F1 score.

Copied to clipboard

Ziqing Yang, Yiming Cui, Chenglei Si, Wanxiang Che, Ting Liu, Shijin Wang, Guoping Hu

SQL Generation via Machine Reading Comprehension (2020.coling-main)

Challenge:	Neural networks are vulnerable to adversarial examples that have been mixed with certain perturbations.
Approach:	They propose a novel adversarial training method that perturbs the embedding matrix instead of word vectors to differentiate the roles of passages and questions.
Outcome:	The proposed method is effective universally and further improves the performance of MRC tasks.

Copied to clipboard

Zeyu Yan, Jianqiang Ma, Yang Zhang, Jianping Shen

Programming Languages in NLP

Challenge:	Text-to-SQL systems can generate SQL queries given natural language questions.
Approach:	They propose a method that formulates a question answering problem as a query answering problem where different slots are predicted by a unified machine reading comprehension (MRC) model.
Outcome:	The proposed method can achieve competitive results on WikiSQL, suggesting it being a promising direction for text-to-SQl.

On the Robustness of Reading Comprehension Models to Entity Renaming (2022.naacl-main)

Copied to clipboard

Jun Yan, Yang Xiao, Sagnik Mukherjee, Bill Yuchen Lin, Robin Jia, Xiang Ren

Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models (2021.acl-short)

Challenge:	SpanBERT model is more robust than RoBERTa, despite having similar accuracy on unperturbed test data.
Approach:	They propose a pipeline to replace entity names with names from a variety of sources.
Outcome:	The proposed model performs worse when entities are renamed, the authors show . SpanBERT, which is pretrained with span-level masking, is more robust than RoBERTa .

Copied to clipboard

Jieyu Lin, Jiajie Zou, Nai Ding

Benchmarking Robustness of Machine Reading Comprehension Models (2021.findings-acl)

Challenge:	Pre-trained language models have achieved human-level performance on many Machine Reading Comprehension (MRC) tasks, but it remains unclear whether these models truly understand language or answer questions by exploiting statistical biases in datasets.
Approach:	They propose a method to attack MRC models by exposing statistical biases in a RACE dataset and propose an augmented training method that can greatly reduce models’ statistical bias.
Outcome:	The proposed method can reduce models’ statistical biases from human-level performance to chance-level.

Copied to clipboard

Chenglei Si, Ziqing Yang, Yiming Cui, Wentao Ma, Ting Liu, Shijin Wang

Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension (2022.acl-long)

Challenge:	Existing benchmarks only evaluate models' robustness under test-time perturbations or adversarial attacks.
Approach:	They propose a model-agnostic benchmark to evaluate models' robustness under adversarial attacks.
Outcome:	The proposed model-agnostic benchmark evaluates models under four different types of adversarial attacks.

Copied to clipboard

Linjuan Wu, Shaojuan Wu, Xiaowang Zhang, Deyi Xiong, Shizhan Chen, Zhiqiang Zhuang, Zhiyong Feng

LogiGraph: Logical Reasoning with Contrastive Learning and Lightweight Graph Networks (2025.coling-main)

Challenge:	Existing methods to zero-shot transfer knowledge from rich-resource to low-resourced languages are limited due to linguistic discrepancies in different languages.
Approach:	They propose a multilingual MRC framework equipped with a Siamese Semantic Disentanglement Model to disassociate semantics from syntax in models learned by multilingual pre-trained models.
Outcome:	The proposed model disassociates semantics from syntax in multilingual models.

Copied to clipboard

Xiang Li, Chen Shi, Yong Xu, Jun Huang

Hypothetical Training for Robust Machine Reading Comprehension of Tabular Context (2023.findings-acl)

Challenge:	Existing methods emphasize contextual semantics while others pay more attention to explicit logical features. Existing models utilize graph convolutional networks (GCN) for node updates, still exhibiting some shortcomings.
Approach:	They propose a logical reasoning method with contrastive learning and lightweight graph networks (LogiGraph) they employ conjunction and punctuation marks as two types of edges to construct a dual graph.
Outcome:	The proposed method improves the GCN and employs conjunction and punctuation marks as two types of edges to construct a dual graph.

Copied to clipboard

Moxin Li, Wenjie Wang, Fuli Feng, Hanwang Zhang, Qifan Wang, Tat-Seng Chua

VlogQA: Task, Dataset, and Baseline Models for Vietnamese Spoken-Based Machine Reading Comprehension (2024.eacl-long)

Challenge:	Counterfactual training is expensive because of the complexity of tabular data.
Approach:	They propose a hypothetical training framework that uses paired examples with different hypothetical questions to supervise the direction of model gradient towards the counterfactual answer change.
Outcome:	The proposed framework improves on tabular MRC datasets.

Copied to clipboard

Thinh Ngo, Khoa Dang, Son Luu, Kiet Nguyen, Ngan Nguyen

Answer Uncertainty and Unanswerability in Multiple-Choice Machine Reading Comprehension (2022.findings-acl)

Challenge:	Existing datasets for machine reading comprehension tasks in Vietnamese focus on written documents, such as Wikipedia articles, online newspapers, or textbooks.
Approach:	They propose to capture Vietnamese spoken language in natural settings and use it to create a machine-learning corpus for machine reading comprehension tasks.
Outcome:	The proposed corpus consists of 10,076 question-answer pairs based on 1,230 transcript documents sourced from YouTube .

Copied to clipboard

Vatsal Raina, Mark Gales

A Frame-based Sentence Representation for Machine Reading Comprehension (2020.acl-main)

Challenge:	Machine reading comprehension (MRC) systems focus on selecting the correct answer to a question given a context paragraph.
Approach:	They propose to use machine reading comprehension (MRC) to assess the ability of systems to understand natural language.
Outcome:	The proposed system outperforms a system built with an NOA option . the results show that the system is not confident about the NOA choice .

Copied to clipboard

Shaoru Guo, Ru Li, Hongye Tan, Xiaoli Li, Yong Guan, Hongyan Zhao, Yueping Zhang

Representation Learning

Challenge:	Existing machine learning approaches do not have above semantic knowledge to address complicated MRC questions.
Approach:	They propose a frame-based Sentence Representation method which integrates frame semantic knowledge to facilitate sentence modelling.
Outcome:	The proposed method performs better than state-of-the-art methods on machine reading comprehension task.

Why Machine Reading Comprehension Models Learn Shortcuts? (2021.findings-acl)

Copied to clipboard

Yuxuan Lai, Chen Zhang, Yansong Feng, Quzhe Huang, Dongyan Zhao

Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension (2020.acl-main)

Challenge:	Existing studies show that many MRC models learn shortcuts to outwit benchmarks, but the performance is unsatisfactory in real-world applications.
Approach:	They propose to use shortcut questions to analyze learning difficulty of MRC models . they propose to analyze the learning difficulty regarding shortcut and challenging questions .
Outcome:	The proposed methods show that a large proportion of shortcut questions in training data make models rely on shortcut tricks excessively.

Copied to clipboard

Fei Yuan, Linjun Shou, Xuanyu Bai, Ming Gong, Yaobo Liang, Nan Duan, Yan Fu, Daxin Jiang

Multilinguality

Challenge:	Existing approaches to improve machine reading comprehension performance on low resource languages are limited due to the lack of sufficient training data.
Approach:	They propose to use a mixed MRC task to translate the question to other languages and build cross-lingual question-passage pairs.
Outcome:	The proposed task improves on two cross-lingual MRC datasets.

Frustratingly Simple Few-Shot Slot Tagging (2021.findings-acl)

Copied to clipboard

Jianqiang Ma, Zeyu Yan, Chang Li, Yang Zhang

RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering (2021.naacl-main)

Challenge:	Existing fewshot methods for slot tagging are weak in encoding slot name semantics and slot dependencies.
Approach:	They propose a simple and effective few-shot model for slot tagging which incorporates machine reading comprehension (MRC) using source domain and target domain data.
Outcome:	The proposed model outperforms state-of-the-art methods on the SNIPS dataset.

Copied to clipboard

Srinivasan Iyer, Sewon Min, Yashar Mehdad, Wen-tau Yih

Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text (2020.emnlp-main)

Challenge:	State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) achieve high recall amongst top few predictions, but low overall accuracy, motivating the need for answer re-ranking.
Approach:	They propose a method to make answer re-ranking successful for span-extraction tasks even beyond large pre-training.
Outcome:	The proposed approach achieves 45.5% Exact Match accuracy on Natural Questions and 61.7% on TriviaQA.

Copied to clipboard

Dongfang Li, Baotian Hu, Qingcai Chen, Weihua Peng, Anqi Wang

GCRC: A New Challenging MRC Dataset from Gaokao Chinese for Explainable Evaluation (2021.findings-acl)

Challenge:	MRC has achieved significant progress on the open domain in recent years due to large-scale pre-trained language models.
Approach:	They propose a machine reading comprehension model which exploits structural medical knowledge and reference medical plain text to improve the exam's accuracy.
Outcome:	The proposed model outperforms existing models with a large margin and passes the exam with 61.8% accuracy rate on the test set.

Copied to clipboard

Hongye Tan, Xiaoyue Wang, Yu Ji, Ru Li, Xiaoli Li, Zhiwei Hu, Yunxiao Zhao, Xiaoqi Han

The Impacts of Unanswerable Questions on the Robustness of Machine Reading Comprehension Models (2023.eacl-main)

Challenge:	Existing machine reading comprehension datasets lack an explainable evaluation of systems' reasoning capabilities.
Approach:	They propose a dataset with multi-choice questions that evaluates MRC systems' reasoning process . they use sentence-level relevant supporting facts, error reason of distractors to evaluate MRC .
Outcome:	The proposed dataset is more challenging and useful for identifying limitations of existing MRC systems in an explainable way.

Copied to clipboard

Son Quoc Tran, Phong Nguyen-Thuan Do, Uyen Le, Matt Kretchmar

Self Question-answering: Aspect-based Sentiment Analysis by Role Flipped Machine Reading Comprehension (2021.findings-emnlp)

Challenge:	Pretrained language models have achieved super-human performances on many Machine Reading Comprehension (MRC) benchmarks.
Approach:	They propose to fine-tune three state-of-the-art language models on SQuAD 1.1 or SQu AD 2.0 and then evaluate their robustness under adversarial attacks.
Outcome:	The proposed model is able to perform better under adversarial attacks than model fine-tuned on SQuAD 1.1 or 2.0.

Copied to clipboard

Guoxin Yu, Jiwei Li, Ling Luo, Yuxian Meng, Xiang Ao, Qing He

OPERA: Operation-Pivoted Discrete Reasoning over Text (2022.naacl-main)

Challenge:	unified Aspect-based Sentiment Analysis (ABSA) aims to couple aspect terms with their corresponding opinion terms, which might make it easier to predict sentiment polarities.
Approach:	They propose a new paradigm to pair aspect terms with their corresponding opinion terms . they propose to use a machine learning paradigm to solve the unified ABSA task .
Outcome:	The proposed framework can solve the ABSA task without any additional data annotation or transformation.

Copied to clipboard

Yongwei Zhou, Junwei Bao, Chaoqun Duan, Haipeng Sun, Jiahui Liang, Yifan Wang, Jing Zhao, Youzheng Wu, Xiaodong He, Tiejun Zhao

DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications (2021.acl-short)

Challenge:	Existing methods to predict logical forms ignore the utilization of symbolic operations and lack reasoning ability and interpretability.
Approach:	They propose an operation-pivoted discrete reasoning framework that uses symbolic operations as neural modules to facilitate reasoning ability and interpretability.
Outcome:	Extensive experiments on DROP and RACENum datasets show the reasoning ability of OPERA.

Copied to clipboard

Hongxuan Tang, Hongyu Li, Jing Liu, Yu Hong, Hua Wu, Haifeng Wang

Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehension (2022.coling-1)

Challenge:	In order to comprehensively verify the robustness and generalization of MRC models, we construct a real-world Chinese dataset - DuReader_robust .
Approach:	They introduce a real-world Chinese dataset to evaluate the robustness and generalization of MRC models from three aspects: over-sensitivity, over-stability and generalisation.
Outcome:	The proposed model fails to perform well on the challenge test set and may provide suggestions for future model development.

Copied to clipboard

Jialin Chen, Zhuosheng Zhang, Hai Zhao

Event Extraction as Machine Reading Comprehension (2020.emnlp-main)

Challenge:	Existing methods of logical reasoning focus on entity-aware information but ignore hierarchical relations that may even have mutual effects.
Approach:	They propose a holistic graph network that deals with context at both discourse-level and word-level as the basis for logical reasoning.
Outcome:	The proposed method improves on logical reasoning QA datasets and natural language inference datasets.

Copied to clipboard

Jian Liu, Yubo Chen, Kang Liu, Wei Bi, Xiaojiang Liu

Entity-Relation Extraction as Multi-Turn Question Answering (P19-1)

Challenge:	Event extraction (EE) is a crucial information extraction task that aims to extract event information in texts.
Approach:	They propose a new learning paradigm for event extraction by explicitly casting it as a machine reading comprehension problem.
Outcome:	The proposed model achieves state-of-the-art performance on the data-scarce scenario, achieving 49.8% in F1 for event argument extraction with only 1% data, compared with 2.2% of the previous method.

Copied to clipboard

Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, Jiwei Li

VAULT: VAriable Unified Long Text Representation for Machine Reading Comprehension (2021.acl-short)

Challenge:	Identifying entities and their relations is the prerequisite of extracting structured knowledge from unstructured raw texts.
Approach:	They propose a new paradigm for the task of entity-relation extraction . they cast the task as a multi-turn question answering problem .
Outcome:	The proposed paradigm significantly outperforms previous best models on the ACE and CoNLL04 datasets.

Copied to clipboard

Haoyang Wen, Anthony Ferritto, Heng Ji, Radu Florian, Avi Sil

Representation Learning

Challenge:	Existing models on Machine Reading Comprehension (MRC) require complex model architecture for effectively modeling long texts with paragraph representation and classification, making inference computationally inefficient for production use.
Approach:	They propose a novel Gaussian distribution-based paragraph representation for Machine Reading Comprehension (MRC) that is light-weight and parallel-efficient.
Outcome:	The proposed model can achieve comparable performance on Wikipedia-based (NQ) and TechNotes (TechQA) with a state-of-the-art (SOTA) complex document modeling approach while being 16 times faster, demonstrating the efficiency of the proposed model.

Learning to Generate Questions by Learning to Recover Answer-containing Sentences (2021.findings-acl)

Copied to clipboard

Seohyun Back, Akhil Kedia, Sai Chetan Chinthakindi, Haejun Lee, Jaegul Choo

A Query-Parallel Machine Reading Comprehension Framework for Low-resource NER (2023.findings-emnlp)

Challenge:	Recent research has focused on synthetically generating a question from a given context and an annotated answer by training an additional generative model.
Approach:	They propose a method that learns to generate contextually rich questions by recovering answer-containing sentences.
Outcome:	The proposed approach improves the quality and accuracy of existing models and achieves comparable results to the state-of-the-art on MS MARCO and NewsQA.

Copied to clipboard

Yuhao Zhang, Yongliang Wang

Benchmarking Machine Reading Comprehension: A Psychological Perspective (2021.eacl-main)

Challenge:	Named entity recognition (NER) is a fundamental task in natural language processing.
Approach:	They propose a query-parallel MRC-based approach to named entity recognition . the model is trained with parameter-efficient tuning technique, making it more data-efficient .
Outcome:	The proposed model performs competitively against strong baseline methods in resource-rich settings and achieves state-of-the-art results in low-resource settings.

Copied to clipboard

Saku Sugawara, Pontus Stenetorp, Akiko Aizawa

EFLLex: A Graded Lexical Resource for Learners of English as a Foreign Language (L18-1)

Challenge:	MRC is a task that tests the ability of a machine to read and understand unstructured text.
Approach:	They propose a theoretical basis for the design of MRC datasets based on psychology and psychometrics and propose shortcut-proof questions and explanations as a part of the task design.
Outcome:	The proposed datasets should evaluate the model's ability to understand context-dependent situations and ensure substantive validity by shortcut-proof questions and explanation as a part of the task design.

Copied to clipboard

Luise Dürlich, Thomas François

Teaching Machine Comprehension with Compositional Explanations (2020.findings-emnlp)

Challenge:	EFLLex describes the use of 15,280 English words in pedagogical materials across proficiency levels.
Approach:	They propose to use a part-of-speech tagger and a robust estimator to compute frequency and do manual post-editing work to improve the resource.
Outcome:	The proposed resource describes the use of 15,280 English words across proficiency levels of the European Framework of Reference for Languages.

Copied to clipboard

Qinyuan Ye, Xiao Huang, Elizabeth Boschee, Xiang Ren

Explainability & Interpretability in NLP

Challenge:	Recent advances in machine reading comprehension rely heavily on large-scale annotated corpora, which are timeconsuming and costly to collect.
Approach:	They propose to use semi-structured explanations to “teach” machines reading comprehension using a small number of semi-structural explanations that explicitly inform machines why answer spans are correct.
Outcome:	The proposed method achieves 70.14% F1 score with supervision from 26 explanations on the SQuAD dataset, comparable to plain supervised learning using 1,100 labeled instances yielding a 12x speed up.

To What Extent Do Natural Language Understanding Datasets Correlate to Logical Reasoning? A Method for Diagnosing Logical Reasoning. (2022.coling-1)

Copied to clipboard

Yitian Li, Jidong Tian, Wenqing Chen, Caoyun Fan, Hao He, Yaohui Jin

End-to-End Chinese Speaker Identification (2022.naacl-main)

Challenge:	Reasoning and knowledge-related skills are considered as fundamental skills for natural language understanding (NLU) tasks.
Approach:	They propose a method to diagnose correlations between an NLU dataset and a specific skill.
Outcome:	The proposed method is able to diagnose correlations between dataset and logical reasoning skill on 8 MRC and 3 NLI datasets.

Copied to clipboard

Dian Yu, Ben Zhou, Dong Yu

Dependency Parsing as MRC-based Span-Span Prediction (2022.acl-long)

Challenge:	Existing methods for speaker identification in texts are incomplete and introduce errors that propagate and seriously affect the final output.
Approach:	They propose to use speaker identification (SI) in texts to identify the speaker(s) for each utterance in texts.
Outcome:	The proposed model can achieve comparable or better than previous state-of-the-art methods on all public SI datasets for Chinese.

Copied to clipboard

Leilei Gan, Yuxian Meng, Kun Kuang, Xiaofei Sun, Chun Fan, Fei Wu, Jiwei Li

Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification (P18-1)

Challenge:	Existing methods for dependency parsing address the issue that edges should be constructed at the text span/subtree level rather than word level.
Approach:	They propose a method that constructs dependency trees by directly modeling span-span relations by modeling subtree-subtree relationships.
Outcome:	The proposed method constructs dependency trees by modeling span-span relations . it can retrieve missing spans in the span proposal stage, which leads to higher recall .

Copied to clipboard

Yizhong Wang, Kai Liu, Jing Liu, Wei He, Yajuan Lyu, Hua Wu, Sujian Li, Haifeng Wang

An MRC Framework for Semantic Role Labeling (2022.coling-1)

Challenge:	Recent years have seen rapid growth in the MRC community . MRC is believed to be a crucial step in building a general intelligent agent .
Approach:	They propose an end-to-end neural model that enables multiple passages to verify each other based on their content representations.
Outcome:	The proposed model outperforms the baseline on the English MS-MARCO dataset and the Chinese DuReader dataset, and achieves state-of-the-art performance on both datasets.

Copied to clipboard

Nan Wang, Jiwei Li, Yuxian Meng, Xiaofei Sun, Han Qiu, Ziyao Wang, Guoyin Wang, Jun He

Semantic Parsing

Challenge:	Existing work on semantic role labels ignores the semantic connection between the two tasks . et al. (2010) defined two types of semantic roles: core roles and non-core roles.
Approach:	They propose to use machine reading comprehension to bridge the gap between these two tasks . they formalize predicate disambiguation as multiple-choice machine reading understanding .
Outcome:	The proposed framework achieves state-of-the-art or comparable results to previous work . it uses the descriptions of candidate senses of a given predicate as options to select the correct sense .

Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources (2021.findings-acl)

Copied to clipboard

Taolin Zhang, Chengyu Wang, Minghui Qiu, Bite Yang, Zerui Cai, Xiaofeng He, Jun Huang

What If Sentence-hood is Hard to Define: A Case Study in Chinese Reading Comprehension (2021.findings-emnlp)

Challenge:	MRC is a popular task in NLP, aiming to understand a passage and answer the relevant questions.
Approach:	They propose a multi-target machine learning task for the medical domain that predicts answers to medical questions and corresponding support sentences from medical information sources simultaneously.
Outcome:	The proposed model outperforms baselines by fusing context-aware and knowledge-awful token representations.

Copied to clipboard

Jiawei Wang, Hai Zhao, Yinggong Zhao, Libin Shen

KECP: Knowledge Enhanced Contrastive Prompting for Few-shot Extractive Question Answering (2022.emnlp-main)

Challenge:	Explicit Span-Sentence Predication solves location unit ambiguity problem in many languages, allowing model to determine which sentence contains the answer span when sentence itself has not been clearly defined at all.
Approach:	They propose a machine-learning reader with Explicit Span-Sentence Predication to solve this problem by analyzing Chinese sentences.
Outcome:	The proposed reader achieves state-of-the-art on Chinese MRC benchmark and shows great potential in dealing with other languages.

Copied to clipboard

Jianing Wang, Chengyu Wang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Jun Huang, Ming Gao

Interactive Machine Comprehension with Information Seeking Agents (2020.acl-main)

Challenge:	Extractive Question Answering (EQA) is one of the most essential tasks in Machine Reading Comprehension (MRC).
Approach:	They propose a framework that transforms extractive question answering into a non-autoregressive Masked Language Modeling (MLM) generation problem.
Outcome:	The proposed framework outperforms state-of-the-art approaches in few-shot learning scenarios by a large margin.

Copied to clipboard

Xingdi Yuan, Jie Fu, Marc-Alexandre Côté, Yi Tay, Chris Pal, Adam Trischler

COM-MRC: A COntext-Masked Machine Reading Comprehension Framework for Aspect Sentiment Triplet Extraction (2022.emnlp-main)

Challenge:	Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA).
Approach:	They propose a method that reframes existing machine reading comprehension (MRC) datasets as interactive, partially observable environments.
Outcome:	The proposed method "occludes" the majority of a document’s text and adds context-sensitive commands that reveal "glimpses" of the hidden text to a model.

Copied to clipboard

Zepeng Zhai, Hao Chen, Fangxiang Feng, Ruifan Li, Xiaojie Wang

Machine Reading Comprehension as Data Augmentation: A Case Study on Implicit Event Argument Extraction (2021.emnlp-main)

Challenge:	Aspect Sentiment Triplet Extraction (ASTE) aims to extract sentiment triplets from sentences, but when faced with multiple aspect terms, the MRC-based methods could fail due to the interference from other aspect terms.
Approach:	They propose a COntext-Masked MRC framework for Aspect Sentiment Triplet Extraction (ASTE) which aims to extract sentiment triplets from sentences .
Outcome:	The proposed framework outperforms state-of-the-art methods on benchmark datasets and shows that it can extract sentiment triplets from multiple aspect terms.

Copied to clipboard

Jian Liu, Yufeng Chen, Jinan Xu

Aspect-based Sentiment Analysis as Machine Reading Comprehension (2022.coling-1)

Challenge:	Existing datasets are too small to train a model for capturing regularities underlying how event arguments are extracted.
Approach:	They propose to bridge implicit EAE with machine reading comprehension (MRC) by building a unified training framework and explicit data augmentation regimes via MRC.
Outcome:	The proposed method obtains state-of-the-art performance on two benchmarks and demonstrates superior results in a data-low scenario.

Copied to clipboard

Yifei Yang, Hai Zhao

StuBot: Learning by Teaching a Conversational Agent Through Machine Reading Comprehension (2022.findings-emnlp)

Challenge:	Existing approaches to aspect-based sentiment analysis stack multiple modules and result in severe error propagation.
Approach:	They propose a MRC-PrOmpt mOdeL framework where multiple sentiment aspects are elicited by a machine reading comprehension model and their corresponding sentiment polarities are classified in a prompt learning way.
Outcome:	The proposed framework significantly outperforms existing state-of-the-art models or achieves comparable performance on widely-used benchmark datasets.

Copied to clipboard

Nayoung Jin, Hana Lee

Answer Span Correction in Machine Reading Comprehension (2020.findings-emnlp)

Challenge:	StuBot provides adaptive feedback for learning by teaching .
Approach:	They propose a text-based conversational agent that provides adaptive feedback for learning by teaching.
Outcome:	The proposed agent improves learning performance, immersion, and overall experience by providing adaptive feedback to the users who input the explanation text.

Copied to clipboard

Revanth Gangi Reddy, Md Arafat Sultan, Efsun Sarioglu Kayi, Rong Zhang, Vittorio Castelli, Avi Sil

Enhancing Pre-Trained Language Representations with Rich Knowledge for Machine Reading Comprehension (P19-1)

Challenge:	Existing MRC systems produce only partially correct answers when presented with answerable questions.
Approach:	They propose a method that yields statistically significant performance improvements over existing MRC systems.
Outcome:	The proposed method yields statistically significant performance improvements over state-of-the-art MRC systems in monolingual and multilingual evaluation.

Copied to clipboard

An Yang, Quan Wang, Jing Liu, Kai Liu, Yajuan Lyu, Hua Wu, Qiaoqiao She, Sujian Li

Document-level Event Factuality Identification via Machine Reading Comprehension Frameworks with Transfer Learning (2022.coling-1)

Challenge:	Recent results show pre-trained language models (LMs) can improve machine reading comprehension (MRC) Experimental results indicate that KT-NET offers significant and consistent improvements over BERT .
Approach:	They propose a method that leverages external knowledge bases to improve machine reading comprehension (MRC) KT-NET employs an attention mechanism to select desired knowledge from KBs and fuses selected knowledge with BERT to enable context- and knowledge-aware predictions.
Outcome:	The proposed model outperforms baseline models on ReCoRD and SQuAD1.1 benchmarks and ranks 1st on the ReCoDR and SQUAD1.1 leaderboards.

Copied to clipboard

Zhong Qian, Heng Zhang, Peifeng Li, Qiaoming Zhu, Guodong Zhou

A Vietnamese Dataset for Evaluating Machine Reading Comprehension (2020.coling-main)

Challenge:	Document-level Event Factuality Identification (DEFI) is a fundamental and crucial task in NLP.
Approach:	They propose a framework for document-level event factuality identification (DEFI) they propose to use Span-Extraction and Multiple-Choice to model DEFI as machine reading comprehension tasks .
Outcome:	The proposed model outperforms state-of-the-art models on a document-based event factuality task . it uses Span-Extraction (Ext) and Multiple-Choice (Mch) knowledge to extract knowledge from large-scale MRC corpus .

Copied to clipboard

Kiet Nguyen, Vu Nguyen, Anh Nguyen, Ngan Nguyen

A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension (D18-1)

Challenge:	despite the lack of benchmark datasets for Vietnamese, there are few studies on machine reading comprehension (MRC) . MRC is an essential core for a range of natural language processing applications such as search engines and intelligent agents.
Approach:	They propose to use Vietnamese Question Answering Dataset to evaluate machine reading comprehension in Vietnamese . they use over 23,000 human-generated question-answer pairs based on 5,109 Vietnamese articles .
Outcome:	The proposed dataset includes over 23,000 human-generated question-answer pairs based on 5,109 passages of 174 Vietnamese articles from Wikipedia.

Copied to clipboard

Jiahua Liu, Wan Wei, Maosong Sun, Hao Chen, Yantao Du, Dekang Lin

Learn with Noisy Data via Unsupervised Loss Correction for Weakly Supervised Reading Comprehension (2020.coling-main)

Challenge:	Existing models of machine reading comprehension (MRC) are based on cloze style questions or crowdworkers given a short passage from well-edited sources.
Approach:	They propose a multi-answer multi-task framework that uses multiple reference answers for multiple questions.
Outcome:	The proposed model increases the ROUGE-L score on the DuReader dataset from 44.18, the previous state-of-the-art, to 51.09 .

Copied to clipboard

Xuemiao Zhang, Kun Zhou, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, Junfei Liu

Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension (2020.coling-main)

Challenge:	Existing approaches to filter noise for machine reading comprehension (MRC) are difficult to control and introduce noisy data.
Approach:	They propose a hierarchical loss correction strategy to avoid fitting noise and enhance clean supervision signals by using an unsupervisedly fitted Gaussian mixture model and a hard bootstrapping loss method.
Outcome:	The proposed methods can help improve models significantly on weakly supervised machine reading comprehension datasets.

Copied to clipboard

Shaoru Guo, Yong Guan, Ru Li, Xiaoli Li, Hongye Tan

Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structure (2020.coling-main)

Challenge:	Existing methods for machine reading comprehension rely on manually defined features and are difficult to generalize to other tasks.
Approach:	They propose a Syntax and Frame Semantics model for Machine Reading Comprehension which takes full advantage of syntax and frame semantics to get richer text representation.
Outcome:	The proposed model outperforms ten state-of-the-art models on machine reading comprehension tasks.

Copied to clipboard

Jiaqi Li, Ming Liu, Min-Yen Kan, Zihao Zheng, Zekun Wang, Wenqiang Lei, Ting Liu, Bing Qin

Learning Invariant Representation Improves Robustness for MRC Models (2022.findings-emnlp)

Challenge:	Multiparty dialog applications such as discourse parsing and meeting summarization are now mainstream research.
Approach:	They propose to annotate a machine reading comprehension dataset with discourse structure built over multiparty dialog using a modified Segmented Discourse Representation Theory (SDRT) style.
Outcome:	The proposed dataset contributes large-scale discourse dependency annotations in a modified Segmented Discourse Representation Theory (SDRT) style for all of its multiparty dialogs, and achieves only 67.7% F1 on Molweni’s questions, a 20+% significant drop as compared against its SQuAD 2.0 performance.

Copied to clipboard

Yu Hai, Liang Wen, Haoran Meng, Tianyu Liu, Houfeng Wang

BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis (N19-1)

Challenge:	Existing approaches to improve machine reading comprehension models are vulnerable and not robust to adversarial examples.
Approach:	They propose to construct positive example pairs which have same answer by augmentation and then introduce stability and contrastive loss to improve invariance of representation.
Outcome:	The proposed approach boosts the robustness of QA models across different tasks and attack sets significantly and consistently.

Copied to clipboard

Hu Xu, Bing Liu, Lei Shu, Philip Yu

Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation (2020.coling-main)

Challenge:	Existing work on question-answering has limited training examples for RRC . question-announced questions are a key component of online commerce .
Approach:	They propose to turn customer reviews into a large source of knowledge that can be exploited to answer user questions.
Outcome:	The proposed approach improves review reading comprehension on popular language model BERT . it also improves aspect extraction and aspect sentiment classification tasks .

Copied to clipboard

Junhao Liu, Linjun Shou, Jian Pei, Ming Gong, Min Yang, Daxin Jiang

Multilinguality

Challenge:	Cross-lingual Machine Reading Comprehension (CLMRC) is a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi, and Vietnamese.
Approach:	They propose a novel approach to augment cross-lingual machine reading comprehension by combining knowledge from multiple language branch models into a single model for all target languages.
Outcome:	Extensive experiments on two CLMRC benchmarks show the proposed method is effective and robust to data noises.

Conversational Machine Comprehension: a Literature Review (2020.coling-main)

Copied to clipboard

Somil Gupta, Bhanu Pratap Singh Rawat, Hong Yu

Span Selection Pre-training for Question Answering (2020.acl-main)

Challenge:	Conversational machine comprehension (CMC) is a research track in conversational AI.
Approach:	They propose to synthesize a generic framework for CMC models and highlight differences in recent approaches.
Outcome:	The proposed model will be used as a compendium for future research.

Copied to clipboard

Michael Glass, Alfio Gliozzo, Rishav Chakravarti, Anthony Ferritto, Lin Pan, G P Shrivatsa Bhargav, Dinesh Garg, Avi Sil

Robust Machine Reading Comprehension by Learning Soft labels (2020.coling-main)

Challenge:	Pre-trained BERTs provide large gains across many language understanding tasks, achieving a new state-of-the-art (SOTA).
Approach:	They propose a new pre-training task inspired by reading comprehension to better align the pre- training from memorization to understanding.
Outcome:	The proposed model outperforms BERT-BASE and BERT LARGE on a new dataset and improves answer prediction F1 by 4 points and supporting fact prediction F1.

Copied to clipboard

Zhenyu Zhao, Shuangzhi Wu, Muyun Yang, Kehai Chen, Tiejun Zhao

BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels (D19-1)

Challenge:	Neural models have achieved great success on the task of machine reading comprehension, which are typically trained on hard labels.
Approach:	They propose a robust training method for machine reading comprehension models to address label sparseness problem by using three strategies to train models on soft labels.
Outcome:	The proposed method improves the baseline model performance and achieves state-of-the-art performance on NewsQA and QUOREF.

Copied to clipboard

Yimin Jing, Deyi Xiong, Zhen Yan

Multilinguality

Challenge:	Using BiPaR, we build monolingual, multilingual and cross-lingual MRC on novels.
Approach:	They propose a bilingual parallel novel-style machine reading comprehension dataset BiPaR . they collect 3,667 bilingual parallel paragraphs from Chinese and English novels .
Outcome:	The proposed dataset supports multilingual and cross-lingual reading comprehension.

NumNet: Machine Reading Comprehension with Numerical Reasoning (D19-1)

Copied to clipboard

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, Zhiyuan Liu

Adversarial Domain Adaptation for Machine Reading Comprehension (D19-1)

Challenge:	Existing numerical MRC models are weak in numerical reasoning, such as addition, subtraction, sorting and counting.
Approach:	They propose a numerical MRC model that integrates numerical reasoning into existing MRC models and achieves an EM-score of 64.56% on the DROP dataset.
Outcome:	The proposed model outperforms all existing machine reading comprehension models by considering the numerical relations among numbers on the DROP dataset.

Copied to clipboard

Huazheng Wang, Zhe Gan, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Hongning Wang

Improving Machine Reading Comprehension with General Reading Strategies (N19-1)

Challenge:	Existing models for machine reading comprehension rely on large amounts of human-annotated in-domain data.
Approach:	They propose an unsupervised domain adaptation framework for Machine Reading Comprehension where the source domain has a large amount of labeled data, while only unlabeled passages are available in the target domain.
Outcome:	The proposed framework can be generalizable to different MRC models and datasets and can be extended to semi-supervised learning.

Copied to clipboard

Kai Sun, Dian Yu, Dong Yu, Claire Cardie

Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension (N19-1)

Challenge:	Recent studies have shown that reading strategies improve comprehension levels for readers lacking adequate prior knowledge.
Approach:	They propose three general strategies to improve machine reading comprehension (MRC) by fine-tuning a pre-trained model with strategies and a target task.
Outcome:	The proposed models improve non-extractive machine reading comprehension (MRC) on the largest general domain multiple-choice dataset RACE.

Copied to clipboard

Yichong Xu, Xiaodong Liu, Yelong Shen, Jingjing Liu, Jianfeng Gao

Interactive Language Learning by Question Answering (D19-1)

Challenge:	Existing models for Machine Reading Comprehension (MRC) are small, compared to their size, and there are many studies on using pre-trained word embeddings and back-translation approaches to improve model generalization.
Approach:	They propose a multi-task learning framework to learn a machine reading comprehension model that can be applied to a wide range of MRC tasks in different domains.
Outcome:	The proposed model can be applied to a wide range of MRC tasks in different domains.

Copied to clipboard

Xingdi Yuan, Marc-Alexandre Côté, Jie Fu, Zhouhan Lin, Chris Pal, Yoshua Bengio, Adam Trischler

Enhancing Multiple-choice Machine Reading Comprehension by Punishing Illogical Interpretations (2021.emnlp-main)

Challenge:	Existing machine reading comprehension tasks lack interactive information-seeking component of comprehension.
Approach:	They propose a question-asking task that asks questions in a text-based environment . they propose QAit, which uses a game generator to build models that include deep reinforcement learning agents.
Outcome:	The proposed task poses questions about existence, location, and attributes of objects found in environment.

Copied to clipboard

Yiming Ju, Yuanzhe Zhang, Zhixing Tian, Kang Liu, Xiaohuan Cao, Wenting Zhao, Jinlong Li, Jun Zhao

Is the Understanding of Explicit Discourse Relations Required in Machine Reading Comprehension? (2021.eacl-main)

Challenge:	Multiple-choice MRC is one of the most studied tasks in MRC due to the convenience of evaluation and the flexibility of answer format.
Approach:	They propose to use multiple-choice MRC to explain a trained model and reveal how it arrives at the prediction by punishing illogical attributions.
Outcome:	The proposed method improves model performance without external information and model structure change without any external information.

Copied to clipboard

Yulong Wu, Viktor Schlegel, Riza Batista-Navarro

Synonym Knowledge Enhanced Reader for Chinese Idiom Reading Comprehension (2020.coling-main)

Challenge:	Existing benchmarks for machine reading comprehension (MRC) are insufficient to assess models for their capabilities to read and comprehend .
Approach:	They propose an ablation-based method to assess the extent to which MRC datasets evaluate the understanding of explicit discourse relations.
Outcome:	The proposed method shows that the model's performance drops on three large-scale datasets . the results suggest that most of the answers do not require understanding the discourse structure of the text.

Copied to clipboard

Siyu Long, Ran Wang, Kun Tao, Jiali Zeng, Xinyu Dai

How Many Answers Should I Give? An Empirical Study of Multi-Answer Reading Comprehension (2023.findings-acl)

Challenge:	Experimental results show that our model achieves state-of-the-art performance for Chinese idiom comprehension.
Approach:	They propose a model that can mitigate the inconsistency between literal and literal meanings by incorporating the synonym knowledge enhanced reader into the model.
Outcome:	The proposed model achieves state-of-the-art on a Chinese idiom reading comprehension dataset.

Copied to clipboard

Chen Zhang, Jiuheng Lin, Xiao Liu, Yuxuan Lai, Yansong Feng, Dongyan Zhao

A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction (2020.acl-main)

Challenge:	Despite recent progress in multi-answer MRC, there is no systematic analysis of how this phenomenon arises and how to better address it.
Approach:	They develop a taxonomy to categorize commonly-seen multi-answer MRC instances and examine how well different paradigms deal with different types of multi-announced questions.
Outcome:	The proposed taxonomy categorizes commonly-seen multi-answer instances and analyzes how well different paradigms deal with different types of multi-announced instances.

Copied to clipboard

Yilin Niu, Fangkai Jiao, Mantong Zhou, Ting Yao, Jingfang Xu, Minlie Huang

Feeding What You Need by Understanding What You Learned (2022.acl-long)

Challenge:	Existing models for machine reading comprehension lack evidence labels for training models.
Approach:	They propose a method which supervises the evidence extractor with auto-generated evidence labels in an iterative process.
Outcome:	The proposed method improves on three MRC tasks on seven datasets.

Copied to clipboard

Xiaoqiang Wang, Bang Liu, Fangli Xu, Bo Long, Siliang Tang, Lingfei Wu

Green & Sustainable NLP

Challenge:	Existing research on machine reading comprehension rely heavily on large-size models and corpus to improve performance.
Approach:	They propose a framework that assesses model capabilities in an explainable and multi-dimensional manner.
Outcome:	The proposed framework achieves an 11.22% / 8.71% improvement of EM / F1 on MRC tasks.

Dataset for the First Evaluation on Chinese Machine Reading Comprehension (L18-1)

Copied to clipboard

Yiming Cui, Ting Liu, Zhipeng Chen, Wentao Ma, Shijin Wang, Guoping Hu

BioRead: A New Dataset for Biomedical Reading Comprehension (L18-1)

Challenge:	Existing reading comprehension datasets are mostly in English .
Approach:	They propose a Chinese reading comprehension dataset to add diversity to existing reading comprehension data . proposed dataset contains cloze-style reading comprehension and user query reading comprehension .
Outcome:	The proposed dataset is based on a Chinese reading comprehension dataset . it includes two types of cloze-style and user query reading comprehension . the proposed dataset hosted the 1st Evaluation on Chinese Machine Reading Comprehension (CMRC-2017)

Copied to clipboard

Dimitris Pappas, Ion Androutsopoulos, Haris Papageorgiou

Coreference Reasoning in Machine Reading Comprehension (2021.acl-long)

Challenge:	BioRead is a publicly available cloze-style biomedical machine reading comprehension (MRC) dataset with 16.4 million passage-question instances.
Approach:	They propose to build a cloze-style biomedical machine reading comprehension (MRC) dataset with 16.4 million passage-question instances.
Outcome:	The proposed method outperforms baselines on bioReadLite and bioASQ, and is currently the best on BioReadLite.

Copied to clipboard

Mingzhu Wu, Nafise Sadat Moosavi, Dan Roth, Iryna Gurevych

What Makes Reading Comprehension Questions Easier? (D18-1)

Challenge:	Existing datasets for machine reading comprehension do not reflect the natural distribution and, consequently, the challenges of coreference reasoning.
Approach:	They propose to use existing coreference resolution datasets to train machine reading comprehension models to better reflect the natural distribution and, consequently, the challenges of coreference reasoning.
Outcome:	The proposed method improves the performance of state-of-the-art models on a set of coreference-related datasets.

Copied to clipboard

Saku Sugawara, Kentaro Inui, Satoshi Sekine, Akiko Aizawa

Contrastive Distant Supervision for Debiased and Denoised Machine Reading Comprehension (2023.findings-emnlp)

Challenge:	Recent studies have shown that questions require a deeper understanding of language to answer beyond using superficial cues.
Approach:	They propose to use simple heuristics to split MRC datasets into easy and hard subsets and manually annotate questions from each subset with validity and reasoning skills to investigate which skills explain the difference between easy and harder questions.
Outcome:	The proposed model performs better for hard and easy questions than for easy questions.

Copied to clipboard

Ning Bian, Hongyu Lin, Xianpei Han, Ben He, Le Sun

IDK-MRC: Unanswerable Questions for Indonesian Machine Reading Comprehension (2022.emnlp-main)

Challenge:	Distant supervision (DS) is a promising learning approach for machine reading comprehension (MRC) however, the annotated dataset will inevitably lead to mislabeled instances, resulting in answer bias and context noise problems.
Approach:	They propose an algorithm that can learn to distinguish confusing and noisy instances via confidence-aware contrastive learning.
Outcome:	The proposed algorithm can learn to distinguish confusing and noisy instances via confidence-aware contrastive learning.

Copied to clipboard

Rifki Afina Putri, Alice Oh

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space (2020.emnlp-main)

Challenge:	Existing MRC datasets in Indonesian are inadequate because of the small size and limited question types.
Approach:	They propose to combine automatic and manual unanswerable question generation to minimize the cost of manual dataset construction while maintaining the dataset quality.
Outcome:	The proposed dataset significantly improves the performance of Indonesian MRC models, showing a large improvement for unanswerable questions.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou

Text Generation

Challenge:	Existing data augmentation techniques for natural language processing tasks are difficult to design.
Approach:	They propose a controllable rewriting based question data augmentation method for machine reading comprehension, question generation and question-answering natural language inference tasks.
Outcome:	The proposed method generates high-quality, high-quality question data samples on machine reading comprehension, question generation, and question-answering natural language inference tasks.

Muppet: Massive Multi-task Representations with Pre-Finetuning (2021.emnlp-main)

Copied to clipboard

Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, Sonal Gupta

IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning (2023.findings-acl)

Challenge:	Recent work shows gains from pre-training and fine-tuning that are multi-task . but it can be difficult to know which intermediate tasks will best transfer .
Approach:	They propose a large-scale learning stage for pre-finetuning between pre-training and fine-tun.
Outcome:	The proposed model improves performance on pretrained discriminators and generation models on a wide range of tasks while improving sample efficiency during fine-tuning.

Copied to clipboard

Zihang Xu, Ziqing Yang, Yiming Cui, Shijin Wang

Generating Extractive Answers: Gated Recurrent Memory Reader for Conversational Question Answering (2023.findings-emnlp)

Challenge:	Existing systems for logical reasoning have surpassed the average performance of humans in many tasks like SQuAD but there is still a long way to go when it comes to logical reasoning.
Approach:	They propose an InDicator-Oriented Logic Pre-training task which logically strengthens pre-trained models with the help of 6 types of logical indicators and a logicalally rich dataset.
Outcome:	The proposed task achieves state-of-the-art on ReClor and LogiQA, the two most representative benchmarks in logical reasoning MRC.

Copied to clipboard

Xuanyu Zhang, Qing Yang

A Unified MRC Framework for Named Entity Recognition (2020.acl-main)

Challenge:	Conversational question answering (CQA) requires models to extract answers from given contents to answer follow-up questions according to conversation history.
Approach:	They propose a novel architecture that integrates extractive MRC models into a generalized sequence-to-sequence framework.
Outcome:	The proposed architecture can use less storage space and consider historical memory deeply and selectively.

Copied to clipboard

Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, Jiwei Li

Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension (2024.emnlp-main)

Challenge:	Named entity recognition is divided into nested NER and flat NER depending on whether entities are nesting.
Approach:	They propose to formulate named entity recognition task as machine reading comprehension task instead of sequence labeling problem .
Outcome:	The proposed framework achieves vast amount of performance boost over current models on nested and flat NER datasets.

Copied to clipboard

Lin Ai, Zheng Hui, Zizhou Liu, Julia Hirschberg

Have You Seen That Number? Investigating Extrapolation in Question Answering Models (2021.emnlp-main)

Challenge:	Extractive Machine Reading Comprehension (MRC) is a challenging field in the field of Natural Language Processing.
Approach:	They propose a Question-Attended Span Extraction module to address the limitations of generative approaches for extractive machine reading comprehension (MRC) . module significantly enhances performance of pre-trained generative language models, enabling them to surpass the extractive capabilities of advanced Large Language Models (LLMs)
Outcome:	The QASE module surpasses state-of-the-art models in few-shot settings.

Copied to clipboard

Jeonghwan Kim, Giwon Hong, Kyung-min Kim, Junmo Kang, Sung-Hyon Myaeng

Guiding LLM to Fool Itself: Automatically Manipulating Machine Reading Comprehension Shortcut Triggers (2023.findings-emnlp)

Challenge:	Numerical reasoning in machine reading comprehension (MRC) has shown drastic improvements over the past few years.
Approach:	They propose an E-digit number form that alleviates the lack of extrapolation in numerical MRC models.
Outcome:	The proposed model can't extrapolate to unseen numbers, the authors say . they also show that the model needs to treat numbers differently from regular words .

Copied to clipboard

Mosh Levy, Shauli Ravfogel, Yoav Goldberg

Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian (2020.coling-main)

Challenge:	Recent applications of LLMs in Machine Reading Comprehension systems have shown impressive results, but the use of shortcuts has emerged as a potential threat to their reliability.
Approach:	They propose a framework that guides an editor to add potential shortcuts-triggers to samples.
Outcome:	The proposed framework can edit trigger shortcuts in samples that fool LLMs . it also shows that GPT4 can be deceived by its own edits (15% drop in F1).

Copied to clipboard

Alena Fenogenova, Vladislav Mikhailov, Denis Shevelev

Enhancing Extractive Question Answering in Multiparty Dialogues with Logical Inference Memory Network (2025.coling-main)

Challenge:	MRC in other languages, including Russian, has not been well-addressed due to the lack of high-quality and large-scale datasets.
Approach:	They propose two Russian machine reading comprehension datasets that require reasoning over multiple sentences and commonsense knowledge to infer the answer.
Outcome:	The proposed datasets are more complex than the original ones for Russian . the results show that the proposed models are challenging for advanced models .

Copied to clipboard

Shu Zhou, Rui Zhao, Zhengda Zhou, Haohan Yi, Xuhui Zheng, Hao Wang

Incremental Transformer: Efficient Encoder for Incremented Text Over MRC and Conversation Tasks (2025.coling-main)

Challenge:	Existing models for multiparty dialogue question answering (QA) do not consider logical inference relations in multiparty dialogs, leading to suboptimal performance.
Approach:	They propose a memory network with logical inference for extractive QA in multiparty dialogues.
Outcome:	The proposed model achieves state-of-the-art on Molweni and FriendsQA benchmarks.

Copied to clipboard

Weisheng Li, Yuechen Wang, Jiaxin Shi, Wengang Zhou, Qi Tian, Houqiang Li

Lite Unified Modeling for Discriminative Reading Comprehension (2022.acl-long)

Challenge:	Existing encoders that encode incremented inputs have to re-encode the whole text to obtain the encoding of the extended input.
Approach:	They propose an efficient encoder dedicated for faster encoding of incremented input . it takes only added input as input but attends to cached representations of original input a lower layer .
Outcome:	The proposed encoder achieves 6.2x speedup over current encoders . it takes only added input as input but attends to cached representations of original input .

Copied to clipboard

Yilin Zhao, Hai Zhao, Libin Shen, Yinggong Zhao

ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations (2021.emnlp-main)

Challenge:	generative and discriminative MRCs focus on answer generation, extractive MRC on answer extraction.
Approach:	They propose a lightweight POS-Enhanced Iterative Co-Attention Network to handle diverse discriminative MRC tasks synchronously.
Outcome:	The proposed model improves on four discriminative MRC benchmarks.

Copied to clipboard

Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, Nanyun Peng

Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge (2022.acl-long)

Challenge:	Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations.
Approach:	They propose a machine reading comprehension dataset that leverages natural language queries to reason about the five most common event semantic relations.
Outcome:	The proposed dataset shows that current SOTA systems achieve 22.1%, 63.3% and 83.5% for token-based exact-match, F1 and event-based HIT@1 scores.

Copied to clipboard

Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Claire Cardie

A Span-Extraction Dataset for Chinese Machine Reading Comprehension (D19-1)

Challenge:	Existing methods to improve machine reading comprehension (MRC) tasks require unstated knowledge to perform well.
Approach:	They propose to extract a new kind of structured knowledge from scripts and use it to improve machine reading comprehension (MRC) They propose a teacher-student paradigm to facilitate the transfer of knowledge in weakly-labeled MRC data.
Outcome:	The proposed method outperforms methods that use weakly-labeled data and improves a state-of-the-art baseline by 4.3% in accuracy on a Chinese multiple-choice MRC dataset C3.

Copied to clipboard

Yiming Cui, Ting Liu, Wanxiang Che, Li Xiao, Zhipeng Chen, Wentao Ma, Shijin Wang, Guoping Hu

Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension (2020.acl-main)

Challenge:	Existing reading comprehension datasets are mostly in English . MRC is a new field of research that aims to comprehend the context of articles and answer the questions based on them.
Approach:	They propose a Span-Extraction dataset for Chinese machine reading comprehension to add language diversities to existing reading comprehension datasets.
Outcome:	The proposed dataset is composed of 20,000 real questions annotated on Wikipedia paragraphs by human experts.

Copied to clipboard

Hongyu Gong, Yelong Shen, Dian Yu, Jianshu Chen, Dong Yu

MCˆ2: Multi-perspective Convolutional Cube for Conversational Machine Reading Comprehension (P19-1)

Challenge:	Existing approaches to machine reading comprehension (MRC) on long texts typically chunk text into equally-spaced segments without considering information from other segments.
Approach:	They propose to let a model learn to chunk in a more flexible way via reinforcement learning.
Outcome:	The proposed model extracts a text span from document and query as answer . previous models can only take a fixed-length (e.g., 512) text as input .

Copied to clipboard

Xuanyu Zhang

HRCA+: Advanced Multiple-choice Machine Reading Comprehension Method (2022.lrec-1)

Challenge:	Existing models combine previous questions for conversation understanding and only employ recurrent neural networks (RNN) for reasoning.
Approach:	They propose a multi-perspective convolutional cube model that integrates 1D and 2D convolutions with recurrent neural networks (RNN) to understand context from different perspectives.
Outcome:	The proposed model is based on the Conversational Question Answering (CoQA) dataset and achieves state-of-the-art results.

Copied to clipboard

Yuxiang Zhang, Hayato Yamana

Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction (2024.findings-acl)

Challenge:	Multiple-choice question answering (MCQA) requires a model to understand natural languages and understand textual representations.
Approach:	They propose a model that uses human reading comprehension attention to increase accuracy for machine reading comprehension.
Outcome:	The proposed model outperforms state-of-the-art models on the Semeval-2018 Task 11 dataset and on the DREAM dataset.

Copied to clipboard

Albert Sawczyn, Katsiaryna Viarenich, Konrad Wojtasik, Aleksandra Domogała, Marcin Oleksy, Maciej Piasecki, Tomasz Kajdanowicz

A Framework for Evaluation of Machine Reading Comprehension Gold Standards (2020.lrec-1)

Challenge:	Existing KBQA datasets are outdated and inefficient in human labor, and assisting tools like Large Language Models (LLM) are not utilized to reduce the workload.
Approach:	They propose a semi-automated question answering task that uses structured knowledge graphs to answer extensive knowledge-intensive questions.
Outcome:	The proposed approach includes KBQA, MRC, and Information Retrieval tasks for low-resource languages.

Copied to clipboard

Viktor Schlegel, Marco Valentino, Andre Freitas, Goran Nenadic, Riza Batista-Navarro

Knowing-how & Knowing-that: A New Task for Machine Comprehension of User Manuals (2023.findings-acl)

Challenge:	Existing literature on machine reading comprehension (MRC) data is limited on the data design of gold standards.
Approach:	They propose a framework to investigate linguistic features, lexical cues and ambiguity in MRC gold standards.
Outcome:	The proposed framework investigates the present linguistic features, required reasoning and background knowledge and factual correctness on the one hand, and the presence of lexical cues as a lower bound for the requirement of understanding on the other.

Copied to clipboard

Hongru Liang, Jia Liu, Weihong Du, Dingnan Jin, Wenqiang Lei, Zujie Wen, Jiancheng Lv

To Test Machine Comprehension, Start by Defining Comprehension (2020.acl-main)

Challenge:	Existing methods for machine reading comprehension of user manuals have trouble answering complex questions.
Approach:	They propose a knowing-how & knowing-that task that requires the model to answer factoid-style, procedure-style and inconsistent questions about user manuals.
Outcome:	The proposed model can answer factoid-style, procedure-style and inconsistent questions about user manuals.

Copied to clipboard

Jesse Dunietz, Greg Burnham, Akash Bharadwaj, Owen Rambow, Jennifer Chu-Carroll, Dave Ferrucci

Natural Response Generation for Chinese Reading Comprehension (2023.findings-emnlp)

Challenge:	Existing approaches to machine reading comprehension do not adequately define comprehension, authors argue . authors argue that existing systems are not up to the task of narrative understanding as they define it .
Approach:	They propose a detailed definition of comprehension for short narratives . they argue existing systems are not up to the task of narrative understanding .
Outcome:	The proposed task definitions suggest existing systems are not up to the task of narrative understanding as they define it.

Copied to clipboard

Nuo Chen, Hongguang Li, Yinan Bao, Baoyuan Wang, Jia Li

Text Generation

Challenge:	MRC models trained on labeled answers are limited in generating human-like responses in real QA scenarios.
Approach:	They construct a dataset called Penguin to promote machine reading comprehension . they use 200k training data with fluent, well-informed responses to train models .
Outcome:	The proposed dataset is the first benchmark towards natural response generation in Chinese MRC on a relatively large scale.

Explaining Interactions Between Text Spans (2023.emnlp-main)

Copied to clipboard

Sagnik Ray Choudhury, Pepa Atanasova, Isabelle Augenstein

Interpreting Themes from Educational Stories (2024.lrec-main)

Challenge:	Existing highlight-based explanations focus on identifying individual important features or interactions only between adjacent tokens or tuples of tokens.
Approach:	They propose a multi-annotator dataset of human span interaction explanations for NLU and FC.
Outcome:	The proposed method compares human reasoning processes to those of a fine-tuned large language model.

Copied to clipboard

Yigeng Zhang, Fabio Gonzalez, Thamar Solorio

Explainability & Interpretability in NLP

Challenge:	Recent advances in machine reading comprehension (MRC) have centered on literal comprehension, referring to the surface-level understanding of content.
Approach:	They propose a dataset specifically designed for interpretive comprehension of educational narratives, providing corresponding well-edited theme texts.
Outcome:	The proposed dataset spans genres and cultural origins and includes human-annotated theme keywords with varying levels of granularity.

Causal Intervention for Mitigating Name Bias in Machine Reading Comprehension (2023.findings-acl)

Copied to clipboard

Jiazheng Zhu, Shaojuan Wu, Xiaowang Zhang, Yuexian Hou, Zhiyong Feng

HeQ: a Large and Diverse Hebrew Reading Comprehension Benchmark (2023.findings-emnlp)

Challenge:	Existing MRC models may overuse name information to make predictions, causing name bias .
Approach:	They propose a Causal Interventional paradigm for MRC to mitigate name bias by analyzing pre-trained knowledge and context representations.
Outcome:	The proposed model is robust to names and performs competitively on the original SQuAD.

Copied to clipboard

Amir Cohen, Hilla Merhav-Fine, Yoav Goldberg, Reut Tsarfaty

A Causal View of Entity Bias in (Large) Language Models (2023.findings-emnlp)

Challenge:	Current benchmarks for Hebrew Natural Language Processing (NLP) focus mainly on morpho-syntactic tasks, neglecting the semantic dimension of language understanding.
Approach:	They propose to use Hebrew machine reading comprehension (MRC) as extractive Question Answering to address this problem.
Outcome:	The proposed benchmark features 30,147 question-answer pairs derived from both Hebrew Wikipedia articles and Israeli tech news.

Copied to clipboard

Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, Muhao Chen

MRC-based Nested Medical NER with Co-prediction and Adaptive Pre-training (2024.lrec-main)

Challenge:	Entity bias affects pretrained (large) language models, causing them to rely on (biased) parametric knowledge to make unfaithful predictions.
Approach:	They propose a structured causal model whose parameters are easier to estimate . they propose to perturb the original entity with neighboring entities .
Outcome:	The proposed model reduces biasing information pertaining to the original entity while still preserving sufficient semantic information from similar entities.

Copied to clipboard

Xiaojing Du, Hanjie Zhao, Danyan Xing, Yuxiang Jia, Hongying Zan

Select High-quality Synthetic QA Pairs to Augment Training Data in MRC under the Reward Guidance of Generative Language Models (2024.lrec-main)

Challenge:	Experimental evaluations conducted on the CMeEE, a benchmark for Chinese nested medical named entity recognition (NER) model outperforms the compared state-of-the-art (SOTA) models.
Approach:	They propose a model based on machine reading comprehension that uses a task-adaptive pre-training strategy to improve the model’s capability in the medical field.
Outcome:	The proposed model outperforms the compared state-of-the-art models on the CMeEE, a benchmark for Chinese nested medical NER.

Copied to clipboard

Jing Jin, Houfeng Wang

Title-based Extractive Summarization via MRC Framework (2024.lrec-main)

Challenge:	Existing approaches focus on downstream metrics to select QA pairs, which lack generalization across different datasets.
Approach:	They propose a general selection method that uses a large pre-trained language model as a reward model in a Reinforcement Learning framework for the training of the selection agent.
Outcome:	The proposed method improves performance on generative and extractive datasets.

Copied to clipboard

Hongjin Kim, Jai-Eun Kim, Harksoo Kim