SPAN

Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering (2021.findings-emnlp)

Challenge:	Pretraining has reduced many complex task-specific NLP models to simple lightweight layers.
Approach:	They propose a lightweight end-to-end coreference model that removes the dependency on span representations, handcrafted features, pruning heuristics, and more.
Outcome:	The proposed model performs competitively with the current standard model, while being simpler and more efficient.

Copied to clipboard

Chenyu You, Nuo Chen, Yuexian Zou

Multimodality

Challenge:	Experimental results show that our model achieves state-of-the-art results on three SQA benchmarks.
Approach:	They propose a self-supervised training stage and a contrastive representation learning stage for spoken question answering with auxiliary tasks and augmentation strategies.
Outcome:	The proposed model achieves state-of-the-art results on three SQA benchmarks.

AutoRC: Improving BERT Based Relation Classification Models via Architecture Search (2021.acl-srw)

Copied to clipboard

Wei Zhu

Mediators in Determining what Processing BERT Performs First (2021.naacl-main)

Challenge:	Existing literature on relation classification models shows no consensus on optimal architecture .
Approach:	They propose a search space for BERT based relation classification models and employ an ENAS method to find better architectures.
Outcome:	The proposed method can find better architectures than baseline models on eight benchmark RC tasks.

Copied to clipboard

Aviv Slobodkin, Leshem Choshen, Omri Abend

HiTRANS: A Hierarchical Transformer Network for Nested Named Entity Recognition (2021.findings-emnlp)

Challenge:	Probing neural models for the ability to perform downstream tasks using their activation patterns is often used to localize what parts of the network specialize in performing which tasks.
Approach:	They propose to consider the prediction’s context length as a potential mediating factor and consider the length of the span whose processing is minimally required to perform the prediction.
Outcome:	The proposed model can get 196 different rankings when probing with seven tasks, the authors show .

Copied to clipboard

Zhiwei Yang, Jing Ma, Hechang Chen, Yunke Zhang, Yi Chang

Self-Cleaning: Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances (2024.findings-naacl)

Challenge:	Existing studies for named entity recognition focus on flat NER, i.e., without nested entities, by sequence labeling methods.
Approach:	They propose a Hierarchical Transformer network which decomposes the input sentence into multi-grained spans and enhances the representation learning in a hierarchical manner.
Outcome:	The proposed method achieves much better performance than the state-of-the-art approaches on GENIA, ACE-2004, ace-2005 and NNE datasets.

Copied to clipboard

Zhendong Chu, Ruiyi Zhang, Tong Yu, Rajiv Jain, Vlad Morariu, Jiuxiang Gu, Ani Nenkova

Dual-Channel Span for Aspect Sentiment Triplet Extraction (2023.emnlp-main)

Challenge:	Existing methods to train named entity recognition models on noisy data are expensive and time-intensive to accumulate.
Approach:	They propose to denoise noisy NER data with guidance from a small set of clean instances.
Outcome:	The proposed method can improve on large-scale datasets with a small guidance set.

Copied to clipboard

Pan Li, Ping Li, Kai Zhang

Sentiment Analysis

Challenge:	Existing approaches to extract sentiment triplets are too noisy and enumerate all possible spans.
Approach:	They propose a dual-channel span generation method to constrain the search space of span candidates.
Outcome:	The proposed method reduces span enumeration by nearly half on two versions of public datasets.

A Robustly Optimized BMRC for Aspect Sentiment Triplet Extraction (2022.naacl-main)

Copied to clipboard

Shu Liu, Kaiwen Li, Zuhe Li

Robustness in NLP

Challenge:	Aspect sentiment triplet extraction (ASTE) is a challenging subtask in aspect-based sentiment analysis.
Approach:	They propose a bidirectional machine reading comprehension method to extract triplets of aspects, opinions and sentiments with complex correspondence from the context.
Outcome:	The proposed method achieves state-of-the-art on multiple benchmark datasets.

Dynamic Dialogue Policy for Continual Reinforcement Learning (2022.coling-1)

Copied to clipboard

Christian Geishauser, Carel van Niekerk, Hsien-chin Lin, Nurul Lubis, Michael Heck, Shutong Feng, Milica Gašić

Dialogue Systems & Conversational Agents

Challenge:	Continual reinforcement learning of the dialogue policy has remained unaddressed . lack of a framework with training protocols, baseline models and suitable metrics has hindered research in this direction.
Approach:	They propose a continual learning algorithm, baseline architectures and metrics for assessing continual reinforcement learning models.
Outcome:	The proposed architecture can integrate new knowledge seamlessly and achieve significant zero-shot performance when exposed to unseen domains.

CAPID: Context-Aware PII Detection for Question-Answering Systems (2026.eacl-srw)

Copied to clipboard

Mariia Ponomarenko, Sepideh Abedini, Masoumeh Shafieinejad, D. B. Emerson, Shubhankar Mohapatra, Xi He

Natural Language Interfaces

Challenge:	Existing approaches mainly redact all PII, disregarding the fact that some may be contextually relevant to the user’s question, resulting in a degradation of response quality.
Approach:	They propose a method that fine-tunes a locally owned small language model that filters sensitive information before it is passed to LLMs for QA.
Outcome:	The proposed approach outperforms baselines in span, relevance and type accuracy while preserving significantly higher utility under anonymization.

Writing Mentor: Self-Regulated Writing Feedback for Struggling Writers (C18-2)

Copied to clipboard

Nitin Madnani, Jill Burstein, Norbert Elliot, Beata Beigman Klebanov, Diane Napolitano, Slava Andreyev, Maxwell Schwartz

Challenge:	Writing Mentor provides feedback on various aspects of writing using natural language processing (NLP) methods and resources.
Approach:	They propose to use natural language processing (NLP) to generate feedback on various aspects of writing using categorized and actionable resources.
Outcome:	The add-on provides feedback on various aspects of writing using natural language processing (NLP) methods and resources.

Tempo-Lexical Context Driven Word Embedding for Cross-Session Search Task Extraction (N18-1)

Copied to clipboard

Procheta Sen, Debasis Ganguly, Gareth Jones

ET5: A Novel End-to-end Framework for Conversational Machine Reading Comprehension (2022.coling-1)

Challenge:	Existing work on task extraction has focused on identifying tasks within a single session . but, we aim to identify tasks that span across multiple sessions.
Approach:	They propose to embed query words into query vectors to capture task semantics . they propose to use query vector embedding to predict whether a session is a part of a broader search task .
Outcome:	The proposed method improves task extraction efficiency over existing methods . it can predict whether a session is part of a broader complex search task .

Copied to clipboard

Xiao Zhang, Heyan Huang, Zewen Chi, Xian-Ling Mao

Reasoning

Challenge:	Existing methods require three steps to understand text, but span extraction and question rephrasing steps are not fully exploited.
Approach:	They propose a framework for conversational machine reading comprehension based on shared parameter mechanism . experimental results show the proposed framework achieves new state-of-the-art results on the ShARC leaderboard .
Outcome:	The proposed framework achieves state-of-the-art on the ShARC leaderboard with the BLEU-4 score of 55.2.

SCV: Light and Effective Multi-Vector Retrieval with Sequence Compressive Vectors (2025.coling-industry)

Copied to clipboard

Cheoneum Park, Seohyeong Jeong, Minsang Kim, KyungTae Lim, Yong-Hun Lee

Information Retrieval

Challenge:	Recent advances in language models (LMs) have driven progress in information retrieval (IR), effectively extracting semantically relevant information.
Approach:	They propose a multi-vector retrieval framework with sequence compression vectors and a coarse-to-fine vector search framework that uses span compressive vectors to score documents.
Outcome:	The proposed approach achieves the fastest latency compared to other state-of-the-art models and can obtain competitive performance on both in-domain and out-of domain benchmark datasets.

Decoding Text Spans for Efficient and Accurate Named-Entity Recognition (2026.acl-industry)

Copied to clipboard

Andrea Maracani, Savas Ozkan, Junyi Zhu, Sinan Mutlu, Mete Ozay

Green & Sustainable NLP

Challenge:	Named Entity Recognition (NER) is a key component in industrial information extraction pipelines, where systems must satisfy strict latency and throughput constraints in addition to strong accuracy.
Approach:	They propose a span-based NER framework that can be used to compute span representations at the final transformer stage, avoiding redundant computation in earlier layers.
Outcome:	The proposed framework matches competitive baselines while improving throughput and reducing computational cost.

T 2 -NER: A Two-Stage Span-Based Framework for Unified Named Entity Recognition with Templates (2023.tacl-1)

Copied to clipboard

Peixin Huang, Xiang Zhao, Minghao Hu, Zhen Tan, Weidong Xiao

Linear-time Constituency Parsing with RNNs and Dynamic Programming (P18-2)

Challenge:	Named Entity Recognition (NER) has evolved from flat to overlapped and discontinuous . NER is a text recognition task that recognizes mentions that represent entities in text .
Approach:	They propose a two-stage span-based framework to solve a unified NER task using two stages . they extract entity spans, classify over all entity span pairs and combine them to train two stages.
Outcome:	The proposed framework beats all the current competitive baselines on eight benchmark datasets, obtaining the best performance of unified NER.

Copied to clipboard

Juneki Hong, Liang Huang

Multimodality

Challenge:	Existing span-based constituency parsers are too slow for longer sentences and for applications beyond sentence boundaries.
Approach:	They propose a linear-time constituency parser with RNNs and dynamic programming using graph-structured stack and beam search.
Outcome:	The proposed parser is faster for long sentences and faster for discourse parsing.

SlideGuard: AI-Driven Evaluation of Graduate Student Presentation Materials (2026.acl-demo)

Copied to clipboard

Nikolay Alekseevich Butakov, Maria Khodorchenko, Mazein Nikita, Daniil Gareev, Yuri Falevskiy, Georgii Konev, Denis Nasonov

Multimodality

Challenge:	Effective communication is a core objective of graduate education in AI and machine learning (ML).
Approach:	They propose an evaluation agent that assesses slide decks against a framework of expert-defined criteria using a visual language model.
Outcome:	The evaluation agent detects the majority of expert-identified issues on a dataset of 150 annotated slide decks and shows strong results on structural and visual criteria and known limitations on subjective dimensions such as research quality.

Comparing Test Sets with Item Response Theory (2021.acl-long)

Copied to clipboard

Clara Vania, Phu Mon Htut, William Huang, Dhara Mungra, Richard Yuanzhe Pang, Jason Phang, Haokun Liu, Kyunghyun Cho, Samuel R. Bowman

Linguistic Theories

Challenge:	Recent results from large pretrained models show that many datasets are saturated and unlikely to detect further progress.
Approach:	They evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples.
Outcome:	The proposed datasets are saturated and unlikely to detect future improvements.

Event Pattern-Instance Graph: A Multi-Round Role Representation Learning Strategy for Document-Level Event Argument Extraction (2025.findings-acl)

Copied to clipboard

Qizhi Wan, LiuTao LiuTao, Changxuan Wan, Rong Hu, Keli Xiao, Yuxin Shuai

Representation Learning

Challenge:	Existing role-based span selection strategies ignore interrelations between events . authors propose a multi-round role representation learning strategy for document-level event argument extraction .
Approach:	They propose a pattern-instance graph to capture role semantics embedded in various associations . they also propose re-inventing the role representations learned from previous analyzed documents .
Outcome:	The proposed model captures role semantics embedded in various associations . iteratively updates representations of role nodes and edges to enrich their semantic information . the model improves prediction performance in subsequent rounds of span selection .

Higher-Order Coreference Resolution with Coarse-to-Fine Inference (N18-2)

Copied to clipboard

Kenton Lee, Luheng He, Luke Zettlemoyer

diaNED: Time-Aware Named Entity Disambiguation for Diachronic Corpora (P18-2)

Challenge:	a new approach to coreference resolution uses a span-ranking architecture as an attention mechanism to iteratively refine span representations.
Approach:	They propose a fully-differentiable approximation to higher-order inference for coreference resolution . they propose introducing a coarse-to-fine approach that incorporates a less accurate but more efficient bilinear factor .
Outcome:	The proposed model significantly improves accuracy on the English OntoNotes benchmark while being far more computationally efficient.

Copied to clipboard

Prabal Agarwal, Jannik Strötgen, Luciano del Corro, Johannes Hoffart, Gerhard Weikum

Challenge:	Named Entity Disambiguation (NED) systems perform well on news articles but quality drops when inputs span long time periods.
Approach:	They propose a time-aware method that resolves ambiguities even when mention contexts give only few cues.
Outcome:	The proposed method improves on a newly created diachronic corpus.

A Novel Three-stage Framework for Few-shot Named Entity Recognition (2024.lrec-main)

Copied to clipboard

Shengjie Ji, Fang Kong

Appraisal Framework for Clinical Empathy: A Novel Application to Breaking Bad News Conversations (2024.lrec-main)

Challenge:	Existing methods for Named Entity Recognition (NER) rely on labeled data, but data scarcity is a major challenge.
Approach:	They propose a framework for Few-shot Named Entity Recognition that can learn from limited labeled data and generalize to new domains.
Outcome:	The proposed framework surpasses existing methods on several benchmarks.

Copied to clipboard

Allison Claire Lahnala, Béla Neuendorf, Alexander Thomin, Charles Welch, Tina Stibane, Lucie Flek

Natural Language Interfaces

Challenge:	Empathy is essential in healthcare communication.
Approach:	They propose an annotation approach that draws on well-established frameworks for clinical empathy and breaking bad news conversations for considering the dynamic dynamics of discourse relations.
Outcome:	The proposed model can be used to train models to detect causal relations involving empathy, a feature of systems that can provide feedback to medical professionals in training.

Pre-training Entity Relation Encoder with Intra-span and Inter-span Information (2020.emnlp-main)

Copied to clipboard

Yijun Wang, Changzhi Sun, Yuanbin Wu, Junchi Yan, Peng Gao, Guotong Xie

Span-based Named Entity Recognition by Generating and Compressing Information (2023.eacl-main)

Challenge:	Existing pre-trained models do not handle text spans and relation among text span pairs.
Approach:	They propose to integrate span-related information into pre-trained encoder for entity relation extraction task.
Outcome:	The proposed pre-training method outperforms distantly supervised pre-trained models on two entity relation extraction benchmark datasets.

Copied to clipboard

Nhung T. H. Nguyen, Makoto Miwa, Sophia Ananiadou

Sentence-Level Resampling for Named Entity Recognition (2022.naacl-main)

Challenge:	Existing work on Named Entity Recognition (NER) only used generative or information compression models to improve performance.
Approach:	They propose to combine two types of IB models into one system to enhance Named Entity Recognition (NER) they incorporate unsupervised generative components span reconstruction and synonym generation into a span-based NER system.
Outcome:	The proposed model focuses on learning span representation, which is applicable not only to span-based NER but also to other span-related tasks such as event coreference resolution and question answering.

Copied to clipboard

Xiaochen Wang, Yue Wang

Span Fine-tuning for Pre-trained Language Models (2021.findings-emnlp)

Challenge:	named entity recognition (NER) tasks are often dominated by the majority of non-entity tokens in text . a data imbalance problem is causing the NER models to ignore named entities .
Approach:	They propose a set of sentence-level resampling methods to reduce data imbalance . they use a training sentence to compute the importance of each training sentence based on its tokens and entities .
Outcome:	The proposed methods outperform sub-sentence-level resampling, data augmentation, and loss functions on multiple corpora.

Copied to clipboard

Rongzhou Bao, Zhuosheng Zhang, Hai Zhao

A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning (D19-1)

Challenge:	Existing methods to fine-tune pre-trained language models are time-consuming and lack flexibility.
Approach:	They propose a span fine-tuning method which allows for a more efficient and efficient way of incorporating span-level information into pre-training.
Outcome:	Experiments on GLUE benchmark show that the proposed method significantly enhances the PrLM and offers more flexibility in an efficient way.

Copied to clipboard

Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li

Reasoning

Challenge:	Existing models for reading comprehension and question answering do not support discrete reasoning abilities.
Approach:	They propose a reading comprehension model that uses a multi-type answer predictor and a multiple-span extraction method to produce one or multiple text spans.
Outcome:	The proposed model achieves 79.9 F1 on the DROP hidden test set, creating new state-of-the-art results.

SCL-RAI: Span-based Contrastive Learning with Retrieval Augmented Inference for Unlabeled Entity Problem in NER (2022.coling-1)

Copied to clipboard

Shuzheng Si, Shuang Zeng, Jiaxing Lin, Baobao Chang

Should You Mask 15% in Masked Language Modeling? (2023.eacl-main)

Challenge:	Existing methods to solve Unlabeled Entity Problem (UEP) in Named Entities Recognition datasets are not effective in real-world datasets.
Approach:	They propose to decrease the distance of span representations with the same label while increasing it for different ones via span-based contrastive learning.
Outcome:	The proposed method outperforms the previous method on two real-world datasets.

Copied to clipboard

Alexander Wettig, Tianyu Gao, Zexuan Zhong, Danqi Chen

Salient Span Masking for Temporal Understanding (2023.eacl-main)

Challenge:	Masked language models (MLMs) traditionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations.
Approach:	They revisit the 15% masking rate of MLMs to examine the role of masking in linguistic training.
Outcome:	The proposed masking rate outperforms BERT-large size models on GLUE and SQUAD while maintaining 95% accuracy.

Copied to clipboard

Jeremy R. Cole, Aditi Chaudhary, Bhuwan Dhingra, Partha Talukdar

Few-Shot Question Answering by Pretraining Span Selection (2021.acl-long)

Challenge:	Salient Span Masking (SSM) has shown to be effective for closed-book question answering . authors of this study found that SSM alone improves performance on temporal tasks .
Approach:	They introduce Temporal Span Masking (TSM) to improve performance on temporal tasks . they find that SSM alone improves the downstream performance by +5.8 points .
Outcome:	The proposed approach improves performance on three temporal tasks by +5.8 points . the additional targeted spans achieved by adding the TSM task are the best .

Copied to clipboard

Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, Omer Levy

Fusing Heterogeneous Factors with Triaffine Mechanism for Nested Named Entity Recognition (2022.findings-acl)

Challenge:	Pretraining models with recurring span selection are effective, but perform poorly in a few-shot setting.
Approach:	They propose recurring span selection scheme that asks model to select correct span in passage with multiple sets of recurring recurrings.
Outcome:	The proposed model achieves 72.7 F1 on multiple benchmarks while maintaining competitive performance in the high-resource setting.

Copied to clipboard

Zheng Yuan, Chuanqi Tan, Songfang Huang, Fei Huang

Towards Exploiting Background Knowledge for Building Conversation Systems (D18-1)

Challenge:	Named entity recognition (NER) is a fundamental natural language processing task that extracts entities from texts.
Approach:	They propose a triaffine mechanism which integrates heterogeneous factors into a single model to fuse these factors into one model to achieve better span representation.
Outcome:	The proposed method outperforms previous span-based methods and achieves state-of-the-art F1 scores on nested NER datasets GENIA and KBP2017.

Copied to clipboard

Nikita Moghe, Siddhartha Arora, Suman Banerjee, Mitesh M. Khapra

Dialogue Systems & Conversational Agents

Challenge:	Existing dialog datasets contain a sequence of utterances without any explicit background knowledge associated with them.
Approach:	They propose to use movie chats to generate responses by copying unstructured background knowledge . they use a dataset of 9K conversations to test whether responses are generated by copy-and-modify models .
Outcome:	The proposed model mimics human process of conversing by copying and/or modifying sentences from unstructured background knowledge.

An Empirical Study on Finding Spans (2022.emnlp-main)

Copied to clipboard

Weiwei Gu, Boyuan Zheng, Yunmo Chen, Tongfei Chen, Benjamin Van Durme

Challenge:	Various information extraction tasks require a span finding component, which either directly yields the output or serves as an essential component of downstream linking.
Approach:	They propose methods for span finding, the selection of consecutive tokens in text for some downstream tasks.
Outcome:	The proposed methods perform better on masked language models and pre-trained encoders than on encoder-decoder models.

A Span-level Bidirectional Network for Aspect Sentiment Triplet Extraction (2022.emnlp-main)

Copied to clipboard

Yuqi Chen, Chen Keming, Xian Sun, Zequn Zhang

Sentiment Analysis

Challenge:	Aspect Sentiment Triplet Extraction (ASTE) is a new fine-grained sentiment analysis task . recent studies have focused on solving aspects term extraction, opinion term extraction and aspect-level sentiment classification tasks individually or in combination of two subtasks.
Approach:	They propose a span-level bidirectional network which utilizes all possible spans as input and extracts triplets from spans bidirectionally.
Outcome:	The proposed framework outperforms state-of-the-art methods and improves performance . it can extract triplets of aspect terms, sentiments, and opinion terms from review sentences .

A general framework for information extraction using dynamic span graphs (N19-1)

Copied to clipboard

Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, Hannaneh Hajishirzi

Scented-EAE: Stage-Customized Entity Type Embedding for Event Argument Extraction (2024.findings-acl)

Challenge:	Existing frameworks for information extraction use a pipeline approach to identify entities and then use the detected entity spans for relation extraction and coreference resolution.
Approach:	They propose a framework for several information extraction tasks that share span representations using dynamically constructed span graphs.
Outcome:	The proposed framework significantly outperforms state-of-the-art on multiple information extraction tasks across multiple datasets reflecting different domains.

Copied to clipboard

Yu Yang, Jinyu Guo, Kai Shuang, Chenrui Mao

How Much Context Span is Enough? Examining Context-Related Issues for Document-level MT (2022.lrec-1)

Challenge:	Existing methods for incorporating entities into EAE rely on prompts or NER . weak semantic associations due to missing role-entity correspondence cues . one-sided semantic understanding relying solely on argument role semantics a problem .
Approach:	They propose an EAE model with stage-customized entity type embedding to explore the role of entity types.
Outcome:	The proposed model achieves state-of-the-art performance on mainstream benchmarks and robustness in low-resource settings.

Copied to clipboard

Sheila Castilho

Multilinguality

Challenge:	Recent advances in neural machine translation (NMT) have made it possible to include discourse into translation systems.
Approach:	They use the DELA corpus to examine the context span needed to translate from English into Portuguese.
Outcome:	The shortest span to disambiguate issues can appear in different positions in the document including preceding, following, global, world knowledge.

Packed Levitated Marker for Entity and Relation Extraction (2022.acl-long)

Copied to clipboard

Deming Ye, Yankai Lin, Peng Li, Maosong Sun

Zero-shot Cross-lingual Alignment for Embedding Initialization (2024.findings-acl)

Challenge:	Existing work on entity and relation extraction ignores the interrelation between spans . a novel approach to extract better span representations from pre-trained languages is needed .
Approach:	They propose a span representation approach that packs Levitated Markers to consider interrelation between spans.
Outcome:	The proposed model improves on baselines on six NER benchmarks and achieves a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models.

Copied to clipboard

Xi Ai, Zhiyong Huang

ReFACT: A Benchmark for Scientific Confabulation Detection with Positional Error Annotations (2026.eacl-long)

Challenge:	CrossInit initializes embeddings into similar geometrical structures across languages in unsupervised manner.
Approach:	They propose a method that initializes embeddings into similar geometrical structures across languages in an unsupervised manner.
Outcome:	The proposed method demostrates similar patterns in low-resource and dissimilar languages.

Copied to clipboard

Yindong Wang, Martin Preiß, Margarita Bugueño, Jan Vincent Hoffbauer, Abdullatif Ghajar, Tolga Buz, Gerard de Melo

Challenge:	Evaluating 9 state-of-the-art LLMs reveals two critical limitations: 61% of incorrect span predictions are semantically unrelated to actual errors.
Approach:	They propose a benchmark of 1,001 expert-annotated question-answer pairs with span-level error annotations derived from Reddit's r/AskScience.
Outcome:	Evaluating 9 state-of-the-art LLMs, we find that comparative judgment is paradoxically harder than independent detection when comparing answers side-by-side.

Dissecting Span Identification Tasks with Performance Prediction (2020.emnlp-main)

Copied to clipboard

Sean Papay, Roman Klinger, Sebastian Padó

Challenge:	Span identification tasks are a staple of applied NLP, but there is little insight on how their properties influence their difficulty.
Approach:	They propose to build a model to predict span ID performance for unseen span ID tasks that can support architecture choices.
Outcome:	The proposed model predicts span ID tasks for unseen span ID task in English, and the meta model predictable span ID performance.

Parsing All: Syntax and Semantics, Dependencies and Spans (2020.findings-emnlp)

Copied to clipboard

Junru Zhou, Zuchao Li, Hai Zhao

Syntactic Text Processing

Challenge:	Syntactic and semantic structures are key linguistic contextual clues, but few studies have explored how they can be used to improve syntactical parsing.
Approach:	They propose a syntactic and semantic parsing model which integrates syntaktic information in the encoder of neural network and benefits from two representation formalisms in a uniform way.
Outcome:	The proposed model achieves state-of-the-art or competitive results on both span and dependency representations and on Penn Treebank.

LIMIT-BERT : Linguistics Informed Multi-Task BERT (2020.findings-emnlp)

Copied to clipboard

Junru Zhou, Zhuosheng Zhang, Hai Zhao, Shuailiang Zhang

SpanAlign: Sentence Alignment Method based on Cross-Language Span Prediction and ILP (2020.coling-main)

Challenge:	Existing language models are usually trained on large amounts of unlabeled text data.
Approach:	They propose a multi-task language representations learning framework for multi-linguistics tasks by Multi-Task Learning.
Outcome:	The proposed model outperforms the baseline Whole Word Masking BERT on both dependency and constituent syntactic/semantic parsing, GLUE benchmark, and SNLI task.

Copied to clipboard

Katsuki Chousa, Masaaki Nagata, Masaaki Nishino

Multilinguality

Challenge:	Existing methods for automatic sentence alignment assume monotonic alignments, but they can handle non-monotonic alignments.
Approach:	They propose a method to automatically extract parallel sentences from noisy parallel documents by embeddings and encoding each source and target sentence.
Outcome:	The proposed method improves translation accuracy by 4.1 BLEU scores on English-Japanese . it can predict spans in target document from sentences in source document .

Dialogue-Based Relation Extraction (2020.acl-main)

Copied to clipboard

Dian Yu, Kai Sun, Claire Cardie, Dong Yu

An Empirical Study of Span Representations in Argumentation Structure Parsing (P19-1)

Challenge:	Existing dialogue-based relation extraction tasks focus on texts from formal genres such as professionally written and edited news reports or well-edited websites.
Approach:	They propose to use DialogRE to study cross-sentence relation extraction . they propose to annotate 36 possible relation types between arguments in dialogues .
Outcome:	The proposed dataset supports the prediction of relation(s) between two arguments that appear in a dialogue.

Copied to clipboard

Tatsuki Kuribayashi, Hiroki Ouchi, Naoya Inoue, Paul Reisert, Toshinori Miyoshi, Jun Suzuki, Kentaro Inui

Representation Learning

Challenge:	Argumentation structure parsing (ASP) is a task of identifying argumentation structures in argumentative text.
Approach:	They propose to exploit neural network-based span representations for ASP to improve performance . they also propose task-dependent extensions for a parser that can be used to parse arguments .
Outcome:	The proposed model outperforms neural network-based approaches for argumentation structure parsing (ASP) it also provides some challenging types of instances to be parsed.

Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction (2022.acl-long)

Copied to clipboard

Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, Jing Shao

TGEA: An Error-Annotated Dataset and Benchmark Tasks for TextGeneration from Pretrained Language Models (2021.acl-long)

Challenge:	Using a prompt-based model, we find that event argument extraction is efficient and generalized well to few-shot settings.
Approach:	They propose a model PAIE for event argument extraction using prompt tuning for extractive objectives.
Outcome:	The proposed model can extract arguments with the same role instead of heuristic threshold tuning.

Copied to clipboard

Jie He, Bo Peng, Yi Liao, Qun Liu, Deyi Xiong

Knowledgeable Parameter Efficient Tuning Network for Commonsense Question Answering (2023.acl-long)

Challenge:	Using pretrained language models, we propose an error-annotated dataset for text generation . we use carefully selected prompt words to guide GPT-2 to generate candidate sentences .
Approach:	They propose an error-annotated dataset with multiple benchmark tasks for text generation from pretrained language models.
Outcome:	The proposed dataset covers 24 types of errors according to common sense and linguistics.

Copied to clipboard

Ziwang Zhao, Linmei Hu, Hanyu Zhao, Yingxia Shao, Yequan Wang

Compressive Summarization with Plausibility and Salience Modeling (2020.emnlp-main)

Challenge:	Existing commonsense question answering models incur prohibitive computation costs and poor interpretability .
Approach:	They propose a parameter efficient tuning network to pair PLMs with external knowledge for commonsense question answering.
Outcome:	The proposed adapter integrates entity- and query-related knowledge at a small cost.

Copied to clipboard

Shrey Desai, Jiacheng Xu, Greg Durrett

Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search (2021.acl-long)

Challenge:	a new method to learn which compressions to apply is based on syntactic rules for deleting spans . plausibility and salience are the two main criteria for determining which compression to apply . a recent study shows that the plausability model generally selects for grammatical and factual deletions compared to extractive methods .
Approach:	They propose to leave the decision about what to delete to two data-driven criteria . they show that plausibility and salience are the most important criteria if a span is deleted .
Outcome:	The proposed method achieves strong in-domain results on benchmark datasets and human evaluation shows that plausibility model generally selects for grammatical and factual deletions.

Copied to clipboard

Gyuwan Kim, Kyunghyun Cho

Information Retrieval

Challenge:	Pre-trained language models have achieved notable improvements in various NLP tasks.
Approach:	They propose a Length-Adaptive Transformer that can be used for various inference scenarios after one-shot training.
Outcome:	The proposed model can be used for various inference scenarios after one-shot training.

SA-DETR:Span Aware Detection Transformer for Moment Retrieval (2025.coling-main)

Copied to clipboard

Tianheng Xiong, Wei Wei, Kaihe Xu, Dangyang Chen

Empowering the Fact-checkers! Automatic Identification of Claim Spans on Twitter (2022.emnlp-main)

Challenge:	Moment Retrieval aims to locate video segments related to text.
Approach:	They propose a method that leverages the importance of instance related span anchors . they initialize span anchor using instance related fuse token and supervise them with GT labels .
Outcome:	The proposed method achieves competitive results on QVHighlights, Charades-STA and TACoS.

Copied to clipboard

Megha Sundriyal, Atharva Kulkarni, Vaibhav Pulastya, Md. Shad Akhtar, Tanmoy Chakraborty

AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation (2024.findings-emnlp)

Challenge:	Current vogue is to employ manual fact-checkers to efficiently classify and verify such data to combat this avalanche of misinformation and fake news.
Approach:	They propose a large-scale Twitter corpus with token-level claim spans on more than 7.5k tweets and a model that automatically detects and extracts the snippets of misinformation.
Outcome:	The proposed model outperforms baseline systems on several evaluation metrics, improving by 1.5 points.

Copied to clipboard

Zhitao He, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Jiexin Xu, Huaijun Li, Kang Liu, Jun Zhao

Hit the Sweet Spot! Span-Level Ensemble for Large Language Models (2025.coling-main)

Challenge:	Recent advances in deep learning have significantly impacted the legal domain.
Approach:	They propose a multi-agent framework for judicial decision-making that simulates the court trial process . they propose 420 Chinese judgment documents to support their framework and build a large-scale legal knowledge base .
Outcome:	The proposed framework outperforms existing methods in various aspects, especially in generating legal articles.

Copied to clipboard

Yangyifan Xu, Jianghao Chen, Junhong Wu, Jiajun Zhang

Robustness in NLP

Challenge:	a recent study focused on sample-level and token-level ensembles, which hinder dynamic correction and enhancement of outputs during the generation process.
Approach:	They propose a span-level ensemble method that balances real-time adjustments and accurate ensemble decisions.
Outcome:	The proposed method improves performance across language generation tasks significantly.

Entity, Relation, and Event Extraction with Contextualized Span Representations (D19-1)

Copied to clipboard

David Wadden, Ulme Wennberg, Yi Luan, Hannaneh Hajishirzi

Split or Merge: Which is Better for Unsupervised RST Parsing? (D19-1)

Challenge:	Existing frameworks for named entity recognition, relation extraction, and event extraction can be easily adapted for new tasks or datasets.
Approach:	They propose a framework that enumerates, refins, and scores text spans to capture local (within-sentence) and global (cross-sentent) context.
Outcome:	The proposed framework achieves state-of-the-art results on four datasets from a variety of domains.

Copied to clipboard

Naoki Kobayashi, Tsutomu Hirao, Kengo Nakamura, Hidetaka Kamigaito, Manabu Okumura, Masaaki Nagata

Deep Span Representations for Named Entity Recognition (2023.findings-acl)

Challenge:	Rhetorical Structure Theory (RST) parsers have been based on supervised learning approaches that require an annotated corpus of sufficient size and quality.
Approach:	They propose two unsupervised methods that build an optimal RST tree based on a dissimilarity score function for splitting a text span into smaller ones and a similarity score for merging two adjacent spans into a large one.
Outcome:	The proposed method achieves the best score on English and German RST treebanks, around 0.8 F1 score, close to the previous supervised parsers.

Copied to clipboard

Enwei Zhu, Yiyang Liu, Jinpeng Li

Downstream Datasets Make Surprisingly Good Pretraining Corpora (2023.acl-long)

Challenge:	Existing span-based NER systems shallowly aggregate token representations to span representations, causing performance degradations.
Approach:	They propose a method that shallowly aggregates token representations to span representations . DSpERT produces span representation of deep semantics with weight initialization .
Outcome:	The proposed model performs better than or competitive with state-of-the-art systems on six NER benchmarks.

Copied to clipboard

Kundan Krishna, Saurabh Garg, Jeffrey Bigham, Zachary Lipton

Improving Span Representation by Efficient Span-Level Attention (2023.findings-emnlp)

Challenge:	a dominant practice is to fine tune large pretrained transformer models using smaller downstream datasets . performance gains are not always attributable to the use of external data in massive amounts .
Approach:	They propose to use the same (downstream) training data for pretraining and finetuning to compare models.
Outcome:	The proposed model outperforms standard pretraining on the BookWiki corpus on 7 and 5 datasets.

Copied to clipboard

Pengyu Ji, Songlin Yang, Kewei Tu

PersonaLM: Language Model Personalization via Domain-distributed Span Aggregated K-Nearest N-gram Retrieval Augmentation (2023.findings-emnlp)

Challenge:	Existing methods for generating high-quality span representations are limited by subset of tokens . span-span interactions should play an important role in span encoding, authors argue .
Approach:	They propose to introduce span-span interactions and more comprehensive span-token interactions to improve span representations.
Outcome:	The proposed model outperforms baseline models on span-related tasks and shows superior performance.

Copied to clipboard

Puneet Mathur, Zhe Liu, Ke Li, Yingyi Ma, Gil Keren, Zeeshan Ahmed, Dinesh Manocha, Xuedong Zhang

Subject-level Inference for Realistic Text Anonymization Evaluation (2026.acl-long)

Challenge:	Existing language modeling tools for automatic speech recognition (ASR) are difficult to personalize.
Approach:	They propose a domain-distributed Span-Aggregated K-nearest N-gram retrieval augmentation to improve language modeling for automatic speech recognition (ASR) personalization.
Outcome:	The proposed model outperforms baselines on Wikitext-103, UserLibri, and ASAP datasets with a 10-16% improvement in perplexity and a 5-8% reduction in word error rates.

Copied to clipboard

Myeong Seok Oh, Dong-Yun Kim, Hanseok Oh, Chaean Kang, Joeun Kang, Xiaonan Wang, Hyunjung Park, Young Cheol Jung, Hansaem Kim

AttenWalker: Unsupervised Long-Document Question Answering via Attention-based Graph Walking (2023.findings-acl)

Challenge:	Existing text anonymization evaluations assume only a single data subject, ignoring multi-subject scenarios.
Approach:	They propose a benchmark that shifts the unit of evaluation from text spans to individuals . they show that subject-level inference protection drops as low as 33% when masked .
Outcome:	The proposed benchmark reduces the amount of protection available when PII spans are masked.

Copied to clipboard

Yuxiang Nie, Heyan Huang, Wei Wei, Xian-Ling Mao

Low-Resource NLP

Challenge:	Existing methods for annotating long-document question answering are based on short documents and can hardly incorporate long-range information.
Approach:	They propose an unsupervised method to generate long-document question answering pairs . they propose a method to aggregate and generate answers with long-range dependency .
Outcome:	The proposed method outperforms existing methods on NarrativeQA and Qasper.

A Boundary Offset Prediction Network for Named Entity Recognition (2023.findings-emnlp)

Copied to clipboard

Minghao Tang, Yongquan He, Yongxiu Xu, Hongbo Xu, Wenyuan Zhang, Yang Lin

Membership and Memorization in LLM Knowledge Distillation (2025.emnlp-main)

Challenge:	Named entity recognition (NER) is a fundamental task in natural language processing . span-based methods assign entity types to text spans, resulting in imbalanced sample space .
Approach:	They propose a method that predicts boundary offsets between candidate and nearest spans . the method integrates entity type and span representations to generate type-aware boundary offset .
Outcome:	The proposed method outperforms existing methods on eight widely-used NER datasets.

Copied to clipboard

Ziqi Zhang, Ali Shahin Shamsabadi, Hanxiao Lu, Yifeng Cai, Hamed Haddadi

Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals (2024.lrec-main)

Challenge:	Recent advances in Knowledge Distillation (KD) aim to mitigate the high computational demands of Large Language Models (LLMs).
Approach:	They characterize and investigate membership privacy risks inherent in six LLM KD techniques . they use instruction-tuning settings that span seven NLP tasks and three teacher model families and various size student models to examine the extent of privacy risks.
Outcome:	The proposed methods carry membership and memorization privacy risks from the teacher to students, but differ across different techniques.

Copied to clipboard

Rui Zheng, Yuhao Zhou, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang

Robustness in NLP

Challenge:	Existing models that extract discrete inputs into fixed-length representations are vulnerable to adversarial attacks that place perturbations on clean inputs to fool DNNs.
Approach:	They propose to inspect the subspaces of sample features through spectral analysis to better understand adversarial attacks.
Outcome:	The proposed strategy enables the model to inherently suppress adversaries, which boosts model robustness and motivates new directions of effective adversarial defense.

A Multi-Level Benchmark for Causal Language Understanding in Social Media Discourse (2025.emnlp-main)

Copied to clipboard

Xiaohan Ding, Kaike Ping, Buse Çarık, Eugenia Rho

Reasoning

Challenge:	Existing datasets focus on explicit causality in structured text, providing limited support for detecting implicit causal expressions.
Approach:	They propose a dataset of Reddit posts annotated across four causal tasks . they use a binary causal classification, explicit vs. implicit causality, cause–effect span extraction and causal gist generation to bridge causal detection and reasoning over informal discourse.
Outcome:	The proposed dataset analyzes 10,120 Reddit posts discussing public health related to the COVID-19 pandemic.

SRA: Span Representation Alignment for Large Language Model Distillation (2026.acl-long)

Copied to clipboard

Quoc Phong Dao, Hoang Son Nguyen, Pham Khanh Chi, Tung Nguyen, Linh Ngo Van, Nguyen Thi Ngoc Diep, Trung Le