Papers by Lin Qiu

Exploring Diverse Expressions for Paraphrase Generation (D19-1)

Challenge:	Existing benchmarks for large language models (LLMs) do not accurately uncover safety vulnerabilities in LLMs.
Approach:	They propose a value alignment benchmark called Flames that encompasses both harmlessness principles and a unique morality dimension that integrates specific Chinese values such as harmony.
Outcome:	The proposed model performs poorly on Flames, particularly in safety and fairness dimensions.

Copied to clipboard

Lihua Qian, Lin Qiu, Weinan Zhang, Xin Jiang, Yong Yu

Text Generation

Challenge:	Existing neural paraphrase generation methods focus on single paraphrases while ignoring the fact that diversity is essential for enhancing generalization capability and robustness of downstream applications.
Approach:	They propose a novel approach with two discriminators and multiple generators to generate a variety of different paraphrases.
Outcome:	The proposed model gains significant diversity and improves quality over state-of-the-art datasets.

LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (2023.emnlp-main)

Copied to clipboard

Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu

Balanced Data Sampling for Language Model Training with Clustering (2024.findings-acl)

Challenge:	Large language models (LLMs) are increasingly lengthy and require longer prompts . this paper presents a coarse-to-fine prompt compression method to reduce cost and increase performance.
Approach:	They propose a coarse-to-fine prompt compression method that maintains semantic integrity under high compression ratios and a token-level iterative compression algorithm to better model the interdependence between compressed contents.
Outcome:	The proposed method yields state-of-the-art performance and allows for up to 20x compression with little performance loss over four datasets from different scenarios.

Copied to clipboard

Yunfan Shao, Linyang Li, Zhaoye Fei, Hang Yan, Dahua Lin, Xipeng Qiu

StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding (2023.emnlp-main)

Challenge:	Large Language Models (LLMs) are a fundamental part of the training process.
Approach:	They propose to use clustering to balance the text distribution of training data for better model training.
Outcome:	Extensive experiments validate the effectiveness of ClusterClip Sampling under various training datasets and large language models.

Copied to clipboard

Cheng Jiayang, Lin Qiu, Tsz Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, Zheng Zhang

Unified Demonstration Retriever for In-Context Learning (2023.acl-long)

Challenge:	Analogy-making between narratives is crucial for human reasoning . despite its importance, there has been limited research on story analogies .
Approach:	They construct a large-scale story-level analogy corpus with 24K story pairs . they find that the tasks are incredibly difficult for large language models such as ChatGPT .
Outcome:	The proposed corpus contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory.

Copied to clipboard

Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, Xipeng Qiu

Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration (2025.acl-long)

Challenge:	In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction.
Approach:	They propose a single model to retrieve demonstrations for a wide range of tasks by combining training signals from various tasks into a unified list-wise ranking formulation by language model’s feedback.
Outcome:	The proposed model outperforms baselines on 30+ tasks across 13 task families and multiple data domains.

Copied to clipboard

Shao Zhang, Xihuai Wang, Wenhao Zhang, Chaoran Li, Junru Song, Tingyu Li, Lin Qiu, Xuezhi Cao, Xunliang Cai, Wen Yao, Weinan Zhang, Xinbing Wang, Ying Wen

Linguistic Theories

Challenge:	Large language models (LLMs) excel in turn-by-turn human-AI collaboration but struggle with simultaneous tasks requiring real-time interaction.
Approach:	They propose a language agent framework that integrates System 1 and System 2 for efficient real-time simultaneous human-AI collaboration.
Outcome:	The proposed framework improves on existing LLM-based agents and human collaborators by integrating Theory of Mind and asynchronous reflection to infer human intentions and perform reasoning-based autonomous decisions.

Distributed Marker Representation for Ambiguous Discourse Markers and Entangled Relations (2023.acl-long)

Copied to clipboard

Dongyu Ru, Lin Qiu, Xipeng Qiu, Yue Zhang, Zheng Zhang

Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus (2023.emnlp-main)

Challenge:	Discourse markers are natural representations of discourse in our daily language.
Approach:	They propose to use unlimited discourse marker data to learn a Distributed Marker Representation by bridging markers with sentence pairs.
Outcome:	The proposed model outperforms existing models on the implicit discourse relation recognition task and provides strong interpretability.

Copied to clipboard

Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, Luoyi Fu

Instance-level Randomization: Toward More Stable LLM Evaluations (2025.findings-emnlp)

Challenge:	Existing methods for detecting hallucinations in LLMs rely on external knowledge for reference retrieval or require sampling multiple responses for consistency verification.
Approach:	They propose a reference-free, uncertainty-based method for detecting hallucinations in Large Language Models that imitates human focus in factuality checking from three aspects: focus on the most informative keywords; focus on unreliable tokens in historical context; focus of token properties such as token type and token frequency.
Outcome:	The proposed method achieves state-of-the-art performance across all evaluation metrics and eliminates the need for additional information.

Copied to clipboard

Yiyang Li, Yonghuang Wu, Ying Luo, Liangtai Sun, Zishu Qin, Lin Qiu, Xuezhi Cao, Xunliang Cai

Code Needs Comments: Enhancing Code LLMs with Comment Augmentation (2024.findings-acl)

Challenge:	Evaluations of large language models suffer from instability, where small changes of random factors can lead to drastic fluctuations of scores and even model rankings.
Approach:	They propose an instance-level randomization method to reduce variance and improve fairness in evaluations by randomizing all factors that affect evaluation scores for every single instance.
Outcome:	The proposed method reduces variance and improves fairness in model comparisons by using instance-level randomization.

Copied to clipboard

Demin Song, Honglin Guo, Yunhua Zhou, Shuhao Xing, Yudong Wang, Zifan Song, Wenwei Zhang, Qipeng Guo, Hang Yan, Xipeng Qiu, Dahua Lin

Programming Languages in NLP

Challenge:	Large Language Models (LLMs) require a deep understanding of programming languages and their correlation with natural languages (NLs).
Approach:	They propose a data augmentation method that generates comments for existing code and a filtering strategy that filters out code data poorly correlated with natural language.
Outcome:	The proposed method outperforms the model trained on the augmented data and the model further trained on data without augmentation on two widely-used programming skill benchmarks.

Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models? (2024.emnlp-main)

Copied to clipboard

Siyu Yuan, Cheng Jiayang, Lin Qiu, Deqing Yang

Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Space (2020.findings-emnlp)

Challenge:	Analogical reasoning plays a critical role in human cognition, enabling us to understand new concepts by associating them with familiar ones.
Approach:	They propose to use free-form analogies to aid students in understanding scientific concepts . they also show that analogies generated by student LMs can improve their own performance .
Outcome:	The proposed model can help students understand scientific concepts, the authors show .

Copied to clipboard

Dongyu Ru, Jiangtao Feng, Lin Qiu, Hao Zhou, Mingxuan Wang, Weinan Zhang, Yong Yu, Lei Li

Turn Waste into Worth: Rectifying Top-k Router of MoE (2024.emnlp-main)

Challenge:	Existing uncertainty sampling methods are time-consuming and can't be executed frequently.
Approach:	They propose adversarial uncertainty sampling in discrete space to find informative unlabeled text samples for annotation using adversarials.
Outcome:	The proposed approach outperforms baselines on effectiveness on five datasets.

Copied to clipboard

Zhiyuan Zeng, Qipeng Guo, Zhaoye Fei, Zhangyue Yin, Yunhua Zhou, Linyang Li, Tianxiang Sun, Hang Yan, Dahua Lin, Xipeng Qiu

Green & Sustainable NLP

Challenge:	Top-k router suffers from redundancy computation and memory costs due to unbalanced routing . some experts are overflow, where exceeding tokens are dropped, while others are empty, which are padded with zeros, negatively impacting model performance.
Approach:	They propose a top-k router that is unbalanced and uses a multi-gPU system to handle dropped tokens and padding.
Outcome:	The proposed model surpasses the top-1 router by 4.7% in terms of performance . the top-k router suffers from redundancy computation and memory costs .

Multi-Programming Language Sandbox for LLMs (2025.acl-demo)

Copied to clipboard

Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang

Evaluating Psychological Safety of Large Language Models (2024.emnlp-main)

Challenge:	MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs).
Approach:	They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models.
Outcome:	The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs).

Copied to clipboard

Xingxuan Li, Yutong Li, Lin Qiu, Shafiq Joty, Lidong Bing

Minos: A Multimodal Evaluation Model for Bidirectional Generation Between Image and Text (2026.findings-acl)

Challenge:	a recent study evaluated the psychological safety of large language models.
Approach:	They designed unbiased prompts to evaluate the psychological safety of large language models.
Outcome:	The proposed prompts showed that they were fine-tuned with behavioral metrics to reduce toxicity.

Copied to clipboard

Junzhe Zhang, Huixuan Zhang, Xinyu Hu, Li Lin, Mingqi Gao, Shi Qiu, Xiaojun Wan

Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce (P18-2)

Challenge:	Existing evaluation models struggle to achieve consistent performance across image-to-text (I2T) and text-to image (T2I) tasks.
Approach:	They construct a multimodal evaluation model using a large multimodal dataset and rigorous quality control strategies to train it.
Outcome:	The proposed model achieves state-of-the-art evaluation performance across 16 out-of domain datasets covering both I2T and T2I tasks among all open-source multimodal evaluation models and remain competitive with closed-source models.

Copied to clipboard

Minghui Qiu, Liu Yang, Feng Ji, Wei Zhou, Jun Huang, Haiqing Chen, Bruce Croft, Wei Lin

Natural Language Interfaces

Challenge:	Recent researches focus on deep learning and reinforcement learning for multi-turn information seeking conversation systems.
Approach:	They propose an efficient and effective multi-turn conversation model based on convolutional neural networks and extend it to adapt the knowledge learned from a resource-rich domain to enhance the performance.
Outcome:	The proposed model performs better than the existing model on an industrial chatbot called AliMe Assist.

Mitigate Position Bias in LLMs via Scaling a Single Hidden States Channel (2025.findings-acl)

Copied to clipboard

Yijiong Yu, Huiqiang Jiang, Xufang Luo, Qianhui Wu, Chin-Yew Lin, Dongsheng Li, Yuqing Yang, Yongfeng Huang, Lili Qiu

Challenge:	Long-context language models exhibit position bias, also known as "lost in the middle" research shows that even long-contemporary LLMs fail to utilize all context information effectively .
Approach:	They propose a method to mitigate position bias by scaling positional hidden states . they propose to use a channel of hidden states to modify positional Hidden states a LCLM's positional bias .
Outcome:	The proposed method can improve performance by 15.2% in a "lost in the middle" benchmark.

LongWanjuan: Towards Systematic Measurement for Long Text Quality (2024.findings-emnlp)

Copied to clipboard

Xiaoran Liu, Kai Lv, Qipeng Guo, Hang Yan, Conghui He, Xipeng Qiu, Dahua Lin

Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)

Challenge:	Existing efforts to improve data quality have focused on deduplication and the evaluation of data diversity and difficulty.
Approach:	They propose a set of metrics to evaluate the quality of long texts by evaluating three fundamental linguistic dimensions: coherence, cohesion, and complexity.
Outcome:	The proposed model improves on long-text tasks with over 160B tokens and categorizes long texts into holistic, aggregated, and chaotic types.

Copied to clipboard

Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong

Robustness in NLP

Challenge:	Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise.
Approach:	They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble.
Outcome:	The proposed technique improves ensemble performance and robustness against erroneous signals.

SnapNTell: Enhancing Entity-Centric Visual Question Answering with Retrieval Augmented Multimodal LLM (2024.findings-emnlp)

Copied to clipboard

Jielin Qiu, Andrea Madotto, Zhaojiang Lin, Paul Crook, Yifan Xu, Babak Damavandi, Xin Dong, Christos Faloutsos, Lei Li, Seungwhan Moon

PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)

Challenge:	Vision-extended LLMs have made significant strides in VQA, but they still encounter significant difficulties in handling queries involving long-tail entities.
Approach:	They propose a benchmark to test models' ability to identify entities and provide detailed, entity-specific knowledge by combining 10 images and 10 knowledge-intensive QA pairs.
Outcome:	The proposed model outperforms existing methods on the SnapNTell dataset, achieving a 66.5% improvement in the BELURT score.

Copied to clipboard

Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei

Attribution-Based Analysis and Optimization of Modular Agentic Workflows (2026.findings-acl)

Challenge:	Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning.
Approach:	They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios.
Outcome:	The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics.

Copied to clipboard

Yingxuan Yang, Bo Huang, Siyuan Qi, Chao Feng, Haoyi Hu, Yuxuan Zhu, Jinbo Hu, Haoran Zhao, Ziyi He, Xiao Liu, ZongYu Wang, Muning Wen, Lin Qiu, Xuezhi Cao, Xunliang Cai, Yong Yu, Weinan Zhang

Challenge:	Large Language Models (LLMs) have driven the rise of agentic workflows . yet, how can we attribute performance gains to individual upgrades and their interactions?
Approach:	They propose a game-theoretic framework that models component upgrades as players and evaluates component coalitions to compute Shapley values.
Outcome:	The proposed framework provides interaction-aware attribution and recommendation for model allocation under a fixed workflow structure.

PersLEARN: Research Training through the Lens of Perspective Cultivation (2023.acl-demo)

Copied to clipboard

Yu-Zhe Shi, Shiqian Li, Xinyi Niu, Qiao Xu, Jiawen Liu, Yifan Xu, Shiyu Gu, Bingru He, Xinyang Li, Xinyu Zhao, Zijian Zhao, Yidong Lyu, Zhen Li, Sijia Liu, Lin Qiu, Jinhao Ji, Lecheng Ruan, Yuxi Ma, Wenjuan Han, Yixin Zhu

Challenge:	PersLEARN is a tool designed to facilitate the cultivation of scientific perspectives . junior researchers struggle to identify the perspectives reflected in the literature and struggle to develop their own viewpoints.
Approach:	They propose a tool to facilitate the cultivation of scientific perspectives by interacting with a prompt-based model and allowing students to develop their own perspectives explicitly.
Outcome:	The proposed tool outperforms baseline approaches across multiple domains of literature from different perspectives.

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models (2025.acl-long)

Copied to clipboard

Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin

Programming Languages in NLP

Challenge:	Existing Mixture-of-Experts training frameworks use a micro-batch to calculate LBL . micro-batches are restricted to a single sequence, preventing expert specialization .
Approach:	They propose to use a global-batch to loosen the load balance constraint for MoEs models . they propose to synchronize fi across micro-batches and then use it to calculate the LBL .
Outcome:	The proposed global-batch LBL improves the domain specialization of experts . the micro-battery LBL is almost at the sequence level, and the router is pushed to distribute the token evenly .

MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models (2026.findings-acl)

Copied to clipboard

Boqi Chen, Xudong Liu, Jiachuan Peng, Marianne Frey-Marti, Kyle Lam, Bang Zheng, Lin Li, Jianing Qiu

Over-Searching in Search-Augmented Large Language Models (2026.eacl-long)

Challenge:	Existing benchmarks for multimodal large language models do not capture real-world clinical complexity.
Approach:	They evaluate multilingual, multimodal multimodal models of clinical cases with up to 7 distinct visual clinical evidence types per case.
Outcome:	The proposed model outperforms human models on differential diagnosis (DDx) generation and final diagnosis (FDx) selection.

Copied to clipboard

Roy Xie, Deepak Gopinath, David Qiu, Dong Lin, Haitian Sun, Saloni Potdar, Bhuwan Dhingra

Multimedia Event Extraction with LLM Knowledge Editing (2025.emnlp-main)

Challenge:	Search-augmented large language models (LLMs) excel at knowledge-intensive tasks by integrating external retrieval.
Approach:	They conduct a systematic evaluation of over-searching across multiple dimensions including query types, model categories, retrieval conditions, and multi-turn conversations.
Outcome:	The proposed model improves answer accuracy on answerable queries but harms abstention on unanswerable ones .

Copied to clipboard

Jiaao Yu, Yijing Lin, Zhipeng Gao, Xuesong Qiu, Lanlan Rui

Event Extraction

Challenge:	Existing multimodal event extraction methods focus on weakly aligning features from wellpretrained unimodal encoders, resulting in redundant feature perception.
Approach:	They propose a multimodal event extraction strategy with a redundant feature selection mechanism that enhances event understanding ability of multimodal large language models.
Outcome:	The proposed method outperforms the state-of-the-art (SOTA) baselines on the M2E2 benchmark.

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards (2024.acl-long)

Copied to clipboard

Haoxiang Wang, Yong Lin, Wei Xiong, Rui Yang, Shizhe Diao, Shuang Qiu, Han Zhao, Tong Zhang

Challenge:	Reinforcement Learning from Human Feedback (RLHF) relies on scalar rewards to capture user preferences.
Approach:	They propose a framework that integrates multi-objective reward modeling to represent diverse preference profiles.
Outcome:	The proposed method improves performance across reward objectives and targets.

Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information (2020.emnlp-main)

Copied to clipboard

Zehui Lin, Xiao Pan, Mingxuan Wang, Xipeng Qiu, Jiangtao Feng, Hao Zhou, Lei Li

Multilinguality

Challenge:	Existing pre-training methods are not effective for machine translation tasks.
Approach:	They propose a method to pre-train a universal multilingual neural machine translation model . they use random aligned substitution technique to bring words and phrases with similar meanings closer in the representation space.
Outcome:	The proposed approach improves translation quality on low, medium, rich resource languages.

Knowledge-Centric Hallucination Detection (2024.emnlp-main)

Copied to clipboard

Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Yue Zhang, Zheng Zhang

EasyNLP: A Comprehensive and Easy-to-use Toolkit for Natural Language Processing (2022.emnlp-demos)

Challenge:	Large Language Models (LLMs) have shown impressive capabilities but a tendency to hallucinate.
Approach:	They propose a framework that introduces claim-triplets to represent claims in LLM responses and evaluates them against a reference.
Outcome:	The proposed framework outperforms prior methods by 18.2 to 27.2 points on a benchmark spanning various NLP tasks and annotated 11k claim-triplets from 2.1k responses by seven LLMs.

Copied to clipboard

Chengyu Wang, Minghui Qiu, Taolin Zhang, Tingting Liu, Lei Li, Jianing Wang, Ming Wang, Jun Huang, Wei Lin

Learning Logic Rules for Document-Level Relation Extraction (2021.emnlp-main)

Challenge:	Pre-Trained Models (PTMs) have reshaped the development of natural language processing (NLP) but it is not easy to obtain high-performing PTMs without a large amount of labeled training data and deploy them online with fast inference speed.
Approach:	They propose to make it easy to build NLP applications with knowledge-enhanced pre-training and knowledge distillation.
Outcome:	EasyNLP supports a comprehensive suite of NLP algorithms and features knowledge-enhanced pre-training, knowledge distillation and few-shot learning functionalities.

Copied to clipboard

Dongyu Ru, Changzhi Sun, Jiangtao Feng, Lin Qiu, Hao Zhou, Weinan Zhang, Yong Yu, Lei Li

Information Extraction & Text Mining

Challenge:	Existing models for document-level relation extraction relied on implicitly powerful representations, which makes the model less transparent.
Approach:	They propose a probabilistic model for document-level relation extraction by learning logic rules.
Outcome:	The proposed model outperforms baseline models in relation performance and logical consistency.

MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)

Copied to clipboard

Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie

NumNet: Machine Reading Comprehension with Numerical Reasoning (D19-1)

Challenge:	Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models.
Approach:	They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts.
Outcome:	The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings .

Copied to clipboard

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, Zhiyuan Liu

OpenResearcher: Unleashing AI for Accelerated Scientific Research (2024.emnlp-demo)

Challenge:	Existing numerical MRC models are weak in numerical reasoning, such as addition, subtraction, sorting and counting.
Approach:	They propose a numerical MRC model that integrates numerical reasoning into existing MRC models and achieves an EM-score of 64.56% on the DROP dataset.
Outcome:	The proposed model outperforms all existing machine reading comprehension models by considering the numerical relations among numbers on the DROP dataset.

Copied to clipboard

Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, Pengfei Liu

Information Retrieval

Challenge:	Global scientific publications are growing annually by about 4%-5% (Pinedo et al., 2024).
Approach:	They introduce an AI-assisted platform that answers diverse questions from researchers using Retrieval-Augmented Generation (RAG) they develop various tools to understand queries, search from the scientific literature, filter retrieved information, provide accurate and comprehensive answers, and self-refine answers.
Outcome:	OpenResearcher is built on Retrieval-Augmented Generation (RAG) to integrate Large Language Models (LLMs) with up-to-date, domain-specific knowledge.

Dynamically Fused Graph Network for Multi-hop Reasoning (P19-1)

Copied to clipboard

Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, Yong Yu

F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods (2024.acl-long)

Challenge:	Text-based question answering (TBQA) has been studied extensively in recent years.
Approach:	They propose a Dynamically Fused Graph Network to answer questions requiring multiple scattered evidence and reasoning over them.
Outcome:	The proposed method achieves competitive results on a public TBQA dataset and produces interpretable reasoning chains.

Copied to clipboard

Yu Sun, Keyuchen Keyuchen, Shujie Wang, Peiji Li, Qipeng Guo, Hang Yan, Xipeng Qiu, Xuanjing Huang, Dahua Lin

Challenge:	Large language models (LLMs) have been evaluated for their instruction-following capabilities but lack references to their fundamental abilities.
Approach:	They propose a bilingual evaluation benchmark to evaluate the fundamental abilities of large language models including expression, commonsense and logic.
Outcome:	The proposed evaluation methods show higher correlation coefficients and larger distinction than other evaluators.

EgoMemory: Memory-Augmented Personalized Retrieval for Long-Context Egocentric Video (2026.findings-acl)

Copied to clipboard

Yuanmin Tang, Jue Zhang, Xiaoting Qin, Jing Yu, Meikang Qiu, Gaopeng Gou, Gang Xiong, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Wu

N-GLARE: An Non-Generative Latent Representation-Efficient LLM Safety Evaluator (2026.acl-long)

Challenge:	Existing egocentric video datasets do not support the personalization and long-context reasoning required for episodic memory retrieval.
Approach:	They propose a benchmark framework that uses MLLMs and reflective Chain-of-Thought to ground user queries in personal memory explicitly.
Outcome:	The proposed framework outperforms state-of-the-art benchmarks on three benchmarks . it can be used to generate detailed target video descriptions in long-context contexts based on user-specific object annotations enriched with user-specified object annotation data .

Copied to clipboard

Zheyu Lin, Jirui Yang, Yukui Qiu, Yubing Bao, Hengqi Guo, Yao Guan

Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction (2021.findings-acl)

Challenge:	Evaluating the safety robustness of LLMs is critical for their deployment.
Approach:	They propose to use latent representations to characterize hidden layer dynamics by analyzing the APT of latent models and introducing the JSS metric.
Outcome:	The proposed method exploits the APT (Angular-Probabilistic Trajectory) of latent representations and introduces the JSS (Jensen-Shannon Separability) metric.

Copied to clipboard

Tianyu Gao, Xu Han, Yuzhuo Bai, Keyue Qiu, Zhiyu Xie, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

Information Extraction & Text Mining

Challenge:	Distantly supervised relation extraction (RE) has attracted much attention in the past few years . previous methods to evaluate models manually or directly on autolabeled data have produced inaccurate evaluations .
Approach:	They propose to use distant supervision to generate large-scale autolabeled data . they build manually-annotated test sets for two DS-RE datasets and evaluate models .
Outcome:	The proposed method produces 53% wrong labels at the entity pair level in the popular NYT10 dataset.

Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translation (2020.acl-main)

Copied to clipboard

Qiu Ran, Yankai Lin, Peng Li, Jie Zhou

Multilinguality

Challenge:	Existing non-autoregressive neural machine translation models suffer from multi-modality problem . despite their autoregressivity, most NMT models suffer with slow decoding speed .
Approach:	They propose a semi-autoregressive model which generates a translation as a sequence of segments while each segment is predicted token-by-token.
Outcome:	The proposed model can achieve 4 times speedup while maintaining comparable performance.

EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs (2024.lrec-main)

Copied to clipboard

Cheng Jiayang, Lin Qiu, Chunkit Chan, Xin Liu, Yangqiu Song, Zheng Zhang

InteGround: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding (2025.findings-emnlp)

Challenge:	Existing frameworks for leveraging background knowledge of narratives are limited.
Approach:	They propose a framework to ground free-texts to eventuality-centric KGs for narrative reasoning . their framework is based on a set of probabilistic probabilistic models that are grounded in the real world .
Outcome:	The proposed framework outperforms baseline models while providing interpretable evidence.

Copied to clipboard

Cheng Jiayang, Qianqian Zhuang, Haoran Li, Chunkit Chan, Xin Liu, Lin Qiu, Yangqiu Song

Information Retrieval

Challenge:	Existing grounding approaches work well for simple queries, but many real-world information needs require synthesizing multiple pieces of evidence.
Approach:	They introduce "integrative grounding" to evaluate the ability to ground large language models in external knowledge sources.
Outcome:	The proposed approach is robust to redundant evidence, but rationalizes using internal knowledge when information is incomplete.

Glancing Transformer for Non-Autoregressive Neural Machine Translation (2021.acl-long)

Copied to clipboard

Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, Lei Li

LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression (2024.findings-acl)

Challenge:	Existing non-autoregressive neural machine translation methods are either inferior to Transformer or require multiple decoding passes, leading to reduced speedup.
Approach:	They propose a Glancing Language Model (GLM) for single-pass parallel generation models and Glancing Transformer (GLAT) with only single- pass decoding, GLAT is able to generate high-quality translation with 8-15 speedup.
Outcome:	The proposed model outperforms all previous non-autoregressive methods on multiple language directions and is nearly comparable to Transformer.

Copied to clipboard

Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang

Soft-Labeled Contrastive Pre-Training for Function-Level Code Representation (2022.findings-emnlp)

Challenge:	Existing approaches to compress prompts only leverage unidirectional context, causing suboptimal results.
Approach:	They propose a task-agnostic prompt compression method that takes tokens from context . they use a Transformer encoder to capture all essential information needed for prompt compression .
Outcome:	The proposed method is 3x-6x faster than existing prompt compression methods and faster than baselines.

Copied to clipboard

Xiaonan Li, Daya Guo, Yeyun Gong, Yun Lin, Yelong Shen, Xipeng Qiu, Daxin Jiang, Weizhu Chen, Nan Duan

CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks (2026.acl-long)

Challenge:	Existing methods for contrastive pre-training ignore the relevance between codes in large code corpus.
Approach:	They propose a Soft-labeled contrastive pre-training framework with positive sample construction methods to learn functional-level code representation.
Outcome:	The proposed framework can obtain fine-grained soft-labels through an iterative adversarial manner and use them to learn better code representation.

Copied to clipboard

Lingyue Fu, Hao Guan, Bolun Zhang, Haowei Yuan, Yaoming Zhu, Lin Qiu, ZongYu Wang, Xuezhi Cao, Xunliang Cai, Weiwen Liu, Weinan Zhang, Yong Yu

Challenge:	Existing large language models for software engineering rely on coarse-grained pass rates obscuring specific cognitive bottlenecks.
Approach:	They propose a repository-level benchmark that dissects coding capabilities through atomized tasks.
Outcome:	The proposed framework achieves a 78.55% validity yield, surpassing the 31.7% retention rate of SWE-bench-Verified.

OSCR-Attack: One-Shot Character Level Attacks through Self-Optimizing Continuous Relaxation (2026.findings-acl)

Copied to clipboard

Lingyi Kong, Zhuo Liu, Zhanghao Hu, Qilong Qiu, Yutao Yang, Jingjing Xue, Zheng Wang, Lin Gui, Feiping Nie

Robustness in NLP

Challenge:	Character-level adversarial attacks preserve semantics but are costly and inefficient . generative LLMs are gaining popularity due to their uncertainty and vulnerability to textual adversarials .
Approach:	They propose an end-to-end framework that transforms discrete choices into continuous representations and a conflict resolution strategy that maps them back into discrete insertion operations.
Outcome:	The proposed framework improves ASR by 21.45% points and accelerates the attack by 3.66 times compared to baselines.

ECON: On the Detection and Resolution of Evidence Conflicts (2024.emnlp-main)

Copied to clipboard

Cheng Jiayang, Chunkit Chan, Qianqian Zhuang, Lin Qiu, Tianhang Zhang, Tengxiao Liu, Yangqiu Song, Yue Zhang, Pengfei Liu, Zheng Zhang

Case2Code: Scalable Synthetic Data for Code Generation (2025.coling-main)

Challenge:	Recent studies have shown that AI generated content is more likely to dominate search results, making it difficult to detect when compared to human-produced content.
Approach:	They propose a method for generating diverse, validated evidence conflicts to simulate real-world misinformation scenarios.
Outcome:	The proposed method enables the detection of conflicting information in real-world scenarios and shows that weaker models struggle with similar answer conflicts while stronger models show robust performance.

Copied to clipboard

Yunfan Shao, Linyang Li, Yichuan Ma, Peiji Li, Demin Song, Qinyuan Cheng, Shimin Li, Xiaonan Li, Pengyu Wang, Qipeng Guo, Hang Yan, Xipeng Qiu, Xuanjing Huang, Dahua Lin

Programming Languages in NLP

Challenge:	Large Language Models (LLMs) have shown outstanding breakthroughs in code generation.
Approach:	They propose a case-to-code induction task that exploits the expressiveness and correctness of programs by incorporating LLMs into their training.
Outcome:	The proposed task improves distribution case-to-code induction and various coding generation tasks.

LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression (2024.acl-long)

Copied to clipboard

Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu