Papers by Qi Shen

MaCP: Minimal yet Mighty Adaptation via Hierarchical Cosine Projection (2025.acl-long)

Challenge:	Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.
Approach:	They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks.
Outcome:	The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting.

Copied to clipboard

Yixian Shen, Qi Bi, Jia-hong Huang, Hongyi Zhu, Andy D. Pimentel, Anuj Pathania

Challenge:	MaCP is a new adaptation method for large foundation models that requires minimal parameters and memory for fine-tuning.
Approach:	They propose a method that exploits the superior energy compaction and decorrelation properties of cosine projection to improve model efficiency and accuracy.
Outcome:	The proposed method improves model efficiency and accuracy across a wide range of single-modality tasks including natural language understanding, natural language generation, text summarization, and multi-modalities such as image classification and video understanding.

Profiling-Free Mixed-Precision Quantization for MoE LLMs via Fuzzy Rule Interpolation (2026.acl-long)

Copied to clipboard

Huachen Qi, Ruiyu Zhuo, Bowen Shi, Xiang Chang, Fei Chao, Changjing Shang, Qiang Shen

One Battle After Another: Probing LLMs’ Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework (2026.acl-long)

Challenge:	Large Language Models are scaling in size and capability, driving substantial computational and memory costs.
Approach:	They propose a mixed-precision quantization framework that uses fuzzy rule interpolation to predict quantization error from only sparse samples.
Outcome:	The proposed framework accelerates the profiling phase by up to 15.7 on DeepSeek-V2 while achieving comparable or slightly superior zero-shot accuracy.

Copied to clipboard

Qi Jia, Ye Shen, Xiujie Song, Kaiwei Zhang, Shibo Wang, Dun Pei, Xiangyang Zhu, Guangtao Zhai

Dialogue Systems & Conversational Agents

Challenge:	Existing benchmarks for instruction-following in multi-topic dialogues are limited to a fixed number of turns, susceptible to saturation and failing to account for users’ interactive experience.
Approach:	They propose a framework featuring a three-layer tracking mechanism and a query synthesis agent to mimic sequential user behaviors.
Outcome:	The proposed framework outperforms existing benchmarks in the evaluation of instruction following in multi-topic dialogues and demonstrates deficiencies in failure recovery and fine-grained instruction following.

GeAR: Graph-enhanced Agent for Retrieval-augmented Generation (2025.findings-acl)

Copied to clipboard

Zhili Shen, Chenxin Diao, Pavlos Vougiouklis, Pascual Merita, Shriram Piramanayagam, Enting Chen, Damien Graux, Andre Melo, Ruofei Lai, Zeren Jiang, Zhongyang Li, Ye Qi, Yang Ren, Dandan Tu, Jeff Z. Pan

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Challenge:	Retrieval-augmented Generation (RAG) relies on effective retrieval capabilities, yet traditional sparse and dense retrievers struggle with multi-hop retrieval scenarios.
Approach:	They propose a graph expansion mechanism that augments any conventional base retriever and an agent framework that incorporates the resulting graph-based retrieval into a multi-step retrieval framework.
Outcome:	The proposed system achieves state-of-the-art results on three multi-hop question answering datasets while consuming fewer tokens and requiring fewer iterations than existing multi-step retrieval systems.

Copied to clipboard

Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong

Explainability & Interpretability in NLP

Challenge:	Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach:	They propose a survey structured around the pipeline to identify and improve MI models.
Outcome:	The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.

Governance in Motion: Co-evolution of Constitutions and AI models for Scalable Safety (2025.emnlp-main)

Copied to clipboard

Chenhao Huang, Ziyu Shen, Yicong Ren, Huiyuan Zheng, Jiazheng Zhang, Mingxu Chai, Ming Zhang, Shihan Dou, Fan Mo, Jie Shi, Tao Gui, Qi Zhang, Xuanjing Huang

Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating with Neighbors (2023.findings-acl)

Challenge:	Existing approaches to align large language models with human preferences lack flexibility . static alignment preferences lack the ability to correct misaligned behaviors as they emerge .
Approach:	They propose a framework that enables dynamic and continuous alignment of large language models with human preferences.
Outcome:	The proposed framework improves safety and accuracy of a 7B model with human annotations.

Copied to clipboard

Peng Qi, Yuyang Zhao, Yufeng Shen, Wei Ji, Juan Cao, Tat-Seng Chua

The Mirage of Model Editing: Revisiting Evaluation in the Wild (2025.acl-long)

Challenge:	Existing frameworks for detecting fake news videos are limited . a new approach is proposed to integrate neighborhood information of new videos .
Approach:	They propose a framework for automatically detecting fake news videos . it integrates neighborhood relationship of new videos belonging to same event .
Outcome:	The proposed framework improves performance of existing detectors and graph aggregation and debunking rectification modules.

Copied to clipboard

Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Qi Cao, Dawei Yin, Huawei Shen, Xueqi Cheng

Question Answering

Challenge:	despite near-perfect results, effectiveness of model editing in real-world applications remains unclear.
Approach:	They propose QAEdit and WILD to better reflect real-world use of model editing . they propose a benchmark aligned with widely used question answering datasets and a task-agnostic evaluation framework .
Outcome:	The proposed QAEdit benchmark and WILD evaluation framework show that current models perform worse than previously reported.

TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities (2024.emnlp-main)

Copied to clipboard

Ming Zhang, Caishuang Huang, Yilong Wu, Shichun Liu, Huiyuan Zheng, Yurui Dong, Yujiong Shen, Shihan Dou, Jun Zhao, Junjie Ye, Qi Zhang, Tao Gui, Xuanjing Huang

Dialogue Systems & Conversational Agents

Challenge:	Current datasets cater to user-led systems and are limited to predefined specific scenarios and slots.
Approach:	They propose to use a Chinese dialogue dataset to train a model that authentically simulates human-computer dialogues in 30 popular life service scenarios.
Outcome:	The proposed model achieves a joint accuracy of 75.09% in out-of-domain evaluations . it also achieves notable abilities in slot filling and questioning .

A Systematic Survey of Automatic Prompt Optimization Techniques (2025.emnlp-main)

Copied to clipboard

Kiran Ramnath, Kang Zhou, Sheng Guan, Soumya Smruti Mishra, Xuan Qi, Zhengyuan Shen, Shuai Wang, Sangmin Woo, Sullam Jeoung, Yawei Wang, Haozhu Wang, Han Ding, Yuzhe Lu, Zhichao Xu, Yun Zhou, Balasubramaniam Srinivasan, Qiaojing Yan, Yueyan Chen, Haibo Ding, Panpan Xu, Lin Lee Cheong

P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (2024.findings-emnlp)

Challenge:	Recent advances in prompt engineering have created impediments for end users to adopt . however, prompt engineering remains an impedance due to rapid advances in models, tasks, and associated best practices.
Approach:	They propose to define APO as a 5-part unifying framework and categorize all relevant works based on their salient features.
Outcome:	The proposed framework aims to improve the performance of large language models on various tasks.

Copied to clipboard

Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan

Information Extraction & Text Mining

Challenge:	Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales.
Approach:	They propose to annotate a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans.
Outcome:	The proposed model outperforms existing methods on understanding the capabilities of LLMs in logical reasoning by 10% or more.

PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts (2025.findings-acl)

Copied to clipboard

Ming Zhang, Yuhui Wang, Yujiong Shen, Tingyi Yang, Changhao Jiang, Yilong Wu, Shihan Dou, Qinhao Chen, Zhiheng Xi, Zhihao Zhang, Yi Dong, Zhen Wang, Zhihui Fei, Mingyang Wan, Tao Liang, Guojun Ma, Qi Zhang, Tao Gui, Xuanjing Huang

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) have shown remarkable progress in dialogue and reasoning, but they struggle to solve strictly constrained dialogue tasks.
Approach:	They construct a dataset that contains 12,705 high-quality Chinese dialogue instructions from 440 flowcharts containing 5,055 process nodes.
Outcome:	The proposed model outperforms GPT-4o models on backward transitions and outperformed GPT-42 models on the same dataset.

SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation (2025.naacl-long)

Copied to clipboard

Yixian Shen, Qi Bi, Jia-hong Huang, Hongyi Zhu, Andy D. Pimentel, Anuj Pathania

Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction (2026.findings-acl)

Challenge:	Low-rank adaptation (LoRA) has been demonstrated effective in reducing the trainable parameter number when fine-tuning a large foundation model (LLM).
Approach:	They propose a low-rank adaptation approach that reduces the number of trainable parameters while enhancing model performance.
Outcome:	The proposed approach outperforms existing parameter-efficient fine-tuning methods while achieving substantial reductions in computational cost and memory requirements.

Copied to clipboard

Xu Shen, Qi Zhang, Song Wang, Zhen Tan, Xinyu Zhao, Laura Yao, Vaishnav Tadiparthi, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Kwonjoon Lee, Tianlong Chen

To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion (2023.acl-long)

Challenge:	Large Language Model based multi-agent systems (MAS) excel at collaborative problem solving but remain brittle to cascading errors.
Approach:	They propose a metacognitive framework that enables step-level error detection and self-correction in Large Language Model based multi-agent systems (MAS) .
Outcome:	The proposed framework outperforms baselines on the Who When benchmark and delivers consistent gains on AgentErrorBench.

Copied to clipboard

Rui Li, Xu Chen, Chaozhuo Li, Yanming Shen, Jianan Zhao, Yujing Wang, Weihao Han, Hao Sun, Weiwei Deng, Qi Zhang, Xing Xie

Adaptive Knowledge-Enhanced Bayesian Meta-Learning for Few-shot Event Detection (2021.findings-acl)

Challenge:	Existing methods for embedding knowledge graphs implicitly memorize relation rules to infer missing links, but they are difficult to memorize due to the inherent deficiencies of such implicit memorization strategy.
Approach:	They propose a vertical learning paradigm that allows to explicitly copy target information from related factual triples for more accurate prediction.
Outcome:	The proposed model improves generalization ability and makes distant link prediction significantly easier.

Copied to clipboard

Shirong Shen, Tongtong Wu, Guilin Qi, Yuan-Fang Li, Gholamreza Haffari, Sheng Bi

BoundRL: Efficient Token-level Structured Text Segmentation through Reinforced Boundary Generation (2026.findings-acl)

Challenge:	Event detection typically does not have sufficient labelled data, thus can be formulated as a few-shot learning problem.
Approach:	They propose a knowledge-based fewshot event detection method which introduces external event knowledge as the knowledge prior of event types.
Outcome:	Experiments show that the proposed method outperforms baselines by 15 F 1 points . event detection is an important task in information extraction .

Copied to clipboard

Haoyuan Li, Zhengyuan Shen, Sullam Jeoung, Yueyan Chen, Jiayu Li, Qi Zhu, Shuai Wang, Vassilis N. Ioannidis, Huzefa Rangwala

Responsible & Trustworthy NLP

Challenge:	Structured texts often contain elements beyond plain language, such as code snippets, which conventional sentence-level segmentation methods cannot handle effectively.
Approach:	They propose a token-level approach that performs efficient token-based text segmentation and label prediction for long structured texts.
Outcome:	The proposed approach outperforms existing models on short-shot prompts and SFT and standard RLVR models on complex LLM prompts.

Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow (2026.findings-acl)

Copied to clipboard

Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao

REAP: Towards Effective Training-Free Chemical Reasoning with Explicit Atomic Priors (2026.findings-acl)

Challenge:	Autoregressive (AR) language models dominate modern natural language processing due to strong likelihood-based training objectives and reliable left-to-right decoding.
Approach:	They characterize MDLM behavior along two dimensions: parallelism strength and generation order . authors propose a Generate-then-Edit paradigm that mitigates dependency loss .
Outcome:	The proposed model improves on tasks that require "backward information" the Generate-then-Edit paradigm improves parallel decoding efficiency while reducing dependency loss.

Copied to clipboard

Mingxu Zhang, Dazhong Shen, Qi Zhang, Ying Sun

CSP:Code-Switching Pre-training for Neural Machine Translation (2020.emnlp-main)

Challenge:	Current approaches to instill explicit priors into LLMs often suffer from an information bottleneck .
Approach:	They propose a training-free framework that equips LLMs with an external knowledge base, enabling them to reason over retrieved chemical priors dynamically.
Outcome:	Experiments show that REAP outperforms current reasoning methods and rivals state-of-the-art training-based models.

Copied to clipboard

Zhen Yang, Bojie Hu, Ambyera Han, Shen Huang, Qi Ju

Multilinguality

Challenge:	Existing methods to train neural machine translation models are data-hungry and low-resource . et al., 2018; Radford e.t., 2019; Yang ee.,2019) proposes a new pre-training method for NMT .
Approach:	They propose a new pre-training method which randomly replaces some words in the input sentence with their translation words in target language.
Outcome:	The proposed method improves on unsupervised and supervised NMT models by making full use of monolingual corpora.

PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)

Copied to clipboard

Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei

Reasoning

Challenge:	Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning.
Approach:	They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios.
Outcome:	The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics.

Fine-Grained Data Ordering Improves Fine-Tuning for Large Language Models (2026.findings-acl)

Copied to clipboard

Xiaomeng Hu, Yixuan Tang, Haoze Li, Hao Chen, Qi Zhang, Zhanming Shen, Yiming Zhang, Haobo Wang, Junbo Zhao

Dynamic Curriculum Learning for Low-Resource Neural Machine Translation (2020.coling-main)

Challenge:	Prior work focused on data preprocessing, focusing on filtering and cleaning data . a study aimed to improve fine-grained scheduling of data order in epochs .
Approach:	They propose a fine-grained scheduling method of data order in epochs to fill this gap . they define data difficulty based on relevance between data and model .
Outcome:	The proposed method improves on pre-training and small-scale fine-tuning experiments 2.4% over baselines.

Copied to clipboard

Chen Xu, Bojie Hu, Yufan Jiang, Kai Feng, Zeyang Wang, Shen Huang, Qi Ju, Tong Xiao, Jingbo Zhu

LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models (2026.acl-long)

Challenge:	Recent work on neural machine translation (NMT) has demonstrated impressive performance improvements and became the de-facto standard.
Approach:	They propose a dynamic curriculum learning method to reorder training samples in training using a Transformer-based system.
Outcome:	The proposed method outperforms baselines on three low-resource machine translation benchmarks and different sized data of WMT’16 En-De.

Copied to clipboard

Ming Zhang, Yujiong Shen, Jingyi Deng, Yuhui Wang, Huayu Sha, Kexin Tan, Qiyuan Peng, Yue Zhang, Junzhe Wang, Shichun Liu, Yueyuan Huang, Jingqi Tong, Changhao Jiang, Yilong Wu, Zhihao Zhang, Mingqi Wu, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

Responsible & Trustworthy NLP

Challenge:	Existing evaluation of Large Language Models on static benchmarks is vulnerable to data contamination and leaderboard overfitting.
Approach:	LLMEval-Fair framework provides a framework for dynamic evaluation of Large Language Models . evaluators use a proprietary bank of 220k graduate-level questions to analyze model data .
Outcome:	LLMEval-Fair provides robust and credible evaluation framework for Large Language Models . it provides a strong empirical validation for the dynamic evaluation paradigm .

Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement (2025.emnlp-industry)

Copied to clipboard

Haotan Guo, Jianfei He, Jiayuan Ma, Hongbin Na, Zimu Wang, Haiyang Zhang, Qi Chen, Wei Wang, Zijing Shi, Tao Shen, Ling Chen

Responsible & Trustworthy NLP

Challenge:	Phonetic Cloaking Replacement (PCR) is a problem in content moderation in China.
Approach:	They organize PCR into a four-way surface-form taxonomy and compile PCR-ToxiCN, a dataset of 500 phonetically cloaked offensive posts gathered from the RedNote platform.
Outcome:	The proposed model achieves only an F1-score and zero-shot chain-of-thought prompting pushes performance even lower.

When to Trust LLMs: Aligning Confidence with Response Quality (2024.findings-acl)

Copied to clipboard

Shuchang Tao, Liuyi Yao, Hanxing Ding, Yuexiang Xie, Qi Cao, Fei Sun, Jinyang Gao, Huawei Shen, Bolin Ding

Challenge:	Existing methods express reliability by confidence level, but lack objective guidance . Existing approaches express reliability but lack guidance on when to trust LLMs .
Approach:	They propose a reward-based approach to align confidence with quality to ensure reliability . they propose 'conqORD' to help model to verbalize greater confidence for higher quality responses .
Outcome:	Experiments show that CONQORD significantly improves confidence and response accuracy . the proposed approach can be used to determine reliability of large language models .

CA-GAR: Context-Aware Alignment of LLM Generation for Document Retrieval (2025.findings-acl)

Copied to clipboard

Heng Yu, Junfeng Kang, Rui Li, Qi Liu, Liyang He, Zhenya Huang, Shuanghong Shen, Junyu Lu

Information Retrieval

Challenge:	Recent techniques such as Generation-Augmented Retrieval (GAR) and Generative Document Retrieleval (GDR) leverage LLMs to enhance retrieval performance but face key challenges: GAR’s generated content may not always align with the target document corpus, while GDR limits the generative capacity of LLM.
Approach:	They propose a Context-Aware Generation-Augmented Retrieval approach which integrates corpus information into their generation process.
Outcome:	Experimental results show that CA-GAR outperforms existing methods on seven tasks and four non-English languages.

Open-World Attribute Mining for E-Commerce Products with Multimodal Self-Correction Instruction Tuning (2025.acl-long)

Copied to clipboard

Jiaqi Li, Yanming Li, Xiaoli Shen, Chuanyi Zhang, Guilin Qi, Sheng Bi

Modeling and Solving Stable Matching under Probabilistic Preferences with Large Language Models (2026.findings-acl)

Challenge:	Current AM methods focus on extracting attributes from unimodal text, underutilizing multimodal data.
Approach:	They propose a framework for multimodal self-correction instruction tuning to extract new attributes from images and text with Multimodal Large Language Models.
Outcome:	The proposed framework outperforms state-of-the-art methods on two datasets.

Copied to clipboard

Yuqi Kong, Shiyu Liu, Jiaxu Li, Hongtao Liu, Qi Qi, Weiran Shen

PKAD: Pretrained Knowledge is All You Need to Detect and Mitigate Textual Backdoor Attacks (2024.findings-emnlp)

Challenge:	Large language models (LLMs) have shown strong capability in understanding and simulating humans’ decisions, suggesting a new way to use LLMs as tools to study social systems.
Approach:	They propose a Hybrid GS–LLM matching method that integrates Gale–Shapley with probabilistic acceptance decisions.
Outcome:	The proposed method outperforms classical baselines in terms of stability and improves robustness under uncertainty.

Copied to clipboard

Yu Chen, Qi Cao, Kaike Zhang, Xuchao Liu, Huawei Shen

Robustness in NLP

Challenge:	Current defense methods can be classified into inference-time and training-time ones based on their execution phase.
Approach:	They propose a two-stage poison detection strategy using pre-trained language models to detect poisoned samples before model training.
Outcome:	The proposed method achieves better performance than current methods more quickly and with fewer training costs.

The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models (2023.emnlp-main)

Copied to clipboard

Jingyuan Qi, Zhiyang Xu, Ying Shen, Minqian Liu, Di Jin, Qifan Wang, Lifu Huang

Geo-BERT Pre-training Model for Query Rewriting in POI Search (2021.findings-emnlp)

Challenge:	Chain-of-Thought (CoT) prompting relies on the initial decisions, causing errors in early steps to accumulate and impact the final answers.
Approach:	They propose a divide-and-conquer style algorithm that leverages large language models to raise and answer sub-questions until collecting enough information to tackle the original one.
Outcome:	The proposed algorithm is more robust to errors and errors than CoT prompting and Tree-of-Thought prompting methods.

Copied to clipboard

Xiao Liu, Juan Hu, Qi Shen, Huan Chen

Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders (2021.acl-long)

Challenge:	Existing methods to solve the word mismatch between queries and documents are often inadequate to integrate geographic information into the pre-training model.
Approach:	They propose to train a pre-training model to integrate semantics and geographic information in the pre-trained representations of POIs.
Outcome:	The proposed model achieves excellent accuracy on a wide range of real-world datasets of map services.

Copied to clipboard

Chen Xu, Bojie Hu, Yanyang Li, Yuhao Zhang, Shen Huang, Qi Ju, Tong Xiao, Jingbo Zhu

Reward Modeling Requires Automatic Adjustment Based on Data Quality (2024.findings-emnlp)

Challenge:	End-to-end Speech Translation (E2E ST) encoders lack global context representation, whereas MT encoder lacks it.
Approach:	They propose a Stacked Acoustic-and-Textual Encoding method for speech translation . they propose an adaptor module to alleviate representation inconsistency .
Outcome:	The proposed method achieves state-of-the-art BLEU scores of 18.3 and 25.2 on two ST tasks.

Copied to clipboard

Binghai Wang, Rui Zheng, Lu Chen, Zhiheng Xi, Wei Shen, Yuhao Zhou, Dong Yan, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values.
Approach:	They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values .
Outcome:	The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets.

Evaluating Document Coherence Modeling (2021.tacl-1)

Copied to clipboard

Aili Shen, Meladel Mistica, Bahar Salehi, Hang Li, Timothy Baldwin, Jianzhong Qi

GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)

Challenge:	a new study examines pretrained language models' ability to model discourse and pragmatic phenomena.
Approach:	They propose a sentence intrusion detection task using a dictionary dataset . they show that pretrained LMs perform impressively in in-domain evaluation .
Outcome:	The proposed dataset shows that pretrained LMs perform impressively in in-domain evaluation, but experience a substantial drop in the cross-domain setting, indicating limited generalization capacity.

Copied to clipboard

Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou

OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024.acl-long)

Challenge:	Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work.
Approach:	They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations.
Outcome:	The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work.

Copied to clipboard

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, Maosong Sun

Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025.emnlp-main)

Challenge:	Large Language Models (LLMs) and Large Multimodal Models have exceeded general human capabilities in various tasks.
Approach:	They present an Olympiad-level bilingual multimodal scientific benchmark featuring 8,476 problems from Olympiad level mathematics and physics competitions.
Outcome:	The best performing model, GPT-4V, attains an average score of 17.97% on OlympiadBench, with a mere 10.74% in physics, highlighting the benchmark rigor and the intricacy of physical reasoning.

Copied to clipboard

Hexiang Tan, Fei Sun, Sha Liu, Du Su, Qi Cao, Xin Chen, Jingang Wang, Xunliang Cai, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng

Challenge:	Existing detection methods fail to account for self-consistent error . study identifies self-consistency errors and evaluates them .
Approach:	They propose a method that fuses hidden state evidence from an external verifier LLM to detect self-consistent errors.
Outcome:	The proposed method significantly enhances performance on self-consistent errors across three LLM families.

Simple or Complex? Complexity-controllable Question Generation with Soft Templates and Deep Mixture of Experts Model (2021.findings-emnlp)

Copied to clipboard

Sheng Bi, Xiya Cheng, Yuan-Fang Li, Lizhen Qu, Shirong Shen, Guilin Qi, Lu Pan, Yinlin Jiang

Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extraction (2026.findings-eacl)

Challenge:	Existing work on complex questions does not consider controlling complexity of generated questions.
Approach:	They propose an end-to-end neural complexity-controllable question generation model that incorporates a mixture of experts as the selector of soft templates to capture question similarity while avoiding the expensive construction of actual templates.
Outcome:	The proposed model is superior to state-of-the-art methods in both automatic and manual evaluations on two benchmark QA datasets.

Copied to clipboard

Chong Zhang, Yixi Zhao, Yulu Xie, Chenshu Yuan, Yi Tu, Ya Guo, Mingxu Chai, Ziyu Shen, Yue Zhang, Qi Zhang

LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation (2025.findings-emnlp)

Challenge:	PTLMs have shown remarkable success in multiple information extraction tasks . however, their performance in real-world scenarios falls short of expectations .
Approach:	They propose to use an entity-centric dataset to evaluate PTLMs' performance . they find that inadequate annotations in benchmark datasets lead to spurious correlations .
Outcome:	The proposed dataset disentangles the falsely-coupled segment and entity annotations that arises from the block-level annotation of FUNSD.

Copied to clipboard

Ming Zhang, Yujiong Shen, Zelin Li, Huayu Sha, Binze Hu, Yuhui Wang, Chenhao Huang, Shichun Liu, Jingqi Tong, Changhao Jiang, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors (2026.acl-long)

Challenge:	Current medical benchmarks have limitations in question design, data sources and evaluation methods.
Approach:	They propose a new benchmark covering five core medical areas . it includes 2,996 questions created from real-world electronic health records .
Outcome:	The proposed model covers five core medical areas and includes 2,996 questions created from real-world electronic health records and expert-designed clinical scenarios.

Copied to clipboard

Yuqing Yang, Qi Zhu, Zhen Han, Boran Han, Zhengyuan Shen, Shuai Wang, Vassilis N. Ioannidis, Huzefa Rangwala

VisFinEval: A Scenario-Driven Chinese Multimodal Benchmark for Holistic Financial Understanding (2025.emnlp-main)

Challenge:	Large language models (LLMs) perform well on table tasks, but they still make data referencing errors (DREs) prior studies have only offered limited, small-scale analyses.
Approach:	They propose inference-time strategies and lightweight critics to mitigate data referencing errors.
Outcome:	The proposed model achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-difference DREs and assists inference for larger models.

Copied to clipboard

Zhaowei Liu, Xin Guo, Haotian Xia, Lingfeng Zeng, Fangqi Lou, Jinyi Niu, Mengping Li, Qi Qi, Jiahuan Li, Wei Zhang, Yinglong Wang, Weige Cai, Weining Shen, Liwen Zhang

Low-Entropy Watermark Detection via Bayes’ Rule Derived Detector (2025.findings-acl)

Challenge:	Existing benchmarks focus on text comprehension, but MLLMs lack the ability to integrate visual data over financial visuals.
Approach:	They evaluate 21 state-of-the-art multimodal large language models in a zero-shot setting . they use an annotated question–answer pair from eight common financial image modalities .
Outcome:	The new benchmark outperforms existing models but trailed financial experts by 14 percentage points.

Copied to clipboard

Beining Huang, Du Su, Fei Sun, Qi Cao, Huawei Shen, Xueqi Cheng

Numerical Reasoning

Challenge:	Existing methods for text watermarking ignore strong evidences embedded in low-entropy tokens, causing statistical measures to falsely indicate the absence of a watermark.
Approach:	They propose a Bayes' Rule derived watermark Detector which exploits watermark information from every token by leveraging the posterior probability of watermark’s presence.
Outcome:	The proposed method achieves 50% and 70% relative improvements over baselines in code generation and math problem-solving tasks.

SQL-Trail: Multi-Turn Reinforcement Learning with Interleaved Feedback for Text-to-SQL (2026.acl-long)

Copied to clipboard

Harper Hua, Zhen Han, Zhengyuan Shen, Meng-Chieh Lee, Sheng Guan, Qi Zhu, Sullam Jeoung, Yueyan Chen, Yunfei Bai, Shuai Wang, Vassilis N. Ioannidis, Huzefa Rangwala

Programming Languages in NLP

Challenge:	Recent large language models (LLMs) have significantly improved Text-to-SQL generation, but a gap remains between AI systems and human experts on challenging benchmarks such as BIRD-Sql.
Approach:	They propose a multi-turn reinforcement learning agentic framework for Text-to-SQL that uses execution feedback to iteratively refine its predictions.
Outcome:	The proposed framework outperforms proprietary systems on 7B and 14B models by **5% on average, underscoring the effectiveness of interactive, agentic workflows for robust Text-to-SQL generation.

Event Causality Identification via Derivative Prompt Joint Learning (2022.coling-1)

Copied to clipboard

Shirong Shen, Heng Zhou, Tongtong Wu, Guilin Qi

Low-Resource NLP

Challenge:	Existing methods for event causality identification lack annotated data, and they lack the ability to identify explicit and implicit causality.
Approach:	They propose a derivative prompt joint learning model which leverages potential causal knowledge in the pre-trained language model to tackle the data scarcity problem.
Outcome:	The proposed model can identify explicit and implicit causality on two benchmark datasets and it has great advantages over previous methods.

Hierarchical Chinese Legal event extraction via Pedal Attention Mechanism (2020.coling-main)

Copied to clipboard

Shirong Shen, Guilin Qi, Zhen Li, Sheng Bi, Lusheng Wang

Information Extraction & Text Mining

Challenge:	Existing methods for event extraction cannot express connections between arguments, which are crucial in legal events.
Approach:	They propose a dynamic event structure for Chinese legal events to distinguish between similar events by hierarchical event features for event detection and a pedal attention mechanism to extract the semantic relation between two words through their dependent adjacent words.
Outcome:	The proposed model surpasses state-of-the-art models on a Chinese legal event dataset.

CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search (2022.emnlp-main)

Copied to clipboard

Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang, Weizhu Chen, Nan Duan

Modelling Uncertainty in Collaborative Document Quality Assessment (D19-55)

Challenge:	Existing code pre-training approaches often adopt (masked) language modeling as the training objective which targets on learning to predict (macked) tokens in a given code context.
Approach:	They propose a code-text contrastive learning model which learns function-level code semantic representations through large-scale code corpus.
Outcome:	The proposed model achieves new state-of-the-art with significant improvement over existing pre-trained models on eleven domain/language-specific code search tasks with six programming languages in different code granularity.

Copied to clipboard

Aili Shen, Daniel Beck, Bahar Salehi, Jianzhong Qi, Timothy Baldwin

Challenge:	Existing work on document quality assessment relies on predicting the quality of a document relative to a putative gold standard, without paying attention to the subjectivity of this task.
Approach:	They propose to use Gaussian processes and random forests to measure the uncertainty in document quality predictions.
Outcome:	The proposed methods can predict the quality of Wikipedia articles while providing an estimate of uncertainty when there is inconsistency in the quality labels from the contributors.

Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback (2023.findings-emnlp)

Copied to clipboard

Wei Shen, Rui Zheng, Wenyu Zhan, Jun Zhao, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang

DocFusion: A Unified Framework for Document Parsing Tasks (2025.findings-acl)

Challenge:	Experimental results prove that language models can learn from human feedback better, irrespective of sequence length . emergence of length bias often induces the model to favor longer outputs .
Approach:	They propose to separate reward modeling from the influence of sequence length by using the Product-of-Experts technique.
Outcome:	The proposed approach shows that language models perform better regardless of sequence length . the main expert is focused on understanding human intents, while the biased expert targets the identification and capture of length bias.

Copied to clipboard

Mingxu Chai, Ziyu Shen, Chong Zhang, Yue Zhang, Xiao Wang, Shihan Dou, Jihua Kang, Jiazheng Zhang, Qi Zhang

Challenge:	Existing methods for document parsing often employ multiple models, limiting performance . Existing models often employ discrete tokens, whereas recognition relies on continuous coordinates .
Approach:	They propose a Gaussian-Kernel Cross-Entropy Loss (GK-CEL) that unifies detection and recognition by enabling generative frameworks to handle both tasks simultaneously.
Outcome:	The proposed model performs competitively across four core document parsing tasks.

SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent (2024.findings-emnlp)

Copied to clipboard

Jiarui Ji, Yang Li, Hongtao Liu, Zhicheng Du, Zhewei Wei, Qi Qi, Weiran Shen, Yankai Lin

Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)

Challenge:	Existing research on the allocation of public scarce resources has limitations due to data scarcity and data scariness.
Approach:	They propose a framework that integrates Large Language Models into economic simulations . they conduct extensive policy simulation experiments to verify the framework's effectiveness .
Outcome:	The proposed framework bridges the gap between theoretical models and real-world dynamics by integrating large language models into economic simulations.

Copied to clipboard

Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training (2026.acl-long)

Challenge:	Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations.
Approach:	They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference.
Outcome:	The proposed method significantly mitigates object hallucinations across various model architectures.

Copied to clipboard

Changhao Jiang, Ming Zhang, Yifei Cao, Junjie Ye, Xiaoran Fan, Shihan Dou, Zhiheng Xi, Jiajun Sun, Yi Dong, Yujiong Shen, Jingqi Tong, Baoyu Fan, Tao Gui, Qi Zhang, Xuanjing Huang