Papers by Yiming Liu

DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Management (2025.findings-emnlp)

Challenge:	Large language models are ideal for decision-making, but they can be difficult to process when they are verbose and include repetition, hedging, and vagueness.
Approach:	They propose a framework that constructs probabilistic factor profiles from complex scenarios and integrates them with analogical reasoning to guide LLMs in making decisions in new situations.
Outcome:	The proposed framework separates the tasks of quantifying uncertainty and incorporating it into LLM decision-making.

Copied to clipboard

Kai Yin, Xiangjue Dong, Chengkai Liu, Lipai Huang, Yiming Xiao, Zhewei Liu, Ali Mostafavi, James Caverlee

Information Retrieval

Challenge:	Existing information retrieval benchmarks focus on general or specialized domains, such as medicine or finance, neglecting the unique linguistic complexity and diverse information needs encountered in disaster management scenarios.
Approach:	DisastIR is the first comprehensive IR evaluation benchmark specifically tailored for disaster management.
Outcome:	DisastIR covers 48 retrieval tasks derived from six search intents and eight general disaster categories . evaluations show no single model excelling universally .

CMQA: A Dataset of Conditional Question Answering with Multiple-Span Answers (2022.coling-1)

Copied to clipboard

Yiming Ju, Weikang Wang, Yuanzhe Zhang, Suncong Zheng, Kang Liu, Jun Zhao

Cross-Lingual Machine Reading Comprehension (D19-1)

Challenge:	Existing QA datasets only contain unconditional and parallel answers . conditional question answering with hierarchical multi-span answers is challenging for the community to solve .
Approach:	They propose a conditional question answering task with hierarchical multi-span answers . they propose CMQA, which contains conditional and hierarchic samples .
Outcome:	The proposed task can be used to build more reliable and sophisticated QA systems.

Copied to clipboard

Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, Guoping Hu

Machine Reading Comprehension

Challenge:	Existing work on machine reading comprehension task is focused on English, but there are few efforts on other languages due to the lack of large-scale training data.
Approach:	They propose a cross-lingual machine reading comprehension task for other languages . they propose cloze-style reading comprehension and various neural network approaches .
Outcome:	The proposed model improves reading comprehension performance of Chinese datasets over state-of-the-art systems by a large margin over existing systems.

DARM: Distribution-Aware Reward Modeling by Alleviating Biases from Low Preference-Context Dependency Data (2026.acl-long)

Copied to clipboard

Shaofan Liu, Guoqiang Zhang, Shihan Dou, Huiyuan Zheng, Yiming Zhou, Junjie Ye, Shaowen Wang, Shichun Liu, Jiazheng Zhang, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Existing methods for training reward models are vulnerable to context neglect and degraded accuracy.
Approach:	They propose distribution-aware reward modeling that augments the RM objective with a conditional mutual information regularizer that maximizes context and the predicted reward conditioned on the response.
Outcome:	The proposed model improves performance in RLHF and improves accuracy in other settings.

Adapt to Thrive! Adaptive Power-Mean Policy Optimization for Improved LLM Reasoning (2026.findings-acl)

Copied to clipboard

Yiming Huang, Zhenbo Shi, Shuzheng Gao, Cuiyun Gao, Peiyi Han, Chuanyi Liu

Free Energy-Driven Reinforcement Learning with Adaptive Advantage Shaping for Unsupervised Reasoning in LLMs (2026.acl-long)

Challenge:	Existing methods for reinforcement learning with verifiable rewards (RLVR) rely on static objective functions and rigid clipping strategies that misalign with the model’s evolving reasoning capabilities.
Approach:	They propose to incorporate Power-Mean Policy Optimization (PMPO) and Feedback-Adaptive Clipping (FAC) to overcome limitations of static mechanisms.
Outcome:	Extensive experiments on nine reasoning tasks show the proposed paradigm outperforms state-of-the-art methods.

Copied to clipboard

Yiming Huang, Zhenbo Shi, Xin-Cheng Wen, Jichuan Zeng, Cuiyun Gao, Peiyi Han, Chuanyi Liu

FinDVer: Explainable Claim Verification over Long and Hybrid-content Financial Documents (2024.emnlp-main)

Challenge:	Existing unsupervised reinforcement learning methods lack the capacity to adapt to the model’s evolving reasoning capabilities during training.
Approach:	They propose an unsupervised reinforcement learning algorithm that adapts rewards to balance consensus and exploration based on the Free Energy Principle.
Outcome:	Empirical evaluations on nine datasets show that FREIA outperforms baseline methods on reasoning tasks.

Copied to clipboard

Yilun Zhao, Yitao Long, Tintin Jiang, Chengye Wang, Weiyuan Chen, Hongjun Liu, Xiangru Tang, Yiming Zhang, Chen Zhao, Arman Cohan

GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Art (2025.acl-long)

Challenge:	FinDVer is a benchmark to evaluate the explainable claim verification capabilities of LLMs . financial documents are typically long, intricate and dense, and they include both quantita and numerical reasoning.
Approach:	They propose a benchmark to evaluate the explainable claim verification capabilities of LLMs . they assess 25 LLM systems under long-context and RAG settings .
Outcome:	The proposed benchmark can be used to evaluate the explainable claim verification capabilities of LLMs in financial documents.

Copied to clipboard

Yiming Lei, Chenkai Zhang, Zeming Liu, Haitao Leng, ShaoGuo Liu, Tingting Gao, Qingjie Liu, Yunhong Wang

NegVQA: Can Vision Language Models Understand Negation? (2025.findings-acl)

Challenge:	Existing benchmarks for video comment art are constrained by their limited modalities and insufficient categories, hindering creativity in video-based comment art creation.
Approach:	They propose a benchmark that integrates video and text modalities to evaluate MLLMs’ abilities to compose video Comment art.
Outcome:	The proposed framework integrates video and text modalities to evaluate MLLMs’ abilities to compose video comment art.

Copied to clipboard

Yuhui Zhang, Yuchang Su, Yiming Liu, Serena Yeung-Levy

DisastQA: A Comprehensive Benchmark for Evaluating Question Answering in Disaster Management (2026.findings-acl)

Challenge:	NegVQA is a visual question answering (VQA) benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions.
Approach:	They propose a visual question answering benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions.
Outcome:	The proposed model fails to correctly interpret negation, leading to critical errors in interactive AI systems.

Copied to clipboard

Zhitong Chen, Kai Yin, Xiangjue Dong, Chengkai Liu, Xiangpeng Li, Bo Li, Junwei Ma, Yiming Xiao, Ali Mostafavi, James Caverlee

Adversarial Training for Machine Reading Comprehension with Virtual Embeddings (2021.starsem-1)

Challenge:	Existing benchmarks for question answering (QA) are lacking in a high-stakes environment.
Approach:	They propose a rigorously verified benchmark of 3,000 expert-annotated questions . they propose 'keypoint-based evaluation protocol' emphasizing factual completeness over verbosity .
Outcome:	Experiments with 20 models reveal substantial divergences from general-purpose models such as MMLU-Pro.

Copied to clipboard

Ziqing Yang, Yiming Cui, Chenglei Si, Wanxiang Che, Ting Liu, Shijin Wang, Guoping Hu

SEAD: A Surrogate-free Label-only Membership Inference Attack against Pre-trained LLMs with Semantic-Aware Density (2026.findings-acl)

Challenge:	Neural networks are vulnerable to adversarial examples that have been mixed with certain perturbations.
Approach:	They propose a novel adversarial training method that perturbs the embedding matrix instead of word vectors to differentiate the roles of passages and questions.
Outcome:	The proposed method is effective universally and further improves the performance of MRC tasks.

Copied to clipboard

Biao Yi, Jiahao Li, Yiming Li, Yu He, Baolei Zhang, Zheli Liu, Dacheng Tao

Robustness in NLP

Challenge:	Existing membership inference attacks require access to complete logits, but such access is often unavailable in real-world deployments where only the generated text is exposed.
Approach:	They propose a surrogate-free label-only MIA approach that directly estimates token probabilities through Monte Carlo sampling of the target model.
Outcome:	The proposed approach outperforms existing label-only attacks and serves as a foundational density estimator in the label-exclusive setting.

CSLM: A Framework for Question Answering Dataset Generation through Collaborative Small Language Models (2024.findings-emnlp)

Copied to clipboard

Yiming Wang, Yang Liu, Lingchen Wang, An Xiao

Semantic Text Processing

Challenge:	Collecting high-quality question-answer (QA) pairs is vital for training large language models, but computational demands and associated costs often render such approaches prohibitive for the average researcher.
Approach:	They propose a small-scaled, open-source solution that generates QA pairs from documents or raw corpora using large-scale models like Llama-70B.
Outcome:	Experiments on domain-specific datasets show that the proposed model can generate high-quality QA pairs, making it accessible to a broader range of researchers.

KwaiChat: A Large-Scale Video-Driven Multilingual Mixed-Type Dialogue Corpus (2025.findings-naacl)

Copied to clipboard

Xiaoming Shi, Zeming Liu, Yiming Lei, Chenkai Zhang, Haitao Leng, Chuan Wang, Qingjie Liu, Wanxiang Che, Yunhong Wang

S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Model (2024.naacl-long)

Challenge:	Currently, video-based dialogue systems rely on a single dialogue type, hindering their versatility in practical applications.
Approach:	They propose to generate video-driven multilingual mixed-type dialogues using KwaiChat . they propose to create a video-based multilingual mix of 4 dialogue types, 30 domains, 4 languages, 13 topics .
Outcome:	The proposed model performs best on KwaiChat but is not perfect in this situation.

Copied to clipboard

Fangyu Lei, Qian Liu, Yiming Huang, Shizhu He, Jun Zhao, Kang Liu

LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces (2026.findings-acl)

Challenge:	Existing benchmarks fail to evaluate extremely long-context LLMs or analyze their limitations.
Approach:	They propose a Synthetic, Scalable, Systematic evaluation suite for LLMs using SQL execution.
Outcome:	The proposed evaluation suite is able to scale text length and difficulty across scenarios and provides strong correlations with real-world benchmarks.

Copied to clipboard

Yukang Feng, Jianwen Sun, Zelai Yang, Jiaxin Ai, Chuanhao Li, Zizhen Li, Fanrui Zhang, Kang He, Rui Ma, Jifan Lin, Jie Sun, Yang Xiao, Sizhuo Zhou, Wenxiao Wu, Yiming Liu, Pengfei Liu, Shenglin Zhang, Kaipeng Zhang

SARA: Salience-Aware Reinforced Adaptive Decoding for Large Language Models in Abstractive Summarization (2025.acl-long)

Challenge:	Existing benchmarks for agentic programming in long-horizon command-line interface tasks are limited by short task horizons, data contamination from GitHub scraping, and a lack of fine-grained evaluation metrics.
Approach:	They propose a benchmark to evaluate agentic capabilities across long-horizon command-line interface tasks.
Outcome:	The proposed benchmarks cover four engineering categories: from scratch, feature addition, bug fixing, and refactoring.

Copied to clipboard

Nayu Liu, Junnan Zhu, Yiming Ma, Zhicong Lu, Wenlei Xu, Yong Yang, Jiang Zhong, Kaiwen Wei

HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks (2026.findings-acl)

Challenge:	Existing decoding strategies neglect the explicit use of salient contextual information and rely on static hyperparameters to fix the balance between contextual and prior knowledge.
Approach:	They propose a salience-aware reinforced adaptive decoding (SARA) which incorporates salient contextual information and allows the model to determine reliance on source document's context, salient context, and model's prior knowledge based on pointwise mutual information.
Outcome:	The proposed model improves the quality and faithfulness of summaries across LLMs without modifying their weights.

Copied to clipboard

Yiming Zeng, Jinghan Cao, Zexin Li, Wanhao Yu, Zhankai Ye, Dawei Xiang, Ting Hua, Xin Liu, Shangqian Gao, Tingting Yu

Text Generation

Challenge:	Existing approaches treat instruction-based text editing as a generic text generation problem. Existing methods either over-edit or fail to apply modifications consistently.
Approach:	They propose a framework that processes each editing request to best align with it.
Outcome:	The proposed framework achieves 9% improvement over the state-of-the-art model.

CharBERT: Character-aware Pre-trained Language Model (2020.coling-main)

Copied to clipboard

Wentao Ma, Yiming Cui, Chenglei Si, Ting Liu, Shijin Wang, Guoping Hu

Semantic Text Processing

Challenge:	Pre-trained language models (PLMs) construct word representations at subword level with Byte-Pair Encoding (BPE) or its variations . but these methods split a word into subword units and make it incomplete and fragile .
Approach:	They propose a character-aware pre-trained language model to tackle OOV problems . they construct contextual word embedding for each token from sequential character representations .
Outcome:	The proposed model improves on the existing models on multiple NLP benchmarks.

CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)

Copied to clipboard

Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan

Challenge:	Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages .
Approach:	They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models .
Outcome:	The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English .

RAM-SD: Retrieval-Augmented Multi-agent framework for Sarcasm Detection (2026.acl-long)

Copied to clipboard

Ziyang Zhou, Ziqi Liu, Yan Wang, Yiming Lin, Yangbin Chen

Noise-injected Consistency Training and Entropy-constrained Pseudo Labeling for Semi-supervised Extractive Summarization (2022.coling-1)

Challenge:	Existing approaches to sarcastic detection use a uniform reasoning strategy . existing approaches lack a framework to deal with the diverse analytical demands of sarcasm .
Approach:	They propose a Retrieval-Augmented Multi-Agent framework for Sarcasm Detection . the framework provides transparent and interpretable reasoning traces .
Outcome:	The proposed framework outperforms existing methods on four benchmarks and outperformed the strong GPT-4o+CoC baseline.

Copied to clipboard

Yiming Wang, Qianren Mao, Junnan Liu, Weifeng Jiang, Hongdong Zhu, Jianxin Li

Dataset for the First Evaluation on Chinese Machine Reading Comprehension (L18-1)

Challenge:	Existing studies on semi-supervised learning methods focus on how to effectively utilize abundant unlabeled data.
Approach:	They propose a semi-supervised consistency training method to regularize model predictions and a pseudo-labeling strategy to obtain high-confidence labels from unlabeled predictions.
Outcome:	The proposed method improves extractive summarization over an insufficient labeled dataset.

Copied to clipboard

Yiming Cui, Ting Liu, Zhipeng Chen, Wentao Ma, Shijin Wang, Guoping Hu

LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models (2025.findings-acl)

Challenge:	Existing reading comprehension datasets are mostly in English .
Approach:	They propose a Chinese reading comprehension dataset to add diversity to existing reading comprehension data . proposed dataset contains cloze-style reading comprehension and user query reading comprehension .
Outcome:	The proposed dataset is based on a Chinese reading comprehension dataset . it includes two types of cloze-style and user query reading comprehension . the proposed dataset hosted the 1st Evaluation on Chinese Machine Reading Comprehension (CMRC-2017)

Copied to clipboard

Jiayi Gui, Yiming Liu, Jiale Cheng, Xiaotao Gu, Xiao Liu, Hongning Wang, Yuxiao Dong, Jie Tang, Minlie Huang

A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020.coling-main)

Challenge:	Large Language Models (LLMs) have demonstrated notable capabilities across various tasks, showcasing complex problem-solving abilities.
Approach:	They propose a benchmark to evaluate the rule-based logical reasoning capabilities of Large Language Models (LLMs) they create simulated scenarios in which models execute or plan operations to achieve specific outcomes.
Outcome:	The proposed benchmark evaluates the performance of large language models on a variety of scenarios with varying difficulty levels.

Copied to clipboard

Yiming Cui, Ting Liu, Ziqing Yang, Zhipeng Chen, Wentao Ma, Wanxiang Che, Shijin Wang, Guoping Hu

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices (2020.acl-main)

Challenge:	Using cloze-style reading comprehension, Chinese machine reading comprehension datasets are becoming more and more popular . a new task is proposed to fill the right candidate sentence into the passage with several blanks .
Approach:	They propose a Chinese task to fill the right candidate sentence into a passage with blanks . they build a dataset to evaluate the difficulty of the task and make fake candidates .
Outcome:	The proposed task fills the right candidate sentence into the passage with blanks . the proposed dataset contains over 100K blanks within over 10K passages based on Chinese narrative stories .

Copied to clipboard

Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, Denny Zhou

TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing (2020.acl-demos)

Challenge:	Empirical studies show that MobileBERT is 4.3x smaller and 5.5x faster than BERT_BASE . BERT is one of the largest models ever in NLP, but suffers from heavy model size and high latency .
Approach:	They propose a tool to compress and accelerate the popular BERT model by task-agnostic application.
Outcome:	The proposed model is 4.3x smaller and 5.5x faster than BERT_BASE . it achieves competitive results on well-known benchmarks .

Copied to clipboard

Ziqing Yang, Yiming Cui, Zhipeng Chen, Wanxiang Che, Ting Liu, Shijin Wang, Guoping Hu

M2PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning (2024.emnlp-main)

Challenge:	Large pre-trained language models have hundreds of millions of parameters and take several gigabytes of memory to train and inference.
Approach:	They propose an open-source knowledge distillation toolkit designed for natural language processing that provides a set of predefined distillation methods and can be extended with custom code.
Outcome:	The proposed method is comparable with or even higher than the public distilled BERT models with similar numbers of parameters.

Copied to clipboard

Taowen Wang, Yiyang Liu, James Liang, Junhan Zhao, Yiming Cui, Yuning Mao, Shaoliang Nie, Jiahao Liu, Fuli Feng, Zenglin Xu, Cheng Han, Lifu Huang, Qifan Wang, Dongfang Liu

Exploiting Contextual Knowledge in LLMs through 𝒱-usable Information based Layer Enhancement (2025.acl-long)

Challenge:	Multimodal Large Language Models (MLLMs) exhibit remarkable performance across a wide range of domains.
Approach:	They propose a multimodal prompt tuning approach for efficient instruction tuning of MLLMs.
Outcome:	The proposed approach shows superior performance on multimodal evaluation datasets compared to state-of-the-art methods.

Copied to clipboard

Xiaowei Yuan, Zhao Yang, Ziyang Huang, Yequan Wang, Siqi Fan, Yiming Ju, Jun Zhao, Kang Liu

Challenge:	Existing approaches to enhance the context-faithfulness of Large Language Models (LLMs) ignore the fundamental mechanism of how contextual information is processed within LLMs’ internal states.
Approach:	They propose a method that enhances the utilization of contextual knowledge within LLMs’ internal representations by employing V-usable information analysis.
Outcome:	The proposed method improves context-faithfulness generation in Question-Answering tasks, particularly in scenarios involving unknown or conflicting contextual knowledge.

MHGRL: An Effective Representation Learning Model for Electronic Health Records (2024.lrec-main)

Copied to clipboard

Feiyan Liu, Liangzhi Li, Xiaoli Wang, Feng Luo, Chang Liu, Jinsong Su, Yiming Qian

Live-Aid: A Large-Scale Dialogue Dataset and Benchmark for Interleaved Multi-party Interactions in Live Streaming (2026.findings-acl)

Challenge:	Effective EHR representations are key to achieving high performance in healthcare applications.
Approach:	They propose a multimodal heterogeneous graph-enhanced representation learning to learn EHR representations using medical ontology and textual notes.
Outcome:	The proposed model outperforms baseline models on two real clinical datasets in downstream tasks.

Copied to clipboard

Yiming Lei, Yize Fan, Zeming Liu, Jiaji Dong, Hui Qiu, Haitao Leng, Qingjie Liu, Kehai Chen, Tingting Gao, Yunhong Wang

Dynamic Transformers Provide a False Sense of Efficiency (2023.acl-long)

Challenge:	Existing Multimodal Large Language Models struggle with dynamic interactions due to the scarcity of high-quality interleaved data.
Approach:	They propose a large-scale interleaved live interaction Chinese dataset with human-annotated video responses.
Outcome:	The proposed model can be used to evaluate live interactions in Chinese over 1,100 hours and 80,037 dialogue turns.

Copied to clipboard

Yiming Chen, Simin Chen, Zexin Li, Wei Yang, Cong Liu, Robby Tan, Haizhou Li

ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Challenge:	Pre-trained language models typically lead to high computational cost during inference.
Approach:	They propose a slowdown attack framework that can reduce inference efficiency by 80% by leveraging existing adversarial attacks targeting model accuracy.
Outcome:	The proposed framework can reduce the efficiency of multi-exit models by 80% on average, validating its effectiveness and generalization ability.

Copied to clipboard

Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo

A Span-Extraction Dataset for Chinese Machine Reading Comprehension (D19-1)

Challenge:	Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach:	They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome:	The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.

Copied to clipboard

Yiming Cui, Ting Liu, Wanxiang Che, Li Xiao, Zhipeng Chen, Wentao Ma, Shijin Wang, Guoping Hu

Machine Reading Comprehension

Challenge:	Existing reading comprehension datasets are mostly in English . MRC is a new field of research that aims to comprehend the context of articles and answer the questions based on them.
Approach:	They propose a Span-Extraction dataset for Chinese machine reading comprehension to add language diversities to existing reading comprehension datasets.
Outcome:	The proposed dataset is composed of 20,000 real questions annotated on Wikipedia paragraphs by human experts.

Context-Sensitive Generation of Open-Domain Conversational Responses (C18-1)

Copied to clipboard

Weinan Zhang, Yiming Cui, Yifa Wang, Qingfu Zhu, Lingzhi Li, Lianqiang Zhou, Ting Liu

Dialogue Systems & Conversational Agents

Challenge:	Existing studies on single-turn conversation generation focus on coherence and context-sensitive generation of open-domain conversational responses.
Approach:	They propose static and dynamic attention based approaches for context-sensitive generation of open-domain conversational responses.
Outcome:	The proposed model outperforms all baselines on automatic and human evaluation on two public datasets.

Understanding the Dark Side of LLMs’ Intrinsic Self-Correction (2025.acl-long)

Copied to clipboard

Qingjie Zhang, Di Wang, Haoting Qian, Yiming Li, Tianwei Zhang, Minlie Huang, Ke Xu, Hewu Li, Liu Yan, Han Qiu

Challenge:	Recent studies show that LLMs’ intrinsic self-correction fails without oracle labels as feedback.
Approach:	They propose to use one simple task and three complex tasks with state-of-the-art LLMs like ChatGPT, Llama, and DeepSeek to interpret LLM's intrinsic self-correction.
Outcome:	The proposed methods reveal the dark side of LLMs’ intrinsic self-correction for different tasks, especially for those failure cases.

Can MLLMs Understand the Deep Implication Behind Chinese Images? (2025.acl-long)

Copied to clipboard

Chenhao Zhang, Xi Feng, Yuelin Bai, Xeron Du, Jinchang Hou, Kaixin Deng, Guangzeng Han, Qinrui Li, Bingli Wang, Jiaheng Liu, Xingwei Qu, Yifei Zhang, Qixuan Zhao, Yiming Liang, Ziqiang Liu, Feiteng Fang, Min Yang, Wenhao Huang, Chenghua Lin, Ge Zhang, Shiwen Ni

Is Graph Structure Necessary for Multi-hop Question Answering? (2020.emnlp-main)

Challenge:	MLLMs perform poorly on traditional culture images, indicating limitations in understanding high-level semantics and lacking a deep knowledge base of Chinese traditional culture.
Approach:	They propose to use Chinese images to assess MLLMs' higher-order perception and understanding of Chinese visual content.
Outcome:	The proposed model incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, to ensure the authenticity of the Chinese context.

Copied to clipboard

Nan Shao, Yiming Cui, Ting Liu, Shijin Wang, Guoping Hu

Structured Data in NLP

Challenge:	Existing studies focus on multi-hop question answering across multiple documents or paragraphs.
Approach:	They propose a graph neural network to deal with graph structure in textual multi-hop reasoning . they propose 'self-attention' and propose removing entire graph structure may not hurt the final results .
Outcome:	The proposed model shows that graph-attention or the entire graph structure can be replaced by self-attention . hotpotQA is a widely used benchmark for multi-hop question answering .

STRUX: An LLM for Decision-Making with Structured Explanations (2025.naacl-short)

Copied to clipboard

Yiming Lu, Yebowen Hu, Hassan Foroosh, Wei Jin, Fei Liu

CasEE: A Joint Learning Framework with Cascade Decoding for Overlapping Event Extraction (2021.findings-acl)

Challenge:	a new LLM decision-making framework is designed to help users understand how and why decisions are made.
Approach:	They introduce a new LLM decision-making framework called STRUX that provides structured explanations for LLM decisions.
Outcome:	The proposed framework improves decision-making by providing structured explanations . it has been evaluated on the task of forecasting stock investment decisions based on earnings call transcripts - superior performance against strong baselines compared with previous frameworks based upon earnings call transcriptions demonstrating superior performance .

Copied to clipboard

Jiawei Sheng, Shu Guo, Bowen Yu, Qian Li, Yiming Hei, Lihong Wang, Tingwen Liu, Hongbo Xu

Revisiting Pre-Trained Models for Chinese Natural Language Processing (2020.findings-emnlp)

Challenge:	Existing methods assume that events appear in sentences without overlaps . overlapping event extraction is a challenging task in natural language understanding .
Approach:	They propose a joint learning framework with cascade decoding for overlapping event extraction . they sequentially perform type detection, trigger extraction and argument extraction based on the specific former prediction .
Outcome:	The proposed framework improves on a public event extraction benchmark . it sequentially performs type detection, trigger extraction and argument extraction .

Copied to clipboard

Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, Guoping Hu

EfficientLLM: Unified Pruning-Aware Pretraining for Auto-Designed Compact Language Models (2026.acl-long)

Challenge:	Existing pre-trained language models have shown tremendous improvements across various NLP tasks.
Approach:	They propose to revisit Chinese pre-trained language models to examine their effectiveness in a non-English language and release the Chinese pretrained model series to the community.
Outcome:	The proposed model improves on RoBERTa in several ways, especially the masking strategy that adopts MLM as correction (Mac).

Copied to clipboard

Xingrun Xing, Zheng Liu, Shitao Xiao, Boyan Gao, Yiming Liang, Haokun Lin, Xianlin Zeng, Guoqi Li, Jiajun Zhang

FRAME: Feedback-Refined Agent Methodology for Enhancing Medical Research Insights (2025.findings-acl)

Challenge:	Large language models (LLMs) driven by scaling laws can be developed in large model sizes.
Approach:	They propose a pruning-aware pretraining approach that decouples LLM pruning from direct pretraining.
Outcome:	The proposed model outperforms pretraining models with 100M 1B parameters in commen sense benchmarks.

Copied to clipboard

Chengzhang Yu, Yiming Zhang, Zhixin Liu, Zenghui Ding, Yining Sun, Zhanpeng Jin

Self-Evolving GPT: A Lifelong Autonomous Experiential Learner (2024.acl-long)

Challenge:	Existing approaches to automate scientific research are limited by human cognitive constraints and timeintensive workflows.
Approach:	They propose a framework that enhances medical paper generation through iterative refinement and structured feedback.
Outcome:	The proposed framework achieves significant improvements over conventional methods across multiple models and evaluation dimensions.

Copied to clipboard

Jinglong Gao, Xiao Ding, Yiming Cui, Jianbai Zhao, Hepeng Wang, Ting Liu, Bing Qin

GATE: Graph-based Adaptive Tool Evolution Across Diverse Tasks (2026.acl-long)

Challenge:	Existing approaches to provide LLMs with textual task-solving experience rely on manual efforts to acquire and apply such experience for each task.
Approach:	They propose a lifelong autonomous experiential learning framework based on LLMs that learns and accumulates experience through experience transfer and induction.
Outcome:	The proposed framework performs reliably in each intermediate step and improves GPT-3.5 and GPT-4 on widely used NLP datasets.

Copied to clipboard

Jianwen Luo, Yiming Huang, Jinxiang Meng, Fangyu Lei, Shizhu He, Xiao Liu, Shanshan Jiang, Bin Dong, Jun Zhao, Kang Liu

COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)

Challenge:	Existing toolsets that use large language models are limited to single-task settings.
Approach:	They propose a framework that dynamically constructs and evolves a hierarchical graph of reusable tools across multiple scenarios.
Outcome:	The proposed framework achieves up to 4.3 faster milestone completion in Minecraft compared to the previous state-of-the-art method and provides an average improvement of 9.23% over existing tool-making methods in code generation tasks and 10.03% in agent tasks.

Copied to clipboard

Siwei Wu, JinCheng Ren, Xeron Du, Shuyue Guo, Xingwei Qu, Yiming Liang, Jie Liu, Yunwen Li, Tyler Loakman, Tianyu Zheng, Boyu Feng, Huaqing Yuan, Zili Wang, Jiaheng Liu, Wenhao Huang, Chenglin Cai, Haoran Que, Jian Yang, Yuelin Bai, Zekun Moore Wang, Zhouliang Yu, Qunshu Lin, Ding Pan, Yuchen Eleanor Jiang, Tiannan Wang, Wangchunshu Zhou, Shenzhi Wang, Xingyuan Bu, Minghao Liu, Guoyin Wang, Ge Zhang, Chenghua Lin

Do BERT-Like Bidirectional Models Still Perform Better on Text Classification in the Era of LLMs? (2025.findings-emnlp)

Challenge:	Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation.
Approach:	They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets.
Outcome:	The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark.

Copied to clipboard

Junyan Zhang, Yiming Huang, Shuliang Liu, Yubo Gao, Xuming Hu

D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension (D19-58)

Challenge:	Rapid adoption of LLMs has overshadowed the potential advantages of traditional BERT-like models in text classification.
Approach:	They compare BERT-like models fine-tuning, LLM internal state utilization, and LLM zero-shot inference across six datasets.
Outcome:	The proposed method outperforms LLMs on six challenging datasets.

Copied to clipboard

Hongyu Li, Xiyuan Zhang, Yibing Liu, Yiming Zhang, Quan Wang, Xiangyang Zhou, Jing Liu, Hua Wu, Haifeng Wang

Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese (2025.emnlp-main)

Challenge:	MRC requires machines to understand text and answer questions about the text.
Approach:	They propose a simple system Baidu submitted for MRQA 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models.
Outcome:	The proposed system is ranked at top 1 of all participants in terms of averaged F1 score.

Copied to clipboard

Yikang Liu, Wanyang Zhang, Yiming Wang, Jialong Tang, Pei Zhang, Baosong Yang, Fei Huang, Rui Wang, Hai Hu

Machine Translation

Challenge:	Translationese is a linguistic property that is often introduced in the translation process that is different from those of original texts.
Approach:	They propose to use synthesized translations and translations in the wild to evaluate T-index's generalizability in cross-domain settings and its validity against human judgments.
Outcome:	The proposed measure can generalize to unseen genres, authors, and language pairs.

R-Judge: Benchmarking Safety Risk Awareness for LLM Agents (2024.findings-emnlp)

Copied to clipboard

Tongxin Yuan, Zhiwei He, Lingzhong Dong, Yiming Wang, Ruijie Zhao, Tian Xia, Lizhen Xu, Binglin Zhou, Fangqi Li, Zhuosheng Zhang, Rui Wang, Gongshen Liu

Challenge:	Large language models (LLMs) have shown compelling abilities in reasoning, decision-making, and instruction following.
Approach:	They propose a benchmark to evaluate the proficiency of large language models (LLMs) in judging and identifying safety risks given agent interaction records.
Outcome:	The proposed model outperforms the best-performing model, GPT-4o, while no other models significantly exceed the random.

Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting (2020.emnlp-main)

Copied to clipboard

Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu, Xiangzhan Yu

Uncertainty-Calibrated Elastic Alignment for Multimodal Sentiment Analysis with Missing Modalities (2026.findings-acl)

Challenge:	Existing methods to fine-tune deep pretrained language models face catastrophic forgetting problems.
Approach:	They propose a recall and learn mechanism which integrates pretraining and downstream tasks into a single mechanism.
Outcome:	The proposed method achieves state-of-the-art performance on the GLUE benchmark and better average performance than directly fine-tuning of BERT-large.

Copied to clipboard

Kang He, Yuzhe Ding, Rao Fu, Yukang Feng, Kaipeng Zhang, Yiming Liu, Fei Li, Chong Teng, Donghong Ji

S3HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering (2023.acl-short)

Challenge:	Existing methods for multimodal sentiment analysis are often dynamically incomplete.
Approach:	They propose a new uncertainty-calibrated elastic alignment framework to address these issues by employing probabilistic imputation to capture cross-modal ambiguity and leverage the estimated uncertainty to drive elastic alignment.
Outcome:	The proposed framework outperforms state-of-the-art models in multiple benchmarks and consistently outperformed existing models.

Copied to clipboard

Fangyu Lei, Xiang Li, Yifan Wei, Shizhu He, Yiming Huang, Jun Zhao, Kang Liu

TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction (2023.findings-emnlp)

Challenge:	Existing question answering systems use a retriever-reader framework to answer multi-hop questions . existing models lack retrieval, selector, and reasoner capabilities .
Approach:	They propose a three-stage text tableQA framework which comprises of retriever, selector, and reasoner.
Outcome:	The proposed framework outperforms baseline methods in the few-shot setting and ranks first on the HybridQA leaderboard.

Copied to clipboard

Junyi Liu, Liangzhi Li, Tong Xiang, Bowen Wang, Yiming Qian

Benchmarking Robustness of Machine Reading Comprehension Models (2021.findings-acl)

Challenge:	ChatGPT and GPT-4 are commercial large language models (LLMs) however, they may produce vague responses or incorrect answers in certain specialized domains.
Approach:	They propose a token compression scheme that uses summarization and semantic compression to reduce the token size of LLMs.
Outcome:	The proposed method reduces token size by doing summarization and semantic compression while reducing token size with only 1.6% accuracy drop.

Copied to clipboard

Chenglei Si, Ziqing Yang, Yiming Cui, Wentao Ma, Ting Liu, Shijin Wang

Robustness in NLP

Challenge:	Existing benchmarks only evaluate models' robustness under test-time perturbations or adversarial attacks.
Approach:	They propose a model-agnostic benchmark to evaluate models' robustness under adversarial attacks.
Outcome:	The proposed model-agnostic benchmark evaluates models under four different types of adversarial attacks.

A Hierarchical Explanation Generation Method Based on Feature Interaction Detection (2023.findings-acl)

Copied to clipboard

Yiming Ju, Yuanzhe Zhang, Kang Liu, Jun Zhao

Explainability & Interpretability in NLP

Challenge:	Existing work on hierarchical attributions tends to limit text groups to a continuous text span, which is difficult for humans to read.
Approach:	They propose a method which captures feature interactions and converts non-hierarchical explanations into hierarchical versions.
Outcome:	The proposed method can convert ubiquitous non-hierarchical explanations into their corresponding hierarchical versions.

Aligning Large Multimodal Models with Factually Augmented RLHF (2024.findings-acl)

Copied to clipboard

Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liangyan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, Trevor Darrell

MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Models (2026.findings-eacl)

Challenge:	Large Multimodal Models (LMMs) are built across modalities and the misalignment between two modality can result in "hallucination" . developing LMMs faces challenges such as a lack of data and a limited number of data sets.
Approach:	They propose a new algorithm that augments the reward model with additional factual information such as image captions and ground-truth multi-choice options.
Outcome:	The proposed approach improves on the LLaVA-Bench dataset with the 96% performance level of the text-only GPT-4 and an improvement of 60% on MMHAL-BENCH over other baselines.

Copied to clipboard

Siwei Wu, King Zhu, Yu Bai, Yiming Liang, Yizhi Li, Haoning Wu, Jiaheng Liu, Ruibo Liu, Xingwei Qu, Xuxin Cheng, Ge Zhang, Wenhao Huang, Chenghua Lin

Enhancing Multiple-choice Machine Reading Comprehension by Punishing Illogical Interpretations (2021.emnlp-main)

Challenge:	Current multimodal benchmarks focus on facts within individual images, but neglect associative relations among multiple images.
Approach:	They propose a multi-image relational association task and a MMRA benchmark to evaluate LVLMs.
Outcome:	The proposed benchmarks show that entity-level multi-image perception tasks pose greater challenges than image-level tasks.

Copied to clipboard

Yiming Ju, Yuanzhe Zhang, Zhixing Tian, Kang Liu, Xiaohuan Cao, Wenting Zhao, Jinlong Li, Jun Zhao

DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models (2024.emnlp-main)

Challenge:	Multiple-choice MRC is one of the most studied tasks in MRC due to the convenience of evaluation and the flexibility of answer format.
Approach:	They propose to use multiple-choice MRC to explain a trained model and reveal how it arrives at the prediction by punishing illogical attributions.
Outcome:	The proposed method improves model performance without external information and model structure change without any external information.

Copied to clipboard

Yiming Huang, Jianwen Luo, Yan Yu, Yitong Zhang, Fangyu Lei, Yifan Wei, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, Kang Liu

Programming Languages in NLP

Challenge:	DA-Code is a code generation benchmark designed to assess LLMs on agent-based data science tasks.
Approach:	They propose a code generation benchmark specifically designed for LLMs on agent-based data science tasks.
Outcome:	The benchmark performs better than existing frameworks, but lacks accuracy . it is based on real-world data, and includes examples that cover a wide range of tasks .

Conversational Word Embedding for Retrieval-Based Dialog System (2020.acl-main)

Copied to clipboard

Wentao Ma, Yiming Cui, Ting Liu, Dong Wang, Shijin Wang, Guoping Hu

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)

Challenge:	Existing word embedding methods for retrieval-based dialog systems are based on co-occurrence statistics and train them based upon the same co-existence statistics.
Approach:	They propose a conversational word embedding method which uses the conversation pairs post, reply, and 'reply' they introduce a word alignment model from statistical machine translation and train it on word-level and sentence-level.
Outcome:	The proposed method improves the quality of the selected response on retrieval-based dialog systems.

Copied to clipboard

Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang

BadActs: A Universal Backdoor Defense in the Activation Space (2024.findings-acl)

Challenge:	Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns.
Approach:	They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users.
Outcome:	The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks.

Copied to clipboard

Biao Yi, Sishuo Chen, Yiming Li, Tong Li, Baolei Zhang, Zheli Liu

Data or Language Supervision: What Makes CLIP Better than DINO? (2025.findings-emnlp)

Challenge:	Backdoor attacks pose an increasingly severe security threat to Deep Neural Networks . existing methods focused on the word space are ineffective against feature-space triggers - a recent study has shown .
Approach:	They propose a backdoor defense that purifies backdoor samples in the activation space . they aim to eliminate backdoor triggers while preserving the integrity of clean data .
Outcome:	The proposed method achieves state-of-the-art against backdoor attacks on clean data.

Copied to clipboard

Yiming Liu, Yuhui Zhang, Dhruba Ghosh, Ludwig Schmidt, Serena Yeung-Levy

Competition-Level Problems are Effective LLM Evaluators (2024.findings-acl)

Challenge:	CLIP outperforms self-supervised models like DINO as vision encoders for vision-language models (VLMs) but it remains unclear whether this advantage stems from CLIP’s language supervision or its much larger training data.
Approach:	Embedding analysis shows CLIP captures high-level semantics while DINO is more responsive to low-level features like colors and styles.
Outcome:	Embedding analysis shows that CLIP captures high-level semantics, while DINO is more responsive to low-level features like colors and styles.

Copied to clipboard

Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Shuai Lu, Fangyu Lei, Yaobo Liang, Yelong Shen, Chen Lin, Nan Duan, Weizhu Chen

OpenRLHF: A Ray-based Easy-to-use, Scalable and High-performance RLHF Framework (2025.emnlp-demos)

Challenge:	Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet there is ongoing debate about their capabilities and the potential data contamination problem.
Approach:	They propose to evaluate the reasoning capabilities of large language models in solving recent competition-level programming problems in Codeforces.
Outcome:	The proposed model has experienced a cliff-like decline in problems after September 2021, which shows the potential data contamination and the challenges for any existing LLM to solve unseen complex reasoning problems.

Copied to clipboard

Jian Hu, Xibin Wu, Wei Shen, Jason Klein Liu, Weixun Wang, Songlin Jiang, Haoran Wang, Hao Chen, Bin Chen, Wenkai Fang, null Xianyu, Yu Cao, Haotian Xu, Yiming Liu

Logic Traps in Evaluating Attribution Scores (2022.acl-long)

Challenge:	Existing RLHF frameworks face inference bottlenecks and complexity barriers restricting their accessibility for newcomers.
Approach:	They propose an open-source RLHF framework that can be used to train large language models.
Outcome:	The proposed framework achieves superior training efficiency with speedups ranging from 1.22 to 1.68 across different model sizes compared to state-of-the-art frameworks, while requiring significantly fewer lines of code for implementation.

Copied to clipboard

Yiming Ju, Yuanzhe Zhang, Zhao Yang, Zhongtao Jiang, Kang Liu, Jun Zhao