Papers by Zheng Zhou
CLEME2.0: Towards Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction (2025.acl-long)
Copied to clipboard
Jingheng Ye, Zishan Xu, Yinghui Li, Linlin Song, Qingyu Zhou, Hai-Tao Zheng, Ying Shen, Wenhao Jiang, Hong-Gee Kim, Ruitong Liu, Xin Su, Zifei Shan
| Challenge: | Existing studies have focused on the interpretability of Grammatical Error Correction (GEC) evaluation metrics, but the interpretabilty of these metrics has been neglected. |
| Approach: | They propose a reference-based metric that describes four aspects of GEC systems: hit-correction, wrong-corrections, under-correcties, and over-corrects. |
| Outcome: | The proposed metric reveals critical qualities and locates drawbacks of GEC systems. |
ToolGrad: Efficient Tool-use Dataset Generation with Textual “Gradients” (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work synthesizes tool-use LLM datasets by first generating a user query, then complex tool-using annotations like DFS. |
| Approach: | They propose an agentic framework that synthesizes user queries and generates valid tool-use chains . they propose a dataset with more complex tool use, lower cost, and almost 100% pass rate . |
| Outcome: | Experiments show that tools trained on ToolGrad outperform expensive baseline datasets and proprietary LLMs. |
Word Segmentation by Separation Inference for East Asian Languages (2022.findings-acl)
Copied to clipboard
| Challenge: | Chinese Word Segmentation (CWS) is a sequence labeling task that divides sentences into words . despite diverse tagging schemas, they all carry implicit position information. |
| Approach: | They propose to model the separation state of every two consecutive characters by tagging them as two tags. |
| Outcome: | The proposed framework outperforms state-of-the-art on Japanese and Korean Word Segmentation datasets. |
Unified Thinker: A General Reasoning Core for Image Generation (2026.acl-long)
Copied to clipboard
Sashuai Zhou, Qiang Zhou, Jijin Hu, Hanqing Yang, Yue Cao, Junpeng Ma, Yinchao Ma, Jun Song, Tiezheng Ge, Cheng Yu, Bo Zheng, Zhou Zhao
| Challenge: | generative models struggle with logic-intensive instruction following, exposing a persistent reasoning–execution gap. |
| Approach: | They propose a task-agnostic reasoning architecture for general image generation . they propose pixel-level feedback to ground the Thinker's policy in pixel feedback . |
| Outcome: | The proposed system significantly improves image reasoning and generation quality. |
Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characters (2024.acl-long)
Copied to clipboard
Yinghui Li, Zishan Xu, Shaoshen Chen, Haojing Huang, Yangning Li, Shirong Ma, Yong Jiang, Zhongli Li, Qingyu Zhou, Hai-Tao Zheng, Ying Shen
| Challenge: | Existing studies focus on misspelled characters, ignoring faked characters which are more common and difficult to correct. |
| Approach: | They propose to use Chinese character checking to identify and correct wrong characters in texts by human annotation. |
| Outcome: | The proposed dataset is the first real-world visual and the largest human-crafted dataset for the Chinese character checking scenario. |
v-HUB: A Benchmark for Video Humor Understanding from Vision and Sound (2026.acl-long)
Copied to clipboard
Zhengpeng Shi, Yanpeng Zhao, Jianqun Zhou, Yuxuan Wang, Qinrong Cui, Wei Bi, Song-Chun Zhu, Bo Zhao, Zilong Zheng
| Challenge: | Humor enriches our daily lives and appears in many forms, from jokes and cartoons to comedies and viral videos. |
| Approach: | They introduce a video humor understanding benchmark to test their ability to understand humor from visual cues. |
| Outcome: | The proposed video humor understanding benchmark is based on a collection of short videos . it features rich annotations and a study of environmental sound that can enhance humor . |
FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs’ Responsiveness to Human Feedback (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing research focuses on benchmarking LLMs in single-turn dialogues, neglecting the nuanced nature of human feedback within real-world usage scenarios. |
| Approach: | They propose a fine-grained, multi-task benchmark designed to evaluate LLMs’ responsiveness to human feedback under real-world usage scenarios in Chinese. |
| Outcome: | The proposed benchmarks show that human feedback can significantly impact LLMs’ responsiveness in real-world usage scenarios. |
Contextual Knowledge Learning for Dialogue Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Incorporating conversational context and knowledge into dialogue generation models has been essential for improving the quality of the generated responses. |
| Approach: | They propose a method to incorporate conversational context and knowledge into dialogue generation models . they use Latent Vectors to capture the relationship between context and knowing . |
| Outcome: | The proposed approach improves performance with two standard datasets and human evaluations. |
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)
Copied to clipboard
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM. |
| Approach: | They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks. |
| Outcome: | The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting. |
Enhancing Extractive Question Answering in Multiparty Dialogues with Logical Inference Memory Network (2025.coling-main)
Copied to clipboard
| Challenge: | Existing models for multiparty dialogue question answering (QA) do not consider logical inference relations in multiparty dialogs, leading to suboptimal performance. |
| Approach: | They propose a memory network with logical inference for extractive QA in multiparty dialogues. |
| Outcome: | The proposed model achieves state-of-the-art on Molweni and FriendsQA benchmarks. |
Real, Fake, or Manipulated? Detecting Machine-Influenced Text (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on machine generated text detection focused on identifying whether document was human or machine written, ignoring these fine-grained uses. |
| Approach: | They propose a machine-influenced text detector that learns to separate text samples from four primary types . the detector uses a subcategory guidance module to help separate the fine-grained categories . |
| Outcome: | The proposed detector outperforms the state-of-the-art in five LLMs and six domains. |
RubricBench: Aligning Model-Generated Rubrics with Human Standards (2026.acl-long)
Copied to clipboard
Junyi Zhou, Qiyuan Zhang, Yufei Wang, Fuyuan Lyu, Yidong Ming, Can Xu, Qingfeng Sun, Kai Zheng, Peng Kang, Xue Liu, Chen Ma
| Challenge: | Existing benchmarks lack discriminative complexity and ground-truth rubric annotations required for rigorous evaluation. |
| Approach: | They propose a curated benchmark with 1,147 pairwise comparisons to assess the reliability of rubric-based evaluation. |
| Outcome: | The proposed benchmarks show that they support diverse domains, exhibit discriminative ability, provide high-quality annotations, and include human-authored rubrics. |
DrAgent: Empowering Large Language Models as Medical Agents for Multi-hop Medical Reasoning (2025.findings-emnlp)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xin Liu, Zhengyang Wang, Xianfeng Tang, Shiyang Li, Xiang He, Ruijie Wang, Bing Yin, Xiao Gu, Lei Clifton, David A. Clifton
| Challenge: | commercial LLMs can be difficult to use in real-world clinical decision-making . a lightweight LLM can be used to collaborate with diverse clinical tools . |
| Approach: | They propose a lightweight LLM that can be used to build medical LLMs as agents . they use recursive curriculum learning to optimize the LLM in an easy-to-hard progression . |
| Outcome: | The proposed approach outperforms human experts in medical examinations on diverse datasets. |
TexSmart: A System for Enhanced Natural Language Understanding (2021.acl-demo)
Copied to clipboard
Lemao Liu, Haisong Zhang, Haiyun Jiang, Yangming Li, Enbo Zhao, Kun Xu, Linfeng Song, Suncong Zheng, Botong Zhou, Dick Zhu, Xiao Feng, Tao Chen, Tao Yang, Dong Yu, Feng Zhang, ZhanHui Kang, Shuming Shi
| Challenge: | TexSmart supports fine-grained named entity recognition (NER) Large-scale fine-granular entity types are expected to provide richer semantic information for downstream NLP applications. |
| Approach: | They introduce TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities. |
| Outcome: | The proposed system supports fine-grained named entity recognition (NER) and enhanced semantic analysis functions. |
Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant strides towards Artificial General Intelligence, but training them from scratch is prohibitively expensive. |
| Approach: | They propose to continuously pre-train LLMs from existing pre-trained LLM models by using a set of parameters instead of randomly initializing them. |
| Outcome: | The proposed approach saves significant resources and accelerates convergence and performance. |
Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning (2024.emnlp-main)
Copied to clipboard
Lu Chen, Rui Zheng, Binghai Wang, Senjie Jin, Caishuang Huang, Junjie Ye, Zhihao Zhang, Yuhao Zhou, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Current methods rely on ranking losses to teach reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions. |
| Approach: | They propose a method that incorporates contrastive learning into the reward modeling process to enhance generalization and stabilize the reinforcement learning training process. |
| Outcome: | The proposed method enhances generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences. |
Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)
Copied to clipboard
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu
| Challenge: | Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature. |
| Approach: | They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management. |
| Outcome: | The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench. |
Reinforcement Learning with Semantic Rewards Enables Low-Resource Language Expansion without Alignment Tax (2026.findings-acl)
Copied to clipboard
Zeli Su, Ziyin Zhang, Zhou Liu, Xuexian Song, Zhankai Xu, Longfei Zheng, Xiaolu Zhang, Rong Fu, Guixian Xu, Wentao Zhang
| Challenge: | Extending large language models to low-resource languages often incurs an "alignment tax" token-level fine-tuning enforces token-level surface imitation on narrow and biased data distributions. |
| Approach: | They propose a semantic-space alignment paradigm powered by group-level semantic rewards instead of likelihood maximization. |
| Outcome: | The proposed model acquires low-resource capa- bilities while mitigating alignment tax on Tibetan–Chinese machine translation and Ti- betan headline generation. |
CLEME: Debiasing Multi-reference Evaluation for Grammatical Error Correction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Evaluating the performance of Grammatical Error Correction systems is a challenging task due to its subjectivity. |
| Approach: | They propose a method to evaluate GEC systems in multi-reference evaluation setting . they use consistent edit boundaries to eliminate bias caused by inconsistent edit boundaries . |
| Outcome: | The proposed evaluation metric eliminates bias caused by inconsistent edit boundaries on six English reference sets. |
DARM: Distribution-Aware Reward Modeling by Alleviating Biases from Low Preference-Context Dependency Data (2026.acl-long)
Copied to clipboard
Shaofan Liu, Guoqiang Zhang, Shihan Dou, Huiyuan Zheng, Yiming Zhou, Junjie Ye, Shaowen Wang, Shichun Liu, Jiazheng Zhang, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing methods for training reward models are vulnerable to context neglect and degraded accuracy. |
| Approach: | They propose distribution-aware reward modeling that augments the RM objective with a conditional mutual information regularizer that maximizes context and the predicted reward conditioned on the response. |
| Outcome: | The proposed model improves performance in RLHF and improves accuracy in other settings. |
MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)
Copied to clipboard
Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
AdaV: Adaptive Text-visual Redirection for Vision-Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Vision-language models often generate excessive visual tokens, leading to poor performance . a novel training-free visual token pruning method is proposed to improve performance despite the computational cost associated with VLMs. |
| Approach: | They propose a training-free visual token pruning method that reduces biased token pruning . they plan to open-source the code upon publication . |
| Outcome: | The proposed method reduces biased token pruning and enhances model robustness with limited visual token budget. |
Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus (2023.emnlp-main)
Copied to clipboard
Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, Luoyi Fu
| Challenge: | Existing methods for detecting hallucinations in LLMs rely on external knowledge for reference retrieval or require sampling multiple responses for consistency verification. |
| Approach: | They propose a reference-free, uncertainty-based method for detecting hallucinations in Large Language Models that imitates human focus in factuality checking from three aspects: focus on the most informative keywords; focus on unreliable tokens in historical context; focus of token properties such as token type and token frequency. |
| Outcome: | The proposed method achieves state-of-the-art performance across all evaluation metrics and eliminates the need for additional information. |
PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (2026.acl-long)
Copied to clipboard
Jingcheng Hu, Yinmin Zhang, Shijie Shang, Xiaobo Yang, Yue Peng, Zhewei Huang, Hebin Zhou, Xin Wu, Jie Cheng, Fanqi Wan, Xiangwen Kong, Chengyuan Yao, Kaiwen Yan, Ailin Huang, Hongyu Zhou, Qi Han, Zheng Ge, Xiangyu Zhang, Heung-Yeung Shum
| Challenge: | Parallel Coordinated Reasoning (PaCoRe) overcomes a central limitation of contemporary language models: their inability to scale test-time compute (TTC) far beyond sequential reasoning under a fixed context window. |
| Approach: | They propose a training-and-inference framework to overcome a central limitation of language models: their inability to scale test-time compute (TTC) under a fixed context window. |
| Outcome: | The proposed model scales to multi-million-token effective TTC without exceeding context limits. |
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models (2025.emnlp-main)
Copied to clipboard
Jiajun Zhou, Yifan Yang, Kai Zhen, Ziyue Liu, Yequan Zhao, Ershad Banijamali, Athanasios Mouchtaris, Ngai Wong, Zheng Zhang
| Challenge: | Large Language Models (LLMs) are quantized to lower precision to reduce memory cost and latency in inference. |
| Approach: | They propose a quantized zeroth-order framework for fine-tuning Large Language Models (LLMs) using low-precision forward passes. |
| Outcome: | The proposed method achieves comparable results to first-order methods in FP8 and superior accuracy in INT8 and INT4 training. |
Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension (2022.acl-long)
Copied to clipboard
Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer
| Challenge: | Existing QA datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. |
| Approach: | They propose to use FairytaleQA to generate 10,580 questions based on 278 children-friendly stories to assess model's fine-grained learning skills. |
| Outcome: | The proposed dataset consists of 10,580 questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations. |
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback (2025.findings-acl)
Copied to clipboard
Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Dayiheng Liu, Chang Zhou, Wen Xiao, Tianyu Liu, Baobao Chang
| Challenge: | Existing mathematical verifiers are trained with binary classification labels, which are not informative enough for the model to accurately assess the solutions. |
| Approach: | They propose a natural language feedback-enhanced verifier that can validate the correctness of response generated by policy models by constructing automatically generated training data and a two-stage training paradigm. |
| Outcome: | The proposed verifier significantly improves in verification and reinforcement learning and alleviates data-demanding problems of the reward model. |
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement (2023.findings-emnlp)
Copied to clipboard
Zhiheng Xi, Senjie Jin, Yuhao Zhou, Rui Zheng, Songyang Gao, Jia Liu, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing prompting methods have been used to enhance multistep reasoning capabilities of large language models, but they have overlooked the potential of formulating higher-quality problems. |
| Approach: | They propose a method that starts from the problem side and refines problems to be more comprehensible and solvable for models. |
| Outcome: | The proposed method achieves notable and consistent effectiveness on five reasoning benchmarks across different models. |
AutoMIR: Effective Zero-Shot Medical Information Retrieval without Relevance Labels (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Effective zero-shot dense retrieval in the medical domain remains difficult due to the scarcity of relevance-labeled data. |
| Approach: | They propose a framework that leverages large language models to generate hypothetical documents . they also propose 'CMIRB' to provide a rigorous evaluation suite . |
| Outcome: | The proposed framework outperforms HyDE in retrieval accuracy and generalization . it leverages large language models to generate hypothetical documents conditioned on a query . |
Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to build a visual dialog (VD) Questioner do not provide explicit guidance for questioner to generate visually related and informative questions. |
| Approach: | They propose a Related entity enhanced Questioner that learns entity-based questioning strategy from human dialogs. |
| Outcome: | The proposed approach achieves state-of-the-art performance on image-guessing task and question diversity. |
StructuThink: Reasoning with Task Transition Knowledge for Autonomous LLM-Based Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | StructuThink framework enhances LLMs' ability to ground decisions in domain-specific scenarios. |
| Approach: | They propose a knowledge-structured reasoning framework that enhances LLM-based agents with explicit decision constraints. |
| Outcome: | The proposed framework achieves higher task success rates and more efficient action sequences than baseline methods. |
Multi-Programming Language Sandbox for LLMs (2025.acl-demo)
Copied to clipboard
Shihan Dou, Jiazheng Zhang, Jianxiang Zang, Yunbo Tao, Weikang Zhou, Haoxiang Jia, Shichun Liu, Yuming Yang, Shenxi Wu, Zhiheng Xi, Muling Wu, Rui Zheng, Changze Lv, Limao Xiong, Shaoqing Zhang, Lin Zhang, Wenyu Zhan, Rongxiang Weng, Jingang Wang, Xunliang Cai, Yueming Wu, Ming Wen, Yixin Cao, Tao Gui, Xipeng Qiu, Qi Zhang, Xuanjing Huang
| Challenge: | MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs). |
| Approach: | They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models. |
| Outcome: | The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs). |
VIGIL: Defending LLM Agents Against Tool-Stream Injection via Verify-Before-Commit (2026.acl-long)
Copied to clipboard
| Challenge: | Existing defenses for indirect prompt injection are limited by static protection mechanisms . existing models prioritize injected rules due to strict alignment, whereas static protections sever the feedback loop required for adaptive reasoning. |
| Approach: | They propose a framework that shifts the paradigm from restrictive isolation to a verify-before-commit protocol. |
| Outcome: | The proposed framework outperforms state-of-the-art dynamic defenses by reducing the attack success rate by over 22% while more thandoubling utility under attack compared to static baselines. |
ProcessBench: Identifying Process Errors in Mathematical Reasoning (2025.acl-long)
Copied to clipboard
Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin
| Challenge: | Existing models fail to generalize to more challenging math problems, authors say . existing benchmarks related to assessing language models' reasoning process are limited . |
| Approach: | They propose a tool to measure language models' ability to identify erroneous steps in reasoning . they use two types of models: process reward models and critic models . |
| Outcome: | The proposed model outperforms existing models in evaluating language models' reasoning process . the best open-source model has demonstrated the critique capability competitive with the proprietary model . |
A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Check (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years, Chinese Spelling Check (CSC) has been greatly improved by designing task-specific pre-training methods or introducing auxiliary tasks. |
| Approach: | They propose to decompose Chinese Spelling Check into detection, reasoning, and searching subtasks and to train a module that is compatible with existing CSC models. |
| Outcome: | The proposed module can be trained for one model and benefit other models. |
From Implicit Graph Encoding to Explicit Evidence: A Training-Free LLM Framework for Temporal Knowledge Graph Reasoning (2026.findings-acl)
Copied to clipboard
Guo Tang, Ke Cheng, Huiming Fan, Heng Chang, Wenxiang Zheng, Xianhao Ou, Junjia Xiang, Ming Liu, Yujun Zhou, Li Lanyu, Bing Qin
| Challenge: | Existing Large Language Models (LLMs) struggle with implicit modality alignment and suboptimal graph linearization. |
| Approach: | They propose a training-free, test-time adaptive framework that reframes TKG prediction as explicit evidence-driven reasoning. |
| Outcome: | ExE-LLM outperforms fully trained graph neural networks on four benchmarks . it achieves SOTA performance in inductive settings, significantly outperforming fully trained neural networks . |
Locally Differentially Private In-Context Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Large pretrained language models (LLMs) have shown surprising In-Context Learning ability. |
| Approach: | They propose a locally differentially private framework of in-context learning for LLMs that can be augmented with a private database for some specific task. |
| Outcome: | The proposed framework can predict labels without additional parameter modifications without input-label pairs . |
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)
Copied to clipboard
Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Mengmeng Yang, Zheng Fu, Jinyu Miao, Yining Shi, He Zhe Lim, Li Liu, Tianbao Zhou, Hongyi Wang, Huang Yu, Yifei Hu, Guang Li, Guang Chen, Hao Ye, Lijun Sun, Diange Yang
| Challenge: | Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning. |
| Approach: | AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks. |
| Outcome: | Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% . |
Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples (2020.acl-main)
Copied to clipboard
| Challenge: | Previously studies focused on semantic tasks such as sentiment analysis, question answering and reading comprehension. |
| Approach: | They propose two approaches to study where and how adversarial examples exist in dependency parsing . they use a state-of-the-art parser to find adversarials in existing texts . |
| Outcome: | The proposed approaches show that adversarial examples exist in dependency parsing . they show that up to 77% of input examples admit adversarials . |
MegaPairs: Massive Data Synthesis for Universal Multimodal Retrieval (2025.acl-long)
Copied to clipboard
Junjie Zhou, Yongping Xiong, Zheng Liu, Ze Liu, Shitao Xiao, Yueze Wang, Bo Zhao, Chen Jason Zhang, Defu Lian
| Challenge: | despite the growing demand for multimodal retrieval, there is a lack of training data. |
| Approach: | They propose a data synthesis method that leverages vision language models and open-domain images to generate high-quality data. |
| Outcome: | The proposed method outperforms baseline models on 70 more datasets and can scale up. |
Empathetic Dialogue Generation via Sensitive Emotion Recognition and Sensible Knowledge Selection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Empathy is a key trait of everyday human conversations. |
| Approach: | They propose a serial encoding and Emotion-Knowledge interaction method for empathetic dialogue generation which is more sensitive to emotion dynamics in conversations. |
| Outcome: | The proposed method outperforms baseline evaluations on the utterance-level annotated EMPATHETICDIALOGUES. |
Transcending Scaling Laws with 0.1% Extra Compute (2023.emnlp-main)
Copied to clipboard
Yi Tay, Jason Wei, Hyung Chung, Vinh Tran, David So, Siamak Shakeri, Xavier Garcia, Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc Le, Mostafa Dehghani
| Challenge: | Existing scaling of language models is expensive and requires significant computational costs. |
| Approach: | They propose a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. |
| Outcome: | The proposed method significantly improves existing language models and their scaling curves with a relatively tiny amount of extra compute. |
VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multimodal retrieval are mostly text-oriented, which lack the capability to process visual information. |
| Approach: | They propose a multi-modal multi-text embedding model VISTA which extends a powerful text encoder with the image understanding capability by introducing visual token embedds. |
| Outcome: | The proposed model achieves superior performance across a variety of multi-modal retrieval tasks in zero-shot and supervised settings. |
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System (2025.acl-long)
Copied to clipboard
Haoyang Su, Renqi Chen, Shixiang Tang, Zhenfei Yin, Xinzhe Zheng, Jinzhe Li, Biqing Qi, Qi Wu, Hui Li, Wanli Ouyang, Philip Torr, Bowen Zhou, Nanqing Dong
| Challenge: | Recent AI methods have shown promise in tasks such as hypothesis generation and experimental design, but they fail to replicate the collaborative nature of real-world scientific practices. |
| Approach: | They propose a virtual scientific system that mimics the collaborative nature of scientific research by organizing a team of agents to generate, evaluate, and refine research ideas. |
| Outcome: | The proposed system outperforms the state-of-the-art method in producing new scientific ideas and offers valuable insights to guide future research. |
Cross-Lingual Dependency Parsing by POS-Guided Word Reordering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual dependency parsing rely on large corpus size and cost. |
| Approach: | They propose a cross-lingual dependency parsing approach based on word reordering . they propose to train a model that transfers knowledge learned in one or multiple languages to target languages . |
| Outcome: | The proposed approach outperforms the baseline approach in Hindi and Latin by 15.3% and 6.7%. |
Reinforcement Learning on Pre-Training Data (2026.acl-long)
Copied to clipboard
Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang
| Challenge: | Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings. |
| Approach: | They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data. |
| Outcome: | Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base. |
Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models (2025.acl-long)
Copied to clipboard
Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
| Challenge: | Existing Mixture-of-Experts training frameworks use a micro-batch to calculate LBL . micro-batches are restricted to a single sequence, preventing expert specialization . |
| Approach: | They propose to use a global-batch to loosen the load balance constraint for MoEs models . they propose to synchronize fi across micro-batches and then use it to calculate the LBL . |
| Outcome: | The proposed global-batch LBL improves the domain specialization of experts . the micro-battery LBL is almost at the sequence level, and the router is pushed to distribute the token evenly . |
X-LeBench: A Benchmark for Extremely Long Egocentric Video Understanding (2025.findings-emnlp)
Copied to clipboard
Wenqi Zhou, Kai Cao, Hao Zheng, Yunze Liu, Xinyi Zheng, Miao Liu, Per Ola Kristensson, Walterio W. Mayol-Cuevas, Fan Zhang, Weizhe Lin, Junxiao Shen
| Challenge: | Existing benchmark datasets focus on short to moderately long videos, leaving a substantial gap in evaluating extensive, ultra-long egocentric video recordings. |
| Approach: | X-LeBench is a benchmark dataset designed to evaluate long egocentric video recordings . it uses a life-logging pipeline to produce realistic, coherent daily plans . |
| Outcome: | X-LeBench is a new benchmark dataset designed to evaluate long-form egocentric video understanding . the approach produces realistic, coherent daily plans aligned with real-world video data . |
DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis (2022.coling-1)
Copied to clipboard
Xian Wu, Shuxin Yang, Zhaopeng Qiu, Shen Ge, Yangtian Yan, Xingwang Wu, Yefeng Zheng, S. Kevin Zhou, Li Xiao
| Challenge: | X-ray and CT are the gold standard for COVID-19 diagnosis and treatment . however, due to the excessive number of patients, writing reports becomes a heavy burden for radiologists. |
| Approach: | They propose to use X-ray and CT to generate medical reports automatically . they evaluate DeltaNet on a COVID-19 dataset, where it outperforms state-of-the-art approaches . |
| Outcome: | The proposed system outperforms state-of-the-art methods on a COVID-19 dataset. |
Towards Unifying Multi-Lingual and Cross-Lingual Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on multilingual summarization and cross-lingual summmarization has been limited due to their different definitions. |
| Approach: | They propose to unify MLS and CLS into a more general setting, i.e. many-to-many summarization. |
| Outcome: | The proposed model outperforms the state-of-the-art models in the zero-shot directions. |
MovieChats: Chat like Humans in a Closed Domain (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, open-domain chatbots are far from satisfactory. |
| Approach: | They propose a unified, readily scalable neural approach which reconciles all subtasks like intent prediction and knowledge retrieval. |
| Outcome: | The proposed approach outperforms commercial systems replying on complex rules on static and interactive tests and shows that the results are remarkably good. |
KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing knowledge-driven dialog data is limited due to the lack of dialog data which consists of multi-turn conversations on multiple topics and with knowledge annotations. |
| Approach: | They propose a Chinese multi-domain knowledge-driven conversation dataset which grounds the topics in multi-turn conversations to knowledge graphs. |
| Outcome: | The proposed dataset can be enhanced by introducing background knowledge, but there is still a large space for leveraging knowledge to model multi-turn conversations for further research. |
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark (2022.findings-acl)
Copied to clipboard
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, Minlie Huang
| Challenge: | Dialogue safety problems severely limit the real-world deployment of generative conversational models. |
| Approach: | They propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings. |
| Outcome: | The proposed taxonomy captures unsafe behaviors in human-bot dialogue settings with rich context-sensitive unsafe examples. |
On the Transferability of Adversarial Attacks against Neural Text Classifier (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that deep neural networks are vulnerable to adversarial examples . a small perturbation to an input alters the model prediction . |
| Approach: | They propose a genetic algorithm to find models that can induce adversarial examples to fool models . they propose word replacement rules that can be used for model diagnostics from these examples . |
| Outcome: | The proposed model can fool almost all existing models, while ignoring the data bias in the training set. |
Diffusion Language Model Inference with Monte Carlo Tree Search (2026.findings-eacl)
Copied to clipboard
Zheng Huang, Kiran Ramnath, Yueyan Chen, Aosong Feng, Sangmin Woo, Balasubramaniam Srinivasan, Zhichao Xu, Kang Zhou, Shuai Wang, Haibo Ding, Lin Lee Cheong
| Challenge: | Existing methods for inference use heuristics to determine which positions to unmask and which tokens to commit . MEDAL is an inference-time scaling framework that integrates Monte Carlo Tree SEarch initialization for Diffusion Language Model inference. |
| Approach: | They propose a framework that integrates Monte Carlo Tree SEarch initialization for Diffusion Language Model inference. |
| Outcome: | The proposed framework achieves 22.0% improvement over existing inference strategies across multiple benchmarks. |
Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for retrieval augmentation work with chunked contexts, which leads to poor quality of semantic representation and incomplete retrieval of useful information. |
| Approach: | They propose a method for retrieval augmentation of long-context language modeling using landmark embedding. |
| Outcome: | The proposed method outperforms existing retrieval methods with a notable advantage. |
CASE: Aligning Coarse-to-Fine Cognition and Affection for Empathetic Response Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing empathetic dialogue models only consider the affective aspect of empathy, which limits the capability of emotional response generation. |
| Approach: | They propose a model that aligns the user's cognition and affection at both the coarse-grained and fine-grounded levels and then automatically and manually evaluates the model. |
| Outcome: | The proposed model outperforms state-of-the-art models and generates more empathetic and informative responses. |
Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that by applying our framework, we can easily learn effective FGET models for low-resource languages. |
| Approach: | They propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages. |
| Outcome: | The proposed framework can learn effective FGET models for low-resource languages even without human-labeled data. |
Imperfectly Cooperative Human-AI Interactions: Comparing the Impacts of Human and AI Attributes in Simulated and User Studies (2026.findings-acl)
Copied to clipboard
Myke C. Cohen, Mingqian Zheng, Neel Bhandari, Hsien-Te Kao, Xuhui Zhou, Daniel Nguyen, Laura Cassani, Maarten Sap, Svitlana Volkova
| Challenge: | In simulations, personality traits and AI attributes were comparatively influential, but with actual human subjects, AI attributes – particularly transparency – were much more impactful. |
| Approach: | They compare a purely simulated dataset and a parallel human subjects experiment to examine how human personality traits and AI design characteristics jointly shape interaction outcomes in imperfectly cooperative scenarios. |
| Outcome: | The results show that personality traits and AI attributes are comparatively influential in simulations, but with actual human subjects, they are much more impactful. |
Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models that extract discrete inputs into fixed-length representations are vulnerable to adversarial attacks that place perturbations on clean inputs to fool DNNs. |
| Approach: | They propose to inspect the subspaces of sample features through spectral analysis to better understand adversarial attacks. |
| Outcome: | The proposed strategy enables the model to inherently suppress adversaries, which boosts model robustness and motivates new directions of effective adversarial defense. |
Detecting Adversarial Samples through Sharpness of Loss Landscape (2023.findings-acl)
Copied to clipboard
Rui Zheng, Shihan Dou, Yuhao Zhou, Qin Liu, Tao Gui, Qi Zhang, Zhongyu Wei, Xuanjing Huang, Menghan Zhang
| Challenge: | Existing studies have shown that adversarial samples are more vulnerable than normal ones to textual adversarials. |
| Approach: | They propose a simple and effective sharpness-based detector that can distinguish adversarial samples by maximizing the loss increment within the region where the inference sample is located. |
| Outcome: | The proposed method outperforms previous detection methods by large margins on three text classification tasks. |
SWE-QA-Pro: A Representative Benchmark and Scalable Training Recipe for Repository-Level Code Understanding (2026.findings-acl)
Copied to clipboard
Songcheng Cai, Zhiheng Lyu, Yuansheng Ni, Xiangchao Chen, Baichuan Zhou, Shenzhe Zhu, Yi Lu, Haozhe Wang, Chi Ruan, Benjamin Schneider, Weixu Zhang, Xiang Li, Andy Zheng, Yuyu Zhang, Ping Nie, Wenhu Chen
| Challenge: | Existing benchmarks for agentic repository-level code understanding overlook long tail topics and rely on memorized knowledge. |
| Approach: | They propose a repository-level agentic code understanding benchmark that uses long-tail repositories with executable environments to enforce topical balance. |
| Outcome: | Empirically, a Qwen3-8B model trained with the proposed benchmark outperforms GPT-4o by 2.3 points. |
DimABSA: Building Multilingual and Multidomain Datasets for Dimensional Aspect-Based Sentiment Analysis (2026.acl-long)
Copied to clipboard
Lung-Hao Lee, Liang-Chih Yu, Natalia V Loukachevitch, Ilseyar Alimova, Alexander Panchenko, Tzu-Mi Lin, Zhe-Yu Xu, Jian-Yu Zhou, Guangmin Zheng, Jin Wang, Sharanya Awasthi, Jonas Becker, Jan Philip Wahle, Terry Ruas, Shamsuddeen Hassan Muhammad, Saif M. Mohammad
| Challenge: | Existing ABSA research relies on coarse-grained categorical labels, which limits its ability to capture nuanced affective states. |
| Approach: | They propose a dimensional approach that represents sentiment with continuous valence–arousal (VA) scores, enabling fine-grained analysis at both the aspect and sentiment levels. |
| Outcome: | The proposed approach represents sentiment with continuous valence–arousal (VA) scores, enabling fine-grained analysis at both the aspect and sentiment levels. |
SSEGCN: Syntactic and Semantic Enhanced Graph Convolutional Network for Aspect-based Sentiment Analysis (2022.naacl-main)
Copied to clipboard
| Challenge: | Aspect-based Sentiment Analysis (ABSA) aims to predict sentiment polarity towards aspects in sentences . a novel model for ABSA is proposed, but how to harness it is still a challenge . |
| Approach: | They propose a syntactic and semantic enhanced Graph Convolutional Network (SSEGCN) model for ABSA task using aspect-aware attention mechanism and self-attention. |
| Outcome: | The proposed model outperforms state-of-the-art methods on benchmark datasets. |
Plan Dynamically, Express Rhetorically: A Debate-Driven Rhetorical Framework for Argumentative Writing (2025.emnlp-main)
Copied to clipboard
| Challenge: | Argumentative essay generation (AEG) is a complex task that requires advanced semantic understanding, logical reasoning, and organized integration of perspectives. |
| Approach: | They propose a debate-driven rhetorical framework for argumentative writing that integrates Bitzer’s rhetorical situation theory to improve logical depth, argumentative diversity, and rhetorical persuasiveness. |
| Outcome: | The proposed framework improves logical depth, argumentative diversity, and rhetorical persuasiveness over existing state-of-the-art models. |
CDConv: A Benchmark for Contradiction Detection in Chinese Conversations (2022.emnlp-main)
Copied to clipboard
Chujie Zheng, Jinfeng Zhou, Yinhe Zheng, Libiao Peng, Zhen Guo, Wenquan Wu, Zheng-Yu Niu, Hua Wu, Minlie Huang
| Challenge: | Existing methods for detecting dialogue contradictions are difficult due to contextualization nature of conversations. |
| Approach: | They propose a benchmark for Contradiction Detection in Chinese Conversations . they use automatic conversation generation to simulate common user behaviors . |
| Outcome: | The proposed benchmark simulated the user behaviors that trigger chatbots to make contradictions . the results show that the current state-of-the-art chatbot can be easily goaded into making contradictions. |
Grounding Language Model with Chunking-Free In-Context Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | CFIC retrieval approach eliminates the need for document chunking and provides a more efficient and efficient method for RAG systems. |
| Approach: | They propose a Chunking-Free In-Context retrieval approach specifically tailored for RAG systems . they employ auto-aggressive decoding to accurately identify specific evidence text . |
| Outcome: | The proposed method is better than traditional methods on open question answering datasets. |
GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing LLM leaderboards often reference scores reported in other papers without consistent settings and prompts, which may encourage cherry-picking favored settings and for better results. |
| Approach: | They propose an open-source and reproducible LLM evaluation suite built on top of OpenAI Evals that systematically evaluates 10+ leading LLMs and OpenAI’s legacy models on 20+ curated benchmarks across 7 capability categories. |
| Outcome: | The evaluation suite is built on top of OpenAI Evals and evaluates 10+ leading LLMs and OpenAI’s legacy models on 20+ curated benchmarks across 7 capability categories. |
MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark (2024.findings-acl)
Copied to clipboard
Hongwei Liu, Zilong Zheng, Yuxuan Qiao, Haodong Duan, Zhiwei Fei, Fengzhe Zhou, Wenwei Zhang, Songyang Zhang, Dahua Lin, Kai Chen
| Challenge: | Recent advances in large language models have showcased significant improvements in mathematics, but traditional benchmarks like GSM8k offer a unidimensional perspective. |
| Approach: | MathBench is a benchmark that rigorously assesses the mathematical capabilities of large language models. |
| Outcome: | MathBench spans a wide range of mathematical disciplines, offering a detailed evaluation of both theoretical understanding and practical problem-solving skills. |
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)
Copied to clipboard
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding (2022.acl-long)
Copied to clipboard
Yanan Zheng, Jing Zhou, Yujie Qian, Ming Ding, Chonghua Liao, Li Jian, Ruslan Salakhutdinov, Jie Tang, Sebastian Ruder, Zhilin Yang
| Challenge: | Existing evaluation protocols for few-shot natural language understanding (NLU) tasks are inconsistent and hinder fair comparison and measuring progress. |
| Approach: | They propose an evaluation framework that improves previous evaluation procedures in three key aspects, i.e., test performance, dev-test correlation, and stability. |
| Outcome: | The proposed framework improves evaluation procedures in three key aspects, i.e., performance, dev-test correlation, and stability. |
Reward Modeling Requires Automatic Adjustment Based on Data Quality (2024.findings-emnlp)
Copied to clipboard
Binghai Wang, Rui Zheng, Lu Chen, Zhiheng Xi, Wei Shen, Yuhao Zhou, Dong Yan, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values. |
| Approach: | They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values . |
| Outcome: | The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets. |
Training-Free Test-Time Contrastive Learning for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing training-free alternatives to training-based models are static or depend on external guidance. |
| Approach: | They propose a training-free adaptation framework that enables a frozen LLM to improve online by distilling supervision from its own inference experiences. |
| Outcome: | The proposed framework outperforms existing test-time adaptation methods under online evaluation. |
The Lessons of Developing Process Reward Models in Mathematical Reasoning (2025.findings-acl)
Copied to clipboard
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin
| Challenge: | a recent study shows that process reward models can make mistakes, leading to wrong conclusions. |
| Approach: | They propose a consensus filtering mechanism that integrates MC estimation with LLM-as-a-judge to improve model performance and data efficiency. |
| Outcome: | The proposed model outperforms existing open-source alternatives and provides practical guidelines for future research. |
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosure (2026.acl-demo)
Copied to clipboard
Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhou, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Nenkov Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, YU Chen, Xue Liu, Preslav Nakov, Zhuohan Xie
| Challenge: | FinReporting is an agentic workflow for localized cross-jurisdiction financial reporting . existing approaches assume a single-market setting and overlook structural differences across jurisdictions . |
| Approach: | They propose a workflow that decomposes financial reporting into auditable stages . they use Large Language Models to extract and summarize corporate disclosures . |
| Outcome: | The proposed system decomposes reporting into auditable stages . it improves consistency and reliability under heterogeneous reporting regimes. |
Speculative Contrastive Decoding (2024.acl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit exceptional performance in language tasks, yet their auto-regressive inference is limited due to high computational requirements and is sub-optimal due to the exposure bias. |
| Approach: | They propose a decoding approach that leverages predictions from smaller language models to achieve both decoding acceleration and quality improvement. |
| Outcome: | The proposed method achieves both decoding acceleration and quality improvement on four diverse language tasks. |
AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)
Copied to clipboard
Jiaxin Bai, Wei Fan, Qi Hu, Qing Zong, Chunyang Li, Hong Ting Tsang, Hongyu Luo, Yauwai Yim, Haoyu Huang, Xiao Zhou, Feng Qin, Tianshi Zheng, Xi Peng, Xin Yao, Huiwen Yang, Leijie Wu, JI Yi, Gong Zhang, Renhai Chen, Yangqiu Song
| Challenge: | Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage. |
| Approach: | They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas. |
| Outcome: | The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality. |
Deep Differential Amplifier for Extractive Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to extract summary from document with a disproportionate ratio of selected and unselected sentences are far from human performance. |
| Approach: | They propose a model that rebalances sentence-level extractive summarization by amplifying the semantic difference between each sentence and all other sentences and applying the residual unit as the second item of the differential amplifier to deepen the architecture. |
| Outcome: | The proposed model performs competitively against state-of-the-art methods on two benchmark datasets. |
Sugar-Coated Poison: Benign Generation Unlocks Jailbreaking (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to jailbreak large language models rely on black-box manipulation of prompt templates, resulting in high costs and poor generalizability. |
| Approach: | They propose a sugar-coated poison attack paradigm that uses a "semantic reversal" strategy to induce the model into a safety response mode. |
| Outcome: | The proposed attack paradigm outperforms baselines in the study. |
LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encoding (2025.emnlp-main)
Copied to clipboard
Yuxuan Hu, Jihao Liu, Ke Wang, Jinliang Zheng, Weikang Shi, Manyuan Zhang, Qi Dou, Rui Liu, Aojun Zhou, Hongsheng Li
| Challenge: | Recent advances in Large Language Models have opened new avenues for solving complex optimization problems, including Neural Architecture Search (NAS). |
| Approach: | They propose a framework that leverages LLMs for cross-domain neural architecture optimization without extensive domain-specific tuning. |
| Outcome: | The proposed framework achieves competitive performance in both in-domain and out-of-domain tasks. |
Approximation of Response Knowledge Retrieval in Knowledge-grounded Dialogue Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have focused on improving dialogue generation models that include knowledge related to the posts. |
| Approach: | They propose to use a novel method to generate responses from posts and related knowledge by injecting knowledge into dialogue generation models. |
| Outcome: | The proposed method outperforms baseline models in terms of knowledge relevance and quality. |
Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework (2026.findings-acl)
Copied to clipboard
Jiaqi Weng, Han Zheng, Hanyu Zhang, Ej Zhou, Qinqin He, Jialing Tao, Hui Xue, Zhixuan Chu, Xiting Wang
| Challenge: | Existing studies on how SAEs derive most fine-grained latent features for safety remain unexplored. |
| Approach: | They propose a framework for interpreting SAE features in safety-critical domains . they train a suite of SAEs with human-readable explanations and systematic evaluations based on pornography, politics, violence, and terror . |
| Outcome: | The proposed framework reduces interpretation cost by 55% and improves safety-critical features. |
How Credible Is an Answer From Retrieval-Augmented LLMs? Investigation and Evaluation With Multi-Hop QA (2025.coling-main)
Copied to clipboard
| Challenge: | Retrieval-augmented large language models (RaLLMs) are reshaping knowledge acquisition, offering long-form, knowledge-grounded answers through advanced reasoning and generation capabilities. |
| Approach: | They propose a benchmarking system to evaluate RaLLMs' correctness and Groundedness to determine their reliability in multi-hop question-answering tasks. |
| Outcome: | The proposed model-based evaluation pipeline for multi-hop question-answering tasks reveals that the model generates inaccuracies when dealing with flawed or partial knowledge. |
Uncertainty-Aware Iterative Preference Optimization for Enhanced LLM Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for enhancing the performance of large language models require expensive manual annotations. |
| Approach: | They propose an offline direct preference optimization method that collects preference pairs through iterative sampling and execution feedback to improve model confidence. |
| Outcome: | The proposed method improves performance on three reasoning tasks and shows a 3.6% improvement over the standard method. |
MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning (2024.acl-long)
Copied to clipboard
Chengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, Chang Zhou
| Challenge: | In math reasoning with large language models, fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective. |
| Approach: | They propose to fine-tune data augmentation by query evolution and diverse reasoning paths. |
| Outcome: | The proposed model achieves new state-of-the-art on GSM8K and MATH. |
Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that biased samples can be brittle for VQA models . however, the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data. |
| Approach: | They propose a contrastive learning approach that exploits biased samples for unbiased information that contributes to reasoning. |
| Outcome: | The proposed method achieves competitive performance on the OOD dataset while maintaining robustness on the ID dataset. |
RecMind: Large Language Model Powered Agent For Recommendation (2024.findings-naacl)
Copied to clipboard
Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Yanbin Lu, Xiaojiang Huang, Yingzhen Yang
| Challenge: | Existing recommendations systems are limited in generalizing to new tasks due to model scale and data size constraints. |
| Approach: | They propose an LLM-powered autonomous recommender agent, RecMind, which is capable of leveraging external knowledge to provide zero-shot personalized recommendations. |
| Outcome: | The proposed model outperforms existing zero/few-shot LLM-based recommendation baseline methods in various tasks and achieves comparable performance to a fully trained recommendation model P5. |
CAPruner: Conceptual-Adjacent Scene Graph Pruner for Enhancing 3D Spatial Reasoning of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing pruning methods rely on spatial proximity and remove relevant relations, thereby undermining reliable spatial reasoning. |
| Approach: | They propose a scene graph pruning model that integrates fuzzy semantic relevance with spatial proximity to estimate the importance of relations. |
| Outcome: | Experiments show that CAPruner outperforms proximity-based pruning with negligible cost savings. |
Robust Lottery Tickets for Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models contain smaller matching subnetworks that are not robust to adversarial examples. |
| Approach: | They propose a method to find robust tickets hidden in pre-trained language models by learning binary weight masks and an adversarial loss objective to guide the search. |
| Outcome: | The proposed method improves on previous work on adversarial robustness evaluation. |
ClaimGen-CN: A Large-scale Chinese Dataset for Legal Claim Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, legal claims are not being used by non-professionals. |
| Approach: | They construct a dataset for Chinese legal claim generation task and then use it to evaluate the generated claims. |
| Outcome: | The proposed dataset is the first for the Chinese legal claim generation task and will be made publicly available. |
Jailbreaking? One Step Is Enough! (2025.acl-long)
Copied to clipboard
Weixiong Zheng, Peijian Zeng, YiWei Li, Hongyan Wu, Nankai Lin, Junhao Chen, Aimin Yang, Yongmei Zhou
| Challenge: | Large language models (LLMs) excel in various tasks but remain vulnerable to jailbreak attacks, where adversaries manipulate prompts to generate harmful outputs. |
| Approach: | They propose a Reverse Embedded Defense Attack mechanism that disguises the attack intention as the "defense" intention against harmful content. |
| Outcome: | The proposed method outperforms existing methods on open-source and closed-source models and enables successful jailbreak in one iteration. |
MPPO: Multi Pair-wise Preference Optimization for LLMs with Arbitrary Negative Samples (2025.coling-main)
Copied to clipboard
Shuo Xie, Fangzhi Zhu, Jiahui Wang, Lulu Wen, Wei Dai, Xiaowei Chen, Junxiong Zhu, Kai Zhou, Bo Zheng
| Challenge: | Existing preference optimization methods such as DPO and KTO are inherently derived from PPO, requiring a reference model that adds GPU memory resources and relies heavily on abundant preference data. |
| Approach: | They propose an algorithm that leverages the average likelihood of model responses to fit the reward function and maximizes the utilization of preference data. |
| Outcome: | The proposed algorithm outperforms DPO, ORPO, and SimPO on MT-Bench and Arena-Hard. |
Uncovering Argumentative Flow: A Question-Focus Discourse Structuring Framework (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing structure modeling approaches fail to capture the author’s rhetorical intent and reasoning process. |
| Approach: | They propose a Question-Focus discourse structuring framework that explicitly models the underlying argumentative flow by anchoring each argumentative unit to a guiding question and a set of attentional foci. |
| Outcome: | The proposed framework outperforms baseline models and curated models on an argument reconstruction task in Chinese think-tank articles and claims coverage. |
Towards Emotional Support Dialog Systems (2021.acl-long)
Copied to clipboard
| Challenge: | Emotional support is a crucial ability for many conversation scenarios, including social interactions, mental health support, and customer service chats. |
| Approach: | They propose an Emotional Support Conversation task and an ESC Framework to train emotional support into dialog systems. |
| Outcome: | The proposed framework provides an example of an Emotional Support Conversation task and shows that it is more effective than existing models. |
ORTicket: Let One Robust BERT Ticket Transfer across Different Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | Pretrained language models are susceptible to subtle perturbations and require multiple adversarial training during fine-tuning to improve their robustness. |
| Approach: | They propose a novel adversarial defense method ORTicket that fine-tunes a model for downstream tasks. |
| Outcome: | The proposed method achieves comparable robustness to other defense methods while maintaining the efficiency of fine-tuning. |
Deep Equilibrium Non-Autoregressive Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | et al., 2017) is the most prevailing neural architecture for sequence-to-sequence learning. |
| Approach: | They propose to solve for the equilibrium state of NAR models with black-box root-finding solvers and back-propagate through the equilibrium point via implicit differentiation with constant memory. |
| Outcome: | The proposed framework can converge to a more accurate prediction on four WMT benchmarks. |
How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition (2024.acl-long)
Copied to clipboard
Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, Jingren Zhou
| Challenge: | supervised fine-tuning (SFT) is a technique used to enhance multiple abilities in large language models. |
| Approach: | They propose to study the interplay of data composition between mathematical reasoning, code generation, and general human-aligning abilities during supervised fine-tuning. |
| Outcome: | The proposed model improves math reasoning and code generation with increasing data amount . the proposed model size and SFT strategies can be used to learn multiple skills with different scaling patterns. |
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David Clifton
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
Skill-Based Few-Shot Selection for In-Context Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods based on pre-trained embeddings can be easily biased by surface features that are not important for the target task. |
| Approach: | They propose a skill-based few-shot selection method for in-context learning . it generates skill-specific descriptions for each test case and candidate example . |
| Outcome: | The proposed method significantly outperforms existing methods in five cross-domain semantic parsing datasets and six backbone models. |
Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) has shown great success in BERT compression. |
| Approach: | They propose a knowledge distillation paradigm that extracts the teacher's hidden state knowledge and then compresses it into three dimensions. |
| Outcome: | The proposed paradigm gives rise to training speedup of 2.7x 3.4x for two kinds of student models and computing devices. |
SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture (2025.naacl-long)
Copied to clipboard
| Challenge: | balancing the training budget, downstream performance, and general capabilities of large language models remains a challenge in many applications. |
| Approach: | They propose a mixture of expert framework based on Soft LoRA and Identity Mixture . SLIM allows dynamic routing between LoRA adapters and identity layers . |
| Outcome: | The proposed framework reduces training cost while maintaining general capabilities . it can be open-sourced upon publication. |
Improving Chinese Pop Song and Hokkien Gezi Opera Singing Voice Synthesis by Enhancing Local Modeling (2023.emnlp-main)
Copied to clipboard
| Challenge: | Singing Voice Synthesis (SVS) synthesizes pleasing vocals based on music scores and lyrics . current acoustic models ignore the significance of local modeling within the sequence and the hard-to-synthesize parts in the predicted mel-spectrogram . |
| Approach: | They propose a method to enhance local modeling in the acoustic model by focusing on phoneme tokens located before and after the phoneme. |
| Outcome: | The proposed method improves local modeling in the acoustic model by focusing on the hard-to-synthesize parts of the predicted mel-spectrogram. |
CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown significant potential in code generation, but they also present challenges regarding the protection of Intellectual Property (IP) related to model architectures, weights, and training data. |
| Approach: | They propose a multi-bit watermarking technique that embeds additional information to preserve provenance details, such as the vendor ID of an LLM. |
| Outcome: | The proposed technique preserves provenance details while maintaining syntactical correctness of generated code. |
Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show vulnerability of deep neural networks to adversarial examples that intentionally fool the networks. |
| Approach: | They propose a method for training a robust model to defense synonym substitution-based attacks by sampling embedding vectors for each word in an input sentence and augmenting them with the training data. |
| Outcome: | The proposed method outperforms other proposed defense methods by a significant margin across different network architectures and multiple data sets. |
GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety (2026.findings-acl)
Copied to clipboard
Changxuan Fan, Xi Yang, Yueyuan Zheng, Bin Zhou, Yuanping Wang, Wenbin Hu, Huihao Jing, Ki Sen Hung, Dazhao Du, Haoran Li, Janet Hui-wen Hsiao, Yangqiu Song
| Challenge: | a survey of older adults shows that many LLMs mishandle elderly-specific contextual risks. |
| Approach: | They propose a framework to assess elderly-specific contextual risks in LLM interactions . they use a taxonomy to identify 50 fine-grained risk types across mental well-being, financial, medical, toxicity, and privacy domains . |
| Outcome: | a new framework assesses elderly-specific contextual risks in LLM interactions . it achieves 96.2% and 90.9% unsafe-prompt detection accuracy, respectively . |
Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022.acl-long)
Copied to clipboard
| Challenge: | a fundamental challenge in modeling math problems is how to fuse semantics of textual description and formulas. |
| Approach: | They propose a method to continually pre-train language models for improving understanding of math problems with syntax-aware memory networks. |
| Outcome: | The proposed approach outperforms competitive baselines on four math tasks. |
ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Text-to-speech (TTS) performance has improved with the advent of denoising Diffusion Probabilistic Models . however, perceived quality of audio depends on content, pitch, rhythm, and energy . |
| Approach: | They propose a visual TTS model with scalable diffusion transformers that complement phoneme sequences with visual information to generate high-perceived audio. |
| Outcome: | The proposed model outperforms existing models regardless of visibility of the scene . it can generate high-perceived audio, opening up new avenues for AR and VR applications . |
RPD: A Distance Function Between Word Embeddings (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing word embeddings are poorly understood, but little is known about how they differ between different sets of word embeds. |
| Approach: | They propose a metric called Relative Pairwise Inner Product Distance to quantify the distance between different word embeddings. |
| Outcome: | The proposed metric measures the distance between different sets of embeddings and investigates the influence of different training processes and corpora. |
ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control (2025.acl-long)
Copied to clipboard
Shengpeng Ji, Qian Chen, Wen Wang, Jialong Zuo, Minghui Fang, Ziyue Jiang, Hai Huang, Zehan Wang, Xize Cheng, Siqi Zheng, Zhou Zhao
| Challenge: | Prior zero-shot TTS models only mimic the speaker’s voice without further control and adjustment capabilities while prior controllable TTS systems cannot perform speaker-specific voice generation. |
| Approach: | They propose a style control module that captures codec representations corresponding to timbre, content, and style in a discrete decoupling codec space. |
| Outcome: | The proposed system can fully clone the speaker's voice and perform speech-specific adjustment and control functions. |
Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
Shirong Ma, Yinghui Li, Rongyi Sun, Qingyu Zhou, Shulin Huang, Ding Zhang, Li Yangning, Ruiyang Liu, Zhongli Li, Yunbo Cao, Haitao Zheng, Ying Shen
| Challenge: | Chinese Grammatical Error Correction (CGEC) is a challenging NLP task and a common application in human daily life. |
| Approach: | They propose a linguistic rules-based approach to construct large-scale CGEC training corpora with automatically generated grammatical errors. |
| Outcome: | The proposed method improves performance of existing CGEC models and the benchmark is excellent resource for further development. |
Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show pre-trained language models contain matching subnetworks that have similar transfer learning performance as the original PLM. |
| Approach: | They propose to prune matching subnetworks using magnitude-based pruning . they propose to optimize the subnetwork structure towards the pre-training objectives . |
| Outcome: | The proposed method is more efficient in searching subnetworks and advantageous when fine-tuning within a range of data scarcity. |
Vocabulary Learning via Optimal Transport for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Empirical results show that VOLT beats widely-used vocabularies in diverse scenarios, including WMT-14 English-German translation, TED bilingual translation, and TED multilingual translation. |
| Approach: | They propose a token dictionary solution that can be used without trial training to find the best dictionary with a proper size. |
| Outcome: | The proposed solution beats widely-used vocabularies in English-German translation, TED bilingual translation, and TED multilingual translation. |
Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models (2026.eacl-long)
Copied to clipboard
Qiao Liang, Yanjiang Liu, Weixiang Zhou, Ben He, Yaojie Lu, Hongyu Lin, Jia Zheng, Xianpei Han, Le Sun, Yingfei Sun
| Challenge: | Existing research treats MLLMs as unified systems optimized through end-to-end training, but the impact of vision encoder’s prior knowledge is seldom investigated. |
| Approach: | They propose a metric to quantify the effect of prior knowledge on MLLM performance by integrating prior knowledge at the vision encoder level into a training framework. |
| Outcome: | The proposed training framework incorporates prior knowledge at the vision encoder level, and significantly boosts visual understanding capabilities of MLLMs. |
A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiences (2025.findings-acl)
Copied to clipboard
Jiaxin Shen, Jinan Xu, Huiqi Hu, Luyi Lin, Guoyang Ma, Fei Zheng, Fandong Meng, Jie Zhou, Wenjuan Han
| Challenge: | a recent study focuses on generating impartial and interpretable judicial judgments based on established criminal fact. |
| Approach: | They propose a law reasoning schema enriched with hierarchical factum probandum, evidence, and implicit experience that enables public scrutiny and preventing bias. |
| Outcome: | The proposed schema enables public scrutiny and prevents bias in the "Intelligent Court" it employs a suite of legal analysis tools to address the challenge task. |
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)
Copied to clipboard
Siwei Wu, JinCheng Ren, Xeron Du, Shuyue Guo, Xingwei Qu, Yiming Liang, Jie Liu, Yunwen Li, Tyler Loakman, Tianyu Zheng, Boyu Feng, Huaqing Yuan, Zili Wang, Jiaheng Liu, Wenhao Huang, Chenglin Cai, Haoran Que, Jian Yang, Yuelin Bai, Zekun Moore Wang, Zhouliang Yu, Qunshu Lin, Ding Pan, Yuchen Eleanor Jiang, Tiannan Wang, Wangchunshu Zhou, Shenzhi Wang, Xingyuan Bu, Minghao Liu, Guoyin Wang, Ge Zhang, Chenghua Lin
| Challenge: | Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation. |
| Approach: | They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets. |
| Outcome: | The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark. |
Improving the Adversarial Robustness of NLP Models by Information Bottleneck (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that adversarial examples can be directly attributed to the presence of non-robust features. |
| Approach: | They propose to capture task-specific robust features while eliminating non-robust ones . they show that models can achieve significant improvement in robust accuracy . |
| Outcome: | The proposed method outperforms all defense methods on SST-2, AGNEWS and IMDB datasets while achieving no performance drop. |
Question-Interlocutor Scope Realized Graph Modeling over Key Utterances for Dialogue Reading Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Compared to standard RC tasks, dialogue reading comprehension (DRC) has raised challenges because of the complex speaker information and noisy dialogue context. |
| Approach: | They propose a new method for dialogue reading comprehension that extracts answers from dialogues by using key-utterances-extracting methods and a Question-Interlocutor Scope Realized Graph. |
| Outcome: | The proposed method achieves state-of-the-art performance against previous works. |
Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checking (2022.findings-emnlp)
Copied to clipboard
Yinghui Li, Shirong Ma, Qingyu Zhou, Zhongli Li, Li Yangning, Shulin Huang, Ruiyang Liu, Chao Li, Yunbo Cao, Haitao Zheng
| Challenge: | Chinese Spell Checking (CSC) aims to detect and correct Chinese spelling errors. |
| Approach: | They propose a framework which renders Chinese Spell Checking model to learn heterogeneous knowledge from the dictionary in terms of phonetics, vision, and meaning. |
| Outcome: | The proposed framework renders the CSC model to learn heterogeneous knowledge from the dictionary in terms of phonetics, vision, and meaning. |
Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing ensemble methods for Large Language Models focus on reward model ranking of outputs, leading to significant computation overhead. |
| Approach: | They propose a reward-guided routing method distilling rewards on training queries to train a routing function. |
| Outcome: | The proposed method outperforms the best single model and ranks first on 44% of tasks. |
PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides (2025.emnlp-main)
Copied to clipboard
Hao Zheng, Xinyan Guan, Hao Kong, Wenkai Zhang, Jia Zheng, Weixiang Zhou, Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun
| Challenge: | Existing methods for generating presentations from documents focus on improving and evaluating content quality in isolation, overlooking visual appeal and structural coherence. |
| Approach: | They propose an edit-based presentation generation system that analyzes and iterates on slides to create new slides. |
| Outcome: | The proposed presentation generation tool outperforms existing methods in three dimensions . it analyzes slides, iterates and generates edit actions based on selected slides . |
RJE: A Retrieval-Judgment-Exploration Framework for Efficient Knowledge Graph Question Answering with LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graph question answering (KGQA) aims to answer natural language questions using knowledge graphs. |
| Approach: | They propose a framework that retrieves refined reasoning paths and evaluates their sufficiency. |
| Outcome: | The proposed framework outperforms existing baselines while enabling small open-source LLMs to achieve competitive results without fine-tuning LLM. |
PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference (2025.acl-long)
Copied to clipboard
Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Alex Qiu, Jiayi Zhou, Kaile Wang, Boxun Li, Sirui Han, Yike Guo, Yaodong Yang
| Challenge: | Using large-scale annotation data, large language models can generate noise, errors and biases, leading to unexpected behaviours. |
| Approach: | They propose a dataset to promote safety alignment in large language models . they separate helpfulness and harmlessness annotations for question-answering pairs . |
| Outcome: | The proposed dataset provides 44.6k prompts and 265k question-answer pairs with safety meta-labels for 19 harm categories and three severity levels, with answers generated by Llama-family models. |
ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have excellent performance in evaluation benchmarks, but struggle in complex reasoning tasks. |
| Approach: | They propose a tool-augmented chain-of-thought reasoning framework for chat-based LLMs . they model chain- of-thoughting reasoning as multi-turn conversations to utilize tools . |
| Outcome: | The proposed framework can outperform state-of-the-art models on complex reasoning tasks. |
More than Text: Multi-modal Chinese Word Segmentation (2021.acl-short)
Copied to clipboard
| Challenge: | Currently, word segmentation is performed in many languages without word delimiters. |
| Approach: | They propose to combine the multi-modality to perform Chinese word segmentation . they propose a time-dependent multi-module interactive model to integrate multi-modality information . |
| Outcome: | The proposed model integrates multi-modal information for word sequence labeling with Chinese language as target . the proposed model performs well on three training sets on Chinese and other languages without word delimiters. |
Collision to Cognition: Hash-Driven Graph Construction for Efficient RAG (2026.acl-long)
Copied to clipboard
Chuang Zhou, Zheng Yuan, Linhao Luo, Zhaozhuo Xu, Yilin Xiao, Junnan Dong, Siyu An, di Yin, Xing Sun, Xiao Huang
| Challenge: | Retrieval-augmented generation (RAG) has been used for enhancing large language models with external knowledge. |
| Approach: | They propose a framework for mining efficient graph structures via hashing to enhance RAG . they adopt an inductive paradigm where global graph structure emerges from local hash collisions . |
| Outcome: | The proposed framework outperforms existing baselines while requiring no GPU resources or token budget. |
DopplerBAS: Binaural Audio Synthesis Addressing Doppler Effect (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for binaural audio synthesis are limited in phase estimation, which is crucial for spatial hearing. |
| Approach: | They propose a method to explicitly address the Doppler effect of the moving speaker . it calculates the radial relative velocity of the speaker in spherical coordinates . |
| Outcome: | The proposed method improves the representative WarpNet and BinauralGrad backbones in phase error metric and reaches a new state of the art (SOTA) it is compared with the current method which is limited in phase estimation . |
TAKE: Topic-shift Aware Knowledge sElection for Dialogue Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work finds that realizing who holds the initiative can help select knowledge . however, there is a strong semantic transition between two rounds, probably leading to initiative misjudgment . |
| Approach: | They propose a topic-shift Aware Knowledge sElector(TAKE) model which locates relevant parts from dialogue history to improve knowledge selection. |
| Outcome: | The proposed model outperforms baseline models on the WoW. |
Noise Robust Named Entity Understanding for Voice Assistants (2021.naacl-industry)
Copied to clipboard
Deepak Muralidharan, Joel Ruben Antony Moniz, Sida Gao, Xiao Yang, Justine Kao, Stephen Pulman, Atish Kothari, Ray Shen, Yinying Pan, Vivek Kaul, Mubarak Seyed Ibrahim, Gang Xiang, Nan Dun, Yidan Zhou, Andy O, Yuan Zhang, Pooja Chitkara, Xuan Wang, Alkesh Patel, Kushal Tayal, Roger Zheng, Peter Grasch, Jason D Williams, Lin Li
| Challenge: | Named Entity Recognition and Entity Linking are challenging for voice assistants . utterances are relatively short, so there is not much context to help disambiguate . |
| Approach: | They propose a Named Entity Understanding system that combines NER and EL in a joint reranking module. |
| Outcome: | The proposed framework improves NER accuracy by up to 3.13% and EL accuracy by 3.6% in F1 score . it also leads to better accuracies in other natural language understanding tasks . |
Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing research focuses on object-level or attribute-level hallucinations, neglecting the more complex relation hallucinosities. |
| Approach: | They propose a comprehensive benchmark targeting relation hallucinations comprising over 20,000 real-world samples and a confidence-based mitigation strategy which reduces the halluciation rate by an average of 9.75% across three datasets. |
| Outcome: | The proposed approach reduces the hallucination rate by an average of 9.75% across three datasets, including Reefknot. |
ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development (2026.findings-acl)
Copied to clipboard
Jie Yang, Honglin Guo, Li Ji, Jiazheng Zhou, Rui Zheng, Zhikai Lei, Shuo Zhang, Zhiheng Xi, Shichun Liu, Yuxin Wang, Bo Wang, Yining Zheng, Tao Gui, Xipeng Qiu
| Challenge: | Large Language Models (LLMs) have redefined the role of AI in software engineering . current benchmarks focus on localized code generation, but neglect dynamic, full-process requirements of real-world engineering. |
| Approach: | They propose a benchmark to evaluate agentic backend coding within a realistic, executable workflow. |
| Outcome: | The ABC-Bench benchmark evaluates agentic backend coding within a realistic, executable workflow. |
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections (2022.emnlp-main)
Copied to clipboard
Chenliang Li, Haiyang Xu, Junfeng Tian, Wei Wang, Ming Yan, Bin Bi, Jiabo Ye, He Chen, Guohai Xu, Zheng Cao, Ji Zhang, Songfang Huang, Fei Huang, Jingren Zhou, Luo Si
| Challenge: | Existing pre-trained vision-language models suffer from inefficiency and linguistic signal overwhelmed by long visual sequences in cross-modal alignment. |
| Approach: | They propose a vision-language foundation model with cross-modal skip-connections that can be pre-trained end-to-end on large-scale image-text pairs with both discriminative and generative objectives. |
| Outcome: | The proposed model achieves state-of-the-art results on a wide range of vision-language downstream tasks, including image captioning, image-text retrieval, visual grounding and visual question answering. |
A Survey on Cross-Lingual Summarization (2022.tacl-1)
Copied to clipboard
| Challenge: | Cross-lingual summarization is a task of generating a summary in one language for a given document in a different language. |
| Approach: | They present a systematic review of the literature on cross-lingual summarization . they summarize previous efforts and compare them with each other . |
| Outcome: | The proposed approach is compared with previous approaches and summarizes them to provide a deeper analysis. |
Taming Language Models for Text-attributed Graph Learning with Decoupled Aggregation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to learning text-attributed graphs neglect interaction between textual and structural information. |
| Approach: | They propose a framework that integrates textual and structural information into TAG learning . they propose combining semantic aggregation and structural aggregations to improve learning a . |
| Outcome: | The proposed framework outperforms state-of-the-art learning methods while requiring less resources. |
StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)
Copied to clipboard
Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective. |
| Approach: | They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality. |
LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)
Copied to clipboard
Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen
| Challenge: | a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented. |
| Approach: | They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs). |
| Outcome: | The proposed library is based on extensive experiments in a variety of evaluation settings. |
RealBehavior: A Framework for Faithfully Characterizing Foundation Models’ Human-like Behavior Mechanisms (2023.findings-emnlp)
Copied to clipboard
Enyu Zhou, Rui Zheng, Zhiheng Xi, Songyang Gao, Xiaoran Fan, Zichu Fei, Jingting Ye, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing studies on human-like behaviors in foundation models do not verify their faithfulness . a simple application of psychological tools cannot faithfully characterize all human-type behaviors . |
| Approach: | They propose a framework to characterize humanoid behaviors in foundation models . they argue that a simple application of psychological tools cannot faithfully characterize all human-like behaviors . |
| Outcome: | The proposed framework assesses the faithfulness of results based on reproducibility, internal consistency, and generalizability. |
Towards Scalable Lightweight GUI Agents via Multi-role Orchestration (2026.findings-acl)
Copied to clipboard
Ziwei Wang, Junjie Zheng, Leyang Yang, Sheng Zhou, Xiaoxuan Tang, Fang Zhouhua, Zhiwei Liu, Dajun Chen, Yong Li, Jiajun Bu
| Challenge: | Advanced GUI agents suffer from prohibitive deployment costs on resource-constrained devices. |
| Approach: | They propose a lightweight GUI agent with GUI-specific knowledge and task scalability . LAMO-3B supports monolithic execution and MAS-style orchestration . |
| Outcome: | The proposed GUI agent LAMO-3B supports monolithic execution and MAS-style orchestration. |
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to machine reading comprehension treat documents at their hierarchical nature, ignoring their dependencies. |
| Approach: | They propose a machine reading comprehension benchmark with two-grained answers . they use graph attention networks to model documents at their hierarchical nature . |
| Outcome: | The proposed framework outperforms existing systems at long and short answer criteria. |
ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to building cross-lingual summarization systems on dialogue documents are limited. |
| Approach: | They propose a benchmark dataset for building cross-lingual summarization systems on dialogue documents. |
| Outcome: | The proposed model outperforms pipeline models on ClidSum and mDialBART. |
Toward Fully Exploiting Heterogeneous Corpus:A Decoupled Named Entity Recognition Model with Two-stage Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental and widely used task in natural language processing. |
| Approach: | They propose a decoupled NER model with two-stage training to take advantage of heterogeneous corpus, including dictionaries, distantly supervised instances, and human-annotated instances. |
| Outcome: | Empirical results show that the proposed model improves against baselines and can be scaled to a large extent. |
Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to learn the transfer from speech to text are unexplored . how to solve the representation discrepancy of speech and text is unexplorable . |
| Approach: | They propose a cooperative acoustic and linguistic representation learning method to fuse and utilize contextual information of speech and text. |
| Outcome: | The proposed method outperforms existing methods on low-resource speech recognition. |
ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models (2024.findings-acl)
Copied to clipboard
Yanan Wu, Jie Liu, Xingyuan Bu, Jiaheng Liu, Zhanhui Zhou, Yuanxing Zhang, Chenchen Zhang, ZhiqiBai ZhiqiBai, Haibin Chen, Tiezheng Ge, Wanli Ouyang, Wenbo Su, Bo Zheng
| Challenge: | ConceptMath evaluates concept-wise mathematical reasoning of Large Language Models (LLMs) Existing benchmarks that evaluate general mathematical reasoning with an average accuracy fail to probe the fine-grained failure modes of mathematical reasoning on specific datasets. |
| Approach: | They introduce a bilingual, fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models. |
| Outcome: | The proposed benchmarks evaluate concept-wise mathematical reasoning of Large Language Models with concept-based accuracies. |
LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning are limited by the growing number of trainable parameters with the rapid deployment of Large Language Models (LLMs). |
| Approach: | They propose a parameter-efficient framework that reduces trainable parameters through tensor-train decomposition. |
| Outcome: | The proposed methods achieve comparable or better performance than most widely used methods with up to 100 fewer parameters on the LLaMA-2-7B models. |
Modality-Aware Integration with Large Language Models for Knowledge-Based Visual Question Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to integrate multimodal knowledge in a modality-agnostic manner can be sub-optimal. |
| Approach: | They propose a modality-aware integration with large language models (LLMs) that leverages multimodal knowledge for both image understanding and knowledge reasoning. |
| Outcome: | The proposed model is able to bridge a tight inter-modal exchange while preserving insightful intra-modal learning. |
COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)
Copied to clipboard
Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang
| Challenge: | Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns. |
| Approach: | They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users. |
| Outcome: | The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks. |
Multimodal Large Language Models for Multi-Subject In-Context Image Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text-to-image (T2I) generation have enabled visually coherent image synthesis from descriptions, but generating images containing multiple given subjects remains challenging. |
| Approach: | They propose a model that enables automatic and scalable data generation without manual annotations to overcome the data scarcity. |
| Outcome: | The proposed model overcomes the data scarcity and lacks manual annotations. |
COLD: A Benchmark for Chinese Offensive Language Detection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Offensive language detection is crucial for maintaining a civilized social media platform and deploying pre-trained language models. |
| Approach: | They propose a benchmark benchmark for Chinese offensive language analysis including a Chinese Offensive Language Dataset and a baseline detector which is trained on the dataset. |
| Outcome: | The proposed benchmark contributes to Chinese offensive language detection which is challenging for existing resources. |
MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues (2024.acl-long)
Copied to clipboard
Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, Wanli Ouyang
| Challenge: | Large Language Models (LLMs) have greatly enhanced dialogue systems, but evaluation of their capabilities remains a challenge. |
| Approach: | They propose a model to evaluate the fine-grained abilities of Large Language Models in multi-turn dialogues. |
| Outcome: | The proposed model evaluates 21 popular chatbots based on MT-Bench-101 . it includes 3 overarching abilities and 13 distinct tasks within multi-turn dialogue scenarios. |
A Decoding Algorithm for Length-Control Summarization Based on Directed Acyclic Transformers (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for length control summarization treat the length requirement as a soft constraint, which may not always be satisfied. |
| Approach: | They propose a novel length-control decoding algorithm based on the directed acyclic Transformer (DAT) their approach allows for multiple plausible sequence fragments and predicts a path to connect them. |
| Outcome: | The proposed algorithm allows for multiple plausible sequence fragments and predicts a path to connect them. |
OneRec-Think: In-Text Reasoning for Generative Recommendation (2026.acl-long)
Copied to clipboard
Zhanyu Liu, Shiyao Wang, Xingmei Wang, Rongzhou Zhang, Jiaxin Deng, Honghui Bao, Jinghao Zhang, Wuchao Li, PengFei Zheng, Xiangyu Wu, Yifei Hu, Qigen Hu, Xinchen Luo, Lejian Ren, Zhang Zixing, Qianqian Wang, Kuo Cai, Yunfan Wu, Hongtao Cheng, Zexuan Cheng, Lu Ren, Huanjie Wang, Yi Su, Ruiming Tang, Kun Gai, Guorui Zhou
| Challenge: | Existing generative models lack the capacity for explicit and controllable reasoning, a key advantage of LLMs. |
| Approach: | They propose a framework that integrates dialogue, reasoning, and personalized recommendation. |
| Outcome: | Experiments across public benchmarks show state-of-the-art performance. |
Measure Children’s Mindreading Ability with Machine Reading (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing scoring models do not take the features of the stories and video clips into account when scoring, which will reduce the accuracy of the models. |
| Approach: | They propose to leverage the features extracted from stories and videos related to the questions being asked during the children’s mindreading evaluation. |
| Outcome: | The proposed framework agrees well with human experts on scores produced by the models. |
Verifiable Format Control for Large Language Model Generations (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods focus on benchmarking general instruction following while overlooking how to improve specific format following ability for small LLMs. |
| Approach: | They propose to synthesize massive datasets to improve LLMs' format following abilities by using a verifiable format following feature. |
| Outcome: | The proposed method improves the format following ability of small LLMs with about 7B parameters. |
Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA (2022.findings-emnlp)
Copied to clipboard
Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou
| Challenge: | Visual Question Answering (VQA) models are prone to learn the shortcut solution formed by dataset biases rather than the intended solution. |
| Approach: | They propose a dataset that considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets. |
| Outcome: | The proposed dataset considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets. |
Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval (2025.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal retrieval models are lacking in visual representations of multimodal data. |
| Approach: | They propose a visualized information retrieval paradigm where multimodal information is represented by a unified visual format called Screenshots for various retrieval applications. |
| Outcome: | The proposed model is based on a large dataset of screenshots from diverse sources . it is compared with existing models and lays a solid foundation for the new model . |
Rethinking Vocabulary Augmentation: Addressing the Challenges of Low-Resource Languages in Multilingual Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to augment vocabularies ignore the disparities between model representation and frequency distributions. |
| Approach: | They propose an Entropy-Consistency Word Selection method which integrates semantic and frequency metrics for vocabulary augmentation. |
| Outcome: | The proposed method improves performance for low-resource languages compared to high-resourced ones . it integrates semantic and frequency metrics for vocabulary augmentation . |
What Factors Influence LLMs’ Judgments? A Case Study on Question Answering (2024.lrec-main)
Copied to clipboard
Lei Chen, Bobo Li, Li Zheng, Haining Wang, Zixiang Meng, Runfeng Shi, Hao Fei, Jun Zhou, Fei Li, Chong Teng, Donghong Ji
| Challenge: | Existing studies indicate that Large Language Models perform at a level comparable to humans with advantages of speed and cost-effectiveness in different fields. |
| Approach: | They propose to introduce four unexplored factors and a new dimension of question difficulty to provide a more comprehensive understanding of LLMs’ judgments across varying question intricacies. |
| Outcome: | The proposed dimensions of question difficulty and answer quantity provide valuable insights into optimizing LLMs’ performance as judges. |
More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing systems struggle with multimodal content where the emergent meaning transcends the aggregation of individual modalities. |
| Approach: | They propose a framework to characterize semantic intent shifts where modalities interact to construct implicit hate from benign cues or neutralize toxicity through semantic inversion. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on H-VLI and on established benchmarks. |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)
Copied to clipboard
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
Privacy-Preserving Reasoning with Knowledge-Distilled Parametric Retrieval Augmented Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing RAG systems require uploading local documents to the cloud, resulting in inference latency and poor generalization on out-of-distribution (OOD) inputs. |
| Approach: | They propose a generalizable knowledge-distilled parametric RAG model aligned with standard RAG in document structure and parameter activation. |
| Outcome: | The proposed model outperforms baselines in accuracy and generalizes well on out-of-distribution (OOD) data. |
AMOA: Global Acoustic Feature Enhanced Modal-Order-Aware Network for Multimodal Sentiment Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods treat three modal features equally, without distinguishing the importance of different modalities. Existing models split the video into frames, leading to missing the global acoustic information. |
| Approach: | They propose a global Acoustic feature enhanced Modal-Order-Aware network to address these problems. |
| Outcome: | The proposed model outperforms state-of-the-art models on two public datasets. |
Enhancing Semantics in Multimodal Chain of Thought via Soft Negative Sampling (2024.lrec-main)
Copied to clipboard
| Challenge: | Chain of thought (CoT) is used for complex reasoning problems, but hallucinations are a problem in multimodal CoT. |
| Approach: | They propose a method to generate soft negative samples with different semantics to mitigate hallucinations in multimodal CoT. |
| Outcome: | The proposed method mitigates hallucinations in multimodal CoT by using soft negative sampling. |
FlipDA: Effective and Robust Data Augmentation for Few-Shot Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text data augmentation are limited to simple tasks and weak baselines. |
| Approach: | They propose a data augmentation method FlipDA that uses a generative model and a classifier to generate label-flipped data. |
| Outcome: | The proposed method improves many tasks while not negatively affecting the others. |
The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking (2022.findings-acl)
Copied to clipboard
Yinghui Li, Qingyu Zhou, Yangning Li, Zhongli Li, Ruiyang Liu, Rongyi Sun, Zizhen Wang, Chao Li, Yunbo Cao, Hai-Tao Zheng
| Challenge: | Chinese Spell Checking (CSC) aims to detect and correct spelling errors, which are caused by the phonological or visual similarity. |
| Approach: | They propose an Error-driven COntrastive Probability Optimization framework to refine the knowledge representations of pre-trained language models to avoid predicting common characters. |
| Outcome: | Extensive experiments and detailed analyses on SIGHAN datasets demonstrate that ECOPO is simple yet effective. |
Knowledge-Grounded Dialogue Generation with Term-level De-noising (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for dialogue generation use terms to describe a post, such as 'question', 'utterance','source' and 'query', but this approach introduces noise and diminishes the effectiveness of the generative models. |
| Approach: | They propose a Knowledge Term Weighting Model that incorporates term-level de-noising of the selected knowledge into the model. |
| Outcome: | The proposed model achieves statistically significant improvements over methods without term weighting on two publicly available datasets Wizard of Wikipedia and Holl-E. |
A Universal Discriminator for Zero-Shot Generalization (2023.acl-long)
Copied to clipboard
| Challenge: | Generative modeling has been the dominant approach for large-scale pretraining and zeroshot generalization. |
| Approach: | They propose a discriminator that predicts whether a text sample comes from the true data distribution and which option has the highest probability of coming from the real data distribution. |
| Outcome: | The proposed discriminative approach outperforms GANs on a number of NLP tasks by 16.0%, 7.8%, and 11.5% respectively. |