Papers by Liang He
How Do Humans Write Code? Large Models Do It the Same Way Too (2024.emnlp-main)
Copied to clipboard
| Challenge: | Program-of-Thought (PoT) replaces natural language-based Chain-ofThough (CoT) but introduces more reasoning errors, such as incorrect formulas or flawed logic, compared to CoT. |
| Approach: | They propose a method that integrates CoT and Program-of-Thought to achieve more accurate reasoning and reinforcement learning. |
| Outcome: | The proposed method achieves an average improvement of 6.5% on the Llama-Base model and 4.3% on the Mistral-Bass model across 8 mathematical calculation datasets. |
A Regularization-based Transfer Learning Method for Information Extraction via Instructed Graph Decoder (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for information extraction (IE) focus on training task-specific models, while common knowledge among different IE tasks is not explicitly modeled. |
| Approach: | They propose a regularization-based transfer learning method for IE via an instructed graph decoder which decodes various complex structures into a graph uniformly based on corresponding instructions. |
| Outcome: | The proposed method can learn common knowledge from existing datasets and transfer it to a new dataset with new labels. |
Composable Text Controls in Latent Space with ODEs (2023.emnlp-main)
Copied to clipboard
Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiaodan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li, Zhiting Hu
| Challenge: | Existing approaches to composable text operations often require plug-and-play . a single LM can perform arbitrary text operation composition in the latent space . |
| Approach: | They propose an efficient approach for composable text operations in the latent space of text . they connect pretrained LMs to the laten space and adapt them to the space . |
| Outcome: | The proposed approach improves on existing methods in the latent space of text. |
JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods achieve promising performance in in-target stance detection when trained and tested on the same datasets. |
| Approach: | They propose a joint contrastive learning framework to generalize stance features for unseen targets. |
| Outcome: | The proposed framework achieves state-of-the-art on three benchmark datasets. |
SKD-NER: Continual Named Entity Recognition via Span-based Knowledge Distillation with Reinforcement Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Continual learning for named entity recognition (CL-NER) aims to enable models to continuously learn new entity types while retaining the ability to recognize previously learned ones. |
| Approach: | They propose a model that leverages knowledge distillation to retain memory and employs reinforcement learning strategies to optimize the soft labeling and distillation losses generated by the teacher model to effectively prevent catastrophic forgetting. |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets showing that it significantly improves the performance of the CL-NER task. |
Enhancing Medical Dialogue Generation through Knowledge Refinement and Dynamic Prompt Adjustment (2025.findings-acl)
Copied to clipboard
| Challenge: | Medical dialogue systems (MDS) struggle to identify relevant medical knowledge and generate accurate responses. |
| Approach: | They propose a medical dialogue system that integrates knowledge refining and dynamic prompt adjustment to improve medical knowledge and accuracy. |
| Outcome: | The proposed system outperforms state-of-the-art systems in both generation quality and medical entity accuracy. |
Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on improving the performance of domain-specific models based on the target dataset. |
| Approach: | They propose a Large Language Model-based Continual Learning (LLM-CL) model for ABSA that learns the target domain’s ability while maintaining the history domains’ abilities. |
| Outcome: | The proposed model obtains new state-of-the-art over 19 datasets. |
ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback (2023.findings-emnlp)
Copied to clipboard
Wenxiang Jiao, Jen-tse Huang, Wenxuan Wang, Zhiwei He, Tian Liang, Xing Wang, Shuming Shi, Zhaopeng Tu
| Challenge: | Large language models (LLMs) like ChatGPT are only accessible through restricted APIs, which creates barriers to new research and advancements in the field. |
| Approach: | They propose a framework to enhance and regulate the translation abilities during chat . they reformulate translation data into the instruction-following style and introduce a "Hint" field . |
| Outcome: | The proposed framework enhances and regulates the translation abilities during chat . it reformulates translation data into the instruction-following style and introduces a "Hint" field . |
Reason from Fallacy: Enhancing Large Language Models’ Logical Reasoning through Logical Fallacy Understanding (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated good performance in many reasoning tasks, but struggle with some more complex reasoning tasks including logical reasoning. |
| Approach: | They propose five concrete tasks from three cognitive dimensions of WHAT, WHY, and HOW to evaluate LLMs’ capability of logical fallacy understanding. |
| Outcome: | The proposed dataset can be used to evaluate LLMs’ LFU capability and to fine-tune LLM models to obtain significantly enhanced performance on logical reasoning. |
Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Conventional speculative decoding methods use a predefined length policy for proposing drafts, but the reality deviates from this assumption. |
| Approach: | They propose a self-verification length policy that adaptively determines the lengths of draft sequences by referring to the draft entropy. |
| Outcome: | The proposed method achieves 17% speedup on MT-Bench and 22% speedup in long-form reasoning. |
Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents (2026.findings-acl)
Copied to clipboard
Ying He, Zhouhong Gu, Zhecheng Hu, Yubo Zhou, Hao Shen, Jiaqing Liang, Zhaoqian Dai, Ma Shuguang, Fei Yu, Yanghua Xiao, Zhixu Li
| Challenge: | Existing LLMs struggle to identify errors in financial documents, a study shows . 18% of financial practitioners make errors daily, one-third make errors several times weekly, and 59% make errors multiple times monthly. |
| Approach: | They introduce FinED-Bench, a publicly available Benchmark for financial error detection . it covers nine real-world financial scenarios and includes over 900 documents in 2025 . supervised fine-tuning can significantly improve the performance of weaker LLMs, they show . |
| Outcome: | The proposed benchmark covers nine real-world financial scenarios and includes over 900 documents reported in 2025 that are unseen by existing language models. |
FlowRAG: Synergizing Explicit Reasoning via Frequency-Aware Multi-Granularity Graph Flow (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for Graph-based retrieval-augmented generation rely on implicit semantic relevance propagation. |
| Approach: | They propose a semantic-aware retrieval framework that improves both semantic recall and explicit reasoning. |
| Outcome: | Extensive experiments show that FlowRAG improves both semantic recall and explicit reasoning. |
SG-FSM: A Self-Guiding Zero-Shot Prompting Paradigm for Multi-Hop Question Answering Based on Finite State Machine (2025.findings-naacl)
Copied to clipboard
Xiaochen Wang, Junqing He, Liang Chen, Gholamreza Haffari, Yiru Wang, Zhe Yang, Xiangdi Meng, Kunhao Pan, Zhifang Sui
| Challenge: | Multi-hop Question Answering (MHQA) is a challenging task that requires models to answer multiple questions with multiple passages. |
| Approach: | They propose a self-guided prompting finite state machine to improve multi-hop reasoning abilities by iterating over multiple questions and correcting itself to improve accuracy. |
| Outcome: | The proposed approach outperforms baselines on Musique and other datasets. |
UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation (2024.acl-long)
Copied to clipboard
Xun Liang, Shichao Song, Simin Niu, Zhiyu Li, Feiyu Xiong, Bo Tang, Yezhaohui Wang, Dawei He, Cheng Peng, Zhonghao Wang, Haiying Deng
| Challenge: | Large language models (LLMs) produce hallucinated text, compromising their practical utility in professional contexts. |
| Approach: | They have developed an unconstrained hallucination generation evaluation benchmark that contains hallucines generated by large language models with minimal restrictions. |
| Outcome: | The proposed benchmarks are based on a Chinese-language dataset that is lacking in the field. |
Order Doesn’t Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Logical reasoning is essential for large language models (LLMs) to ensure accurate and coherent inferences. |
| Approach: | They propose an order-centric data augmentation framework based on commutativity in logical reasoning that randomly shuffles independent premises to introduce condition order augmentation. |
| Outcome: | The proposed framework improves LLMs’ reasoning performance and adaptability to diverse logical structures. |
MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing multi-agent systems lack agent coordination and rely on predefined procedures . existing systems lack adaptive task coordination when task is big and complex . |
| Approach: | They propose a large-scale autonomous LLM-based multi-agent system that generates agents based on task complexity and enables dynamic task decomposition, parallel execution, efficient communication and comprehensive system monitoring. |
| Outcome: | The proposed system outperforms existing systems in task completion efficiency and scalability. |
AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation (2025.acl-long)
Copied to clipboard
Xiechi Zhang, Zetian Ouyang, Linlin Wang, Gerard De Melo, Zhu Cao, Xiaoling Wang, Ya Zhang, Yanfeng Wang, Liang He
| Challenge: | Existing evaluation methods based on large language models (LLMs) are expensive and lack expertise due to limitations in human expertise. |
| Approach: | They propose an open-source automatic evaluation model with 13B parameters specifically engineered to measure the question-answering proficiency of medical LLMs. |
| Outcome: | The proposed model surpasses baselines in terms of correlation with human judgments. |
Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval-augmented generation methods are insufficient for multi-hop question answering . however, they tend to generate hallucinations due to semantic mismatching . |
| Approach: | They propose to optimize question semantic space for dynamic retrieval-augmented multi-hop question answering by optimizing the semantic embeddings. |
| Outcome: | The proposed method outperforms existing RAG methods in both in- and out-of-domain settings. |
MADial-Bench: Towards Real-world Evaluation of Memory-Augmented Dialogue Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for memory-augmented dialogue systems lack practical value . current evaluation methods only consider passive memory retrieval while ignoring diverse memory recall with rich triggering factors. |
| Approach: | They propose to use long-term memory to create human-like dialogues using chatbots. |
| Outcome: | The proposed benchmark covers memory retrieval and memory recognition tasks with both passive and proactive memory recall data. |
Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training (2025.acl-long)
Copied to clipboard
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Jiahao Xu, Tian Liang, Pinjia He, Zhaopeng Tu
| Challenge: | Large Language Models exhibit a level of intelligence that is both impressive and everevolving, but their ability to refuse generating unsafe content is a double-edged sword. |
| Approach: | They propose a method to tackle a refusal position bias within safety tuning data that compromises the models’ ability to appropriately refuse generating unsafe content. |
| Outcome: | The proposed method significantly improves model safety without compromising performance and surpasses baseline methods in defending against attacks. |
HAUSER: Towards Holistic and Automatic Evaluation of Simile Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Similes are a crucial part of creative writing, but there is still a lack of evaluation metrics for simile generation. |
| Approach: | They propose to use similes as a tool to evaluate simile generation metrics . they propose to combine five criteria and automatic metrics for each criterion . |
| Outcome: | The proposed metrics are significantly more correlated with human ratings from each perspective compared with prior automatic metrics. |
Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on equally treating the contribution of each modality or statically using text as the dominant modality to conduct interaction, which neglects the situation where each modal may become dominant. |
| Approach: | They propose a Knowledge-Guided Dynamic Modality Attention Fusion Framework (KuDA) that uses sentiment knowledge to guide the model dynamically selecting the dominant modality and adjusting the contributions of each modality. |
| Outcome: | The proposed model can be used to highlight the contribution of dominant modality through the correlation evaluation loss. |
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach (2021.emnlp-main)
Copied to clipboard
Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao
| Challenge: | Existing approaches to adversarial regularization treat adversarials and defending players equally, which is undesirable because only the defending player contributes to the generalization performance. |
| Approach: | They propose a method which formulates adversarial regularization as a Stackelberg game and induces a competition between a leader and a follower. |
| Outcome: | The proposed method outperforms existing adversarial regularization baselines on a set of machine translation and natural language understanding tasks. |
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation (P19-3)
Copied to clipboard
Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan, Eric Xing
| Challenge: | Texar is an open-source text generation toolkit that supports a broad set of text generation tasks. |
| Approach: | They introduce Texar, an open-source text generation toolkit that supports text generation tasks. |
| Outcome: | Texar supports machine translation, summarization, dialog, content manipulation, and more. |
MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)
Copied to clipboard
Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
TMFN: A Target-oriented Multi-grained Fusion Network for End-to-end Aspect-based Multimodal Sentiment Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for multimodal aspect-based sentiment analysis focus on fusing image regional information and textual words. |
| Approach: | They propose a multimodal aspect-based sentiment analysis method that integrates regional and global image information with global image data. |
| Outcome: | Experiments show that the proposed method outperforms state-of-the-art methods on two benchmark datasets. |
A Multi-Format Transfer Learning Model for Event Argument Extraction via Variational Information Bottleneck (2022.coling-1)
Copied to clipboard
| Challenge: | Event argument extraction (EAE) aims to extract arguments with given roles from texts. |
| Approach: | They propose a multi-format transfer learning model with variational information bottleneck to learn from existing datasets. |
| Outcome: | The proposed model improves on three benchmark datasets and obtains state-of-the-art performance on EAE. |
Graph Reasoning Paradigm: Structured and Symbolic Reasoning with Topology-Aware Reinforcement Learning for Large Language Models (2026.acl-long)
Copied to clipboard
Runxuan Liu, Xianhao Ou, Xinyan Ma, Jiyuan Wang, Jiafeng Liang, Jiaqi Li, Tao He, Zheng Chu, Rongchuan Mu, Zekun Wang, Baoxin Wang, Dayong Wu, Ming Liu, Shijin Wang, Guoping Hu, Bing Qin
| Challenge: | Existing methods for long chain-of-thought (LCoT) are coarse-grained, reward hacking, and poor generalization. |
| Approach: | They propose a Long Chain-of-Thought (LCoT) model that integrates reinforcement learning with verifiable rewards with a process-aware verification approach. |
| Outcome: | The proposed model improves reasoning and code generation tasks while reducing the cost of training and performance bottlenecks. |
A Multilingual Dataset and Empirical Validation for the Mutual Reinforcement Effect in Information Extraction (2026.findings-acl)
Copied to clipboard
Chengguang Gan, Sunbowen Lee, Qingyu Yin, Yunhao Liang, Xinyang He, Hanjun Wei, Younghun Lim, Shijian Wang, Hexiang Huang, QingHao Zhang, Shiwen Ni, Tatsunori Mori
| Challenge: | Existing work on the Mutual Reinforcement Effect in information extraction has not been empirically validated . 76 percent of the 21 sub-datasets exhibit the Mutual Reforcement effect across languages . |
| Approach: | They propose a multilingual MRE mix dataset that integrates 21 sub-datasets covering English, Japanese, and Chinese. |
| Outcome: | The proposed framework reduces manual annotation effort while preserving structural requirements of MRE tasks. |
LLM-based Conversational Recommendation Agents with Collaborative Verbalized Experience (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive zero-shot capabilities in conversational recommender systems (CRS). |
| Approach: | They propose LLM-based CRS-based LLMs with Collaborative Verbalized Experience to enhance historical conversations by sampling trajectories of LLM agents on historical queries and establishing verbalized experience banks . |
| Outcome: | The proposed system improves on existing approaches to enhancing historical conversations by leveraging trajectories and verbalized experiences from LLMs on historical queries and user feedback. |
DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks for hallucination detection are intentionally generated by large language models (LLMs) however, many focus on factuality while ignoring faithfulness. |
| Approach: | They propose a dialogue-level hallucination evaluation benchmark for large language models . they integrate the topic into prompts and facilitate a dialog between two LLMs . |
| Outcome: | The proposed benchmark covers four common multi-turn dialogue domains and five hallucination subtypes, extended from factuality and faithfulness hallucines. |
Token-wise Curriculum Learning for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of “easy” samples from training data at the early stage of training. |
| Approach: | They propose a token-wise curriculum learning approach that creates sufficient amounts of easy samples from training data. |
| Outcome: | The proposed approach outperforms baselines on five language pairs on low-resource languages. |
Let’s Rectify Step by Step: Improving Aspect-based Sentiment Analysis with Diffusion Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Empirical evaluations conducted on eight benchmark datasets underscore the compelling advantages offered by DiffusionABSA when compared against robust baseline models. |
| Approach: | They propose a diffusion model which extracts aspects step by step and learns a denoising process that progressively restores them in a reverse manner. |
| Outcome: | Empirical evaluations on eight benchmark datasets underscore the compelling advantages offered by DiffusionABSA when compared against robust baseline models. |
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization (2021.acl-long)
Copied to clipboard
Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen
| Challenge: | 'lottery tickets' can be trained to match the performance of a full model . subnetwork training can also outperform random sampled subnetworks of the same size . |
| Approach: | They propose to train a subnetwork of 'lottery tickets' to match the full model's performance. |
| Outcome: | The proposed model outperforms subnetworks of the same size in a phase transition phenomenon . the proposed model improves single task fine-tuning by 0.9 points on BERT-base and 1.0 points on GLUE large . |
PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Numerical reasoning is ubiquitous in scientific research and financial analysis, but few benchmarks evaluate them by integrating numerical processing and mathematical reasoning. |
| Approach: | They propose a numerically-integrated hierarchical benchmark with 27,215 questions derived from 7,404 math word problems that spans 4 key cognitive aspects, 14 subcategories, and 2 modalities. |
| Outcome: | The proposed model improves Qwen-2.5 score with SOLVE and IRPO training. |
UMRSpell: Unifying the Detection and Correction Parts of Pre-trained Models towards Chinese Missing, Redundant, and Spelling Correction (2023.acl-long)
Copied to clipboard
| Challenge: | Chinese Spelling Correction (CSC) is a task of detecting and correcting misspelled charac- ters in Chinese texts. |
| Approach: | They propose a model to learn detection and correction parts together from a multi-task learning perspective. |
| Outcome: | The proposed model can learn detection and correction parts together from a multi-task learning perspective. |
Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond (2025.acl-industry)
Copied to clipboard
Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Tanglifu Tanglifu, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, Xiangzheng Zhang
| Challenge: | Experimental results show that opensource curriculum training is more effective when distinct datasets are available for different training stages. |
| Approach: | They propose an opensource suite for training long reasoning models using publicdata and models. |
| Outcome: | The proposed model outperforms DeepSeek-R1-DistillQwen-32B models in math reasoning. |
Simultaneous Translation (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | Simultaneous translation is a problem that has long been considered one of the hardest problems in AI . this tutorial will provide a deep understanding of the history and the recent advances in simultaneous translation. |
| Approach: | This tutorial will examine the design and evaluation of policies for simultaneous translation . it will provide an overview of the history and recent advances in simultaneous translation. |
| Outcome: | This tutorial will examine the design and evaluation of policies for simultaneous translation . |
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework (P19-1)
Copied to clipboard
Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, Haifeng Wang
| Challenge: | Simultaneous translation is notoriously dif- ficult due to word-order differences. |
| Approach: | They propose a prefix-to-prefix framework that implicitly learns to anticipate in a single translation model. |
| Outcome: | The proposed framework achieves low latency and reasonable qual- ity on 4 directions. |
Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments show that MEO significantly improves computational efficiency . compared to dense networks, sparsely activated networks only employ a few parameters for each input . |
| Approach: | They propose a method that merges multiple experts into one to reduce computation costs . they demonstrate that a sparse Mixture of Experts (MoE) can reduce the cost by activating a small subset of parameters for each input . |
| Outcome: | The proposed approach reduces the computational cost to that of a single expert by 83.3% compared to 82.6% in vanilla MoE. |
A Generative Model for End-to-End Argument Mining with Reconstructed Positional Encoding and Constrained Pointer Mechanism (2022.emnlp-main)
Copied to clipboard
| Challenge: | Argument mining (AM) is a challenging task as it requires recognizing complex argumentation structures involving multiple subtasks. |
| Approach: | They propose a generative framework where expected outputs of AM are framed as a simple target sequence. |
| Outcome: | The proposed framework achieves state-of-the-art on two AM benchmarks. |
Mixup-Transformer: Dynamic Data Augmentation for NLP Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work on data augmentation techniques that interpolate inputs and labels shows strong effectiveness in image classification. |
| Approach: | They propose to integrate mixup to transformer-based pre-trained architecture for NLP tasks while keeping the whole end-to-end training system. |
| Outcome: | The proposed framework improves on GLUEbenchmark and transformer-based learning models while keeping the whole end-to-end training system. |
CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)
Copied to clipboard
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan
| Challenge: | Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages . |
| Approach: | They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models . |
| Outcome: | The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English . |
RGAR: Recurrence Generation-augmented Retrieval for Factual-aware Medical Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrieval approaches often overlook patient-specific factual knowledge embedded in EHRs . existing retrieval frameworks often overlook this factual information, limiting its effectiveness in clinical decision-making. |
| Approach: | They propose a recurrence generation-augmented retrieval framework that synergizes factual and conceptual knowledge from dual sources. |
| Outcome: | The proposed framework improves on factual-aware medical QA benchmarks. |
PAD-Net: An Efficient Framework for Dynamic Networks (2023.acl-long)
Copied to clipboard
| Challenge: | Dynamic networks can significantly improve the model’s representation power with acceptable computational cost. |
| Approach: | They propose a partially dynamic network to transform redundant dynamic parameters into static ones and iterative mode partition to partition dynamic and static parameters efficiently. |
| Outcome: | The proposed network surpasses fully dynamic networks by +0.7% top-1 acc with only 30% dynamic parameters for DY-Conv and +1.9% average score in language understanding with only 50% dynamic parameters. |
Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion (2025.acl-long)
Copied to clipboard
Jianqing Zhu, Huang Huang, Zhihang Lin, Juhao Liang, Zhengyang Tang, Khalid Almubarak, Mosen Alharthi, Bang An, Juncai He, Xiangbo Wu, Fei Yu, Junying Chen, Ma Zhuoheng, Yuhao Du, He Zhang, Saied Alshahrani, Emad A. Alghamdi, Lian Zhang, Ruoyu Sun, Haizhou Li, Benyou Wang, Jinchao Xu
| Challenge: | In the evolving landscape of large language models, the predominant focus has been on English and Chinese. |
| Approach: | They propose to utilize Arabic-specific vocabulary in the tokenizer to accelerate decoding. |
| Outcome: | The proposed model achieves decent performance comparable to the best Arabic LLMs across various Arabic benchmarks. |
UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model (2023.findings-emnlp)
Copied to clipboard
Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, Qin Jin, Liang He, Xin Lin, Fei Huang
| Challenge: | Existing studies for visually-situated language understanding have shown shallow zero-shot visual text recognition ability when fed a low-resolution image with salient text information. |
| Approach: | They propose a model for universal OCR-free visually-situated language understanding based on the Multimodal Large Language Model (MLLM) their model is jointly finetuned on a wide range of visually situated language understanding tasks via a unified instruction format. |
| Outcome: | The proposed model achieves state-of-the-art ocr-free performance in 8 out of 10 visually-situated language understanding tasks across 5 domains: documents, tables, charts, natural images, and webpage screenshots. |
ARCH: Efficient Adversarial Regularized Training with Caching (2021.findings-emnlp)
Copied to clipboard
Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
| Challenge: | Existing approaches to regularize models require generating a perturbation for each sample in each epoch. |
| Approach: | They propose an adversarial regularization method where perturbations are generated and cached once every several epochs. |
| Outcome: | The proposed method significantly eases the computational burden (saves up to 70% of computational time) it produces a notably better (in most of the tasks) or comparable model generalization. |
PsychEval: A Multi-Session and Multi-Therapy Benchmark for High-Realism AI Psychological Counselor (2026.findings-acl)
Copied to clipboard
Qianjun Pan, Junyi Wang, Jie Zhou, Yutao Yang, Junsong Li, Kaiyin Xu, Yougen Zhou, Yihan Li, JingYuan Zhao, Qin Chen, Ningning Zhou, Kai Chen, Liang He
| Challenge: | Existing models focus on a single therapy, but complex cases require flexible strategies among various therapies. |
| Approach: | They propose a multi-session, multi-therapy, and highly realistic benchmark . it is designed to address three key challenges: 1) can we train a highly realistic AI counselor? 2) How to systematically evaluate an AI counselor?" |
| Outcome: | The proposed benchmark is annotated with extensive professional skills and includes over 677 meta-skills and 4577 atomic skills. |
TamEdit: Trajectory-Aware Meta-Learning for Specificity-Preserving Continual Knowledge Editing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for continual knowledge editing focus on single edits or preventing knowledge forgetting. |
| Approach: | They propose a meta-learning method that preserves specificity for continual knowledge editing by capturing relationships between different single edits within the trajectory. |
| Outcome: | Experiments show that TamEdit outperforms baselines in continual editing while preserving general capabilities. |
MixRED: A Mix-lingual Relation Extraction Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing research focuses on monolingual relation extraction, but there is a significant gap in understanding relation extraction in the mix-lingual scenario. |
| Approach: | They propose a task of considering relation extraction in the mix-lingual scenario . they construct a human-annotated dataset to support the task . |
| Outcome: | The proposed task evaluates state-of-the-art supervised models and large language models on the human-annotated dataset MixRED. |
COPR: Continual Human Preference Learning via Optimal Policy Regularization (2025.findings-acl)
Copied to clipboard
Han Zhang, Lin Gui, Yu Lei, Yuanzhao Zhai, Yehong Zhang, Zhuo Zhang, Yulan He, Hui Wang, Yue Yu, Kam-Fai Wong, Bin Liang, Ruifeng Xu
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) is effective for aligning Large Language Models with human preferences, but its complex process limits its ability to continually learn human feedback. |
| Approach: | They propose a non-RL offline method to convert historical optimal policies into optimization constraints when continually learning new preferences. |
| Outcome: | The proposed method outperforms strong CL baselines in terms of reward-based evaluations and human assessment. |
What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time (2026.acl-long)
Copied to clipboard
| Challenge: | Existing TTRL methods rely on positive pseudo-labeling strategies to enhance reasoning capabilities. |
| Approach: | They propose a test-time reinforcement learning framework that mitigates label noise amplification by deriving pseudo-rewards from majority voting consensus. |
| Outcome: | The proposed framework mitigates label noise amplification by implementing selective positive pseudo-labeling and entropy-gated negative p-labeled pruning. |
Is There a One-Model-Fits-All Approach to Information Extraction? Revisiting Task Definition Biases (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Definition bias is a negative phenomenon that can mislead models. |
| Approach: | They propose a framework that measures definition bias, bias-aware fine-tuning and task-specific bias mitigation to mitigate definition bias in information extraction. |
| Outcome: | The proposed framework mitigates definition bias in information extraction tasks by measuring definition bias, bias-aware fine-tuning, and task-specific bias mitigation. |
P-React: Synthesizing Topic-Adaptive Reactions of Personality Traits via Mixture of Specialized LoRA Experts (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on personalized large language models focus on modeling explicit character profiles, while ignoring the underlying personality traits that truly shape behaviors and decision-making. |
| Approach: | They propose a personalized large language model (LLM) that captures implicit Big Five personality traits and integrates a Personality Specialization Loss to capture individual trait expressions. |
| Outcome: | The proposed model improves on Big Five personality traits and integrates a Personality Specialization Loss (PSL) to capture individual trait expressions. |
RGR-KBQA: Generating Logical Forms for Question Answering Using Knowledge-Graph-Enhanced Large Language Model (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for Knowledge Base Question Answering (KBQA) face hallucination problems, resulting in low accuracy. |
| Approach: | They propose a retrieval-generate-retrieve framework that uses a Retrieve-Generate framework to retrieve factual knowledge from a knowledge graph. |
| Outcome: | Experimental results show that RGR-KBQA improves on CWQ and WebQSP datasets. |
Exploiting Noisy Data in Distant Supervision Relation Classification (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to relation classification are noisy and time-consuming . RCEND uses noisy data to split noisy data into correctly and incorrectly labeled data . |
| Approach: | They propose a framework to enhance relation classification by exploiting noisy data . they use an instance discriminator with reinforcement learning to split noisy data into correctly and incorrectly labeled data based on the noisy data. |
| Outcome: | The proposed method outperforms the state-of-the-art models on relation classification . the proposed method is based on a semi-supervised learning method . |
Robust Neural Machine Translation with Joint Textual and Phonetic Embedding (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation models are sensitive to noises in input sentences . one special kind of noise is the homophone noise, where words are replaced by other words with similar pronunciations. |
| Approach: | They propose to embed phonetic and textual information into neural machine translation datasets to improve robustness to homophone noises. |
| Outcome: | The proposed method improves the robustness of neural machine translation to homophone noises on clean test sets. |
Learning Intrinsic Dimension via Information Bottleneck for Explainable Aspect-based Sentiment Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Gradient-based explanation methods are increasingly used to interpret neural models in natural language processing (NLP) however, in the context of Aspect-based Sentiment Analysis, only specific dimensions are pertinent. |
| Approach: | They propose a Gradient-based explanation framework that leverages an information bottleneck to refine word embeddings into a concise intrinsic dimension, maintaining essential features and omitting unrelated information. |
| Outcome: | The proposed framework improves both the models’ performance and explanations’ clarity by identifying sentiment-aware features. |
LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration Benchmark (2026.findings-acl)
Copied to clipboard
Guangyi Liu, Pengxiang Zhao, Liang Liu, Zhiming Chen, Yuxiang Chai, Yaozhen Liang, WenHao Wang, Siheng Chen, Zhengxi Lu, Shuai Ren, Hao Wang, Shibo He, Yong Liu, Wenchao Meng
| Challenge: | Mobile GUI agents show promise in automating tasks but face significant generalization challenges in long-tail scenarios. |
| Approach: | They propose a benchmark framework for mobile GUI agents that measures the performance of GUI agents by analyzing their performance. |
| Outcome: | The LearnGUI benchmark outperforms existing methods in offline and online evaluations and demonstrates consistent gains across model architectures. |
C-LLM: Learn to Check Chinese Spelling Errors Character by Character (2024.emnlp-main)
Copied to clipboard
| Challenge: | Chinese Spell Checking (CSC) aims to detect and correct spelling errors in sentences. |
| Approach: | They propose a Chinese Spell Checking method that learns to check errors Character by Character. |
| Outcome: | The proposed method achieves a 2.1% enhancement in general scenarios and a significant improvement in vertical domain scenarios compared to existing methods. |
Vanessa: Visual Connotation and Aesthetic Attributes Understanding Network for Multimodal Aspect-based Sentiment Analysis (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to analyze images focus on superficial features or descriptions, omitting subtle contextual information. |
| Approach: | They propose a Visual Connotation and Aesthetic Attributes Understanding Network (Vanessa) for Multimodal Aspect-based Sentiment Analysis. |
| Outcome: | The proposed network captures both implicit and explicit sentimental cues and can be used to enrich textual sentiment analysis. |
SEEK: Segmented Embedding of Knowledge Graphs (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph embedding can not make a proper trade-off between the model complexity and the model expressiveness, which makes them far from satisfactory. |
| Approach: | They propose a lightweight modeling framework that can achieve highly competitive relational expressiveness without increasing the model complexity. |
| Outcome: | The proposed framework can achieve highly competitive relational expressiveness without increasing model complexity. |
Decomposing Argumentative Essay Generation via Dialectical Planning of Complex Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Argumentative Essay Generation (AEG) is a challenging task in computational argumentation, where detailed logical reasoning and effective rhetorical skills are essential. |
| Approach: | They propose an argumentative planning strategy for prompting large language models to generate high-quality essays by sketch planning and dialectical planning. |
| Outcome: | The proposed method generates more dialectical and persuasive essays with higher diversity compared to baselines. |
ChemAmp: Amplified Chemistry Tools via Composable Agents (2026.findings-acl)
Copied to clipboard
Zhucong Li, Powei Chang, Jin Xiao, Zhijian Zhou, Qianyu He, Jiaqing Liang, Fenglei Cao, Xu Yinghui, Yuan Qi
| Challenge: | LLM-based agents are powerful tools for automating complex scientific workflows, especially in chemistry, but their single-task performance is limited by tool constraints. |
| Approach: | They propose a framework that optimizes the collective capabilities of specialized tools by dynamic coordination within individual tasks. |
| Outcome: | The proposed framework outperforms chemistry-specialized models, generalist LLMs, and agent systems with tool orchestration. |
Don’t Take It Literally: An Edit-Invariant Sequence Loss for Text Generation (2022.naacl-main)
Copied to clipboard
Guangyi Liu, Zichao Yang, Tianhua Tao, Xiaodan Liang, Junwei Bao, Zhen Li, Xiaodong He, Shuguang Cui, Zhiting Hu
| Challenge: | Neural text generation models are typically trained by maximizing log-likelihood with the sequence cross entropy (CE) loss. |
| Approach: | They propose an Edit-Invariant Sequence Loss method which computes the matching loss of a target sequence with all n-grams in the generated sequence. |
| Outcome: | The proposed method outperforms the common CE loss and strong baselines on a wide range of tasks. |
KERS: A Knowledge-Enhanced Framework for Recommendation Dialog Systems with Multiple Subgoals (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing frameworks for multi-subgoal dialogs require a system to build a social bond with users to gain trust and develop affinity. |
| Approach: | They propose a framework for common knowledge-based multi-subgoal dialogs that divides up conversations with multiple subgoals and propose mechanisms to filter noisy knowledge and to include cleaned knowledge in the dialog response generation process. |
| Outcome: | The proposed framework obtains state-of-the-art results on a DuRecDial dataset in both automatic and human evaluation. |
From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) follow instructions with elaborate requirements, yet it remains under-explored how to enhance their ability to follow complex instructions with multiple constraints. |
| Approach: | They propose a method to obtain and utilize effective training data to enhance LLMs' ability to follow complex instructions with multiple constraints. |
| Outcome: | The proposed framework improves models' ability to follow instructions generally and generalize effectively across out-of-domain, in domain, and adversarial settings while maintaining general capabilities. |
Breaking the Reasoning Barrier A Survey on LLM Complex Reasoning through the Lens of Self-Evolution (2025.findings-acl)
Copied to clipboard
Tao He, Hao Li, Jingchang Chen, Runxuan Liu, Yixin Cao, Lizi Liao, Zihao Zheng, Zheng Chu, Jiafeng Liang, Ming Liu, Bing Qin
| Challenge: | OpenAI's O1 and subsequent projects like DeepSeek R1 have significantly advanced research on complex reasoning in LLMs. |
| Approach: | They analyze existing reasoning studies from the perspective of self-evolution and summarize O1-like works from open-source projects like DeepSeek R1 and Kimi-k1.5. |
| Outcome: | The proposed models are based on open-source models and pioneer advanced methodologies like Scaling Reinforcement Learning (RL). |
Metaphor Reasoning is Meta-reasoning (2026.acl-long)
Copied to clipboard
Qianyu He, Junting Lu, Yikai Zhang, Siyu Yuan, Xiaojun Meng, Jiansheng Wei, Jiaqing Liang, Yanghua Xiao
| Challenge: | Existing work on metaphor reasoning's impact on reasoning abilities is limited. |
| Approach: | They propose a system for synthesizing metaphorical riddles that satisfy five quality dimensions: diverse, balanced, reasoning-oriented, challenging, and verifiable. |
| Outcome: | The proposed system improves reasoning abilities across six domains using only thousands of metaphorical riddles. |
Beyond Cross-Modal Alignment: Measuring and Leveraging Modality Gap in Vision-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | a recent study shows that vision-language models have modality gaps that persist even in well-aligned models. |
| Approach: | They propose a modality-dominance score to measure and leverage modality gaps . they propose automatic interpretability metrics to evaluate these features in a scalable manner . |
| Outcome: | The proposed framework allows for training-free probing and editing methods for understanding model perception across genders and generating adversarial examples. |
Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | In real-world scenarios, user instructions often contain soft constraints, which are semantically related and cannot be rule-based verified, posing challenges for large language models. |
| Approach: | They propose a pipeline to construct datasets with high-quality outputs for instructions containing soft constraints automatically and use Direct Preference Optimization (DPO) as the training method. |
| Outcome: | The proposed model improves the LLMs' soft constraint following ability by using direct preference optimization (DPO) and constraint quantity. |
QuAC: Question Answering in Context (D18-1)
Copied to clipboard
Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, Luke Zettlemoyer
| Challenge: | a dataset for Question Answering in Context contains 14K information-seeking QA dialogs . questions are often more open-ended, unanswerable, or only meaningful within the dialog context . |
| Approach: | They propose a dataset for Question Answering in Context that contains 14K dialogs . they use a student to ask questions about a Wikipedia section and a teacher to answer them . |
| Outcome: | The proposed dataset underperforms humans in a number of reference models . the dataset contains 14K information-seeking dialogs over sections from Wikipedia . |
Hypernetwork-Assisted Parameter-Efficient Fine-Tuning with Meta-Knowledge Distillation for Domain Knowledge Disentanglement (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent work on domain adaptation for text summarization fails to account for the huge gap between dialogue and general articles. |
| Approach: | They propose a hypernetwork-assisted encoder-decoder architecture with parameter-efficient fine-tuning to disentangle domain-invariant knowledge from source domains while learning specific knowledge of the target domain. |
| Outcome: | The proposed model can disentangle domain-invariant knowledge from source domains while learning specific knowledge of the target domain. |
SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely applied to cross-domain NLP tasks like sentiment analysis, but fine-tuning them on the source domain tends to overfit, leading to inferior results on the target domain. |
| Approach: | They propose to pre-train a sentiment-aware language model (SentiX) via domain-invariant sentiment knowledge from large-scale review datasets and utilize it for cross-domain sentiment analysis tasks without fine-tuning. |
| Outcome: | The proposed model achieves state-of-the-art in all the cross-domain sentiment analysis tasks and can be trained with only 1% samples and better than BERT with 90% samples. |
Curriculum Prompt Learning with Self-Training for Abstractive Dialogue Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to summarize dialogues are difficult due to insufficient training data and low information density. |
| Approach: | They propose a curriculum-based prompt learning method with self-training that gradually increases the degree of prompt perturbation, improving dialogue understanding and modeling capabilities. |
| Outcome: | The proposed model outperforms baseline models on the AMI and ICSI datasets and human evaluations show it is superior in the quality of the summary generation. |
SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrain-finetuned models are increasingly complex and require more parameters to match the performance of full fine-tuning. |
| Approach: | They propose an efficient Adapter Tuning technique that freezes pretrained language models and fine-tunes a few extra modules. |
| Outcome: | The proposed setting outperforms the standard Adapter Tuning by 80% . the proposed setting is easy to use and has a high sparse ratio . |
MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI Agents (2026.acl-long)
Copied to clipboard
Pengxiang Zhao, Guangyi Liu, Yaozhen Liang, Weiqing He, Zhengxi Lu, WenHao Wang, Yuehao Huang, Yuxiang Chai, Zhaolu Kang, Yaxuan Guo, Hao Wang, Kexin Zhang, Liang Liu, Yong Liu
| Challenge: | Shortcuts such as APIs and deep-links have emerged as efficient complements to flexible GUI operations, but systematic evaluation of GUI–shortcut hybrid agents remains underexplored. |
| Approach: | They propose a benchmark that evaluates GUI-shortcut hybrid agents with a specific focus on the mobile domain. |
| Outcome: | MAS-Bench evaluates agent's ability to generate shortcuts by discovering and creating reusable, low-cost workflows. |
Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models (2026.eacl-long)
Copied to clipboard
Qiao Liang, Yanjiang Liu, Weixiang Zhou, Ben He, Yaojie Lu, Hongyu Lin, Jia Zheng, Xianpei Han, Le Sun, Yingfei Sun
| Challenge: | Existing research treats MLLMs as unified systems optimized through end-to-end training, but the impact of vision encoder’s prior knowledge is seldom investigated. |
| Approach: | They propose a metric to quantify the effect of prior knowledge on MLLM performance by integrating prior knowledge at the vision encoder level into a training framework. |
| Outcome: | The proposed training framework incorporates prior knowledge at the vision encoder level, and significantly boosts visual understanding capabilities of MLLMs. |
Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoning (2026.findings-acl)
Copied to clipboard
Fanding Huang, Guanbo Huang, Xiao Fan, Yi He, Xiao Liang, Xiao Chen, Qinting Jiang, Faisal Nadeem Khan, Jingyan Jiang, Zhi Wang
| Challenge: | Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have substantially improved the reasoning abilities of Large Language Models (LLMs). |
| Approach: | They propose a method that balances exploration and exploitation in the hidden-state space of response trajectories. |
| Outcome: | The proposed model yields consistent improvements across models, algorithms and reasoning benchmarks. |
Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reinforcement learning approaches suffer from dependency on external supervision and sparse reward signals from multi-constraint tasks. |
| Approach: | They propose a self-supervised reinforcement learning framework that eliminates dependency on external supervision by deriving reward signals directly from instructions and generating pseudo-labels for reward model training. |
| Outcome: | The proposed framework achieves strong improvements across 3 in-domain and 5 out-of-domain datasets while maintaining computational efficiency. |
Pun Generation with Surprise (N19-1)
Copied to clipboard
| Challenge: | In this paper, we explore creative generation with a focus on puns. |
| Approach: | They propose an unsupervised approach to generating puns using lots of raw text and a surprisal principle. |
| Outcome: | The proposed approach generates puns 30% of the time, doubles the neural generation baseline. |
LLM-based Open Domain Planning by Leveraging Entity-Attribute-Level Domain Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, large language models (LLMs) based on Open domain Natural language planning have limited application potential. |
| Approach: | They propose a dataset with a baseline for Open domain Natural language planning . the dataset provides the largest dataset for textual procedures to date . |
| Outcome: | The proposed dataset provides the largest dataset for textual procedures to date . it leverages entity-attribute-level action models to reveal relevant physical properties . |
Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training (2024.acl-long)
Copied to clipboard
Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song, LiuYiBo LiuYiBo, Qianguosun Qianguosun, Yuxin Liang, Hao Wang, Enming Zhang, Jiaxing Zhang
| Challenge: | Large language models suffer from severe hallucinations, compromising performance in knowledge-oriented QA, dialogue, and writing. |
| Approach: | They propose to enhance the information searching and reflection ability of large language models by training them in position-agnostic multi-step QA tasks to improve their model's accuracy. |
| Outcome: | The proposed model improves in multi-doc QA and other benchmarks by 13.7% absolute gain in shuffled settings, by 21.5% in passage retrieval task. |
Mitigating Language-Level Performance Disparity in mPLMs via Teacher Language Selection and Cross-lingual Self-Distillation (2024.naacl-long)
Copied to clipboard
| Challenge: | Large-scale multilingual pretrained language models (mPLMs) yield impressive performance on cross-language tasks, yet significant performance disparities exist across different languages within the same mPLm. |
| Approach: | They propose to leverage the learned knowledge from well-performing languages to guide under-performing ones within the same mPLM. |
| Outcome: | The proposed model shows that it can guide under-performing languages while minimizing language-level performance disparities across different mPLMs. |
Multi-Scale Distribution Deep Variational Autoencoder for Explanation Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for generating explanations for recommender systems produce generic explanations that fail to incorporate user and item specific details. |
| Approach: | They propose a multi-scale distribution deepvariational autoencoder with a prior network that eliminates noise while retaining meaningful signals in the input. |
| Outcome: | The proposed models can generate explanations with concrete input-specific contents. |
FocalOrder: Focal Preference Optimization for Reading Order Detection (2026.acl-long)
Copied to clipboard
Fuyuan Liu, Dianyu Yu, He Ren, Nayu Liu, Xiaomian Kang, Delai Qiu, Fa Zhang, Genpeng Zhen, Shengping Liu, Liang Jiaen, null Weihuang, Yining Wang, Junnan Zhu
| Challenge: | Existing methods for document comprehension rely on uniform supervision, resulting in a performance degradation in the intermediate sections. |
| Approach: | They propose a framework driven by Focal Preference Optimization to detect reading order in document layouts. |
| Outcome: | The proposed framework outperforms competing baselines and surpasses large-scale general VLMs. |
ACE-M3: Automatic Capability Evaluator for Multimodal Medical Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing metrics for multimodal large language models only focus on token overlap and may not align with human judgment. |
| Approach: | They propose an open-source model that assesses the question answering abilities of multimodal large language models. |
| Outcome: | Experiments show that the ACE-M3 model performs better than existing models and is more reliable than existing metrics. |
JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse (2025.findings-acl)
Copied to clipboard
| Challenge: | Visual Language Action models have shown promise in decision-making tasks, but have been neglected in previous work . |
| Approach: | They propose a new paradigm for visual language action models that enhances the foundation model prior to action-specific tuning by first post-training it on a curated set of visual and linguistic tasks using self-supervised learning. |
| Outcome: | The proposed model outperforms the best agent baseline on a diverse set of atomic tasks and surpasses imitation learning-based policies in Minecraft. |
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for training pre-trained language models have limited practicality due to latency requirements. |
| Approach: | They propose a method that uses a Mixture-of-Experts structure to increase model capacity and inference speed. |
| Outcome: | The proposed method outperforms existing distillation methods on natural language understanding and question answering tasks. |
BiSyn-GAT+: Bi-Syntax Aware Graph Attention Network for Aspect-based Sentiment Analysis (2022.findings-acl)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis is challenging because a sentence may contain multiple aspects or complicated relationships. |
| Approach: | They propose a bi-syntax aware Graph Attention Network to model the context of every aspect and sentiment relations across aspects for learning. |
| Outcome: | The proposed model outperforms the state-of-the-art methods on four benchmark datasets. |
Attending via both Fine-tuning and Compressing (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that attention mechanisms can improve models' interpretation, but they are not explicable. |
| Approach: | They propose a framework consisting of a learner and a compressor to purify attention scores . they propose to fine-tune and compress the attention mechanism to obtain a more faithful explanation . |
| Outcome: | The proposed framework improves performance and interpretability on eight benchmark datasets. |
OPERA: Operation-Pivoted Discrete Reasoning over Text (2022.naacl-main)
Copied to clipboard
Yongwei Zhou, Junwei Bao, Chaoqun Duan, Haipeng Sun, Jiahui Liang, Yifan Wang, Jing Zhao, Youzheng Wu, Xiaodong He, Tiejun Zhao
| Challenge: | Existing methods to predict logical forms ignore the utilization of symbolic operations and lack reasoning ability and interpretability. |
| Approach: | They propose an operation-pivoted discrete reasoning framework that uses symbolic operations as neural modules to facilitate reasoning ability and interpretability. |
| Outcome: | Extensive experiments on DROP and RACENum datasets show the reasoning ability of OPERA. |
ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing backdoor models are limited in coverage of attack, system integrity and backdoor alignment . ELBA-Bench provides over 1300 experiments encompassing 12 attack methods, 18 datasets, and 12 LLMs. |
| Approach: | They propose a framework that allows attackers to inject backdoor through parameter efficient fine-tuning or without fine-uning techniques. |
| Outcome: | ELBA-Bench provides over 1300 experiments encompassing 12 attack methods, 18 datasets, and 12 LLMs. |
A Survey of Inductive Reasoning for Large Language Models (2026.acl-long)
Copied to clipboard
Kedi Chen, Dezhao Ruan, Yuhao Dan, Yaoting Wang, Siyu Yan, Xuecheng Wu, Yinqi Zhang, Qin Chen, Jie Zhou, Liang He, Biqing Qi, Linyang Li, Qipeng Guo, Xiaoming Shi, Wei Zhang
| Challenge: | Inductive reasoning is an important task for large language models (LLMs). |
| Approach: | They propose a survey of inductive reasoning for large language models . they categorize methods into three main areas: post-training enhancement, test-time exploration, and data augmentation. |
| Outcome: | The proposed method improves inductive reasoning in large language models. |
An Information Minimization Based Contrastive Learning Model for Unsupervised Sentence Embeddings Learning (2022.coling-1)
Copied to clipboard
| Challenge: | Recent contrastive learning methods keep positive pairs similar and push negative pairs apart, which leads to redundant information in sentence embeddings. |
| Approach: | They propose a contrastive learning approach which maximizes mutual information and minimizes the information entropy between positive and negative instances. |
| Outcome: | The proposed model outperforms all previous competitors on supervised and unsupervised tasks. |
Source Critical Reinforcement Learning for Transferring Spoken Language Understanding to a New Language (C18-1)
Copied to clipboard
| Challenge: | a study aims to develop a language transferring system to avoid the trouble of acquiring and labeling a new big SLU corpus . general-purpose translators cannot handle the lot of semantic labels, not to mention cultural differences . a RL-based language transfer method can be used to adapt the adapted translator to a target language . |
| Approach: | They propose to use reinforcement learning to adapt a spoken language understanding model to a target language. |
| Outcome: | The proposed language transferring method improves domain classification accuracy by 22% compared with naive translation . the proposed language transfer method can be used on Chinese to English translators with more proper slot tags . |
Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | a framework to mitigate spurious optimization signals is proposed for test-time reinforcement learning (TTRL) Reinforcement learning with verifiable rewards (RLVR) is an effective paradigm for improving large language models on structured challenging reasoning tasks. |
| Approach: | They propose a framework to mitigate spurious optimization signals from label noise . they propose to use a frequency-based sampling strategy to exclude ambiguous samples . |
| Outcome: | The proposed framework outperforms existing TTRL baselines on three large language models across multiple mathematical reasoning benchmarks. |
Curriculum Learning based Hierarchical Scoring and Analysis Framework for Question Answering Task Evaluation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods rely on rule-based matching with shallow semantic understanding or adopt LLM-as-a-Judge approaches that incur high cost and latency while offering limited error interpretability. |
| Approach: | They propose a curriculum learning based hierarchical framework for QA task evaluation that supports quick scoring and fine-grained error analysis. |
| Outcome: | The proposed framework outperforms baseline methods on quick scoring and error analysis tasks while being 25 faster. |
What Makes an Ideal Quote? Recommending “Unexpected yet Rational” Quotations via Novelty (2026.acl-long)
Copied to clipboard
| Challenge: | Prior systems focus on topical relevance and overlook what makes quotes memorable. |
| Approach: | They propose a system that maps quotations and contexts into deep-meaning labels for label-enhanced retrieval. |
| Outcome: | The proposed system can recommend quotations that are contextually novel while semantically coherent. |
SymbolicThought: Integrating Language Models and Symbolic Reasoning for Consistent and Interpretable Human Relationship Understanding (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing methods for constructing character relationships from plain text are time-consuming and low in coverage. |
| Approach: | They propose a human-in-the-loop framework that combines LLM-based extraction with symbolic reasoning. |
| Outcome: | The proposed framework improves annotation accuracy and consistency while significantly reducing time cost. |
SAS: Dialogue State Tracking via Slot Attention and Slot Information Sharing (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models with excessive information are inefficient and costly . |
| Approach: | They propose to integrate a Dialogue State Tracker with Slot Attention and Slot Information Sharing to reduce redundant information’s interference and improve long dialogue context tracking. |
| Outcome: | The proposed model significantly outperforms existing models on the MultiWOZ dataset. |
Light Up the Shadows: Enhance Long-Tailed Entity Grounding with Concept-Guided Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Multi-Modal Knowledge Graphs (MMKGs) are knowledge graphs that integrate and align information from diverse modalities (e.g., text and images). |
| Approach: | They propose a framework that integrates image-text pairs of long-tailed entities and a concept guidance module that offers explainability and enables human verification. |
| Outcome: | The proposed framework improves the accuracy of recognizing long-tailed image-text pairs compared to baselines and also offers flexibility and explainability. |
ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research (2025.acl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promising results in various domains, but their practical application in industry-relevant operations research presents significant challenges and opportunities. |
| Approach: | They propose a cognitive-inspired framework that enhances optimization through counterfactual reasoning . they use a workflow that transforms requirements into mathematical models and executable solver code . |
| Outcome: | Experiments show that ORMind outperforms existing methods in the NL4Opt dataset and ComplexOR dataset. |
Is “hot pizza” Positive or Negative? Mining Target-aware Sentiment Lexicons (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing sentiment lexicons assume words’ sentiments are invariant within a domain, but this assumption is weak for fine-granularity analyses of text sentiments. |
| Approach: | They propose a "perturb-and-see" method to extract commonsense sentiments from large-scale datasets by binding a word's sentiment to its collocation words instead of domain labels. |
| Outcome: | The proposed framework is able to achieve highly competitive performances on the unsupervised opinion relation extraction task. |
Quantification of Large Language Model Distillation (2025.acl-long)
Copied to clipboard
Sunbowen Lee, Junting Zhou, Chang Ao, Kaige Li, Xeron Du, Sirui He, Haihong Wu, Tianci Liu, Jiaheng Liu, Hamid Alinejad-Rokny, Min Yang, Yitao Liang, Zhoufutu Wen, Shiwen Ni
| Challenge: | Existing studies have revealed the robustness degra-dation caused by data distillation. |
| Approach: | They propose a framework to evaluate and quantify model distillation . they aim to identify identity cognition contradictions and analyse multi-granularity response similarities across models to measure the extent of homogenization. |
| Outcome: | The proposed framework addresses two key aspects: (1) Identifying identity cognition contradictions to assess discrepancies in how models perceive and represent identity-related information; (2) Analyzing multi-granularity response similarities across models to measure the extent of homogenization. |
Efficiently Editing Mixture-of-Experts Models with Compressed Experts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Mixture-of-Experts models allow for efficient scaling of large language models . fewer experts reduce computational costs, while more experts improve performance . |
| Approach: | They propose to activate only a subset of experts during training and inference . they propose compressed experts that preserve the most important experts . |
| Outcome: | The proposed approach preserves the most important experts while replacing other auxiliary activated experts with compressed experts. |
Order Matters: Investigate the Position Bias in Multi-constraint Instruction Following (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing large language models struggle to follow multi-constraint instructions in real-world applications. |
| Approach: | They propose to quantify the difficulty distribution of constraints by a novel Difficulty Distribution Index (CDDI) they find that LLMs are more performant when presented with constraints in a “hard-to-easy” order. |
| Outcome: | The proposed model is more performant when presented with constraints in a “hard-to-easy” order, compared with existing models with different architectures and sizes of parameters. |
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate (2024.emnlp-main)
Copied to clipboard
Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, Zhaopeng Tu
| Challenge: | Modern large language models (LLMs) have shown remarkable performance on general language tasks but struggle on complex reasoning tasks. |
| Approach: | They propose a multi-agent debate framework that encourages divergent thinking in LLMs . they propose to break debate and use a judge to obtain a final solution . |
| Outcome: | The proposed framework encourages divergent thinking in large language models . it is able to generate novel thoughts even if initial stance is incorrect . |
APEX: Learning Adaptive Priorities for Multi-Objective Alignment in Vision-Language Generation (2026.findings-acl)
Copied to clipboard
Dongliang Chen, Xinlin Zhuang, Junjie Xu, Luojian Xie, Zehui Wang, Jiaxi Zhuang, Haolin Yang, Liang Dou, Xiao He, Xingjiao Wu, Ying Qian
| Challenge: | APEX optimizes for text-to-image generation by combining learning potential, conflict penalty, and progress need. |
| Approach: | They propose an algorithm that stabilizes heterogeneous rewards and dynamically schedules objectives . they propose a method that achieves better Pareto trade-offs across four heterogenous objectives based on P3 Adaptive Priorities . |
| Outcome: | The proposed algorithm achieves better pareto trade-offs across four heterogeneous objectives while maintaining competitive OCR accuracy. |
Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer (N18-1)
Copied to clipboard
| Challenge: | Previous work using adversarial methods has struggled to produce high-quality outputs. |
| Approach: | They propose a method that transforms a sentence to alter a specific attribute while preserving its attribute-independent content. |
| Outcome: | The proposed method generates grammatical and appropriate responses on 22% more inputs than the best previous system, averaged over three attribute transfer datasets. |
CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work has resorted to sharing weights among models, but results are not affordable for real-world deployment. |
| Approach: | They propose a consistency-regularized ensemble learning approach based on perturbed models to retain ensemble benefits while maintaining a low memory cost. |
| Outcome: | The proposed approach outperforms the standard ensemble of 8 BERT-base models on the GLUE benchmark by 0.7 with a significantly smaller model size. |
Decoupling Strategy and Generation in Negotiation Dialogues (D18-1)
Copied to clipboard
| Challenge: | Recent work on negotiation trains neural models, but their end-to-end nature makes it hard to control their strategy. |
| Approach: | They propose a modular approach that decouples strategy and generation by coarse dialogue acts . they test their approach on a recently proposed DEALORNODEAL game . |
| Outcome: | The proposed approach can decouple strategy and generation without degeneracy. |
LLM-KT: Enhancing Large Language Models with Knowledge Tracing via Multi-Level Plug-and-Play Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn behavioral sequences fail to capture complex behavioral patterns due to a lack of deep reasoning capabilities and world knowledge. |
| Approach: | They propose a framework that integrates the reasoning power of Large Language Models with the sequential modeling strengths of traditional KT methods via multi-level plug-and-play alignment. |
| Outcome: | Extensive experiments on four standard datasets show that the proposed framework outperforms existing methods on state-of-the-art questions. |
Learning Neural Templates for Recommender Dialogue System (2021.emnlp-main)
Copied to clipboard
Zujie Liang, Huang Hu, Can Xu, Jian Miao, Yingying He, Yining Chen, Xiubo Geng, Fan Liang, Daxin Jiang
| Challenge: | Recent advances in neural models have shown promising progress on this task, but key challenges remain . |
| Approach: | They propose a framework that can decouple dialogue generation from item recommendation . they use a response template generator and item selector to generate a responses template . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on the benchmark ReDial. |
Label-Enhanced Hierarchical Contextualized Representation for Sequential Metaphor Identification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to identify metaphors ignore extra information from data, such as contextual information and broader discourse information. |
| Approach: | They propose a model augmented with hierarchical contextualized representation to extract more information from both sentence-level and discourse-level. |
| Outcome: | The proposed model outperforms state-of-the-art methods on two tasks using a VUA dataset. |
Route Sparse Autoencoder to Interpret Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sparse autoencoders (SAEs) extract interpretable and monosemantic features in large language models . prior work focused on feature extraction from a single layer, failing to capture activations that span multiple layers. |
| Approach: | They propose a framework that integrates a routing mechanism with a shared SAE to efficiently extract features from multiple layers. |
| Outcome: | The proposed framework extracts features from multiple layers while incurring minimal parameter overhead while achieving high interpretability and flexibility. |
MGCL: Multi-Granularity Clue Learning for Emotion-Cause Pair Extraction via Cross-Grained Knowledge Distillation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional methods often rely on coarse-grained clause-level annotations, which overlook valuable fine-grain clues. |
| Approach: | They propose a method that captures fine-grained clues from a weakly-supervised perspective efficiently by using a teacher model to give sub-clause clues without needing fine-grain annotations. |
| Outcome: | The proposed method achieves state-of-the-art performance while offering improved interpretability. |
Multi-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on multimodal sarcasm detection using textual and visual information have been limited to text-only approaches. |
| Approach: | They propose to construct a cross-modal graph for each multi-modal instance to explicitly draw the ironic relations between textual and visual modalities. |
| Outcome: | The proposed model achieves state-of-the-art in multi-modal sarcasm detection. |
CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios (2024.emnlp-main)
Copied to clipboard
| Challenge: | Chinese medical large language models (LLMs) are underperforming on this benchmark, especially where medical reasoning and factual consistency are vital. |
| Approach: | They propose a benchmark with 14 expert-guided clinical scenarios to assess the medical ability of large language models across 7 pivot dimensions. |
| Outcome: | The proposed benchmark has been validated in several ways. |