Papers by Xue Zhang
Collaborative Performance Prediction for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are one of the most important AI research powered by largescale parameters, high computational resources, and massive training data. |
| Approach: | They propose a framework that leverages historical performance of large language models and other design factors to improve prediction accuracy. |
| Outcome: | The proposed framework surpasses scaling laws in predicting performance of large language models . it also facilitates a detailed analysis of factor importance, an area previously overlooked . |
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
OceanGPT: A Large Language Model for Ocean Science Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have transformed the paradigm in ocean science. |
| Approach: | They propose a framework to automatically obtain large volume of ocean domain instruction data, which generates instructions based on multi-agent collaboration. |
| Outcome: | The proposed framework shows a higher level of knowledge expertise for ocean science tasks and gains preliminary embodied intelligence capabilities in ocean technology. |
Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities (2025.findings-acl)
Copied to clipboard
Wenyue Hua, Kaijie Zhu, Lingyao Li, Lizhou Fan, Mingyu Jin, Shuhang Lin, Haochen Xue, Zelong Li, Jindong Wang, Yongfeng Zhang
| Challenge: | Using large language models, large language model models can be used to evaluate reasoning abilities in context-rich scenarios. |
| Approach: | They construct datasets for both propositional logic and abductive logic reasoning with four difficulty levels across 12 distinct domains based on Wikipedia categorization and those with purely abstract variables. |
| Outcome: | The proposed model can be used to benchmark LLMs in real-world scenarios, but not in context-rich scenarios. |
SGA-MCTS: Decoupling Planning from Execution via Training-Free Atomic Experience Retrieval (2026.findings-acl)
Copied to clipboard
| Challenge: | a new framework casts LLM planning as non-parametric retrieval, but high latency of inference-time search and supervised fine-tuning are limitations. |
| Approach: | They propose a framework that casts LLM planning as non-parametric retrieval . they leverage Monte Carlo Tree Search to explore the solution space . |
| Outcome: | Empirical results show that SGA-MCTS can match the performance of SOTA systems without task-specific fine-tuning. |
CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending (2024.acl-long)
Copied to clipboard
| Challenge: | Existing models that use self-attention and position embedding have anomalous behavior that hinder long context window extrapolation. |
| Approach: | They propose a collinear constraint between Q and K to integrate RoPE and self-attention. |
| Outcome: | The proposed model integrates self-attention and position embedding into LLMs without fine-tuning. |
How to Train a Real-World Silicon Concierge? Internalizing Complex Business Workflow to Only OneModel (2026.acl-industry)
Copied to clipboard
| Challenge: | Traditional industrial agents rely on modular workflows that fracture into a labyrinth of ad-hoc patches, leading to cascading errors and high latency. |
| Approach: | They propose a paradigm shift from external workflows to internalized knowledge representation that consolidates complex business logic and SOPs directly into the model’s parameters. |
| Outcome: | The proposed model breaks the impossible triangle of latency, accuracy, and complexity. |
RubricBench: Aligning Model-Generated Rubrics with Human Standards (2026.acl-long)
Copied to clipboard
Junyi Zhou, Qiyuan Zhang, Yufei Wang, Fuyuan Lyu, Yidong Ming, Can Xu, Qingfeng Sun, Kai Zheng, Peng Kang, Xue Liu, Chen Ma
| Challenge: | Existing benchmarks lack discriminative complexity and ground-truth rubric annotations required for rigorous evaluation. |
| Approach: | They propose a curated benchmark with 1,147 pairwise comparisons to assess the reliability of rubric-based evaluation. |
| Outcome: | The proposed benchmarks show that they support diverse domains, exhibit discriminative ability, provide high-quality annotations, and include human-authored rubrics. |
Improving Continual Few-shot Relation Extraction through Relational Knowledge Distillation and Prototype Augmentation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to Continual Relation Extraction (CRE) are limited in handling the rapid emergence of new relations in real-world scenarios. |
| Approach: | They propose a framework that integrates prototype-based data augmentation and relational knowledge distillation to solve the problem of Continual Few-shot Relation Extraction (CFRE). |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on the FewRel and TACRED datasets. |
A Semi-supervised Scalable Unified Framework for E-commerce Query Classification (2025.acl-industry)
Copied to clipboard
Chunyuan Yuan, Chong Zhang, Zhen Fang, Ming Pang, Xue Jiang, Changping Peng, Zhangang Lin, Ching Law
| Challenge: | Existing query classification methods rely on posterior click behavior to construct training samples, resulting in insufficient prior information for modeling. |
| Approach: | They propose a semi-supervised scaleable unified framework that integrates enhanced modules to unify query classification tasks. |
| Outcome: | The proposed framework outperforms the state-of-the-art models in offline and online A/B experiments. |
CR-UTP: Certified Robustness against Universal Text Perturbations on Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing certified robustness methods for certifying input-specific text perturbations have shown promise in certifyling UTPs, but masking only adversarial words can eliminate the attack. |
| Approach: | They propose a method to certify a language model’s robustness against UTPs by using random smoothing. |
| Outcome: | The proposed method achieves high certified accuracy under extensive masking and achieves state-of-the-art results in multiple settings. |
KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | General pre-trained language models (PLMs) leverage relation triples from knowledge graphs (KGs) and integrate external data sources into language models via self-supervised learning. |
| Approach: | They propose to learn Knowledge-Enhanced language representations with Hierarchical Reinforcement Learning (KEHRL) to detect positions for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge. |
| Outcome: | The proposed model can detect essential positions in texts for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge. |
SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defense methods rely on fine-tuning or inefficient post-hoc interventions, limiting their ability to address novel attacks. |
| Approach: | They propose a decoding-level defense mechanism that employs a lightweight discriminator to iteratively steer the decoding process toward safety. |
| Outcome: | The proposed method improves safety performance by up to 33.40% without fine-tuning on multiple MLLMs. |
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)
Copied to clipboard
Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong
| Challenge: | Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored. |
| Approach: | They propose a survey structured around the pipeline to identify and improve MI models. |
| Outcome: | The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency. |
MedOdyssey: A Medical Domain Benchmark for Long Context Evaluation Up to 200K Tokens (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing benchmarks in the generic domain have evaluated long-context capabilities for LLMs. |
| Approach: | They propose a medical long-context benchmark with seven length levels ranging from 4K to 200K tokens. |
| Outcome: | The proposed benchmarks have seven length levels ranging from 4K to 200K tokens. |
MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation (2025.acl-long)
Copied to clipboard
Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, Chengzhi Liu, Zhongxing Xu, Chong Zhang, Chun-Mei Feng, Yutong Xie, Imran Razzak, Zongyuan Ge, Jionglong Su, Junjun He, Yu Qiao
| Challenge: | Existing multimodal large language models lack the ability to memorize, recall, and reason in sustained interactions. |
| Approach: | They propose a multimodal real-world conversation benchmark for evaluating open-ended abilities of multimodal large language models. |
| Outcome: | The proposed benchmarks show that the models perform better in open-ended conversations. |
MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models (2025.findings-acl)
Copied to clipboard
Boyang Xue, Hongru Wang, Rui Wang, Sheng Wang, Zezhong Wang, Yiming Du, Bin Liang, Wenxuan Zhang, Kam-Fai Wong
| Challenge: | Existing studies on LLM confidence estimations in languages other than English have been limited to English. |
| Approach: | They propose to use question-related language to prompt LLMs to assess their confidence in large language models. |
| Outcome: | The proposed model improves on question-related language prompts for LS tasks, while English exhibits notable linguistic dominance in confidence estimations. |
Enhancing Emotion Recognition in Conversation via Multi-view Feature Alignment and Memorization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Emotion recognition in conversation (ERC) is an advanced capability of conversational AI systems. |
| Approach: | They propose a semi-parametric paradigm for Emotion Recognition in conversation that uses supervised contrastive learning to align semantic-view and context-view features. |
| Outcome: | The proposed model achieves state-of-the-art on four widely used benchmarks. |
FAITH: Factuality Alignment through Integrating Trustworthiness and Honestness (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to correct factually inaccurate outputs are lacking the semantic richness needed to properly understand its internal states of trustworthiness and honesty. |
| Approach: | They propose a framework for factuality alignment that integrates natural-language uncertainty signals with external knowledge and computes confidence scores and semantic entropy from LLM outputs. |
| Outcome: | Extensive experiments on four knowledge-intensive benchmarks show that FAITH improves the factual accuracy and truthfulness of Large Language Models (LLMs). |
On the Hallucination in Simultaneous Machine Translation (2024.acl-short)
Copied to clipboard
| Challenge: | Currently, there are no studies which systematically analyze hallucination in SiMT. |
| Approach: | They conduct a comprehensive analysis of hallucination in simultaneous machine translation (SiMT) they find that halluciation is extremely severe, especially as latency increases . |
| Outcome: | The results show that it is possible to alleviate hallucination by decreasing the over usage of target-side information for SiMT. |
ReFL: Reflective Feedback Learning for Hallucination Detection of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for detecting hallucinations depend on external knowledge sources, incurring high computational costs and limiting real-time applicability, or extract the model’s internal states, leading to poor generalization. |
| Approach: | They propose a hallucination detection framework that leverages corrective in-context learning to guide LLMs to recognize their own prediction errors and adjust internal representations, critically without updating model weights. |
| Outcome: | The proposed framework outperforms existing methods on two benchmark datasets and achieves state-of-the-art performance. |
ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback (2026.findings-acl)
Copied to clipboard
| Challenge: | Unlike chatbots, autonomous agents act directly on external environments, making tool invocation safety critical for reliable deployment. |
| Approach: | They develop a benchmark for step-level tool invocation safety detection in LLM agents and a guardrail model that proactively detects unsafe tool invoking actions before execution using multi-task reinforcement learning. |
| Outcome: | The proposed model reduces harmful tool invocations of ReAct-style agents by 65% on average and improves benign task completion by 10% under prompt injection attacks. |
TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for incorporating external knowledge into language models do not prioritize learning embeddings for entity-related tokens. |
| Approach: | They propose a framework for incorporating external knowledge into pre-training models that utilize entity-related tokens. |
| Outcome: | The proposed framework reduces pre-training time by 50% and outperforms other KEPLMs in knowledge probing tasks and multiple knowledge-aware language understanding tasks. |
LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety (2026.acl-long)
Copied to clipboard
Junxiao Yang, Haoran Liu, Jinzhe Tu, Jiale Cheng, Zhexin Zhang, Shiyao Cui, Jiaqi Weng, Jialing Tao, Hui Xue, Hongning Wang, Han Qiu, Minlie Huang
| Challenge: | Large language models (LLMs) have demonstrated better safety performance in high-resource languages than in low-resourced languages. |
| Approach: | They propose language-agnostic semantic alignment (LASA) which anchors safety alignment directly in semantic bottlenecks. |
| Outcome: | The proposed approach significantly improves safety across all languages: average attack success rate drops from 24.7% to 2.8% on LLaMA-3.1-8B-Instruct and remains within 3–4% across Qwen2.5 and Qwend3 Instruct models (7B–32B). |
From What to Why: Improving Relation Extraction with Rationale Graph (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing neural relation extraction models are limited by entity type and textual context. |
| Approach: | They propose a novel RAtionale Graph to organize co-occurrence constraints among entity types, triggers and relations in a holistic graph view. |
| Outcome: | The proposed method outperforms baselines significantly and achieves state-of-the-art performance on document-level and sentence-level RE benchmarks. |
Rationales for Answers to Simple Math Word Problems Confuse Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that large language models have advanced mathematical problem-solving abilities in grade school math word problems. |
| Approach: | They propose to combine fine-tuning and prompt-based methods to improve performance . they propose to use a hybrid algorithm to fine- tune LLMs on specific tasks . |
| Outcome: | The proposed methods improve performance on the proposed reasoning process evaluation benchmarks. |
ProgressLM: Towards Progress Reasoning in Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models for task progress estimation lack long-horizon and dynamic reasoning . estimating how much of a task has been completed requires long-term reasoning based on partial information. |
| Approach: | They propose a benchmark for evaluating progress reasoning from a single observation . they instantiate a two-stage paradigm that combines episodic retrieval with mental simulation . |
| Outcome: | The proposed benchmark improves on 14 VLMs on a small scale and shows common failure patterns. |
M-ABSA: A Multilingual Dataset for Aspect-Based Sentiment Analysis (2025.emnlp-main)
Copied to clipboard
ChengYan Wu, Bolei Ma, Yihong Liu, Zheyu Zhang, Ningyuan Deng, Yanshu Li, Baolan Chen, Yi Zhang, Yun Xue, Barbara Plank
| Challenge: | Existing studies focus on English-centric aspects of sentiment analysis, limiting scope for multilingual evaluation and research. |
| Approach: | They propose to use a multilingual dataset to analyze aspects with associated sentiment elements in text. |
| Outcome: | The proposed dataset is the most extensive multilingual parallel dataset for ABSA to date. |
Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation (2023.emnlp-main)
Copied to clipboard
Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Jie Fu, Jian Lan, Mei Li, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv
| Challenge: | Existing unsupervised paraphrase generation methods require large-scale, manually annotated paraphrase datasets, which are labor-intensive to build. |
| Approach: | They propose a self-supervised pseudo-data construction method that generates diverse pseudo-paraphrases in distinct surface structures for a given sentence. |
| Outcome: | The proposed method generates diverse pseudo-paraphrases in distinct surface structures for a given sentence. |
SHARP: Self-adaptive Harmful Category-aware Prompt Generation for Black-box Jailbreaking (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for jailbreak ignore the semantic differences between categories of harmful questions, leading to inconsistent success rates and reduced overall attack effectiveness. |
| Approach: | They propose a category-aware jailbreak framework that incorporates the semantic category of harmful questions into prompt generation. |
| Outcome: | The proposed framework improves attack success rates and category alignment and achieves better cross-category robustness compared to the state-of-the-art (SOTA) baselines. |
SpanMlt: A Span-based Multi-Task Learning Framework for Pair-wise Aspect and Opinion Terms Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Aspect terms and opinion terms are key problems of fine-grained aspect-based sentiment analysis. |
| Approach: | They propose a method to extract aspect and opinion terms as pairs from a sentence . they use shared spans to extract the terms under supervision of span boundaries . |
| Outcome: | The proposed method outperforms state-of-the-art methods on both aspects and opinion terms extraction tasks. |
BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Xi et al., 2023) show that LLM-based intelligent agents are vulnerable to backdoor attacks. |
| Approach: | They propose to build LLM-based intelligent agents using trained LLMs and fine-tune them on data for the agent task. |
| Outcome: | The proposed methods are robust even after fine-tuning on trusted data. |
Enhancing Advanced Visual Reasoning Ability of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability. |
| Approach: | They propose a novel multi-modal in-context learning methodology to enhance LLMs’ contextual understanding and reasoning. |
| Outcome: | The proposed model achieves SOTA performance among all visual reasoning tasks and achieves a 'higher level of accuracy' than previous models. |
CM-Align: Consistency-based Multilingual Alignment for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current large language models (LLMs) show a significant performance gap in alignment between English and other languages. |
| Approach: | They propose a consistency-based method to construct high-quality multilingual preference data for improving multilingual alignment. |
| Outcome: | The proposed method is based on three LLMs and three common tasks and shows that it performs better than current methods. |
GAM: Hierarchical Graph-based Agentic Memory for LLM Agents (2026.acl-long)
Copied to clipboard
Zhaofen Wu, Hanrong Zhang, Fulin Lin, Wujiang Xu, Xinran Xu, Yankai Chen, Henry Peng Zou, Shaowen Chen, Weizhi Zhang, Xue Liu, Philip S. Yu, Hongwei Wang
| Challenge: | Current unified stream-based memory systems facilitate context updates but remain vulnerable to interference from transient noise. |
| Approach: | They propose a hierarchical Graph-based Agentic Memory framework that explicitly decouples memory encoding from consolidation to resolve conflict between rapid context perception and stable knowledge retention. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on LoCoMo and LongDialQA. |
Reinforcement Learning on Pre-Training Data (2026.acl-long)
Copied to clipboard
Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang
| Challenge: | Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings. |
| Approach: | They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data. |
| Outcome: | Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base. |
MultiPL-MoE: Multi-Programming-Lingual Extension of Large Language Models through Hybrid Mixture-of-Experts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | MultiPL is a special case of multiple natural languages and requires limited computational resources to generate multilingual code. |
| Approach: | They propose to extend LLMs by combining two paired experts to optimize expert selection at token and segment levels. |
| Outcome: | The proposed extension improves the performance of the base LLMs while retaining the most popular ones using limited computational resources. |
Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts (2026.acl-long)
Copied to clipboard
Haolei Xu, Haiwen Hong, Hongxing Li, Rui Zhou, Yang Zhang, Longtao Huang, Hui Xue, Yongliang Shen, Weiming Lu, Yueting Zhuang
| Challenge: | Existing multimodal Mixture-of-Experts models accurately perceive image content yet fail in subsequent reasoning . Seeing but not thinking phenomenon is a puzzling phenomenon . |
| Approach: | They propose a routing-guided intervention method that enhances domain expert activation. |
| Outcome: | The proposed method achieves consistent improvements on visual reasoning tasks. |
WebAggregator: Enhancing Compositional Reasoning Capabilities of Deep Research Agent Foundation Models (2026.acl-long)
Copied to clipboard
Rui Wang, Ce Zhang, Jun-Yu Ma, Jianshu Zhang, Hongru Wang, Yi Chen, Boyang Xue, Tianqing Fang, Zhisong Zhang, Hongming Zhang, Haitao Mi, Dong Yu, Kam-Fai Wong
| Challenge: | Existing agentic systems are retrieval-heavy but reasoning-light . current systems lack compositional reasoning, a key component of deep research . |
| Approach: | They propose a data synthesis pipeline WebAggregator to shift agentic paradigm . they use Proactive Explorer to collect interconnected knowledge and Compositional Logic Proposer to weave knowledge into complex questions . |
| Outcome: | The proposed pipeline surpasses GPT-4.1 and matches Claude-3.7-Sonnet on GAIA, WebWalkerQA, and XBench. |
START: Self-taught Reasoner with Tools (2025.emnlp-main)
Copied to clipboard
Chengpeng Li, Mingfeng Xue, Zhenru Zhang, Jiaxi Yang, Beichen Zhang, Bowen Yu, Binyuan Hui, Junyang Lin, Xiang Wang, Dayiheng Liu
| Challenge: | Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in complex reasoning through long chain-of-thought, yet they struggle with precise computations and algorithmic operations. |
| Approach: | They propose a training-free approach that activates LRMs’ latent tool-use capabilities through artificial hints and a framework that enables models to learn effective tool utilization through diverse hint patterns and rejection-based data synthesis. |
| Outcome: | Experiments show that START significantly improves state-of-the-art LRMs across challenging benchmarks, including competition-level mathematics (AMC23: 95.0%, AIME24: 75.6%) and graduate-level science questions (GPQA: 64.6%). |
Coarse-to-Fine Pre-training for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a task of discovering information entities and identifying their corresponding categories. |
| Approach: | They propose a NER-specific framework to inject coarse-to-fine named entity knowledge into pre-trained models by using a remote supervision strategy. |
| Outcome: | The proposed framework achieves significant improvements against several pre-trained base-lines, demonstrating its effectiveness in label-few and low-resource scenarios. |
Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions (D19-1)
Copied to clipboard
Rui Zhang, Tao Yu, Heyang Er, Sungrok Shim, Eric Xue, Xi Victoria Lin, Tianze Shi, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | Generating SQL queries from user utterances is an important task to help end users acquire information from databases. |
| Approach: | They propose a context-dependent text-to-SQL generation task that edits previous queries . they use an utterance-table encoder and a table-aware decoder to incorporate context . |
| Outcome: | The proposed model is flexible to change individual tokens and robust to error propagation. |
Learning to Prune Dependency Trees with Rethinking for Neural Relation Extraction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to remove noise from dependency trees are not optimal due to complexity and variability of natural language. |
| Approach: | They propose a dynamically pruned Graph Convolutional Network (DP-GCN) that prunes the dependency tree with rethinking in an end-to-end scheme. |
| Outcome: | The proposed model achieves impressive results compared to strong competitors. |
GCPG: A General Framework for Controllable Paraphrase Generation (2022.findings-acl)
Copied to clipboard
Kexin Yang, Dayiheng Liu, Wenqiang Lei, Baosong Yang, Haibo Zhang, Xue Zhao, Wenqing Yao, Boxing Chen
| Challenge: | Existing studies highlight a special condition under two indispensable aspects of controllable paraphrase generation (CPG) individually, lacking a unified circumstance to explore and analyze their effectiveness. |
| Approach: | They propose a general controllable paraphrase generation framework that integrates lexical and syntactical conditions into a text sequence and uniformly processes them in an encoder-decoder paradigm. |
| Outcome: | The proposed framework can combine lexical and syntactical conditions and improve paraphrase generation. |
Neural Ranking Models for Temporal Dependency Structure Parsing (D18-1)
Copied to clipboard
| Challenge: | a new neural temporal dependency parser is being developed for news reports and narrative stories . a similar system is used for other NLP applications such as timeline construction . |
| Approach: | They build a neural temporal dependency parser that parses time expressions and events in a text . their results shed light on the nature of temporal relation structures in different domains . |
| Outcome: | The proposed model beats baselines on news reports and narrative stories on two data domains. |
Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to generate adversarial examples for NMT use the meaning-preserving restriction. |
| Approach: | They propose a new definition for adversarial examples based on the Doubly Round-Trip Translation (DRTT) they introduce masked language models to construct bilingual adversarials based upon DRTT . |
| Outcome: | The proposed approach significantly improves the robustness of the NMT model on clean and noisy test sets. |
Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation (2026.findings-acl)
Copied to clipboard
Rui Qi, Fengran Mo, Yufeng Chen, Xue Zhang, Shuo Wang, Hongliang Li, Xu Jinan, Meng Jiang, Jian-Yun Nie, Kaiyu Huang
| Challenge: | Existing approaches to multilingual retrieval-augmented generation (MRAG) use a single-turn retrieval and subsequent optimization to acquire and integrate beneficial external knowledge from multilingual collections. |
| Approach: | They propose a multilingual search-augmented reinforcement learning framework that integrates a language-coupled Group Relative Policy Optimization into the policy and reward models. |
| Outcome: | The proposed framework achieves competitive performance and is appropriate for various practical scenarios such as constrained training data and retrieval over collections encompassing a large number of languages. |
DPN-LE: Dual Personality Neuron Localization and Editing for Large Language Models (2026.findings-acl)
Copied to clipboard
Lifan Zheng, Xue Yang, Jiawei Chen, Chenyan WU, Jingyuan Zhang, Fanheng Kong, Xinyi Zeng, Xiang Chen, Yu Tian
| Challenge: | Current methods for editing personality traits in large language models can change personalities but reduce performance. |
| Approach: | They propose a novel paradigm for personality editing that locates and edits LLM neurons and enables competitive personality control at inference time. |
| Outcome: | Experiments on LLaMA-3-8B-Instruct and Qwen2.5-7B-instruct show that the proposed approach can improve performance and improve performance. |
MeMoTune: A Measure and Moment-Driven Fine-Tuning Framework for Quantized Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods combine quantization with parameter-efficient fine-tuning but fail to meet practical performance requirements. |
| Approach: | They propose a measure and moment approach to optimize objective function for superior fine-tuning results by scaling the update process through a gradient. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on tasks like text generation, summarization, and understanding. |
Multilingual Knowledge Editing with Language-Agnostic Factual Neurons (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to update factual knowledge overlook connections of same knowledge between different languages, resulting in knowledge conflicts and limited edit performance. |
| Approach: | They propose a method to edit multilingual knowledge simultaneously that avoids knowledge conflicts and improves edit performance. |
| Outcome: | The proposed method avoids knowledge conflicts and improves edit performance on bi-ZsRE and MzsRE benchmarks. |
Don’t Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction (2026.acl-long)
Copied to clipboard
Yuzhe Zhang, Xianwei Xue, Xingyong Wu, Mengke Chen, Chen Liu, Xinran He, Run Shao, Feiran Liu, Huanmin Xu, Qiutong Pan, Haiwei Wang
| Challenge: | Existing GUI agents assume deterministic environment responses, generating actions without verifying whether previous operations succeeded. |
| Approach: | They propose a GUI agent that explicitly models action outcomes and recovery under noisy environments. |
| Outcome: | The proposed agent reduces failure loops and improves recovery success in noisy environments while maintaining competitive standard task performance. |
SparkRA: A Retrieval-Augmented Knowledge Service System Based on Spark Large Language Model (2024.emnlp-demo)
Copied to clipboard
Dayong Wu, Jiaqi Li, Baoxin Wang, Honghong Zhao, Siyuan Xue, Yanjie Yang, Zhijun Chang, Rui Zhang, Li Qian, Bo Wang, Shijin Wang, Zhixiong Zhang, Guoping Hu
| Challenge: | Large language models (LLMs) have shown remarkable achievements across various language tasks. |
| Approach: | They propose a scientific literature LLM and a knowledge service system based on it . they collect scientific literature and then pre-train it using autoregressive training . |
| Outcome: | The proposed system provides literature investigation, paper reading, and academic writing functions. |
A Quality-based Syntactic Template Retriever for Syntactically-Controlled Paraphrase Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing syntactically-controlled paraphrase generation models perform well with human-annotated or well-chosen syntaktic templates. |
| Approach: | They propose a quality-based Syntactic Template Retriever to retrieve templates based on the quality of the to-be-generated paraphrases. |
| Outcome: | The proposed algorithm can generate high-quality paraphrases without sacrificing quality. |
Section-Aware Commonsense Knowledge-Grounded Dialogue Generation with Pre-trained Language Model (2022.coling-1)
Copied to clipboard
| Challenge: | Pre-trained language models can be expected to deepen the fusing of dialogue context and knowledge because of their superior ability of semantic understanding. |
| Approach: | They propose a two-stage framework to integrate a linearized knowledge into plan text using a ranking network PriorRanking to estimate the relevance of a retrieved knowledge fact. |
| Outcome: | The proposed framework improves the performance of pre-trained language models by using section-aware strategies to encode the linearized knowledge. |
DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing systems for rap generation focus on rhyming lyrics but ignore rhythmic beats . rap lyrics need to be semantically meaningful and fashionable to convey interesting stories . |
| Approach: | They develop a Transformer-based rap generation system that can model both rhymes and rhythms. |
| Outcome: | The proposed system generates high-quality raps with rhymes and rhythms . it is based on a Transformer-based language model . |
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)
Copied to clipboard
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
A Unified Joint Approach with Topological Context Learning and Rule Augmentation for Knowledge Graph Completion (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge graph completion methods perform simple linear update on relation representation, and only local neighborhood information is aggregated, making it difficult to capture logic semantic between relations and global topological context information. |
| Approach: | They propose a joint approach with Topological Context learning and Rule Augmentation (TCRA) it uses a topological context learning mechanism and a relation rule context learning system . |
| Outcome: | The proposed approach performs better on three benchmark datasets and is widely used in knowledgeintensive applications. |
Towards Anytime Fine-tuning: Continually Pre-trained Language Models with Hypernetwork Prompts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Continual pre-training has been used for a multitude of domains and tasks . a continually pre-trained model can show a non-decreasing performance on unseen domains . |
| Approach: | They propose a method that generates domain-specific prompts by agreement and disagreement losses. |
| Outcome: | The proposed method achieves improvements of 3.57% and 3.4% on two real-world datasets. |
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosure (2026.acl-demo)
Copied to clipboard
Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhou, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Nenkov Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, YU Chen, Xue Liu, Preslav Nakov, Zhuohan Xie
| Challenge: | FinReporting is an agentic workflow for localized cross-jurisdiction financial reporting . existing approaches assume a single-market setting and overlook structural differences across jurisdictions . |
| Approach: | They propose a workflow that decomposes financial reporting into auditable stages . they use Large Language Models to extract and summarize corporate disclosures . |
| Outcome: | The proposed system decomposes reporting into auditable stages . it improves consistency and reliability under heterogeneous reporting regimes. |
An LLM-based Framework for Biomedical Terminology Normalization in Social Media via Multi-Agent Collaboration (2025.coling-main)
Copied to clipboard
| Challenge: | Experimental results indicate that our approach exhibits competitive performance. |
| Approach: | They propose a tuning-free approach to normalize non-standard terms using large language models . they use a search engine and a domain knowledge base to expand the short texts into accurate descriptions . |
| Outcome: | The proposed approach is based on the "Recall and Re-rank" framework . it can be used to identify the standard term in a specified termbase for non-standardized mentions . |
Solver-Independent Automated Problem Formulation via LLMs for High-Cost Simulation-Driven Design (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to translating ambiguous design requirements into a mathematical optimization formulation are expensive and time-consuming. |
| Approach: | They propose a solver-independent framework that converts engineers’ natural language requirements into executable optimization models. |
| Outcome: | The proposed framework outperforms existing methods in the accuracy of requirement formalization and quality of resulting radiation efficiency curves on antenna design. |
Language Generation via Combinatorial Constraint Satisfaction: A Tree Search Enhanced Monte-Carlo Approach (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Generating natural language under complex constraints is a principled formulation towards controllable text generation. |
| Approach: | They propose a method to specify combinatorial constraints for sentence generation . they use a tree search algorithm embedded into the proposal process of the Markov Chain Monte Carlo . |
| Outcome: | The proposed method achieves consistent and significant improvement on multiple language generation tasks. |
Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework (2026.findings-acl)
Copied to clipboard
Jiaqi Weng, Han Zheng, Hanyu Zhang, Ej Zhou, Qinqin He, Jialing Tao, Hui Xue, Zhixuan Chu, Xiting Wang
| Challenge: | Existing studies on how SAEs derive most fine-grained latent features for safety remain unexplored. |
| Approach: | They propose a framework for interpreting SAE features in safety-critical domains . they train a suite of SAEs with human-readable explanations and systematic evaluations based on pornography, politics, violence, and terror . |
| Outcome: | The proposed framework reduces interpretation cost by 55% and improves safety-critical features. |
PRA-RAG: Provably Robust Aggregation in Retrieval-Augmented Generation against Retrieval Corruption (2026.findings-acl)
Copied to clipboard
Xue Tan, Yi Zheng, Chang Huo, Yunruo Zhang, Yu Liu, Hao Luan, Zhuyang Yu, Jun Dai, Xiaoyan Sun, Ping Chen
| Challenge: | Existing defense mechanisms lack theoretical robustness guarantees and perform unreliably when the LLM has limited knowledge of the retrieved content. |
| Approach: | They propose a provably robust retrieval aggregation algorithm designed to defend against poisoning attacks on retrieved texts. |
| Outcome: | Experiments show that PRA-RAG reduces the attack success rate to as low as 1% while maintaining an accuracy of 71%, significantly outperforming representative state-of-the-art (SOTA) methods. |
Think Natively: Unlocking Multilingual Reasoning with Consistency-Enhanced Reinforcement Learning (2026.acl-long)
Copied to clipboard
Xue Zhang, Yunlong Liang, Fandong Meng, Songming Zhang, Kaiyu Huang, Yufeng Chen, Xu Jinan, Jie Zhou
| Challenge: | Current Large Reasoning Models exhibit two critical limitations when processing non-English languages: (1) They struggle to maintain input-output language consistency; (2) They generally perform poorly with wrong reasoning paths and lower answer accuracy compared to English. |
| Approach: | They propose a language-consistency reward and a cross-lingual thinking alignment reward to improve the model's interpretability and accuracy. |
| Outcome: | The proposed model achieves nearly 100% language consistency and superior performance on two multilingual benchmarks (MMATH and PolyMath). |
Disco-RAG: Discourse-Aware Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
Dongqi Liu, Hang Ding, Qiming Feng, Xurong Xie, Zhucun Xue, Chengjie Wang, Jian Li, Jiangning Zhang, Yabiao Wang
| Challenge: | Existing RAG strategies treat retrieved passages in a flat and unstructured way, which prevents the model from capturing structural cues and constrains its ability to synthesize knowledge from dispersed evidence across documents. |
| Approach: | They propose a framework that explicitly injects discourse signals into the generation process. |
| Outcome: | Experiments on question answering and long-document summarization benchmarks show the efficacy of the proposed framework. |
CONSTRUCTA: Automating Commercial Construction Schedules in Fabrication Facilities with Large Language Models (2025.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods for automating construction scheduling are limited due to their domain knowledge and complexity. |
| Approach: | They propose a framework leveraging LLMs to optimize construction schedules in commercial construction. |
| Outcome: | The proposed framework improves missing value prediction, dependency analysis and planning performance compared to baseline methods. |
Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a method of detecting entity spans and classifying them into predefined categories. |
| Approach: | They propose a method to iteratively perform noisy label refinery by using self-collaborative denoising learning. |
| Outcome: | The proposed learning paradigm exploits reliable labels and communicates with unreliable annotations by collaborative denoising. |
Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questions (2025.naacl-long)
Copied to clipboard
Hongru Wang, Boyang Xue, Baohang Zhou, Tianhua Zhang, Cunxiang Wang, Huimin Wang, Guanhua Chen, Kam-Fai Wong
| Challenge: | Existing studies focus on leveraging internal knowledge of Large Language Models (LLMs) to answer known questions. |
| Approach: | They propose a framework that allows LLMs to choose between internal and external knowledge . they use a dataset to analyze compositional questions that are composed of unknown sub-questions . |
| Outcome: | The proposed framework can achieve comparable or even better performance with much fewer external calls compared with several strong baselines. |
How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularities (2026.acl-long)
Copied to clipboard
Ziwen Xu, Kewei Xu, Haoming Xu, Haiwen Hong, Longtao Huang, Hui Xue, Ningyu Zhang, Yongliang Shen, Guozhou Zheng, Huajun Chen, Shumin Deng
| Challenge: | Large language models (LLMs) are increasingly deployed in socially sensitive domains, yet their unpredictable behaviors pose significant risks. |
| Approach: | They propose a hierarchical benchmark for evaluating LLM controllability across three domains: language features, sentiment, and personality. |
| Outcome: | The proposed framework offers a principled and interpretable framework for safe and controllable LLM behavior serving as a foundation for future research. |
Porous Lattice Transformer Encoder for Chinese NER (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to integrate word boundary information into character-level Chinese NER are inefficient and lack semantic interaction. |
| Approach: | They propose an extension of transformer encoder that is tailored for ChineseNER to incorporate lexicons into character-level Chinese NER by lattices. |
| Outcome: | The proposed extension performs 11.4 times faster than state-of-the-art methods while retaining the rich long-term dependencies. |
V-Oracle: Making Progressive Reasoning in Deciphering Oracle Bones for You and Me (2025.acl-long)
Copied to clipboard
Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Jiapeng Wang, YiFan Zhang, Zhuoma GongQue, Chong Sun, Yida Xu, Yadong Xue, Ye Tian, Zhimin Bao, Lan Yang, Chen Li, Honggang Zhang
| Challenge: | Deciphering oracle bone scripts using AI technology is not an overnight task due to the evolution of written language over millennia. |
| Approach: | They propose a framework that utilizes Large Multi-modal Models (LMMs) for interpreting Oracle Bone Script (OBS). |
| Outcome: | The proposed framework provides quantitative analyses and superior deciphering capability. |
Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Experts (2025.acl-long)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) have remarkable ability in high-resource languages, but their performance in multilingual scenarios is still limited. |
| Approach: | They propose a layer-wise expert allocation algorithm to determine the appropriate number of new experts for each layer. |
| Outcome: | The proposed method outperforms the previous state-of-the-art baseline with 60% fewer experts in the single-expansion setting and 33.3% fewer in the lifelong-expanding setting. |
Kill two birds with one stone: generalized and robust AI-generated text detection via dynamic perturbations (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods focus on model generalization or focus on robustness. |
| Approach: | They propose a model-based AIGT detection method that can be generalized and robust under two adversarial attacks. |
| Outcome: | The proposed method outperforms state-of-the-art methods for generalization and robustness under two text adversarial attacks. |
LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement (2025.acl-long)
Copied to clipboard
Boyi Kang, Xinfa Zhu, Zihan Zhang, Zhen Ye, Mingshuai Liu, Ziqian Wang, Yike Zhu, Guobin Ma, Jun Chen, Longshuai Xiao, Chao Weng, Wei Xue, Lei Xie
| Challenge: | Recent advances in language models have demonstrated strong capabilities in semantic understanding and contextual modeling. |
| Approach: | They propose a LLaMA-based language model that incentivizes generalization capabilities for speech enhancement. |
| Outcome: | The proposed language model outperforms prior task-specific discriminative and generative models in acoustic enhancement tasks. |
AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for LLM alignment optimize tokens using a sparse, response-level reward or preference annotation. |
| Approach: | They propose an RLHF-equivalent distillation method for token-level reward optimization that incorporates the reward learned by DPO into the RLHG objective and builds a token-based teacher distribution. |
| Outcome: | The proposed method bridges the accuracy gap between the reward from the DPO model and the pure reward model by building a contrastive DPO reward with a normal and a reverse DPO. |
MidMed: Towards Mixed-Type Dialogues for Medical Consultation (2023.acl-long)
Copied to clipboard
| Challenge: | Current medical dialogue systems assume that patients have explicit goals but are often unavailable in real-world situations due to the lack of medical knowledge. |
| Approach: | They propose a human-to-human mixed-type medical consultation dialogue corpus . they build benchmarking baselines on MidMed and propose an instruction-guiding framework . Experimental results show the effectiveness of InsMed . |
| Outcome: | The proposed system can help patients clarify their goals in real-world situations . it covers four departments with 8,309 dialogues and provides benchmarking baselines . |
Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization (2021.acl-long)
Copied to clipboard
| Challenge: | Text style transfer aims to alter the style of a sentence while preserving its content. |
| Approach: | They propose to remove style information at token level and fuse it to style representations using conditional layer normalization. |
| Outcome: | The proposed model outperforms the state-of-the-art models in terms of content preservation and fluency. |
Dynamic Voting for Efficient Reasoning in Large Language Models (2023.findings-emnlp)
Copied to clipboard
Mingfeng Xue, Dayiheng Liu, Wenqiang Lei, Xingzhang Ren, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv
| Challenge: | Multi-path voting methods generate multiple reasoning paths for each problem, causing factual errors and illusion generation. |
| Approach: | They propose a multi-path voting technique that effectively reduces the number of reasoning paths during multi-path voting while preserving accuracies. |
| Outcome: | The proposed method outperforms Self-consistency using 24.7% of the number of paths on the LetterConcat task. |
Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulness (2024.findings-acl)
Copied to clipboard
Guangliang Liu, Milad Afshari, Xitong Zhang, Zhiyu Xue, Avrajit Ghosh, Bidhan Bashyal, Rongrong Wang, Kristen Johnson
| Challenge: | Debiasing Pretrained Language Models (PLMs) are task-agnostic and can be generalizable, but its impact on language modeling ability and the risk of relearning social biases remain as the two most significant challenges. |
| Approach: | They propose a framework which can Propagate Socially-fair Debiasing to Downstream Fine-tuning to alleviate the forgetting issue of PLMs by regularizing debiased attention heads based on the PLM’s bias levels from stages of pretraining and debiase. |
| Outcome: | The proposed framework can Propagate Socially-fair Debiasing to Downstream Fine-tuning, indicating that the ineffectiveness of debiase can be alleviated by overcoming the forgetting issue through regularizing successfully debiased attention heads based on the PLMs’ bias levels from stages of pretraining and debiases. |
Constrained Tuple Extraction with Interaction-Aware Network (2023.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge triples lack constraints for their authenticity due to spatial, temporal, or other constraints. |
| Approach: | They propose a constrained tuple extraction task to guarantee the validity of knowledge tifles by using an interaction-aware network to extract constrained text. |
| Outcome: | The proposed model outperforms existing models on the dataset and the public CaRB dataset. |
Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a method of detecting entity spans and classifying them into predefined categories. |
| Approach: | They propose a method to iteratively perform noisy label refinery by using self-collaborative denoising learning. |
| Outcome: | The proposed learning paradigm exploits reliable labels and communicates with unreliable annotations by collaborative denoising. |
Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel in translation and summarization due to the capabilities of transformer architectures. |
| Approach: | They propose to integrate tensorized adapters into model encoder/decoder blocks to improve model adaptability against data heterogeneity. |
| Outcome: | Experiments on large-scale cross-device FL and large-silo FL show that the proposed methods perform on par or even better than existing federated PEFT approaches while reducing communication cost. |
RoChBert: Towards Robust BERT Fine-tuning for Chinese (2022.findings-emnlp)
Copied to clipboard
Zihan Zhang, Jinfeng Li, Ning Shi, Bo Yuan, Xiangyu Liu, Rong Zhang, Hui Xue, Donghong Sun, Chao Zhang
| Challenge: | Pre-trained language models (e.g., BERT) have been proved vulnerable to adversarial texts. |
| Approach: | They propose to fuse Chinese phonetic and glyph features into pre-trained models by using a more comprehensive adversarial graph. |
| Outcome: | The proposed framework outperforms existing methods in significant ways on a wide range of tasks while remaining accurate on benign texts. |
Beyond Length Scaling: Synergizing Breadth and Depth for Generative Reward Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Generative Reward Models have demonstrated that scaling the length of Chain-of-Thought reasoning enhances reliability of evaluation. |
| Approach: | They propose a framework that reconfigures raw rationales into structured Breadth-CoT and Depth-Co T through a modular synthesis pipeline. |
| Outcome: | The proposed framework surpasses open-source RMs by an average of 8.2%. |
AutoRE: Document-Level Relation Extraction with Large Language Models (2024.acl-demos)
Copied to clipboard
| Challenge: | Existing methods for relation extraction are limited to Sentence-level Relation Extraction (SentRE) tasks. |
| Approach: | They propose an end-to-end DocRE model that adopts a novel RE extraction paradigm named RHF (Relation-Head-Facts) Unlike existing approaches, AutoRE does not rely on the assumption of known relation options, making it more reflective of real-world scenarios. |
| Outcome: | The proposed model surpasses TAG by 10.03% and 9.03% on the dev and test set. |
KoCo-Bench: Can Large Language Models Leverage Domain Knowledge in Software Development? (2026.acl-long)
Copied to clipboard
Xue Jiang, Ge Li, Jiaru Qian, Xianjie Shi, Chenjie Li, Hao Zhu, Ziyu Wang, Jielun Zhang, Zeyu Zhao, Kechi Zhang, Jia Li, Wenpin Jiao, Zhi Jin, Yihong Dong
| Challenge: | Existing domain-specific code benchmarks focus on assessing what knowledge LLMs possess rather than how they acquire and apply new knowledge. |
| Approach: | They propose a benchmark to evaluate domain specialization methods in real-world software development. |
| Outcome: | KOCO-bench is a new benchmark for evaluating domain specialization methods in real-world software development. |
Star-Transformer (N19-1)
Copied to clipboard
| Challenge: | Existing models with fully-connected attention connections are heavy and require large training data. |
| Approach: | They propose a lightweight alternative to the Transformer by sparsifying the fully-connected structure with a star-shaped topology. |
| Outcome: | The proposed model achieves significant performance improvements on 22 datasets on four tasks. |
PASUM: A Pre-training Architecture for Social Media User Modeling Based on Text Graph (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have incorporated different digital traces to better learn the representations of social media users, limited by overloaded text information and hard-to-collect social network information. |
| Approach: | They propose a Pre-training Architecture for Social Media User Modeling based on Text Graph and combine microblogs to represent social media users based upon the text graph model. |
| Outcome: | The proposed framework can represent users based on text even without social network information on microblogs. |
RTCFake: Speech Deepfake Detection in Real-Time Communication (2026.findings-acl)
Copied to clipboard
Jun Xue, Zhuolin Yi, Yihuan Huang, Yanzhen Ren, Yujie Chen, Cunhang Fan, Zicheng Su, Yongcheng Zhang, Bo Cai
| Challenge: | Existing detection studies focus on offline simulations and struggle to cope with complex distortions introduced during RTC transmission. |
| Approach: | They propose a large-scale speech deepfake dataset tailored for RTC scenarios . the dataset is constructed by transmitting speech through multiple social media and conferencing platforms . |
| Outcome: | The proposed dataset is constructed by transmitting speech through multiple mainstream social media and conferencing platforms, enabling precise pairing between offline and online speech. |
ConFit v2: Improving Resume-Job Matching using Hypothetical Resume Embedding and Runner-Up Hard-Negative Mining (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to model resume-job fit are sparse since job seekers apply to only a few jobs. |
| Approach: | They propose two techniques to enhance the encoder’s contrastive training process by augmenting job data with hypothetical reference resume generated by a large language model and creating high-quality hard negatives from unlabeled resume/job pairs using a novel hard-negative mining strategy. |
| Outcome: | The proposed method outperforms ConFit and prior methods on two real-world datasets and achieves an average improvement of 13.8% in recall and 17.5% in nDCG across job-ranking and resume-ranker tasks. |
WenetSpeech-Wu: Datasets, Benchmarks, and Models for a Unified Chinese Wu Dialect Speech Processing Ecosystem (2026.findings-acl)
Copied to clipboard
Chengyou Wang, Mingchen Shao, Jingbin Hu, Zeyu Zhu, Hongfei Xue, Bingshen Mu, Xin Xu, Xingyi Duan, Binbin Zhang, Zhu Pengcheng, Chuang Ding, Xiaojun Zhang, Hui Bu, Lei Xie
| Challenge: | despite its linguistic significance, the Wu dialect of Chinese has long been hindered by the lack of large-scale speech data, standardized evaluation benchmarks, and publicly available models. |
| Approach: | They propose to use WenetSpeech-Wu as a large-scale, multi-dimensionally annotated open-source speech corpus for the Wu dialect of Chinese. |
| Outcome: | The proposed dataset includes 8,000 hours of speech data and strong open-source models . the proposed dataset is competitive and empirically validated . |
AutoSDT: Scaling Data-Driven Discovery Tasks Toward Open Co-Scientists (2025.emnlp-main)
Copied to clipboard
Yifei Li, Hanane Nour Moussa, Ziru Chen, Shijie Chen, Botao Yu, Mingyi Xue, Benjamin Burns, Tzu-Yao Chiu, Vishal Dey, Zitong Lu, Chen Wei, Qianheng Zhang, Tianyu Zhang, Song Gao, Xuhui Huang, Xia Ning, Nesreen K. Ahmed, Ali Payani, Huan Sun
| Challenge: | AutoSDT-5K is the only automatically collected and the largest open dataset for data-driven scientific discovery. |
| Approach: | They propose an automatic pipeline that collects high-quality coding tasks in real-world data-driven discovery workflows. |
| Outcome: | The proposed pipeline synthesizes accurate tasks and tasks from a dataset of 5,404 tasks covering four scientific disciplines and 756 Python packages. |
Enhancing LLM-Based Social Bot via an Adversarial Learning Framework (2025.emnlp-main)
Copied to clipboard
| Challenge: | Social media platforms provide an ideal testbed for large language models that exhibit human-like behavior. |
| Approach: | They propose an LLM-based social **Bot that enhances human-like generative capabilities through an adversarial learning framework. |
| Outcome: | The proposed framework generates human-like content aligned with diverse user profiles . it exhibits strong social responsiveness, more accurately modeling opinion dynamics . |
AtTGen: Attribute Tree Generation for Real-World Attribute Joint Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Attribute extraction aims to identify attribute names and the corresponding attribute values from descriptive texts. |
| Approach: | They propose a unified formulation for real-world attribute extraction application, where closed-world, open-world and semi-open attribute extraction tasks are modeled uniformly. |
| Outcome: | The proposed model outperforms existing methods on three datasets and outperformed existing methods by a large margin. |
MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for answering time-sensitive questions lack temporal reasoning . existing methods struggle with these time-intensive questions, authors say . |
| Approach: | They propose a temporal-based question-answering framework that integrates temporal perturbations and gold evidence labels into a question processing framework. |
| Outcome: | The proposed framework outperforms baseline retrieval methods in retrieval performance. |
BayesKD: Bayesian Knowledge Distillation for Compact LLMs in Constrained Fine-tuning Scenarios (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized various domains with their remarkable capabilities, but their massive parameter sizes pose significant challenges for fine-tuning and inference. |
| Approach: | They propose a Bayesian Knowledge Distillation framework for compact Large Language Models in resource-constrained fine-tuning scenarios that employs Logits Dual-Scaling, Knowledge Alignment Module, and Bayes Distillations Optimization. |
| Outcome: | The proposed framework outperforms baseline methods on various state-of-the-art LLMs, including LLaMA, Qwen2, Bloom, and Vicuna. |
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases (D19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, Dragomir Radev
| Challenge: | CoSQL is a corpus for building cross-domain, general-purpose database querying dialogue systems. |
| Approach: | They present a corpus for building cross-domain, general-purpose database querying dialogue systems . they use a Wizard-of-Oz collection of 3k turns plus 10k+ annotated SQL queries . |
| Outcome: | The proposed corpus is based on a Wizard-of-Oz dataset of 3k dialogues querying 200 complex DBs spanning 138 domains. |
InImageTrans: Multimodal LLM-based Text Image Machine Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing multimodal large language models suffer from repetition and omission hallucinations when transferred to text image machine translation task. |
| Approach: | They propose an efficient MLLM named InImageTrans for TiMT and a method for advancing it. |
| Outcome: | The proposed method outperforms existing open-source MLLMs on the MCiT benchmark. |
Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization (2026.acl-long)
Copied to clipboard
Weixu Zhang, Ye Yuan, Changjiang Han, Yuxing Tian, Zipeng Sun, Linfeng Du, Jikun Kang, Hong Kang, Xue Liu, Haolun Wu
| Challenge: | Large Language Models exhibit strong implicit personalization ability, but most approaches treat this behavior as a black box. |
| Approach: | They propose a mechanistic interpretation perspective and propose 'sparse' set of Preference Heads . they compute a Preference Contribution Score for each attention head and compare their predictions . |
| Outcome: | The proposed framework computes a Preference Contribution Score (PCS) for each attention head and measures its causal impact on user aligned outputs. |
Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning (2026.acl-long)
Copied to clipboard
Zelin Tan, Hejia Geng, Xiaohang Yu, Mulei Zhang, Guancheng Wan, Yifan Zhou, Qiang He, Xiangyuan Xue, Heng Zhou, Yutao Fan, Zhong-Zhi Li, Zaibin Zhang, Guibin Zhang, Chen Zhang, Zhenfei Yin, Philip Torr, Lei Bai
| Challenge: | elucidating scaling laws for large language models (LLMs) during pre-training remains unexplored. |
| Approach: | They characterize how model scale, data, and compute interact during pre-training . they find that large models consistently demonstrate superior compute and data efficiency . |
| Outcome: | The proposed scaling laws offer practical guidance for scaling reasoning capabilities through reinforcement learning post-training. |
Structured Interpretation of Temporal Relations (L18-1)
Copied to clipboard
| Challenge: | Temporal relations between events and time expressions are often modeled in an unstructured manner, resulting in inconsistent and incomplete annotation and computational modeling. |
| Approach: | They propose an annotation approach where events and time expressions form a dependency tree in which each dependency relation corresponds to an instance of temporal anaphora. |
| Outcome: | The proposed approach annotates 235 documents in news and narratives with 48 doubly annotated documents. |
MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Calling (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been widely used in medicine but are limited in their ability to fully address the complexities of the real world. |
| Approach: | They propose a universal agent architecture for Large Language Models that integrates a specialized medical toolkit and employs meta-tool and nested calling mechanisms to enhance LLM tool utilization. |
| Outcome: | The proposed framework improves the accuracy and performance of medical calculators in complex medical scenarios. |
Dynamic Evil Score-Guided Decoding: An Efficient Decoding Framework For Red-Team Model (2025.findings-acl)
Copied to clipboard
Cong Gao, Bo Zhang, Linkang Yang, Minghao Hu, Zhunchen Luo, Xiaoying Bai, Guotong Geng, Jun Zhang, Yunhua Xue
| Challenge: | Existing red-teaming methods require expensive fine-tuning, especially for large LLMs. |
| Approach: | They propose a red-teaming method that uses an ‘evil score’ to evaluate the potential of tokens to contribute to harmful outputs during decoding. |
| Outcome: | The proposed method achieves an ASR of 92.83% on the Llama-3.2-3B-Instruct model, compared to 83.48% with adversarial fine-tuning while using less computational resources. |
Response-Anticipated Memory for On-Demand Knowledge Integration in Response Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Neural conversation models generate appropriate but non-informative responses in general. |
| Approach: | They propose to construct a document memory with anticipated responses in mind using a teacher-student framework and a student's input. |
| Outcome: | The proposed model outperforms the state-of-the-art for the Conversing by Reading task. |
Unified Hallucination Detection for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen
| Challenge: | despite significant strides in multimodal tasks, MLLMs are plagued by the critical issue of hallucination. |
| Approach: | They propose a meta-evaluation benchmark to facilitate evaluation of advancements in hallucination detection methods. |
| Outcome: | The proposed framework validates hallucinations robustly and provides strategic insights . MHaluBench is a meta-evaluation benchmark designed to facilitate evaluation . |
Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation (2026.findings-acl)
Copied to clipboard
Zhen Bi, Zhenlin Hu, Xueshu Chen, Mingyang Chen, Cheng Deng, Yida Xue, Zhen Wang, Qing Shen, Ningyu Zhang, Jungang Lou
| Challenge: | Existing data-centric paradigms equate quality with factuality or diversity and ignore the internal logical complexity of training samples. |
| Approach: | They propose a density-aware re-cognizing optimization strategy that prioritizes high-density logical samples to align training with the model's reasoning boundary. |
| Outcome: | The proposed metric outperforms existing methods and improves reasoning performance without increasing total data volume. |
Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique for aligning large language models (LLMs) with human preferences. |
| Approach: | They propose a novel algorithm that uses multiple-gradient descent to optimize LLMs with diverse preferences to maximize trade-offs between objectives. |
| Outcome: | The proposed approach incorporates user preferences across different objectives and achieves Pareto solutions that better align with the user’s specific needs. |
RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization (2026.acl-long)
Copied to clipboard
Yihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu, Kechi Zhang, Lili Mou, Rongyu Cao, Yingwei MA, Jue Chen, Binhua Li, Zhi Jin, Fei Huang, Yongbin Li, Ge Li
| Challenge: | Reinforcement Learning with Verifiable Reward (RLVR) has significantly advanced the complex reasoning abilities of Large Language Models (LLMs). |
| Approach: | They propose a hybrid-policy optimization approach that synergizes internal exploitation with external data to achieve stronger reasoning capabilities. |
| Outcome: | The proposed approach achieves state-of-the-art performance on six math reasoning benchmarks and superior performance on out-of distribution reasoning tasks. |
Escaping the Echo Trap: On Credit Assignment Failure in Multi-turn LLM Self-Reflection (2026.acl-long)
Copied to clipboard
Linxuan Du, Guangquan Xue, Xiaobo Liang, Qipeng Huang, Yuyang Ding, Xinyu Shi, Zhang Yijun, Ji Qi, Wenpeng Zhu, Juntao Li, Min Zhang
| Challenge: | Existing methods for multi-turn self-reflection are limited by the Echo Trap problem . the model is limited by its inherent capabilities and repeats earlier reflections to preserve reward signals . |
| Approach: | They propose a tree-structured extension of GRPO for multi-turn self-reflection which enables more accurate advantage estimation. |
| Outcome: | The proposed method mitigates behavior collapse and improves performance across benchmarks. |
Interactive Evaluation for Medical LLMs via Task-oriented Dialogue System (2025.coling-main)
Copied to clipboard
| Challenge: | In typical medical scenarios, doctors often ask a set of questions to gain a comprehensive understanding of patients’ conditions. |
| Approach: | They propose to use multi-turn medical dialogue evaluation to evaluate proactive communication and diagnostic capabilities of medical Large Language Models (LLMs) . |
| Outcome: | The proposed model outperforms existing models on multi-turn question-answering datasets and is therefore cost-effective. |
Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics (2026.acl-long)
Copied to clipboard
Ziwen Xu, Chenyan WU, Hengyu Sun, Haiwen Hong, Mengru Wang, Yunzhi Yao, Longtao Huang, Hui Xue, Shumin Deng, Zhixuan Chu, Huajun Chen, Ningyu Zhang
| Challenge: | Methods for controlling large language models (LLMs) are often studied in isolation, obscuring connections and making comparison difficult. |
| Approach: | They propose a preference-utility analysis that separates control effects into preference and utility, and measures both on a shared log-odds scale using polarity-paired contrastive examples. |
| Outcome: | The proposed approach improves preference while preserving utility. |
Dual-Space Knowledge Distillation for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) have strong generalization abilities due to their huge model capacities. |
| Approach: | They propose a dual-space knowledge distillation framework that unifies the output spaces of the two models for KD. |
| Outcome: | The proposed framework outperforms existing white-box KD frameworks on task-agnostic instruction-following benchmarks and can automatically align representations of two models with different vocabularies. |
PAC-tuning: Fine-tuning Pre-trained Language Models with PAC-driven Perturbed Gradient Descent (2023.emnlp-main)
Copied to clipboard
| Challenge: | PAC-tuning is a two-stage fine-tune method for pretrained language models . PAC training minimizes the PACBayes generalization bound to learn proper parameter distribution . |
| Approach: | They propose a two-stage fine-tuning method to minimize the PAC-Bayes generalization bound . they use PAC to inject noise with variance learned in the first stage into the model parameters . |
| Outcome: | The proposed method outperforms baseline methods on 5 GLUE benchmark tasks. |
UniPSDA: Unsupervised Pseudo Semantic Data Augmentation for Zero-Shot Cross-Lingual Natural Language Understanding (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies rely on shallow unsupervised data generated by token surface matching regardless of global context-aware semantics of the surrounding text tokens. |
| Approach: | They propose an Unsupervised Pseudo Semantic Data Augmentation mechanism to enrich training data without human intervention. |
| Outcome: | The proposed model improves on general zero-shot cross-lingual understanding tasks on different languages without human intervention. |
ChildEval:WHEN LARGE LANGUAGE MODELS MEET CHILDREN’S PERSONALITIES (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved remarkable success in effectively understanding and generating human language, leading to a revolutionary era in LLMs. |
| Approach: | They propose a benchmark to evaluate LLMs' ability to infer and follow child-centered preferences in long-context conversations. |
| Outcome: | The proposed benchmark spans five top-level and fourteen sub-level categories covering children’s daily lives and development. |
Rhombus: Incentivizing Coordination in Parallel Thinking through Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Ziyuan Nan, Qi Yi, Di Huang, Yutong Wu, Guanhua Huang, Xue Gong, Kejiao Li, Yuhao Jiang, Chenchen Zhang, Zenan Xu, Xing Hu, Bo Zhou
| Challenge: | Parallel thinking is a promising avenue for scaling test-time compute in Large Language Models . however, coordinating the exploration and aggregation stages remains challenging . |
| Approach: | They propose a parallel thinking framework that explicitly incentivizes coordination between components via end-to-end reinforcement learning. |
| Outcome: | The proposed framework improves accuracy by 6.0% over long chain-of-thought baselines while reducing wall-clock latency by 39.4% under matched token budgets. |
Semi-supervised Domain Adaptation for Dependency Parsing (P19-1)
Copied to clipboard
| Challenge: | Currently, most studies on cross-domain parsing focus on unsupervised domain adaptation . however, unsupervised approaches make limited progress due to the intrinsic difficulty of both domain adaptation and parse. |
| Approach: | They propose a semi-supervised domain adaptation problem for Chinese dependency parsing by using newly-annotated large-scale domain-aware datasets. |
| Outcome: | The proposed method is more effective than direct corpus concatenation and multi-task learning. |