Papers by Yao Xu
Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can solve reasoning and mathematical problems using the Chain-of-Thought technique, but require costly and long CoT data and fine-tuning. |
| Approach: | They propose a method that uses Sparse Autoencoders to extract interpretable features from vanilla CoT and use them to steer the LLM's internal states. |
| Outcome: | The proposed method uses Sparse Autoencoders (SAEs) to extract interpretable features from vanilla CoT and steer the LLM's internal states during generation. |
ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency (2024.emnlp-industry)
Copied to clipboard
Yuhang Yao, Han Jin, Alay Shah, Shanshan Han, Zijian Hu, Dimitris Stripelis, Yide Ran, Zhaozhuo Xu, Salman Avestimehr, Chaoyang He
| Challenge: | Large language models (LLMs) are widely used in commercial applications . low latency is crucial due to system latency, query concurrency, and computational resources constraints. |
| Approach: | They propose a system that can be resource-efficiently served by addressing bottlenecks beyond LLM inference . they propose 4.3 speed up over vLLM and 1.5 higher throughput . |
| Outcome: | The proposed system outperforms state-of-the-arts with 1.5 higher throughput . it achieves 4.3 speed up with 64 concurrent requests on Mixtral 8x7B . |
From Outcome to Process: Optimizing MoE Load Balancing with MCTS (2026.findings-acl)
Copied to clipboard
Wenjun Ke, Hengyuan Xu, Ziyu Shang, Yao He, Jiahao Wang, Zijie Xu, Peng Wang, Yuhang Lou, Jiajun Liu
| Challenge: | Existing balancing strategies focus on constraining the final distribution of expert usage, but overlook the routing decisions made at each layer. |
| Approach: | They propose a three-stage framework that leverages process-level rewards to guide balanced expert routing. |
| Outcome: | Extensive experiments show that LayerMoE improves the performance of state-of-the-art LoRA-MoA baselines, yielding an average accuracy gain of 1.39%. |
Dancing Between Success and Failure: Edit-level Simplification Evaluation using SALSA (2023.emnlp-main)
Copied to clipboard
| Challenge: | Traditional human evaluation methods for text simplification often relies on individual, shallow sentence-level ratings, easily affected by the annotator's preference or bias. |
| Approach: | They propose an edit-based human annotation framework that enables holistic and fine-grained text simplification evaluation. |
| Outcome: | The proposed framework is able to predict sentence- and word-level quality simultaneously and report promising results. |
LIFTED: Multimodal Clinical Trial Outcome Prediction via Large Language Models and Mixture-of-Experts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Clinical trials are costly and pivotal processes that require substantial expenses . a new approach to integrate multimodal data for clinical outcome prediction is needed . |
| Approach: | a proposed framework transforms modality-specific data into natural language descriptions . a sparse Mixture-of-Experts mechanism then identifies shared patterns across modalities . |
| Outcome: | a proposed framework outperforms baseline methods in predicting clinical trial outcomes . it transforms modality-specific data into natural language descriptions, encoded via unified encoders . |
V-ALPHASOCIAL: Benchmark and Self-Reflective Chain-of-Thought Generation for Visual Social Commonsense Reasoning (2025.findings-acl)
Copied to clipboard
Zongyu Lin, Zhikun Xu, Xiaohan Song, Yixin Wan, Xingcheng Yao, Tsung-Han Lin, Selina Song, Pranav Subbaraman, Ben Zhou, Kai-Wei Chang, Yizhou Sun
| Challenge: | Social commonsense reasoning is a multimodal task that requires both textual and visual cues. |
| Approach: | They propose a method that integrates visual cues into social commonsense reasoning tasks. |
| Outcome: | The proposed method improves social commonsense reasoning on a multimodal foundation model. |
TensorOpera Router: A Multi-Model Router for Efficient LLM Inference (2024.emnlp-industry)
Copied to clipboard
Dimitris Stripelis, Zhaozhuo Xu, Zijian Hu, Alay Shah, Han Jin, Yuhang Yao, Jipeng Zhang, Tong Zhang, Salman Avestimehr, Chaoyang He
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance across a diverse set of domain-specific tasks. |
| Approach: | They propose a non-monolithic LLM querying system that seamlessly integrates various LLM experts into a single query interface and dynamically routes incoming queries to the most high-performant expert based on query’s requirements. |
| Outcome: | The proposed model improves query efficiency by 40% and costs by 30% while maintaining or enhancing model performance by 10%. |
Towards Explainable Temporal Reasoning in Large Language Models: A Structure-Aware Generative Framework (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on temporal reasoning models neglect the explainable reasoning processes underlying the results. |
| Approach: | They propose a structure-aware generative framework that integrates Graph structures with text for Explainable TEmporal Reasoning. |
| Outcome: | The proposed framework achieves state-of-the-art performance while also demonstrating robust generalization capabilities. |
Generate-on-Graph: Treat LLM as both Agent and KG for Incomplete Knowledge Graph Question Answering (2024.emnlp-main)
Copied to clipboard
Yao Xu, Shizhu He, Jiabei Chen, Zihao Wang, Yangqiu Song, Hanghang Tong, Guang Liu, Jun Zhao, Kang Liu
| Challenge: | Existing methods to integrate LLMs with Knowledge Graphs (KGs) however, these methods are often incomplete to cover all the knowledge required to answer questions. |
| Approach: | They propose to integrate LLMs with Knowledge Graphs (KGs) to address insufficient knowledge and hallucination issues in Large Language Models. |
| Outcome: | The proposed method outperforms existing methods on two datasets. |
Gentopia.AI: A Collaborative Platform for Tool-Augmented LLMs (2023.emnlp-demo)
Copied to clipboard
Binfeng Xu, Xukun Liu, Hua Shen, Zeyu Han, Yuhan Li, Murong Yue, Zhiyuan Peng, Yuchen Liu, Ziyu Yao, Dongkuan Xu
| Challenge: | Existing frameworks for Augmented Language Models lack flexibility, democratization, and holistic evaluation. |
| Approach: | They propose a lightweight and extensible framework for Augmented Language Models called Gentopia. |
| Outcome: | The proposed framework integrates language models, task formats, prompting modules, and plugins into a unified paradigm. |
DetGPT: Detect What You Need via Reasoning (2023.emnlp-main)
Copied to clipboard
Renjie Pi, Jiahui Gao, Shizhe Diao, Rui Pan, Hanze Dong, Jipeng Zhang, Lewei Yao, Jianhua Han, Hang Xu, Lingpeng Kong, Tong Zhang
| Challenge: | Recent advances in the field of computer vision have enabled more effective and sophisticated interactions between humans and machines. |
| Approach: | They propose a reasoning-based object detection paradigm that leverages state-of-the-art multi-modal models and open-vocabulary object detectors to perform reasoning within the context of the user’s instructions and the visual scene. |
| Outcome: | The proposed method enables users to interact with the system using natural language instructions, allowing for a higher level of interactivity. |
Query2Triple: Unified Query Encoding for Answering Diverse Complex Queries over Knowledge Graphs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Complex Query Answering (CQA) is a challenge task of Knowledge Graphs due to incompleteness of KGs. |
| Approach: | They propose a query embedding approach that decouples the training for simple and complex queries. |
| Outcome: | The proposed approach decouples training for simple and complex queries and achieves state-of-the-art performance over three public benchmarks. |
It is AI’s Turn to Ask Humans a Question: Question-Answer Pair Generation for Children’s Story Books (2022.acl-long)
Copied to clipboard
| Challenge: | Existing question answering (QA) techniques are created mainly to answer questions asked by humans, but in educational applications, teachers often need to decide what questions to ask . |
| Approach: | They propose to use a fairytale-themed storybook as input to generate QA pairs that can test a student's comprehension skills. |
| Outcome: | The proposed system outperforms state-of-the-art QAG baseline systems and builds an interactive story-telling application for the future real-world deployment. |
Data Augmentation for Few-Shot Knowledge Graph Completion from Hierarchical Perspective (2022.coling-1)
Copied to clipboard
| Challenge: | Existing knowledge graph completion models require only a few associative triples to complete a relationship. |
| Approach: | They propose to perform data augmentation from two perspectives to solve the FKGC problem by inferring new triple facts from existing models. |
| Outcome: | The proposed framework can be applied to a number of existing models. |
An Auxiliary Task Boosted Multi-task Learning Method for Service Account Retrieval with Limited Human Annotation (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to service account retrieval have limited human annotation, resulting in labor-intensive and time-consuming tasks. |
| Approach: | They propose an Auxiliary task Boosted Multi-Task Learning method which introduces multiple auxiliary tasks and enhances the performance of the main task, service account retrieval. |
| Outcome: | The proposed method improves the performance of the main task, service account retrieval. |
CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | CKnowEdit is the first-ever knowledge editing dataset designed to correct linguistic, factual, and logical errors in Large Language Models. |
| Approach: | They propose a Chinese knowledge editing dataset to correct linguistic, factual, and logical errors in Large Language Models. |
| Outcome: | The proposed dataset highlights the challenges that LLMs face in mastering Chinese . CKnowEdit can correct linguistic, factual, and logical errors in Chinese, the authors show . |
StructMem: Structured Memory for Long-Horizon Behavior in LLMs (2026.acl-short)
Copied to clipboard
| Challenge: | Existing memory systems lack structure and efficiency in capturing relationships between events. |
| Approach: | They propose a structure-enriched hierarchical memory framework that preserves event-level bindings and induces cross-event connections. |
| Outcome: | The proposed framework preserves event-level bindings and induces cross-event connections while reducing token usage, API calls, and runtime compared to prior memory systems. |
Anchoring-Guidance Fine-Tuning (AnGFT): Elevating Professional Response Quality in Role-Playing Conversational Agents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated significant advancements in various fields, notably in Role-Playing Conversational Agents (RPCAs). |
| Approach: | They propose an Anchoring-Guidance Fine-Tuning Framework to integrate relevant expert knowledge into RPCAs' training process to mitigate this issue. |
| Outcome: | The proposed framework significantly improves the RPCAs’ performance in handling role-specific professional queries while preserving their robust role-playing abilities. |
The World in My Mind: Visual Dialog with Adversarial Multi-modal Feature Encoding (N19-1)
Copied to clipboard
| Challenge: | Visual Dialog is a multi-modal task that requires a model to participate in a dialog grounded on an image and generate correct, human-like responses. |
| Approach: | They propose a framework for effective and robust auxiliary training of visual dialog systems using multi-modal encoding. |
| Outcome: | The proposed framework outperforms supervised learning baselines and fine-tuning methods on most metrics of VisDial v0.5/v0.9 generative tasks. |
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents (2025.findings-acl)
Copied to clipboard
Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, Guohao Li
| Challenge: | Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexity of constructing tasks and evaluators. |
| Approach: | They propose a cross-environment agent benchmark framework that integrates graph-based evaluation and task generation methods. |
| Outcome: | The proposed framework supports multiple devices and can be easily extended to any environment with a Python interface. |
Pause or Fabricate? Training Language Models for Grounded Reasoning (2026.findings-acl)
Copied to clipboard
Yiwen Qiu, Linjuan Wu, Yizhou Liu, Yuchen Yan, Jin Ma, Xu Tan, Yao Hu, Daoxin Zhang, Wenqi Zhang, Weiming Lu, Jun Xiao, Yongliang Shen
| Challenge: | Large language models implicitly fabricate information when inputs are incomplete, causing confidence but unreliable conclusions. |
| Approach: | They propose a framework for grounded reasoning under incomplete information that decomposes reasoning into two stages . they propose stage-specific rewards to penalize hallucinations, enabling models to detect gaps, stop proactively, and resume reasoning after clarification. |
| Outcome: | The proposed framework improves premise detection and task success by 30% . it also reduces average response length by over 20% . |
Using active learning to expand training data for implicit discourse relation recognition (D18-1)
Copied to clipboard
| Challenge: | Existing methods to determine semantic relations between text spans are limited in the field of discourse-level relation recognition. |
| Approach: | They propose to expand the training data set using the corpus of explicitly-related arguments by arbitrarily dropping the overtly presented discourse connectives. |
| Outcome: | The proposed model expands the training data set using the corpus of explicitly-related arguments, by arbitrarily dropping the overtly presented discourse connectives. |
kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning (2024.naacl-long)
Copied to clipboard
Wenting Zhao, Ye Liu, Yao Wan, Yibo Wang, Qingyang Wu, Zhongfen Deng, Jiangshu Du, Shuaiqi Liu, Yunlong Xu, Philip Yu
| Challenge: | Recent advances in task-oriented parsing involve formulating the task as a sequence-to-sequence problem, relying on a wealth of labeled data. |
| Approach: | They propose a task-oriented parsing framework that integrates nearest-neighbor learning with a nearest-nearest approach. |
| Outcome: | The proposed model can be used to synthesize computer programs based on a natural-language prompt without additional data or specialized prompts. |
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)
Copied to clipboard
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)
Copied to clipboard
Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Xuemei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang
| Challenge: | i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data. |
| Approach: | They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data. |
| Outcome: | i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks. |
Meta-Information Guided Meta-Learning for Few-Shot Relation Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing meta-learning models rely on implicit instance statistics and are unreliability and weak interpretability. |
| Approach: | They propose a meta-information guided meta-learning framework that uses semantics to guide meta- learning . experimental results demonstrate the effectiveness of the proposed framework . |
| Outcome: | The proposed framework can establish connections between instance-based information and semantic-based data, enabling faster initialization and adaptation. |
A Learning-Exploring Method to Generate Diverse Paraphrases with Multi-Objective Deep Reinforcement Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Paraphrase generation is of great importance for many downstream tasks in natural language processing. |
| Approach: | They propose a method to generate sentences as learning objectives from the learned data distribution and employ reinforcement learning to combine these new learning objectives for model training. |
| Outcome: | The proposed method gains significant diversity and improves generation quality over state-of-the-art datasets. |
Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension (2022.acl-long)
Copied to clipboard
Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer
| Challenge: | Existing QA datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. |
| Approach: | They propose to use FairytaleQA to generate 10,580 questions based on 278 children-friendly stories to assess model's fine-grained learning skills. |
| Outcome: | The proposed dataset consists of 10,580 questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations. |
StorySparkQA: Expert-Annotated QA Pairs with Real-World Knowledge for Children’s Story-Based Learning (2024.emnlp-main)
Copied to clipboard
Jiaju Chen, Yuxuan Lu, Shao Zhang, Bingsheng Yao, Yuanzhe Dong, Ying Xu, Yunyao Li, Qianwen Wang, Dakuo Wang, Yuling Sun
| Challenge: | Existing story reading systems fail to capture the nuances of how education experts think when conducting interactive story reading activities. |
| Approach: | They propose to use existing question-answering (QA) datasets to capture experts' annotations and thinking process to construct a story-based annotation framework. |
| Outcome: | The proposed framework captures experts’ annotations and thinking process and can be used to generate 5, 868 expert-annotated QA pairs with real-world knowledge. |
Walking in Others’ Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Bias (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing prompting methods that require white-box access to the model or substantial training fail to simultaneously lessen toxicity and bias. |
| Approach: | They propose a strategy that encourages LLMs to integrate diverse human perspectives and self-regulate their responses by incorporating diverse human viewpoints. |
| Outcome: | The proposed approach can significantly diminish toxicity (up to 89%) and bias (up 73%) in LLMs’ responses. |
A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for machine reading comprehension lack evidence labels for training models. |
| Approach: | They propose a method which supervises the evidence extractor with auto-generated evidence labels in an iterative process. |
| Outcome: | The proposed method improves on three MRC tasks on seven datasets. |
AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning (2025.emnlp-demos)
Copied to clipboard
Zhong Zhang, Yaxi Lu, Yikun Fu, Yupeng Huo, Shenzhi Yang, Yesai Wu, Han Si, Xin Cong, Haotian Chen, Yankai Lin, Xie Xie, Wei Zhou, Wang Xu, Zhou Su, Zhongwu Zhai, Xiaoming Liu, null Meiyudong, Jianming Xu, Hongyan Tian, Chongyi Wang, Chi Chen, Yuan Yao, Zhiyuan Liu, Maosong Sun
| Challenge: | Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs. |
| Approach: | They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction. |
| Outcome: | The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI. |
Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction (2026.findings-acl)
Copied to clipboard
Xu Shen, Qi Zhang, Song Wang, Zhen Tan, Xinyu Zhao, Laura Yao, Vaishnav Tadiparthi, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Kwonjoon Lee, Tianlong Chen
| Challenge: | Large Language Model based multi-agent systems (MAS) excel at collaborative problem solving but remain brittle to cascading errors. |
| Approach: | They propose a metacognitive framework that enables step-level error detection and self-correction in Large Language Model based multi-agent systems (MAS) . |
| Outcome: | The proposed framework outperforms baselines on the Who When benchmark and delivers consistent gains on AgentErrorBench. |
Improving Large-scale Paraphrase Acquisition and Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing Twitter-based paraphrase datasets lack quality definitions for identification and generation tasks. |
| Approach: | They propose to use two separate definitions of paraphrase for identification and generation tasks in existing Twitter-based paraphrase datasets. |
| Outcome: | The proposed model achieves state-of-the-art performance of 84.2 F1 for automatic paraphrase identification compared to other models fine-tuned on other corpora such as Quora, MSCOCO, and ParaNMT. |
Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that neural natural language processing models are vulnerable to backdoor attacks. |
| Approach: | They propose to inject neural models with backdoors activated by word substitution . their results raise a serious alarm to the security of NLP models, they argue . |
| Outcome: | The proposed backdoors are activated by a learnable combination of word substitution and exhibit higher invisibility than previous methods. |
DIDS: Domain Impact-aware Data Sampling for Large Language Model Training (2025.emnlp-main)
Copied to clipboard
Weijie Shi, Jipeng Zhang, Yaguang Wu, Jingzhi Fang, Shibo Zhang, Yao Zhao, Hao Chen, Ruiyuan Zhang, Yue Cui, Jia Zhu, Sirui Han, Jiajie Xu, Xiaofang Zhou
| Challenge: | Existing approaches for optimizing domain-level sampling strategies struggle with maintaining intra-domain consistency and accurately measuring domain impact. |
| Approach: | They propose to use a Fisher-Information Matrix-guided metric to measure domain impact to ensure intra-domain consistency and accuracy. |
| Outcome: | The proposed model achieves 3.4% higher average performance while maintaining comparable training efficiency. |
code-transformed: The Influence of Large Language Models on Code (2026.findings-eacl)
Copied to clipboard
| Challenge: | Using Large Language Models, code generation capabilities have transformed programming practices. |
| Approach: | They analyze 20,000 GitHub repositories linked to arXiv papers published between 2020 and 2025 . they identify measurable trends in the evolution of coding style that align with LLM-generated code . |
| Outcome: | The proposed study examines 20,000 GitHub repositories linked to arXiv papers . it finds that LLMs influence code style, and that they can be observed in real-world code . |
Ameli: Enhancing Multimodal Entity Linking with Fine-Grained Attributes (2024.eacl-long)
Copied to clipboard
| Challenge: | Experimental results show that understanding attributes of mentions from text descriptions and visual images plays a vital role in multimodal entity linking. |
| Approach: | They propose to integrate attributes into multimodal entity linking using a text-image-based knowledge base. |
| Outcome: | The proposed approach integrates attributes into disambiguation. |
Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations (2022.acl-long)
Copied to clipboard
Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang, Nan Duan
| Challenge: | Existing studies focus on coarse-grained response selection in retrieval-based dialogue systems. |
| Approach: | They propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations. |
| Outcome: | The proposed model improves over baseline methods on two datasets based on the Reddit comments dump and Twitter corpus compared with baseline methods. |
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)
Copied to clipboard
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen
| Challenge: | Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed . |
| Approach: | They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities. |
| Outcome: | The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech . |
A Survey of Post-Training Scaling in Large Language Models (2025.acl-long)
Copied to clipboard
Hanyu Lai, Xiao Liu, Junjie Gao, Jiale Cheng, Zehan Qi, Yifan Xu, Shuntian Yao, Dan Zhang, Jinhua Du, Zhenyu Hou, Xin Lv, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Large language models (LLMs) have demonstrated proficiency in understanding and generating human natural languages. |
| Approach: | They propose a framework for scaling large language models using supervised fine-tuning, RLxF and test-time compute methodologies. |
| Outcome: | The proposed model can be used to understand and generate human natural languages. |
MMDEND: Dendrite-Inspired Multi-Branch Multi-Compartment Parallel Spiking Neuron for Sequence Modeling (2025.acl-long)
Copied to clipboard
Kexin Wang, Yuhong Chou, Di Shang, Shijie Mei, Jiahong Zhang, Yanbin Huang, Man Yao, Bo Xu, Guoqi Li
| Challenge: | Vanilla spiking neurons are simplified from complex biological neurons with dendrites, soma, and synapses into single somatic compartments. |
| Approach: | They propose a multi-branch, multi-compartment parallel spiking dendritic neuron with a proportion-adjustable multi-branched structure that enables long-term temporal dependencies. |
| Outcome: | The proposed model achieves better long-sequence modeling capability with fewer parameters and lower energy consumption. |
Taking Actions Separately: A Bidirectionally-Adaptive Transfer Learning Method for Low-Resource Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to train NMT models rely on sparse parallel data . a variety of PC variants yield significant improvements for low-resource NMT . |
| Approach: | They propose to transfer well-trained NMT models to low-resource languages by bidirectionally-adaptive learning strategy . they divide inner constituents of Parent encoder into two "teams" aiming to adapt to characteristics of low- and high-resourced languages . |
| Outcome: | The proposed method improves on low-resource NMT models with a variety of PC variants. |
Word-level Commonsense Knowledge Selection for Event Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Event Detection (ED) is a task of automatically extracting multi-class trigger words . Xie and Tu, 2022, use a Context-specific Knowledge Selector to select commonsense knowledge of words based on living contexts . |
| Approach: | They use a Context-specific Knowledge Selector to select the exact commonsense knowledge of words from a large knowledge base. |
| Outcome: | The proposed approach achieves the F1-score of about 78.3% on the ACE-2005 dataset. |
Detoxifying Large Language Models via Knowledge Editing (2024.acl-long)
Copied to clipboard
Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, Huajun Chen
| Challenge: | Existing methods to detoxify Large Language Models (LLMs) are limiting, but knowledge editing can be effective. |
| Approach: | They propose a baseline method to detoxify Large Language Models (LLMs) they propose supervised fine-tuning and reinforcement learning from human feedback (RLHF) |
| Outcome: | The proposed method reduces toxicity of large language models with one instance of tuning . it reduces the toxicity, while minimizing the toxins, the authors show . |
LaDiC: Are Diffusion Models Really Inferior to Autoregressive Counterparts for Image-to-Text Generation? (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing models for text-to-image generation have been underperforming in image-totext generation tasks. |
| Approach: | They propose a framework that uses a split BERT to create a dedicated latent space for captions and integrates a regularization module to manage varying text lengths. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the MS COCO dataset with 38.2 BLEU@4 and 126.2 CIDEr . |
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)
Copied to clipboard
| Challenge: | OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch . |
| Approach: | OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment. |
| Outcome: | OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment . |
IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing quantization methods are compromising performance of large language models (LLMs) despite their high computational intensity, LLMs are still demanding intensive computation. |
| Approach: | They propose to generate the KV cache of pivot tokens losslessly from the full-precision model. |
| Outcome: | The proposed method generates the KV cache of pivot tokens losslessly from the full-precision model with no extra inference overhead. |
Balanced Joint Adversarial Training for Robust Intent Detection and Slot Filling (2020.coling-main)
Copied to clipboard
| Challenge: | Existing joint models for intent detection and slot filling show insufficient robustness . however, some small changes of inputs can fool the models to produce wrong predictions . |
| Approach: | They propose a joint adversarial training model that generates adversarials to attack the joint model and trains the model to defend against the adversarial examples. |
| Outcome: | The proposed model achieves significantly higher scores and improves robustness on two datasets. |
Data Contamination Can Cross Language Barriers (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect contamination of public benchmarks are too superficial to reflect deeper forms of contamination. |
| Approach: | They propose generalization-based approaches to unmask a cross-lingual form of contamination that inflates LLMs’ performance while evading current detection methods. |
| Outcome: | The proposed model outperforms existing detection methods while avoiding contamination of public benchmarks in the pre-training data. |
LLaSA: Large Language and Structured Data Assistant (2025.naacl-long)
Copied to clipboard
Yao Xu, Shizhu He, Jiabei Chen, ZengXiangrong ZengXiangrong, Bingning Wang, Guang Liu, Jun Zhao, Kang Liu
| Challenge: | Structured knowledge grounding (SKG) tasks are a key part of many NLP applications. |
| Approach: | They propose a framework for enhancing LLMs' ability to handle structured data . they represent various types of structured data in a unified hypergraph format . |
| Outcome: | The proposed framework outperforms existing methods on SKG tasks using LoRA finetuning. |
EducationQ: Evaluating LLMs’ Teaching Capabilities Through Multi-Agent Dialogue Framework (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used as educational tools, yet evaluating their teaching capabilities remains challenging due to the resource-intensive nature of teacher-student interactions. |
| Approach: | They propose a multi-agent dialogue framework that efficiently assesses teaching capabilities through simulated dynamic educational scenarios. |
| Outcome: | The proposed framework outperforms open-source models on 1,498 questions across 13 disciplines and 10 difficulty levels on 1,400 questions. |
Awakening Augmented Generation: Learning to Awaken Internal Knowledge of Large Language Models for Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies indicate that Large Language Models model rich knowledge, but it is often not activated and awakened. |
| Approach: | They propose a framework that leverages richer context to enhance question answering . Explicit awakening fine-tunes a context generator to create a synthetic, compressed document that functions as symbolic context. |
| Outcome: | The proposed framework mimics the human ability to answer questions using only thinking and recalling to compensate for knowledge gaps. |
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (D18-1)
Copied to clipboard
| Challenge: | Empirical results show that even the most competitive few-shot learning models struggle on this task, especially as compared with humans. |
| Approach: | They propose a Few-Shot Relation Classification Dataset consisting of 70, 000 sentences on 100 relations derived from Wikipedia and annotated by crowdworkers. |
| Outcome: | The proposed methods perform well on the most competitive few-shot learning models, especially as compared with humans. |
Incorporating Image Matching Into Knowledge Acquisition for Event-Oriented Relation Recognition (C18-1)
Copied to clipboard
| Challenge: | Event relation recognition is a challenging language processing task because the query events are selected from different paragraphs in a document or even different documents, so there is lack of explicit clue. |
| Approach: | They propose to use image processing to acquire similar event instances and use image matching to approximate calculation between events. |
| Outcome: | The proposed model performs comparable to CNN while slightly better than LSTM on the ACE-R2 corpus. |
SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)
Copied to clipboard
Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li
| Challenge: | Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy. |
| Approach: | They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration. |
| Outcome: | The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent . |
Optimizing Reasoning for Text-to-SQL with Execution Feedback (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel in many reasoning tasks, but their ability to leverage Chain-of-Thought (CoT) reasoning remains underexplored. |
| Approach: | They propose a framework that iteratively optimizes open-source LLMs by combining CoT reasoning with off-policy and on-poly DPO, relying solely on execution accuracy as feedback. |
| Outcome: | The proposed framework improves execution accuracy on BIRD and Spider datasets. |
Smart “Chef”: Verifying the Effect of Role-based Paraphrasing for Aspect Term Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Aspect Term Extraction (ATE) is a task of automatically extracting aspect terms from sentences. |
| Approach: | They propose to automatically rewrite sentences from virtual experts with different roles . they leverage ChatGPT to determine virtual experts in the considered domains . |
| Outcome: | The proposed method can be used to expand the predictions obtained on the original sentences without retraining or fine-tuning the baseline extractors. |
PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering (2026.acl-long)
Copied to clipboard
Xiangfeng Wang, Hangyu Guo, Yanlin Lai, Mitt Huang, Liang Zhao, Chengyuan Yao, Yinmin Zhang, Qi Han, null Xiaoxiaoren, Chun Yuan, Tong Xu, Zheng Ge, Xiangyu Zhang, Daxin Jiang
| Challenge: | Current outcome-centric verification paradigms neglect potential errors in the derivation process. |
| Approach: | They propose a process-aware RLVR training paradigm utilizing verifiers selected via **PRIME**. |
| Outcome: | The proposed approach outperforms the baseline verification paradigm on AIME24, AIME25, and Beyond-AIME models. |
Enhancing LLM-based Hatred and Toxicity Detection with Meta-Toxic Knowledge Graph (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to address toxicity issues with large language models are inadequate . lack of domain-specific knowledge leads to false negatives and excessive sensitivity to toxic speech limits freedom of speech. |
| Approach: | They propose a method that leverages graph search on a meta-toxic knowledge graph to enhance hatred and toxicity detection. |
| Outcome: | The proposed method lowers false positive rate and improves toxicity detection performance in out-of-domain scenarios. |
Assist Non-native Viewers: Multimodal Cross-Lingual Summarization for How2 Videos (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing multimodal summarization methods are limited to monolingual videos . a proposed task aims to generate cross-lingual summaries from multimodal inputs . |
| Approach: | They propose a task to generate cross-lingual summaries from multimodal inputs of videos . they propose fusion network that integrates multimodal and cross-linguistic information . |
| Outcome: | The proposed task outperforms existing methods on a reorganized How2 dataset on the reorganized How2 data set. |
Cut the Deadwood Out: Backdoor Purification via Guided Module Substitution (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Model NLP models are often trained on datasets from untrusted platforms, posing significant risks of data poisoning attacks. |
| Approach: | They propose a retraining-free method that selectively replaces modules in the victim model based on a trade-off signal between utility and backdoor. |
| Outcome: | The proposed method outperforms even the strongest defense baseline against challenging attacks like LWS. |
Agentic Verification for Ambiguous Query Disambiguation (2026.findings-acl)
Copied to clipboard
Youngwon Lee, Seung-won Hwang, Ruofan Wu, Feng Yan, Danmei Xu, Moutasem Akkad, Zhewei Yao, Yuxiong He
| Challenge: | Prior Diversify-then-Verify pipelines generate interpretations and then retrieve evidence . ambiguous queries require RAG to disambiguate into interpretations that can be answered from corpus . |
| Approach: | They propose a novel approach that unifies diversification with verification by integrating retriever relevance and generator answerability feedback early. |
| Outcome: | The proposed approach improves grounding-aware F1 by 23% over baselines across multiple LLMs. |
Words Worth a Thousand Pictures: Measuring and Understanding Perceptual Variability in Text-to-Image Generation (2024.emnlp-main)
Copied to clipboard
Raphael Tang, Crystina Zhang, Lixinyu Xu, Yao Lu, Wenyan Li, Pontus Stenetorp, Jimmy Lin, Ferhan Ture
| Challenge: | Current diffusion models do not cover recent models, thus we curate three test sets for evaluation. |
| Approach: | They propose a human-calibrated measure of variability in a set of images bootstrapped from existing image-pair perceptual distances. |
| Outcome: | The proposed model outperforms nine baselines by 18 points in accuracy and matches graded human judgements 78% of the time. |
Open Hierarchical Relation Extraction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing OpenRE methods cast different relation types in isolation without considering their hierarchical dependency. |
| Approach: | They propose a framework to establish bidirectional connections between OpenRE and relation hierarchies by integrating hierarchy information into relation representations. |
| Outcome: | The proposed framework outperforms state-of-the-art models on relation clustering and hierarchy expansion. |
ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation (2021.acl-demo)
Copied to clipboard
Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan
| Challenge: | Existing models for pre-training are not convenient for users to find and set them up. |
| Approach: | They propose to extend ProphetNet into other domains and languages by pre-training models . they pre-train a cross-lingual generation model ProphetNet-Multi and a Chinese generation model . |
| Outcome: | The proposed models achieve new state-of-the-art on 10 benchmarks. |
NoteChat: A Dataset of Synthetic Patient-Physician Conversations Conditioned on Clinical Notes (2024.findings-acl)
Copied to clipboard
| Challenge: | NoteChat is a cooperative multi-agent framework for generating patient-physician dialogues . evaluator finds it outperforms state-of-the-art models for generating clinical notes . clinical documentation is largely done by physicians at both steps . |
| Approach: | They propose a cooperative multi-agent framework leveraging Large Language Models to generate patient-physician dialogues. |
| Outcome: | The proposed framework outperforms state-of-the-art models for generating clinical notes . it can engage patients directly and help clinical documentation, a leading cause of physician burnout . |
WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, Lihong Li
| Challenge: | Existing work on reinforcement learning has focused on single-turn tasks such as solving math problems. |
| Approach: | They propose a framework that learns directly from online interactions by asynchronously generating diverse trajectories, guided by binary rewards depending on task success. |
| Outcome: | Experiments on the WebArena-Lite benchmark show that the framework outperforms state-of-the-art methods and strong proprietary models. |
Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data (D19-1)
Copied to clipboard
| Challenge: | Existing methods to extract relational facts from open domain corpora are time-consuming and human-intensive. |
| Approach: | They propose a framework to learn similarity metrics of relations from labeled data . they propose to transfer relational knowledge to identify novel relations in unlabeled data. |
| Outcome: | Experiments on two real-world datasets show that the proposed framework improves compared with state-of-the-art methods. |
Generative Frame Sampler for Long Video Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing video large language models (LMMs) employ an impedance of thousands of frames to understand long videos. |
| Approach: | They propose a plug-and-play module integrated with VideoLLMs to facilitate efficient lengthy video perception. |
| Outcome: | The proposed module boosts the performance of open-source VideoLLMs and proprietary assistants on long-form video benchmarks. |
AEQ-Bench: Measuring Empathy of Omni-Modal Large Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on cognitive abilities, such as knowledge retrieval, complex reasoning, and instruction following, largely overlooking empathy evaluation. |
| Approach: | They propose to benchmark two core empathetic capabilities of omnimodal large models (OLMs) generating empatries by comprehending affective cues from multi-modal inputs and judging empathy of audio responses without relying on text transcription. |
| Outcome: | The proposed benchmark outperforms existing models with audio output capabilities but is unreliable for evaluating fine-grained paralinguistic expressiveness. |
How to Make LMs Strong Node Classifiers? (2026.findings-eacl)
Copied to clipboard
Zhe Xu, Kaveh Hassani, Si Zhang, Hanqing Zeng, Michihiro Yasunaga, Limei Wang, Dongqi Fu, Ning Yao, Bo Long, Hanghang Tong
| Challenge: | Language Models (LMs) are increasingly challenging the dominance of domain-specific models, such as Graph Neural Networks (GNNs) and Graph Transformers (GTs). |
| Approach: | They propose a novel approach that empowers off-the-shelf LMs to achieve performance comparable to state-of-the art (SOTA) GNNs on node classification tasks without requiring any architectural modifications. |
| Outcome: | The proposed approach outperforms existing GNNs on node classification tasks and is open-source upon publication. |
TAG : Type Auxiliary Guiding for Code Comment Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing code comment generation approaches ignore type information of interpretation of the code, e.g., operator, string, etc. Existing approaches ignore the type information due to the hierarchical dependence among the type. |
| Approach: | They propose an encoder-decoder framework which considers the source code as an N-ary tree with type information associated with each node. |
| Outcome: | The proposed framework is based on a Type Auxiliary Guiding encoder-decoder framework and a type-restricted Decoder to resolve training difficulties. |
RICO: Improving Accuracy and Completeness in Image Recaptioning via Visual Reconstruction (2025.emnlp-main)
Copied to clipboard
Yuchi Wang, Yishuo Cai, Shuhuai Ren, Sihan Yang, Linli Yao, Yuanxin Liu, Yuanxing Zhang, Pengfei Wan, Xu Sun
| Challenge: | Existing recaptioning methods suffer from inaccuracies due to missing fine-grained details. |
| Approach: | They propose a framework that refines captions through visual reconstruction using a text-to-image model and a visual reconstruction framework. |
| Outcome: | The proposed framework outperforms baselines on CapsBench and CompreCap by 10%. |
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)
Copied to clipboard
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
Syntactically-Informed Unsupervised Paraphrasing with Non-Parallel Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on syntactically controlled paraphrase generation rely on large-scale parallel data. |
| Approach: | They propose a syntactically-informed unsupervised paraphrasing model based on conditional variational auto-encoder which can generate texts in a specified syntastic structure. |
| Outcome: | The proposed model can generate diverse paraphrases with specified syntactic structure using non-parallel data. |
Multi-Lingual Question Generation with Language Agnostic Language Model (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing training data for question generation in English and Chinese is limited . a language-agnostic model is developed to learn the shared representation from several languages in a single architecture. |
| Approach: | They propose a language-agnostic language model which learns the shared representation from several languages in a single architecture. |
| Outcome: | The proposed model improves multi-lingual question generation over five languages. |
SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? (2025.emnlp-main)
Copied to clipboard
Yao Dou, Michel Galley, Baolin Peng, Chris Kedzie, Weixin Cai, Alan Ritter, Chris Quirk, Wei Xu, Jianfeng Gao
| Challenge: | Large language models (LLMs) are increasingly used in interactive applications, and human evaluation remains the gold standard for assessing their performance in multi-turn conversations. |
| Approach: | They propose to use large language models to simulate users for automatic assistant evaluation. |
| Outcome: | The proposed model outperforms human evaluations on two interactive tasks and achieves Spearman’s of 0.7 on both tasks. |
Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency (2026.acl-long)
Copied to clipboard
Haoming Xu, Ningyuan Zhao, Yunzhi Yao, Weihong Xu, Hongru Wang, Xinle Deng, Shumin Deng, Jeff Z. Pan, Huajun Chen, Ningyu Zhang
| Challenge: | Existing evaluations rely on point-wise confidence, which can mask brittle belief. |
| Approach: | They propose a measure of belief robustness that evaluates coherence across a conceptual neighborhood. |
| Outcome: | The proposed model is more resistant to interference than existing models. |
Toward Optimal LLM Alignments Using Two-Player Games (2025.findings-emnlp)
Copied to clipboard
Rui Zheng, Hongyi Guo, Zhihan Liu, Xiaoying Zhang, Yuanshun Yao, Xiaojun Xu, Zhaoran Wang, Zhiheng Xi, Tao Gui, Qi Zhang, Xuanjing Huang, Yang Liu, Hang Li
| Challenge: | Alignment of large language models (LLM) is a process that ensures the model’s responses to user prompts align with human intentions and social values. |
| Approach: | They propose an alignment method based on a two-agent game consisting of an adversarial agent and a defensive agent. |
| Outcome: | The proposed method improves on a two-agent game with an adversarial agent and a defensive agent. |
EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models (2024.acl-demos)
Copied to clipboard
Peng Wang, Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang, Shengyu Mao, Xiaohan Wang, Siyuan Cheng, Kangwei Liu, Yuansheng Ni, Guozhou Zheng, Huajun Chen
| Challenge: | Large Language Models (LLMs) suffer from knowledge cutoff or fallacy issues, which means they are unaware of unseen events or generate text with incorrect facts owing to outdated/noisy data. |
| Approach: | They propose an easy-to-use knowledge editing framework for Large Language Models that allows users to easily edit updated knowledge and adjust undesired behavior while minimizing the impact on unrelated inputs. |
| Outcome: | The proposed framework surpasses traditional fine-tuning in terms of reliability and generalization. |
Chart2Code53: A Large-Scale Diverse and Complex Dataset for Enhancing Chart-to-Code Generation (2025.emnlp-main)
Copied to clipboard
Tianhao Niu, Yiming Cui, Baoxin Wang, Xiao Xu, Xin Yao, Qingfu Zhu, Dayong Wu, Shijin Wang, Wanxiang Che
| Challenge: | Existing Chart2code-related training datasets suffer from limited scale, limited type coverage, and inadequate complexity. |
| Approach: | They propose to synthesize chart2code-related training datasets using web plotting code and chart images to address these challenges. |
| Outcome: | The proposed dataset exhibits the greatest diversity and higher complexity compared to other open-source Chart2code related datasets. |
Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences (2024.acl-long)
Copied to clipboard
Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jaehong Yoon, Taixi Lu, Fuxiao Liu, Gedas Bertasius, Mohit Bansal, Huaxiu Yao, Furong Huang
| Challenge: | Multimodal Large Language Models (MLLMs) have demonstrated proficiency in handling a variety of visual-language tasks, but their ability to extrapolate from image sequences has been less investigated. |
| Approach: | They propose a new benchmark to assess MLLMs’ sequential image reasoning abilities. |
| Outcome: | The proposed benchmark features 4,761 diverse image sequences with varying lengths. |
Random Smooth-based Certified Defense against Text Adversarial Attack (2024.findings-eacl)
Copied to clipboard
| Challenge: | Textual adversarial examples train models on the worst-case text generated by substituting words in original texts with synonyms, but due to the discrete word embedding representations, the large search space hinders the robust training efficiency. |
| Approach: | They propose to treat the word substitution as a continuous perturbation on the word embedding representation and apply random smooth techniques to approximate the word replacement operation. |
| Outcome: | The proposed method outperforms conventional methods and improves the robustness in training. |
Rose-SQL: Role-State Evolution Guided Structured Reasoning for Multi-Turn Text-to-SQL (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multi-turn Text-to-SQL tasks rely on unstable APIs or expensive fine-tuning. |
| Approach: | They propose a training-free framework that leverages small-scale LRMs through in-context learning to enable accurate context-dependent parsing. |
| Outcome: | The proposed framework outperforms in-context learning baselines at the 4B scale and surpasses state-of-the-art models at the 8B and 14B scales. |
Understanding the Repeat Curse in Large Language Models from a Feature Perspective (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models suffer from repetitive text generation, a phenomenon we refer to as the ”Repeat Curse”. |
| Approach: | They propose a method to induce and analyze the Repeat Curse in large language models by using mechanistic interpretability. |
| Outcome: | The proposed method induces and analyzes the Repeat Curse in large language models using mechanistic interpretability. |
SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection (2026.acl-industry)
Copied to clipboard
Zhiyong Cao, Dunqiang Liu, Qi Dai, Haojun Xu, Huai Yuen Khor, Hao Wang, Huan He, Yafei Liu, Ke Ma, Ruqian Shi, Sicheng Zhou, Sijia Yao
| Challenge: | High-quality data in training proactive dialogue agents is scarce, despite fine-tuning and reinforcement learning . a recent study has shown that the effectiveness of supervised fine-touring is limited by the lack of high-quality, domain-specific training data. |
| Approach: | They propose a framework for training recruitment proactive dialogue agents using a high-fidelity user simulator and a multi-dimensional evaluation framework based on Chain-of-Intention. |
| Outcome: | The proposed framework outperforms existing simulator-based data selection strategies in a real-world recruitment scenario. |
MoDE-CoTD: Chain-of-Thought Distillation for Complex Reasoning Tasks with Mixture of Decoupled LoRA-Experts (2024.lrec-main)
Copied to clipboard
| Challenge: | Current Chain-of-thought Distillation methods hinder CoT reasoning performance . student models are separately distilled from specific reasoning tasks . parameter update of student models severely harms CoT ability on unseen reasoning tasks. |
| Approach: | They propose a method which distills Chain-of-thought reasoning ability of large language models to much smaller student models. |
| Outcome: | The proposed method improves the reasoning ability of large language models on 14 datasets. |
Denoising Relation Extraction from Document-level Distant Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate auto-labeled sentences for relation extraction (RE) are difficult to extend to document-level relation extraction as noise from DS may be even multiplied in documents. |
| Approach: | They propose a pre-trained model which de-emphasizes noisy DS data via multiple pre-training tasks. |
| Outcome: | The proposed model can capture useful information from noisy data and achieve promising results on the large-scale DocRE benchmark. |
When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life (2026.findings-acl)
Copied to clipboard
Xinyue Lou, Xu Jinan, Jingyi Yin, Xiaolong Wang, Zhaolu Kang, null Liaoyouwei, Yixuan Wang, Xiangyu Shi, Fengran Mo, SU Yao, Kaiyu Huang
| Challenge: | Safety impact of Multimodal Large Language Models (MLLMs) on human behavior is evaluated in this study. |
| Approach: | They propose a safety-warning-based evaluation framework that encourages models to provide clear and informative safety warnings, rather than generic refusals. |
| Outcome: | The proposed safety-warning-based evaluation framework encourages models to provide clear and informative safety warnings, rather than generic refusals. |
Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding (2022.findings-emnlp)
Copied to clipboard
Jiahao Zhu, Daizong Liu, Pan Zhou, Xing Di, Yu Cheng, Song Yang, Wenzheng Xu, Zichuan Xu, Yao Wan, Lichao Sun, Zeyu Xiong
| Challenge: | Existing methods for temporal sentence grounding ignore two crucial issues . 1) Boundary-bias: the video downsampling process may lose these two frames . 2) Reasoning-biases: such incorrect new boundary frames lead to the reasoning bias . |
| Approach: | They propose a siamese sampling mechanism to generate additional contextual frames . they use a reasoning strategy to learn the inter-relationship among these frames a . |
| Outcome: | Extensive experiments demonstrate the effectiveness of a new siamese sampling network on three challenging datasets. |
Decomposed Prompt Tuning via Low-Rank Reparameterization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have achieved remarkable performance on various tasks. |
| Approach: | They propose a decomposed prompt tuning approach that utilizes low-rank matrices to initialize the soft prompt. |
| Outcome: | The proposed method significantly reduces the number of trainable parameters while maintaining effectiveness. |
Fine-Grained Legal Argument-Pair Extraction via Coarse-Grained Pre-training (2024.lrec-main)
Copied to clipboard
| Challenge: | Current methods conceptualize LAE as a supervised sentence-pair classification problem and necessitate extensive manual annotations. |
| Approach: | They propose a model that focuses on fine-grained alignment of argument pairs building upon coarse-grain complaint-defense pairs. |
| Outcome: | The proposed model outperforms baseline models by 3.7 and 2.4 points on average. |
Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) provide superior summarization quality, but their high computational resource requirements limit practical use applications. |
| Approach: | They evaluate 19 small language models for news summarization across 2,000 news samples . they find that top-performing models achieve comparable results to those of 70B LLMs . |
| Outcome: | The proposed models achieve comparable results to 70B LLMs while generating more concise summaries. |
TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios (2025.findings-acl)
Copied to clipboard
Xiaokang Zhang, Sijia Luo, Bohan Zhang, Zeyao Ma, Jing Zhang, Yang Li, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, Jifan Yu, Shu Zhao, Juanzi Li, Jie Tang
| Challenge: | TableLLM is a robust large language model capable of handling tabular data manipulation tasks. |
| Approach: | They propose a distant supervision method for training which includes a reasoning process extension strategy and a cross-way validation strategy. |
| Outcome: | The proposed model has 8 billion parameters and is capable of handling tabular data tasks. |
R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Reasoning is a fundamental capability underpinning text-to-image (T2I) generation. |
| Approach: | They propose a benchmark to rigorously assess reasoning-driven T2I generation. |
| Outcome: | Experiments with 16 representative T2I models show limited reasoning performance . a strong pipeline-based framework decouples reasoning and generation . |
Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation (2026.acl-long)
Copied to clipboard
Jiaju Chen, Yuxuan Lu, Xiaojie Wang, Huimin Zeng, Jing Huang, Jiri Gesi, Ying Xu, Bingsheng Yao, Dakuo Wang
| Challenge: | Existing "LLM-as-a-judge" evaluation frameworks are limited by persona descriptions and are not generalizable to other tasks. |
| Approach: | They propose a framework that can automatically construct multiple evaluator personas with distinct dimensions from relevant text documents and instantiate LLM agents with the persona. |
| Outcome: | The proposed framework can believably simulate human evaluators . it extracts stakeholders' diverse perspectives from the provided research papers and constructs personas for the agents . |
Cascaded Mutual Modulation for Visual Reasoning (D18-1)
Copied to clipboard
| Challenge: | Visual reasoning is a multi-step and compositional problem that requires intensive text-vision interactions. |
| Approach: | They propose a visual reasoning model that uses a feature-wise linear modulation technique to enable textual/visual pipelines to mutually control each other. |
| Outcome: | The proposed model outperforms existing models on visual reasoning benchmarks CLEVR and NLVR . it can generate a textual answer to a visual question answering problem with images . |
LENS: A Learnable Evaluation Metric for Text Simplification (2023.acl-long)
Copied to clipboard
| Challenge: | Existing metrics for text simplification are based on unitary or outdated models, making them unsuitable for this approach. |
| Approach: | They present a learnable evaluation metric for text simplification using language models . they also introduce a human evaluation framework that rates simplifications from several models a list-wise manner . |
| Outcome: | The proposed model correlates much better with human judgment than existing metrics. |
SLIM: Subtrajectory-Level Elimination for More Effective Reasoning (2025.findings-emnlp)
Copied to clipboard
Xifeng Yao, Chengyuan Ma, Dongyu Lang, Yinhao Ni, Zhiwei Xu, Huarui Xie, Zihao Chen, Guang Shen, Dandan Tu, Yi Bai, Changzheng Zhang
| Challenge: | Notable examples include OpenAI’s o1/o3/o4 series and DeepSeek-R1 . |
| Approach: | They develop a framework to identify suboptimal subtrajectories based on human-established criteria . they also use a sampling algorithm to select data whose reasoning process is free from suboptimally subtravertories to the highest degree . |
| Outcome: | The proposed method reduces the number of suboptimal subtrajectories by 25.9% during the inference process. |
VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering (2023.acl-demo)
Copied to clipboard
Zijun Yao, Yuanyong Chen, Xin Lv, Shulin Cao, Amy Xin, Jifan Yu, Hailong Jin, Jianjun Xu, Peng Zhang, Lei Hou, Juanzi Li
| Challenge: | Existing knowledge base question answering systems that parse natural language questions into knowledge oriented program language (KoPL) . |
| Approach: | They propose a knowledge base question answering system that integrates human into the loop to edit and debug queries. |
| Outcome: | The proposed system can debug and edit knowledge base questions on a million-entity-level . it provides auto-completion for its knowledge base schema and user interaction can fix a large portion of wrong KoPL programs to acquire the correct answer. |
Fine-grained Conversational Decoding via Isotropic and Proximal Search (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing text decoding methods are not tailoring for dialogue generation. |
| Approach: | They propose a fine-grained conversational decoding method that generates a semantic-concentrated response while maintaining informativeness and discrimination against the context. |
| Outcome: | The proposed method outperforms existing decoding strategies in the dialogue field across both automatic and human evaluation metrics. |
RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction (2026.findings-acl)
Copied to clipboard
Haonan Bian, Zhiyuan Yao, Sen Hu, Zishan Xu, Shaolei Zhang, Yifu Guo, Ziliang Yang, Xueran Han, Huacan Wang, Ronghao Chen
| Challenge: | Existing benchmarks focus on casual conversation or task-oriented dialogue, failing to capture “long-term project-oriented” interactions where agents must track evolving goals. |
| Approach: | They propose a benchmark that simulates the dynamic evolution of memory in real-world projects. |
| Outcome: | The proposed benchmarks simulate the dynamic evolution of memory in real-world projects. |
Prediction and Calibration: Complex Reasoning over Knowledge Graph with Bi-directional Directed Acyclic Graph Neural Network (2023.findings-acl)
Copied to clipboard
| Challenge: | Knowledge graphs (KGs) organize world knowledge as interlinked triples which describe entities and their relationships. |
| Approach: | They propose a bi-directional Directed Acyclic Graph neural network that splits the reasoning process into prediction and calibration. |
| Outcome: | The proposed model outperforms previous QE models on FB15k, FB16k-237, and NELL995 on prediction and calibration. |
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments (2026.acl-long)
Copied to clipboard
Alejandro Hernández-Cano, Alexander Hägele, Allen Hao Huang, Angelika Romanou, Antoni-Joan Solergibert, Barna Pásztor, Bettina Messmer, Dhia Garbaya, Eduard Frank Ďurech, Ido Hakimi, Juan Garcia Giraldo, Mete Ismayilzada, Negar Foroutan, Skander Moalla, Tiancheng Chen, Vinko Sabolčec, Yixuan Xu, Michael Aerni, Badr AlKhamissi, Inés Altemir Marinas, Mohammad Hossein Amani, Matin Ansaripour, Ilia Badanin, Harold Benoit, Emanuela Boros, Nicholas John Browning, Fabian Bösch, Maximilian Böther, Niklas Canova, Camille Challier, Clément Charmillot, Jonathan Coles, Jan Milan Deriu, Arnout Devos, Lukas Drescher, Daniil Dzenhaliou, Maud Ehrmann, Dongyang Fan, Simin Fan, Silin Gao, Miguel Gila, María Grandury, Diba Hashemi, Alexander Miserlis Hoyle, Jiaming Jiang, Mark Klein, Andrei Kucharavy, Anastasiia Kucherenko, Frederike Lübeck, Roman Machacek, Theofilos Ioannis Manitaras, Andreas Marfurt, Kyle Matoba, Simon Matrenok, Henrique Mendonça, Fawzi Roberto Mohamed, Syrielle Montariol, Luca Mouchel, Sven Najem-Meyer, Jingwei Ni, Gennaro Oliva, Matteo Pagliardini, Elia Palme, Andrei Panferov, Léo Paoletti, Marco Passerini, Ivan Pavlov, Auguste Poiroux, Kaustubh Ponkshe, Nathan Ranchin, Javier Rando, Mathieu Sauser, Jakhongir Saydaliev, Mukhammadali Sayfiddinov, Marian Schneider, Stefano Schuppli, Marco Scialanga, Andrei Semenov, Kumar Shridhar, Raghav Singhal, Anna Sotnikova, Alexander Sternfeld, Ayush Kumar Tarun, Paul Teiletche, Jannis Vamvas, Xiaozhe Yao, Hao Zhao, Alexander Ilic, Ana Klimovic, Andreas Krause, Caglar Gulcehre, David Rosenthal, Elliott Ash, Florian Tramèr, Joost VandeVondele, Livio Veraldi, Martin Rajman, Thomas C. Schulthess, Torsten Hoefler, Antoine Bosselut, Martin Jaggi, Imanol Schlag
| Challenge: | Apertus is a fully open suite of large language models (LLMs) designed to address responsibility shortcomings in today’s open model ecosystem, namely data responsibility and global representation. |
| Approach: | They propose to release a fully open suite of large language models (LLMs) that address data responsibility and global representation shortcomings in today’s open model ecosystem. |
| Outcome: | The proposed model is pretrained on openly available data and suppresses verbatim recall of data while retaining task performance. |
Thresh: A Unified, Customizable and Deployable Platform for Fine-Grained Text Evaluation (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing tools for fine-grained human evaluation lack adaptability to different domains or languages, or modify annotation settings according to user needs. |
| Approach: | They propose a unified platform for fine-grained evaluation that is customizable and deployable with a single YAML configuration file. |
| Outcome: | The proposed frameworks are based on a single YAML configuration file and can be easily extended to different domains or languages. |
MAPLE: Enhancing Review Generation with Multi-Aspect Prompt LEarning in Explainable Recommendation (2025.acl-long)
Copied to clipboard
Ching-Wen Yang, Zhi-Quan Feng, Ying-Jia Lin, Che Wei Chen, Kun-da Wu, Hao Xu, Yao Jui-Feng, Hung-Yu Kao
| Challenge: | Existing models that generate generic aspects do not provide personalized informative recommendations. |
| Approach: | They propose a model that integrates aspect category as another input dimension to facilitate memorizing fine-grained aspect terms. |
| Outcome: | The proposed model outperforms baseline model on restaurant review datasets in the restaurant domain. |
JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation (2026.findings-acl)
Copied to clipboard
Zhichao Shi, Xuhui Jiang, Chengjin Xu, Cangli Yao, Shengjie Ma, Yinghan Shen, Zixuan Li, Jian Guo, Yuanzhuo Wang
| Challenge: | Current evaluation methods for large language models rely on static benchmarks . limited knowledge coverage and fixed difficulties hinder the targeted optimizations resulting in superficial evaluations of LLMs - a problem that has been addressed by JudgeAgent . |
| Approach: | They propose a knowledge-driven and dynamic evaluation framework for large language models . judgeAgent leverages LLM agents equipped with context graphs to traverse knowledge structures . |
| Outcome: | The proposed framework can achieve comprehensive evaluations and facilitate effective model iterations. |
Global Context-enhanced Graph Convolutional Networks for Document-level Relation Extraction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to document-level relation extraction are difficult to establish direct connections between distant entity pairs. |
| Approach: | They propose a global context-enhanced Graph Convolutional Network model which captures rich global context information of entities in a document. |
| Outcome: | The proposed model captures rich global context information of entities in a document. |
Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments (2026.findings-acl)
Copied to clipboard
Junjie Ye, Changhao Jiang, Zhengyin Du, Yufei Xu, Xuesong Yao, Zhiheng Xi, Xiaoran Fan, Qi Zhang, Tao Gui, Xuanjing Huang, Jiecao Chen
| Challenge: | Currently, there are no efficient reinforcement learning (RL) frameworks specifically designed for tool use. |
| Approach: | They propose an automated environment construction pipeline that incorporates scenario decomposition, document generation, function integration, complexity scaling, and localized deployment to enable high-quality training environments without external tools. |
| Outcome: | The proposed framework significantly improves the models’ tool-use performance without degrading their general capabilities. |
Making RALM Robust to Irrelevant Contexts via Layer Knowledge Guided Attention (2025.findings-acl)
Copied to clipboard
Weijie Shi, Hao Chen, Jiaming Li, Yao Zhao, Yazhong Zhang, Qijin Chen, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Xiaofang Zhou
| Challenge: | Large language models (LLMs) face factual hallucination and knowledge obsolescence when tackling knowledge-intensive tasks. |
| Approach: | They propose a layer-knowledge guided attention method which harnesses the layer-wise knowledge of large language models to optimize per-layer attention on useful passages. |
| Outcome: | The proposed method outperforms existing methods on RALM benchmarks. |
xLAM: A Family of Large Action Models to Empower AI Agent Systems (2025.naacl-long)
Copied to clipboard
Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Quoc Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, Zhiwei Liu, Yihao Feng, Tulika Manoj Awalgaonkar, Rithesh R N, Zeyuan Chen, Ran Xu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | Autonomous agents powered by large language models (LLMs) have attracted significant research interest, but there are few standards for developing specialized models for agent tasks. |
| Approach: | They propose a series of large action models with dense and mixture-of-expert architectures that unifies, augments, and synthesizes diverse datasets to enhance agent generalizability and performance. |
| Outcome: | The proposed models outperform GPT-4, Claude-3, and many other models in terms of tool use and outperformed GPT-based models on multiple agent ability benchmarks. |
Knowledge Mechanisms in Large Language Models: A Survey and Perspective (2024.findings-emnlp)
Copied to clipboard
Mengru Wang, Yunzhi Yao, Ziwen Xu, Shuofei Qiao, Shumin Deng, Peng Wang, Xiang Chen, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen, Ningyu Zhang
| Challenge: | Using large language models, we can understand knowledge mechanisms in LLMs for learning, storage, utilization, and evolution. |
| Approach: | They propose to analyze knowledge mechanisms in Large Language Models (LLMs) they examine utilization, evolution, and the potential dark knowledge (hypothesis) they hope to help understand knowledge in LLMs and provide insights for future research . |
| Outcome: | The proposed model can be used to analyze the evolution of parametric knowledge in LLMs. |
Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems (2025.acl-long)
Copied to clipboard
| Challenge: | Existing reward models focus on human preferences, neglecting verifiable correctness signals. |
| Approach: | They propose a reward system that combines human preference rewards with verifiable correctness signals to provide reliable rewards. |
| Outcome: | The proposed reward agent significantly outperforms vanilla reward models on benchmarks and inference-time best-of-n searches on real-world tasks. |
EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Models (2025.emnlp-demos)
Copied to clipboard
Ziwen Xu, Shuxun Wang, Kewei Xu, Haoming Xu, Mengru Wang, Xinle Deng, Yunzhi Yao, Guozhou Zheng, Huajun Chen, Ningyu Zhang
| Challenge: | Large Language Models (LLMs) have demonstrated extraordinary capabilities, however, they may still generate unreliable or unsafe outputs. |
| Approach: | They propose a framework that allows plug-and-play adjustability for controlling Large Language Model (LLM) behaviors. |
| Outcome: | The framework is designed to enable plug-and-play adjustability for controlling Large Language Model (LLM) behaviors. |
i-Code Studio: A Configurable and Composable Framework for Integrative AI (2024.emnlp-demo)
Copied to clipboard
Yuwei Fang, Mahmoud Khademi, Chenguang Zhu, Ziyi Yang, Reid Pryzant, Yichong Xu, Yao Qian, Takuya Yoshioka, Lu Yuan, Michael Zeng, Xuedong Huang
| Challenge: | Existing frameworks for Integrative AI lack flexibility and composability to handle multimodal tasks. |
| Approach: | They propose a configurable framework for Integrative AI that orchestrates multiple pre-trained models to conduct complex multimodal tasks. |
| Outcome: | The proposed framework achieves impressive results on zero-shot multimodal tasks . it can communicate and personalize for users, and it can be used in a multimodal agent . |
iPET: An Interactive Emotional Companion Dialogue System with LLM-Powered Virtual Pet World Simulation (2025.acl-demo)
Copied to clipboard
Zheyong Xie, Shaosheng Cao, Zuozhu Liu, Zheyu Ye, Zihan Niu, Chonggang Lu, Tong Xu, Enhong Chen, Zhe Xu, Yao Hu, Wei Lu
| Challenge: | Existing approaches to role-playing emotional companion products lack sustained personalization and contextual adaptability, limiting their effectiveness in real-world settings. |
| Approach: | They propose a virtual pet agent that can enhance user engagement through rich, dynamic pet behaviors and interactions tailored to individual preferences. |
| Outcome: | The proposed system has been deployed in a real-world, non-commercial product for 200 days and has demonstrated its effectiveness in practical applications. |
Automatic and Human-AI Interactive Text Generation (with a focus on Text Simplification and Revision) (2024.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we focus on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and then generates a revision that is improved according to some specific criteria. |
| Approach: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks that takes a piece of text as input and generates a revision that is improved according to some specific criteria. |
| Outcome: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and generates a revision that is improved according to some specificcriteria. |
Improving Minimum Bayes Risk Decoding with Multi-Prompt (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate LLMs with a single ‘best’ prompt are unstable and sub-optimal in practice. |
| Approach: | They propose to decode multiple candidate generations from a prompt bank at inference-time and use Minimum Bayes Risk (MBR) to select a final output. |
| Outcome: | The proposed method improves MBR across a set of conditional generation tasks and models. |
ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)
Copied to clipboard
Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen
| Challenge: | Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models. |
| Approach: | They propose a dataset that provides rigorous evaluation of multi-hop tool use. |
| Outcome: | The proposed model achieves 49.04% accuracy across five model families. |
DAPE-BR: Distance-Aware Positional Encoding for Mitigating Object Hallucination in LVLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Vision–Language Models (LVLMs) suffer from object hallucination, generating descriptions for objects that are absent from the image, which undermines reliability and hinders real-world deployment. |
| Approach: | They propose a positional-alignment scheme that preserves pretrained weight order while globally—- visual–text distances, embeds an isotropic fused patch-distance metric, and applies a patch-delay causal mask to enforce spatial causality. |
| Outcome: | Extensive experiments on POPE, MMStar and SQA show that DAPE-BR reduces hallucinations and boosts performance. |
Reducing Privacy Risks in Online Self-Disclosures with Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Disclosure is a social media activity that can be rewarding but also poses privacy risks. |
| Approach: | They propose to detect and abstract online self-disclosures using a large corpus of 4.8K annotated disclosure spans and a language model to fine-tune for detection. |
| Outcome: | The proposed model can detect and abstract self-disclosures with 80% accuracy, on-par with GPT-3.5. |
Instructional Agents: Reducing Teaching Faculty Workload through Multi-Agent Instructional Design (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing AI-assisted educational tools focus on isolated tasks, but lack end-to-end workflows for instructional design. |
| Approach: | They propose a multi-agent large language model framework to automate end-to-end course material generation. |
| Outcome: | The proposed framework reduces development time and human workload while reducing human involvement. |
Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics (2026.acl-long)
Copied to clipboard
Ziwen Xu, Chenyan WU, Hengyu Sun, Haiwen Hong, Mengru Wang, Yunzhi Yao, Longtao Huang, Hui Xue, Shumin Deng, Zhixuan Chu, Huajun Chen, Ningyu Zhang
| Challenge: | Methods for controlling large language models (LLMs) are often studied in isolation, obscuring connections and making comparison difficult. |
| Approach: | They propose a preference-utility analysis that separates control effects into preference and utility, and measures both on a shared log-odds scale using polarity-paired contrastive examples. |
| Outcome: | The proposed approach improves preference while preserving utility. |
TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents (2026.findings-acl)
Copied to clipboard
Kai Li, Xuanqing Yu, Ziyi Ni, Yi Zeng, Yao Xu, Zheqing Zhang, Xin Li, Jitao Sang, Xiaogang Duan, Xuelei Wang, Chengbao Liu, Jie Tan
| Challenge: | Existing memory frameworks provide limited support for temporally structured information across hierarchical levels, leading to fragmented memories and unstable long-horizon personalization. |
| Approach: | They propose a temporal–hierarchical memory framework that organizes conversations through a Temporal Memory Tree. |
| Outcome: | The proposed framework outperforms baselines while reducing the recalled memory length by 52.20%. |
Search-in-Context: Efficient Multi-Hop QA over Long Contexts via Monte Carlo Tree Search with Dynamic KV Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multihop question answering (MHQA) over long contexts are often neglecting explicit reasoning or incurring expensive computational costs due to full-attention mechanisms over long contextuals. |
| Approach: | They propose a framework that integrates Monte Carlo Tree Search (MCTS) with dynamic key-value retrieval to enable iterative, context-aware reasoning. |
| Outcome: | The proposed framework integrates Monte Carlo Tree Search (MCTS) with dynamic key-value (KV) retrieval to enable iterative, context-aware reasoning. |
Towards Better Value Principles for Large Language Model Alignment: A Systematic Evaluation and Enhancement (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) show remarkable performance across tasks . alignment with human values is critical for their responsible development. |
| Approach: | They propose a framework that evaluates value principles along three desirable properties . they propose supervised fine-tuning, reinforcement learning-based approaches . |
| Outcome: | The proposed framework improves value principles along the three desirable properties of LLMs. |
Position Paper: Data-Centric AI in the Age of Large Language Models (2024.findings-emnlp)
Copied to clipboard
Xinyi Xu, Zhaoxuan Wu, Rui Qiao, Arun Verma, Yao Shu, Jingtan Wang, Xinyuan Niu, Zhenfeng He, Jiangwei Chen, Zijian Zhou, Gregory Kang Ruey Lau, Hieu Dao, Lucas Agussurja, Rachael Hwee Ling Sim, Xiaoqiang Lin, Wenyang Hu, Zhongxiang Dai, Pang Wei Koh, Bryan Kian Hsiang Low
| Challenge: | a paper proposes a data-centric perspective of AI research, focusing on large language models. |
| Approach: | They propose a data-centric viewpoint of AI research, focusing on large language models . they propose four scenarios centered around data, including data curation, attribution, knowledge transfer . |
| Outcome: | The proposed research focuses on large language models with data centric benchmarks . the proposed benchmarks can be used to develop new data curation methods . |
ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent Web (2026.acl-long)
Copied to clipboard
Zhiyuan Yao, Zishan Xu, Yifu Guo, Zhiguang Han, Cheng Yang, Shuo Zhang, Weinan Zhang, Xingshan Zeng, Weiwen Liu
| Challenge: | Existing routers that use hardcoded tools are limited by scalability and generality bottlenecks. |
| Approach: | They propose a pipeline for training history-aware routers to empower precise navigation in large-scale ecosystems. |
| Outcome: | The proposed pipeline can train routers with dynamic context understanding to create the plug-and-play Light Routing Agent. |
A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing dialogue systems have demonstrated impressive performance conducting fluent and natural-sounding conversations, but they are plagued by the Knowledge Hallucination problem. |
| Approach: | They propose a method that exploits the dialogue-knowledge interaction to reduce hallucination by using external knowledge resources to generate more informative responses. |
| Outcome: | The proposed method reduces hallucination without disrupting other dialogue performance while keeping adaptive to different generation models. |
GPT-4 Jailbreaks Itself with Near-Perfect Success Using Self-Explanation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using large language models to jailbreak is important for testing safety and security issues. |
| Approach: | They propose an approach that leverages the reflective capabilities of large language models for jailbreaking with only black-box access. |
| Outcome: | The proposed method achieves jailbreak success rates of 98% on GPT-4, 92% on GTP-4 Turbo, and 94% on Llama-3.1-70B in under 7 queries. |
Learning Structural Information for Syntax-Controlled Paraphrase Generation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Syntax-controlled paraphrase generation aims to produce paraphrase conform to given syntactic patterns. |
| Approach: | They propose a model that captures parent-child and sibling relations and a syntax encoder to capture alignment relations. |
| Outcome: | The proposed model achieves state-of-the-art in terms of semantic and syntactic quality on two popular benchmark datasets. |
DocRED: A Large-Scale Document-Level Relation Extraction Dataset (P19-1)
Copied to clipboard
Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun
| Challenge: | Existing relation extraction methods focus on extracting intra-sentence relations for single entities. |
| Approach: | They propose a relation extraction dataset from Wikipedia and Wikidata with three features . document-level relation extraction is a task to identify relational facts between entities . |
| Outcome: | The proposed dataset is the largest human-annotated dataset for document-level RE from plain text. |
SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network (2024.acl-long)
Copied to clipboard
| Challenge: | SpikeVoice performs high-quality Text-To-Speech (TTS) via SNN . major obstacle to using SNN for such generative tasks lies in the demand for models to grasp long-term dependencies. |
| Approach: | They propose a brain-inspired Spiking Neural Network (SNN) which performs high-quality Text-To-Speech (TTS) via SNN and explores the potential of SNN to "speak". |
| Outcome: | The proposed model achieves comparable results to Artificial Neural Networks (ANN) with only 10.5% energy consumption of ANN. |