Papers by Xu Shi
ProUIE: A Macro-to-Micro Progressive Learning Method for LLM-based Universal Information Extraction (2026.findings-acl)
Copied to clipboard
Wenda Liu, Song Zhigang, Shuai Nie, Guangyao Liu, Lisung Chen, Binyu Yang, Yaran Chen, Peng Zhou, Hongzhen Wang, Yuchen Liu, Wenyue Hu, Jiaming Xu, Runyu Shi, Ying Huang
| Challenge: | ProUIE improves universal information extraction (UIE) without external information . many LLM-based methods rely on extra schema cues, external resources or complex alignment and verification pipelines . |
| Approach: | They propose a Macro-to-Micro progressive learning approach that improves UIE without external information. |
| Outcome: | ProUIE outperforms instruction-tuned baselines on average for NER and RE while using a smaller backbone. |
REAM♯: An Enhancement Approach to Reference-based Evaluation Metrics for Open-domain Dialog Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation metrics for open-domain dialogue systems are limited by the diversity of possible outcomings. |
| Approach: | They propose a method to augment a reference set to improve reliability . they propose BLEU to measure similarity between a predicted response and a small set of references . |
| Outcome: | The proposed model improves the reliability of reference-based metrics with augmented reference sets. |
Cognitive-Uncertainty Guided Knowledge Distillation for Accurate Classification of Student Misconceptions (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for identifying student misconceptions overlook students' reasoning processes, authors report . |
| Approach: | They propose a knowledge distillation framework that mines high-value samples from existing data. |
| Outcome: | The proposed framework outperforms sota LLM and standard fine-tuned 72B models on cross-topic tests. |
MIBench: Evaluating Multimodal Large Language Models over Multiple Images (2024.emnlp-main)
Copied to clipboard
Haowei Liu, Xi Zhang, Haiyang Xu, Yaya Shi, Chaoya Jiang, Ming Yan, Ji Zhang, Fei Huang, Chunfeng Yuan, Bing Li, Weiming Hu
| Challenge: | Existing benchmarks and MLLMs focus on single-image input scenarios, leaving performance of ML models when handling multiple images underexplored. |
| Approach: | They propose a benchmark to evaluate fine-grained abilities of multimodal large language models in multi-image scenarios. |
| Outcome: | The proposed benchmark categorizes the multi-image abilities into three scenarios: MII, MKS and MIC. |
L2Dir: Integrating L_2-Norm and Directional Alignment for Unsupervised Contrastive Representation Learning in Multimodal Retrieval (2026.acl-long)
Copied to clipboard
Tianyu Zong, Rui Dai, Hongzhu Yi, Yuanxiang Wang, Zhenghao Zhang, Zhenyu Guan, Yujia Yang, Bingkang Shi, Yueyang Ding, Xiangxiang Chu, Kaikui Liu, Jungang Xu
| Challenge: | Existing approaches to multimodal representation learning focus on directional alignment and embedding magnitudes (L2-norm) however, these methods often fail to account for the intrinsic role of L2-norm in the contrastive process. |
| Approach: | They propose a plug-and-play framework that optimizes L2-norm alignment and Directional consistency jointly. |
| Outcome: | The proposed framework achieves consistent and significant performance gains over established baselines across 95 tasks using UniIR and VLM2Vec-V2 frameworks. |
Towards Self-Evolving Agents: Enabling Autonomy through Interactive Experience Refinement (2026.findings-acl)
Copied to clipboard
Cheng Yang, Xuemeng Yang, Licheng Wen, Daocheng Fu, Jianbiao Mei, Rong Wu, Pinlong Cai, Yufan Shen, Nianchen Deng, Jia Xu, Botian Shi, Yu Qiao, Haifeng Li
| Challenge: | Large Language Models struggle with complex, multi-step operational tasks because they remain static during inference and cannot learn from past experience. |
| Approach: | They propose a framework that organizes cross-domain insights to facilitate orchestration of long-horizon workflows. |
| Outcome: | The proposed framework outperforms existing methods on the TAC productivity benchmark and shows strong cross-task transferability. |
Self-Criticism: Aligning Large Language Models with their Understanding of Helpfulness, Honesty, and Harmlessness (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models are useful, honest, harmless (HHH) however, RLHF requires high hardware resources and human efforts. |
| Approach: | They propose a framework that allows LLMs to align themselves with HHH . they use IF and reinforcement learning from human feedback to fine-tune their models . |
| Outcome: | The proposed framework achieves similar performance to RLHF and human-generated models with a minimal alignment tax. |
A Survey on Zero Pronoun Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Zero pronouns (ZPs) are often omitted in pro-drop languages, but should be recalled in non-pro-drop language. |
| Approach: | They propose to analyze the literature on zero pronoun translation after the neural revolution . they uncover that data limitation causes learning bias in languages and domains . |
| Outcome: | The proposed method and methods are compared to other models and evaluation metrics on different benchmarks. |
MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehension (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for Referring Expression Comprehension (REC) lack specific domain abilities for precise local visual perception and visual-language alignment. |
| Approach: | They propose a framework for Parameter-Efficient Transfer Learning to localize a visual region via natural language using a prior-guided prior. |
| Outcome: | The proposed framework achieves the best accuracy compared to the current methods with only 1.41% tunable backbone parameters. |
FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data (2024.emnlp-industry)
Copied to clipboard
Haoran Sun, Renren Jin, Shaoyang Xu, Leiyu Pan, null Supryadi, Menglong Cui, Jiangcun Du, Yikun Lei, Lei Yang, Ling Shi, Juesi Xiao, Shaolin Zhu, Deyi Xiong
| Challenge: | Large language models exhibit significant performance discrepancies between high- and low-resource languages. |
| Approach: | They present an open-source multilingual LLM with 8 billion parameters and a multilingual instruction dataset. |
| Outcome: | The proposed model achieves consistent multilingual representations across languages. |
Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset (2022.lrec-1)
Copied to clipboard
Tiezheng Yu, Rita Frieske, Peng Xu, Samuel Cahyawijaya, Cheuk Tung Yiu, Holy Lovenia, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | In this paper, we address the problem of data scarcity for the Hong Kong Cantonese language . due to the popularization of deep learning, ASR technology has led to a significant improvement in recognizing many languages. |
| Approach: | They propose to use a dataset to analyze the data available for the Hong Kong Cantonese language . they use zh-HK as a source and a state-of-the-art ASR model to build a powerful model . |
| Outcome: | The proposed model improves on the biggest existing dataset, Common Voice zh-HK. |
Reinforced Efficient Reasoning via Semantically Diverse Exploration (2026.acl-long)
Copied to clipboard
Ziqi Zhao, Zhaochun Ren, Jiahong Zou, Liu Yang, Zhiwei Xu, Xuri Ge, Zhumin Chen, Xinyu Ma, Daiting Shi, Shuaiqiang Wang, Dawei Yin, Xin Xin
| Challenge: | Existing methods for reinforcement learning with verifiable rewards suffer from limited exploration diversity and inefficient reasoning. |
| Approach: | They propose a method that rewards concise and correct reasoning while penalizing unnecessarily long reasoning chains. |
| Outcome: | Extensive experiments on Qwen and Llama models validate the effectiveness and efficiency of ROSE. |
SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator (2026.acl-long)
Copied to clipboard
Xueyang Zhou, Weidong Wang, Lin Lu, Jiawen Shi, Guiyao Tie, Xu Yongtian, Lixing Chen, Pan Zhou, Neil Zhenqiang Gong, Lichao Sun
| Challenge: | SafeAgent improves agent safety through fully automated synthetic data generation. |
| Approach: | They propose a framework that improves agent safety through fully automated synthetic data generation. |
| Outcome: | The proposed framework outperforms closed-source models on two safety benchmarks and one real-world task. |
Extracting a Knowledge Base of COVID-19 Events from Social Media (2022.coling-1)
Copied to clipboard
| Challenge: | a flood of COVID-19 related information has appeared on social media since December 2019 . this includes reports on public figures who have tested positive/negative for the virus . |
| Approach: | They construct a corpus of 10,000 tweets with annotated public reports of five COVID-19 events, using slot-filling questions to fill in slots. |
| Outcome: | The proposed method can be quickly applied to develop knowledge bases for new domains in response to emerging crises, including natural disasters or future disease outbreaks. |
Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems (2022.emnlp-main)
Copied to clipboard
| Challenge: | Numerical Question Answering is the task of answering questions that require numerical capabilities. |
| Approach: | They propose to conduct numerical capability diagnosis on a series of Numerical Question Answering systems and datasets. |
| Outcome: | The proposed approach relieves existing systems’ lack of robust numerical capabilities. |
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)
Copied to clipboard
Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy. |
| Approach: | They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses . |
| Outcome: | The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples. |
IMTLab: An Open-Source Platform for Building, Evaluating, and Diagnosing Interactive Machine Translation Systems (2023.emnlp-main)
Copied to clipboard
Xu Huang, Zhirui Zhang, Ruize Gao, Yichao Du, Lemao Liu, Guoping Huang, Shuming Shi, Jiajun Chen, Shujian Huang
| Challenge: | Existing systems that use a left-to-right completion paradigm are inefficient and expensive. |
| Approach: | They propose an open-source end-to-end interactive machine translation system platform . they propose to use a prefix-constrained decoding approach to achieve end- to-end evaluation . |
| Outcome: | The proposed system can guarantee high-quality, error-free translations . it uses prefix-constrained decoding and improves on previous systems . |
ACR: Adaptive Context Refactoring via Context Refactoring Operators for Multi-Turn Dialogue (2026.findings-acl)
Copied to clipboard
Jiawei Shen, Jia Zhu, Hanghui Guo, Weijie Shi, Yue Cui, Qingyu Niu, Guoqing Ma, Jingjiang Liu, Yidan Liang, Yilin Wang, Shimin Di, Jiajie Xu
| Challenge: | Existing approaches to multi-turn dialogues lack contextual consistency and dependencies, and models struggle to maintain factual faithfulness as interaction turns increase. |
| Approach: | They propose an adaptive context refactoring framework that monitors and reshapes the interaction history to mitigate contextual inertia and state drift. |
| Outcome: | The proposed model outperforms baselines while reducing token consumption. |
Improving Image Captioning with Better Use of Caption (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to image captioning focus on visual attention, but many do not. |
| Approach: | They propose a framework that explores semantics available in captions and leverages that to enhance both image representation and caption generation. |
| Outcome: | The proposed framework outperforms baselines on the MSCOCO dataset and is state-of-the-art under a wide range of evaluation metrics. |
TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators (2025.findings-acl)
Copied to clipboard
Jianling Li, ShangZhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, WangHaojie WangHaojie, Jianrong Wang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Triton is a high-level Python-like programming language for building efficient GPU kernels. |
| Approach: | They propose a TritonBench benchmark that provides a comprehensive evaluation of Tritonic operators on widely deployed GPUs. |
| Outcome: | The proposed benchmarks show that current LLMs struggle to generate efficient Triton operators on widely deployed GPUs aligned with industry applications. |
Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems (2025.findings-emnlp)
Copied to clipboard
Minghang Zhu, Zhengliang Shi, Zhiwei Xu, Shiguang Wu, Lingjie Wang, Pengjie Ren, Zhaochun Ren, Zhumin Chen
| Challenge: | Existing methods for fine-tuning agents are often inadequate . a multi-agent system can solve complex tasks by dividing responsibilities among specialized agents . |
| Approach: | a new framework is proposed to improve agents collaboration through iterative alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on held-in and held-out tasks. |
From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models (2026.acl-long)
Copied to clipboard
Ling Shi, Xinwei Wu, Xiaohu Zhao, Hao Wang, Heng Liu, Yangyang Liu, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo
| Challenge: | Recent research in mechanistic interpretability has revealed that Large Language models contain disentangled, human-understandable components. |
| Approach: | They propose a framework that first identifies causal task features through frequency recall and interventional filtering, then selects “Feature-Resonant Data” that maximally activates task features for fine-tuning. |
| Outcome: | The proposed framework outperforms existing models on mathematical reasoning, summarization, and translation tasks while using only 50% of the data. |
AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework (2024.lrec-main)
Copied to clipboard
| Challenge: | Currently, ML&DL methods fail to provide reasons for stock trend predictions, lacking interpretability and reasoning processes. large language models (LLMs) suffer from hallucinations and are unable to keep up with the latest information. |
| Approach: | They develop a method to train large language models to handle financial analysis tasks . they use AlphaFin datasets to compare performance with traditional methods . |
| Outcome: | The proposed method improves stock trend prediction and financial question answering tasks. |
ULMR: Unlearning Large Language Models via Negative Response and Model Parameter Average (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) have attracted significant interest from the research community due to their broad applicability in many language-oriented tasks. |
| Approach: | They propose a framework which uses pre-training datasets to rewrite instructions and generate negative responses to preserve the performance of the original LLM. |
| Outcome: | The proposed framework can erase the pre-training data while maintaining the performance of the original model. |
TexSmart: A System for Enhanced Natural Language Understanding (2021.acl-demo)
Copied to clipboard
Lemao Liu, Haisong Zhang, Haiyun Jiang, Yangming Li, Enbo Zhao, Kun Xu, Linfeng Song, Suncong Zheng, Botong Zhou, Dick Zhu, Xiao Feng, Tao Chen, Tao Yang, Dong Yu, Feng Zhang, ZhanHui Kang, Shuming Shi
| Challenge: | TexSmart supports fine-grained named entity recognition (NER) Large-scale fine-granular entity types are expected to provide richer semantic information for downstream NLP applications. |
| Approach: | They introduce TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities. |
| Outcome: | The proposed system supports fine-grained named entity recognition (NER) and enhanced semantic analysis functions. |
MedAdapter: Efficient Test-Time Adaptation of Large Language Models Towards Medical Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have improved generation and reasoning capabilities compared to traditional BERT-sized models due to massive number of parameters and extensive pre-training on vast textual corpora. |
| Approach: | They propose a unified post-hoc adapter for test-time adaptation of large language models . they propose to fine-tune only a small BERT-sized adapter to rank candidate LLMs . |
| Outcome: | The proposed adapter improves performance on four biomedical tasks without requiring computational resources or sharing data with third parties. |
Track-SQL: Enhancing Generative Language Models with Dual-Extractive Modules for Schema and Context Tracking in Multi-turn Text-to-SQL (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to generative language models struggle to handle the increasing complexity of multi-turn Text-to-SQL tasks. |
| Approach: | They propose a framework which enhances generative language models with dual-extractive modules designed to track schema and contextual changes in multi-turn Text-to-SQL. |
| Outcome: | The proposed framework achieves state-of-the-art performance on SparC and CoSQL datasets and significantly improves execution accuracy in multi-turn interactions by 7.1% and 9.55%. |
ESCP: Enhancing Emotion Recognition in Conversation with Speech and Contextual Prefixes (2024.lrec-main)
Copied to clipboard
| Challenge: | Emotion Recognition in Conversation (ERC) aims to analyze the speaker’s emotional state in a conversation. |
| Approach: | They propose to combine a directed acyclic graph and contextual prefixes to model historical utterances in a conversation and incorporate a contextual prefixed containing sentiment and semantics of historical . |
| Outcome: | The proposed model achieves state-of-the-art (SOTA) performance on several public benchmarks. |
Reasons to Reject? Aligning Language Models with Judgments (2024.findings-acl)
Copied to clipboard
| Challenge: | a new framework for aligning large language models with judgments is proposed to help with alignment . a framework that allows for fine-grained inappropriate content detection and correction based on judgments . large language model alignment is critical for making artificial intelligence a reliable ally for humanity . |
| Approach: | They propose a framework that allows for fine-grained inappropriate content detection and correction based on judgments. |
| Outcome: | The proposed framework beats the 175B DaVinci003 and improves on AlpacaEval using judgments. |
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers (2024.emnlp-main)
Copied to clipboard
Ran Xu, Wenqi Shi, Yue Yu, Yuchen Zhuang, Yanqiao Zhu, May Dongmei Wang, Joyce Ho, Chao Zhang, Carl Yang
| Challenge: | Developing effective biomedical retrieval models is important for excelling at knowledge-intensive biomedically tasks but still challenging due to the lack of sufficient publicly annotated biomedic data and computational resources. |
| Approach: | They propose a series of dense retrievers for enhancing biomedical retrieval via unsupervised pre-training on large biomedically corpora, followed by instruction fine-tuning on a combination of labeled datasets and synthetic pairs. |
| Outcome: | Experiments on 5 biomedical tasks across 11 datasets confirm the performance of the retrieval model on various biomedically demanding tasks. |
Auto-Dialabel: Labeling Dialogue Data with Unsupervised Learning (D18-1)
Copied to clipboard
| Challenge: | Existing dialog datasets rely on human labeling, which is expensive, limited in size, and in low coverage. |
| Approach: | They propose a framework to automatically cluster dialogue intents and slots . they collect context features, leverage an autoencoder for feature assembly, and adapt a dynamic hierarchical clustering method for intent and slot labeling. |
| Outcome: | The proposed framework can promote human labeling cost to a great extent and achieve good intent clustering accuracy (84.1%) it also provides reasonable and instructive slot labeling results. |
Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training (2025.naacl-long)
Copied to clipboard
| Challenge: | Current PP methods face severe bottlenecks, including pipeline bubbles and memory footprint. |
| Approach: | They propose a sequence-level one-forward-one-backward (1F1B) PP method for training LLMs on long sequences with high throughput and memory efficiency. |
| Outcome: | The proposed method achieves 1.14X training throughput with half memory footprint compared to baseline methods . it trains an LLM with 30B parameters on sequences up to 64k tokens using 64X NVIDIA A100 GPUs . |
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization (2023.acl-long)
Copied to clipboard
Pengcheng He, Baolin Peng, Song Wang, Yang Liu, Ruochen Xu, Hany Hassan, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang
| Challenge: | Z-Code++ is a pre-trained language model optimized for abstractive text summarization. |
| Approach: | They propose a pre-trained language model optimized for abstractive text summarization that uses a two-phase pre-training technique to improve model's performance. |
| Outcome: | The proposed model outperforms the competing models on low-resource summarization tasks in zero-shot and few-shot settings. |
ReTRE: Benchmarking LLM Transfer Robustness with Structure-Preserving Variants (2026.acl-long)
Copied to clipboard
| Challenge: | Learning transfer theory emphasizes that applying acquired knowledge to novel manifestations is a key signal of deep understanding |
| Approach: | They propose a benchmark that probes transfer robustness along two rewrite levels: Near Transfer and Far Transfer. |
| Outcome: | The proposed benchmark demonstrates that large language models are robust when faced with novel manifestations of the same problem. |
On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions (2025.findings-emnlp)
Copied to clipboard
Weiqi Wang, Tianqing Fang, Haochen Shi, Baixuan Xu, Wenxuan Ding, Liyu Zhang, Wei Fan, Jiaxin Bai, Haoran Li, Xin Liu, Yangqiu Song
| Challenge: | Conceptualization is a fundamental element of human cognition and plays a pivotal role in generalizable reasoning. |
| Approach: | They propose to categorize different types of conceptualizations into four levels based on the types of instances being conceptualized. |
| Outcome: | The proposed categorization of different types of conceptualizations into four levels based on the types of instances being conceptualized . |
EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Records (2024.emnlp-main)
Copied to clipboard
Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce Ho, Carl Yang, May Dongmei Wang
| Challenge: | EHRAgent enables clinicians to interact with EHRs using natural language . reliance on rule-based conversion systems often necessitates additional training or effort from data engineers. |
| Approach: | They propose a large language model agent that generates and executes code in natural language to facilitate clinicians in directly interacting with EHRs. |
| Outcome: | The proposed agent outperforms the strongest baseline by up to 29.6% in success rate on three real-world EHR datasets. |
LLaMA-E: Empowering E-commerce Authoring with Object-Interleaved Instruction Following (2025.coling-main)
Copied to clipboard
| Challenge: | E-commerce authoring requires engaging, diverse, and targeted content . Large language models lack memorization of domain-specific features in e-commerce applications . |
| Approach: | They propose a unified e-commerce authoring models that address contextual preferences of customers, sellers, and platforms . they propose to integrate interleaved features presented by participating objects into the models to empower authoring applications with comprehensive scenario understanding . |
| Outcome: | The proposed models achieve state-of-the-art evaluation performance and exhibit the advantage in zero-shot practical applications. |
iMOVE : Instance-Motion-Aware Video Understanding (2025.findings-acl)
Copied to clipboard
Jiaze Li, Yaya Shi, Zongyang Ma, Haoran Xu, Yandong.bai Yandong.bai, Huihui Xiao, Ruiwen Kang, Fan Yang, Tingting Gao, Di Zhang
| Challenge: | Recent advances in Video Large Language Models have led to rapid development, significantly enhancing the capture of overall video semantics and achieving remarkable performance in general video understanding tasks. |
| Approach: | They propose a large-scale instance-motion-aware video instruction-tuning dataset iMOVE that utilizes Event-awful Spatiotemporal Efficient Modeling to retain informative instance spatiotemporal motion details while maintaining computational efficiency. |
| Outcome: | The proposed model excels in video temporal understanding and general video understanding. |
SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents (2026.findings-acl)
Copied to clipboard
Zonghao Ying, Yangguang Shao, Jianle Gan, Gan Xu, Wenxin Zhang, Quanchen Zou, Junzheng Shi, Zhenfei Yin, Mingchuan Zhang, Aishan Liu, Xianglong Liu
| Challenge: | Existing security benchmarks only cover user-level prompts and environmental threats . however, these models are vulnerable to pop-up attacks and prompt injections . |
| Approach: | They propose a security benchmark that covers a set of six attack vectors that span both user-level and environment-level manipulations. |
| Outcome: | The proposed security benchmarks cover a set of six real-world web environments with 2,970 adversarial trajectories and a multi-layered evaluation protocol dissecting agent failures across internal reasoning, behavioral execution, and task outcomes. |
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)
Copied to clipboard
Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Xuemei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang
| Challenge: | i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data. |
| Approach: | They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data. |
| Outcome: | i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks. |
E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning (2022.findings-acl)
Copied to clipboard
Jiangjie Chen, Rui Xu, Ziquan Fu, Wei Shi, Zhongqiao Li, Xinbo Zhang, Changzhi Sun, Lei Li, Yanghua Xiao, Hao Zhou
| Challenge: | Existing benchmarks to test word analogy do not reveal the underneath process of analogical reasoning of neural models. |
| Approach: | They propose an explanation benchmark for analogical reasoning using a Civil Service exam . they use a free-text explanation scheme to explain whether an analogy should be drawn . |
| Outcome: | The proposed benchmark is very challenging for state-of-the-art models, it is found. |
PersonaX: A Recommendation Agent-Oriented User Modeling Framework for Long Behavior Sequence (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for user profile modeling extract only partial segments from full historical behavior sequence, resulting in incomplete modeling and suboptimal profiling. |
| Approach: | They propose an agent-agnostic LLM-UM framework to augment downstream recommendation agents . it segments complete historical behaviors into clustered groups and performs offline multi-persona profiling . |
| Outcome: | The proposed framework improves agent performance and inference efficiency by 31% and 10% using 30–50% of behavioral data. |
InsLogicBench: An Argumentation Logic Grounded Benchmark for Complex Insurance Claims Adjudication (2026.acl-long)
Copied to clipboard
Jin Liu, Yunpeng Liu, Keyi Wang, Jie Shi, Xiao Xu, Wenkang Huang, Xingzhong Xu, Xin Liang, Yanghua Xiao
| Challenge: | Existing benchmarks for insurance claims adjudication are limited to information retrieval or simple multiple-choice setups. |
| Approach: | They propose a benchmark that provides complete reasoning traces linking factual inputs, relevant policy clauses, and final verdicts. |
| Outcome: | The proposed benchmark shows that models often produce correct decisions but fail to provide precise justifications, highlighting a critical discrepancy between decision accuracy and logical reasoning capabilities. |
KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)
Copied to clipboard
Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals. |
| Approach: | They propose a self-supervised framework that enhances RAG systems through efficient model adaptation. |
| Outcome: | The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models. |
Learning from Sibling Mentions with Scalable Graph Inference in Fine-Grained Entity Typing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models struggle to handle hard mentions due to insufficient contexts, limiting their overall typing performance. |
| Approach: | They propose to exploit sibling mentions to enhance the mention representations by adding unseen test mentions as new nodes for inference. |
| Outcome: | The proposed model outperforms ten strong baseline models and outperformed strong baselines. |
Why Do More Experts Fail? A Theoretical Analysis of Model Merging (2026.acl-long)
Copied to clipboard
Zijing Wang, Xingle Xu, YongKang Liu, Yiqun Zhang, Peiqin Lin, Shi Feng, Daling Wang, Xiaocui Yang, Hinrich Schuetze
| Challenge: | Existing methods for model merging struggle to maintain performance gains as the number of merged models increases. |
| Approach: | They propose a Reparameterized Heavy-Tailed method to extend the merged model’s coverage and enhance performance. |
| Outcome: | The proposed method extends the merged model’s coverage and enhances performance on 19 benchmarks, including knowledge-intensive and general-purpose tasks. |
DIDS: Domain Impact-aware Data Sampling for Large Language Model Training (2025.emnlp-main)
Copied to clipboard
Weijie Shi, Jipeng Zhang, Yaguang Wu, Jingzhi Fang, Shibo Zhang, Yao Zhao, Hao Chen, Ruiyuan Zhang, Yue Cui, Jia Zhu, Sirui Han, Jiajie Xu, Xiaofang Zhou
| Challenge: | Existing approaches for optimizing domain-level sampling strategies struggle with maintaining intra-domain consistency and accurately measuring domain impact. |
| Approach: | They propose to use a Fisher-Information Matrix-guided metric to measure domain impact to ensure intra-domain consistency and accuracy. |
| Outcome: | The proposed model achieves 3.4% higher average performance while maintaining comparable training efficiency. |
code-transformed: The Influence of Large Language Models on Code (2026.findings-eacl)
Copied to clipboard
| Challenge: | Using Large Language Models, code generation capabilities have transformed programming practices. |
| Approach: | They analyze 20,000 GitHub repositories linked to arXiv papers published between 2020 and 2025 . they identify measurable trends in the evolution of coding style that align with LLM-generated code . |
| Outcome: | The proposed study examines 20,000 GitHub repositories linked to arXiv papers . it finds that LLMs influence code style, and that they can be observed in real-world code . |
Generative Multimodal Entity Linking (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing Entity Linking methods focus on designing complex multimodal interaction mechanisms and require fine-tuning all model parameters. |
| Approach: | They propose a framework for multimodal entity linking based on Large Language Models (LLMs) that trains a feature mapper to enable cross-modal interactions. |
| Outcome: | The proposed framework achieves state-of-the-art on two well-established datasets with a performance gain of 7.7% on WikiDiverse and 8.8% on Wikileaks. |
Negation Triplet Extraction with Syntactic Dependency and Semantic Consistency (2024.lrec-main)
Copied to clipboard
| Challenge: | Negation understanding is crucial to many downstream tasks such as sentiment analysis, question answering, Web search and natural language inference. |
| Approach: | They propose a novel negation triplet extraction task which aims to extract negation subject along with negation cue and scope. |
| Outcome: | The proposed model is based on a generative pretrained language model with a multi-task learning framework and achieves the best performance compared to baselines. |
MSVBench: Towards Human-Level Evaluation of Multi-Shot Video Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods for complex multi-shot video are anchored to single-shot paradigms, lacking comprehensive story assets and cross-shot metrics. |
| Approach: | They propose a framework that synergizes the high-level semantic reasoning of Large Multimodal Models with the fine-grained perceptual rigor of domain-specific expert models. |
| Outcome: | The proposed framework synergizes the high-level semantic reasoning of Large Multimodal Models with the fine-grained perceptual rigor of domain-specific expert models. |
Memory-Guided Hard Data Augmentation for Multimodal Named Entity Recognition (2026.findings-acl)
Copied to clipboard
Xinyu Liu, Kai fu, Yinghan Shi, Quanyou Chu, Ming Du, Hongya Wang, Xiaojun Meng, Jiansheng Wei, Yanghua Xiao, Bo Xu
| Challenge: | Existing methods for Named Entity Recognition (NER) ignore the internal state of the target model. |
| Approach: | They propose a framework to repair model-specific errors by using a model-based approach . they employ cross-validation to identify model- specific Hard Data and a memory tree to induce macro-level error patterns from micro-level failures. |
| Outcome: | The proposed framework yields significant performance gains on Twitter and other platforms. |
QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to QA fine-tune language models on QA pairs constructed from CommonSense Knowledge Bases (CSKBs) however, current QA synthesis protocols introduce noise from the CSKB and generate ungrammatical questions and false negative options, which impede the model’s ability to generalize. |
| Approach: | They propose a framework to analyze the training dynamics of each QA pair at both the question level and option level, discarding machine-detectable artifacts and mislabeled or false-negative options. |
| Outcome: | The proposed framework outperforms baseline approaches while using only 33% of the synthetic data. |
RAAMove: A Corpus for Analyzing Moves in Research Article Abstracts (2024.lrec-main)
Copied to clipboard
Hongzheng Li, Ruojin Wang, Ge Shi, Xing Lv, Lei Lei, Chong Feng, Fang Liu, Jinkun Lin, Yangguang Mei, Linnan Xu
| Challenge: | RAAMove is a comprehensive multi-domain corpus dedicated to the annotation of move structures in Research Article (RA) abstracts. |
| Approach: | They propose a multi-domain corpus dedicated to the annotation of move structures in RA abstracts. |
| Outcome: | The proposed corpus is based on a human-annotated dataset and a BERT-based model to verify its effectiveness. |
Searching for Best Practices in Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, especially in specialized domains. |
| Approach: | They propose several strategies for deploying RAG that balance performance and efficiency. |
| Outcome: | The proposed approaches can significantly enhance question-answering capabilities and accelerate the generation of multimodal content using a “retrieval as generation” strategy. |
Balanced Joint Adversarial Training for Robust Intent Detection and Slot Filling (2020.coling-main)
Copied to clipboard
| Challenge: | Existing joint models for intent detection and slot filling show insufficient robustness . however, some small changes of inputs can fool the models to produce wrong predictions . |
| Approach: | They propose a joint adversarial training model that generates adversarials to attack the joint model and trains the model to defend against the adversarial examples. |
| Outcome: | The proposed model achieves significantly higher scores and improves robustness on two datasets. |
Skeletons Matter: Dynamic Data Augmentation for Text-to-Query (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on a single query language, resulting in limited generalizability . a new task paradigm is proposed to unify semantic parsing tasks across different query languages . |
| Approach: | They propose a task paradigm that unifies parsing tasks across query languages . they identify query skeletons as a shared optimization target of Text-to-Query tasks . |
| Outcome: | The proposed method achieves state-of-the-art performance using only a small amount of synthesized data. |
Video2Roleplay: A Multimodal Dataset and Framework for Video-Guided Role-playing Agents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to RPAs focus on static role profiles, overlooking dynamic perceptual abilities inherent to humans. |
| Approach: | They propose a framework that combines adaptive temporal sampling with dynamic and static role profiles. |
| Outcome: | The proposed framework combines adaptive temporal sampling with dynamic and static role profiles. |
CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)
Copied to clipboard
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan
| Challenge: | Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages . |
| Approach: | They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models . |
| Outcome: | The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English . |
EducationQ: Evaluating LLMs’ Teaching Capabilities Through Multi-Agent Dialogue Framework (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used as educational tools, yet evaluating their teaching capabilities remains challenging due to the resource-intensive nature of teacher-student interactions. |
| Approach: | They propose a multi-agent dialogue framework that efficiently assesses teaching capabilities through simulated dynamic educational scenarios. |
| Outcome: | The proposed framework outperforms open-source models on 1,498 questions across 13 disciplines and 10 difficulty levels on 1,400 questions. |
Answering Narrative-Driven Recommendation Queries via a Retrieve–Rank Paradigm and the OCG-Agent (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate narrative-driven recommendation are based on large language models (LLMs) but the RAG paradigm is inherently ill-suited for such special queries. |
| Approach: | They propose a novel retrieve-rank paradigm that generatively retrieves structurally adaptive and semantically aligned candidates, ensuring both extensive candidate coverage and high-quality information. |
| Outcome: | The proposed paradigm outperforms the existing paradigm and the existing one under real-world scenarios. |
Logits-Based Finetuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for developing compact and efficient large language models lack token-level dependencies and linguistic diversity. |
| Approach: | They propose a logits-based fine-tuning framework that integrates supervised learning and knowledge distillation to build enriched training targets using teacher logits and ground truth labels. |
| Outcome: | The proposed method outperforms existing methods on a large-scale logits dataset and a series of science-focused models. |
MoLAN: A Unified Modality-Aware Noise Dynamic Editing Framework for Multimodal Sentiment Analysis (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multimodal sentiment analysis treat entire modality as an independent unit for feature enhancement or denoising, which often suppresses redundant noise at the cost of weakening critical information. |
| Approach: | They propose a ModaLity-aware noise dynAmic editiNg framework that performs modality-awful block partitioning by dividing features of each modality into multiple blocks. |
| Outcome: | Experiments on five models and four datasets show that MoLAN+ achieves the state-of-the-art performance. |
MCPG: A Flexible Multi-Level Controllable Framework for Unsupervised Paraphrase Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on controllable unsupervised paraphrase generation are expensive and require supervised training on large parallel corpora. |
| Approach: | They propose a method for controllable unsupervised paraphrase generation that is flexible to adapt to specific domains without extra training. |
| Outcome: | The proposed method outperforms state-of-the-art unsupervised baselines by a margin. |
AnaMeta: A Table Understanding Dataset of Field Metadata Knowledge Shared by Multi-dimensional Data Analysis Tasks (2023.findings-acl)
Copied to clipboard
Xinyi He, Mengyu Zhou, Mingjie Zhou, Jialiang Xu, Xiao Lv, Tianle Li, Yijia Shao, Shi Han, Zejian Yuan, Dongmei Zhang
| Challenge: | Tabular data analysis is performed everyday across various domains. |
| Approach: | They propose to use a dataset of 467k tables with supervision labels for four types of field metadata. |
| Outcome: | The proposed framework improves the understanding capability of tabular models by incorporating distribution and knowledge information. |
Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval (2024.lrec-main)
Copied to clipboard
Haowei Liu, Yaya Shi, Haiyang Xu, Chunfeng Yuan, Qinghao Ye, Chenliang Li, Ming Yan, Ji Zhang, Fei Huang, Bing Li, Weiming Hu
| Challenge: | Existing methods for video-text retrieval capture fine-grained semantic concepts . however, they lack the ability to capture finer-grain concepts such as objects and actions. |
| Approach: | They propose a dual-encoder architecture for fast video-text retrieval that learns lexicon representations to capture fine-grained semantics. |
| Outcome: | The proposed framework outperforms existing methods with 4.8% and 8.2% improvement on MSR-VTT and DiDeMo respectively. |
PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)
Copied to clipboard
Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei
| Challenge: | Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning. |
| Approach: | They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios. |
| Outcome: | The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics. |
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning (2025.acl-long)
Copied to clipboard
Yujie Feng, Xujia Wang, Zexin Lu, Shenghong Fu, Guangyuan Shi, Yongxin Xu, Yasha Wang, Philip S. Yu, Xu Chu, Xiao-Ming Wu
| Challenge: | Continual learning (CL) is crucial for large language models without costly retraining. |
| Approach: | They propose a framework for recurrent knowledge identification and fusion that enables dynamic estimation of parameter importance distributions to enhance knowledge transfer. |
| Outcome: | The proposed framework mitigates catastrophic forgetting and enhances knowledge transfer. |
Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring (2025.naacl-long)
Copied to clipboard
Honglin Mu, Han He, Yuxin Zhou, Yunlong Feng, Yang Xu, Libo Qin, Xiaoming Shi, Zeming Liu, Xudong Han, Qi Shi, Qingfu Zhu, Wanxiang Che
| Challenge: | Existing black-box jailbreak methods often rely on model feedback . existing methods may be intercepted by content moderators during the search process . |
| Approach: | They propose a method that guides malicious prompt construction by local training a mirror model of the target black-box model through benign data distillation. |
| Outcome: | The proposed method achieves a 92% attack success rate and 80% stealth rate on a subset of AdvBench. |
PersLEARN: Research Training through the Lens of Perspective Cultivation (2023.acl-demo)
Copied to clipboard
Yu-Zhe Shi, Shiqian Li, Xinyi Niu, Qiao Xu, Jiawen Liu, Yifan Xu, Shiyu Gu, Bingru He, Xinyang Li, Xinyu Zhao, Zijian Zhao, Yidong Lyu, Zhen Li, Sijia Liu, Lin Qiu, Jinhao Ji, Lecheng Ruan, Yuxi Ma, Wenjuan Han, Yixin Zhu
| Challenge: | PersLEARN is a tool designed to facilitate the cultivation of scientific perspectives . junior researchers struggle to identify the perspectives reflected in the literature and struggle to develop their own viewpoints. |
| Approach: | They propose a tool to facilitate the cultivation of scientific perspectives by interacting with a prompt-based model and allowing students to develop their own perspectives explicitly. |
| Outcome: | The proposed tool outperforms baseline approaches across multiple domains of literature from different perspectives. |
Reinforcement Learning on Pre-Training Data (2026.acl-long)
Copied to clipboard
Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang
| Challenge: | Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings. |
| Approach: | They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data. |
| Outcome: | Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base. |
DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | . - (EN) |
| Approach: | . - (EN) |
| Outcome: | . - (EN) |
QuickLLaMA: Query-aware Inference Acceleration for Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle with capturing long-distance dependencies within sequences to deeply understand semantics. |
| Approach: | They propose a system that captures relevant information within a fixed window size and provides precise answers to queries. |
| Outcome: | The proposed system can read Harry Potter within 30s and accurately answer the questions. |
MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)
Copied to clipboard
Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun
| Challenge: | Scientific data visualization is an essential process in research, but its use of large language models remains unexplored. |
| Approach: | They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks. |
| Outcome: | The proposed framework improves performance of commercial and open-source models. |
Semantics-enhanced Cross-modal Masked Image Modeling for Vision-Language Pre-training (2024.lrec-main)
Copied to clipboard
Haowei Liu, Yaya Shi, Haiyang Xu, Chunfeng Yuan, Qinghao Ye, Chenliang Li, Ming Yan, Ji Zhang, Fei Huang, Bing Li, Weiming Hu
| Challenge: | Existing methods for vision-language pre-training lack high-level semantics and text is not sufficiently involved in masked modeling. |
| Approach: | They propose a semantics-enhanced cross-modal MIM framework for vision-language representation learning that harvests high-level semantics from global image features via self-supervised agreement learning and transfers them to local patch encodings by sharing the encode space. |
| Outcome: | The proposed model achieves state-of-the-art or competitive performance on multiple vision-language tasks. |
iAgent: LLM Agent as a Shield between User and Recommender Systems (2025.findings-acl)
Copied to clipboard
Wujiang Xu, Yunxiao Shi, Zujie Liang, Xuying Ning, Kai Mei, Kun Wang, Xi Zhu, Min Xu, Yongfeng Zhang
| Challenge: | Traditional recommender systems focus on the user-platform paradigm, where users are directly exposed under the control of the platform's recommendation algorithms. |
| Approach: | They propose a user-agent-platform paradigm where agent serves as the protective shield between user and recommender system that enables indirect exposure. |
| Outcome: | The proposed model improves 16.6% over baselines on four datasets and mitigates echo chamber effects and reduces model bias in disadvantaged users. |
An Empirical Study on Multiple Information Sources for Zero-Shot Fine-Grained Entity Typing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Auxiliary information from multiple sources has been demonstrated to be effective in zero-shot fine-grained entity typing (ZFET) however, there is no comprehensive understanding of how to make better use of the existing information sources and how they affect the performance of ZFET. |
| Approach: | They propose a multi-source fusion model targeting auxiliary information from multiple sources to improve zero-shot fine-grained entity typing (ZFET) |
| Outcome: | The proposed model achieves 11.42% and 22.84% gains over state-of-the-art baselines on BBN and Wiki respectively with regard to macro F1 scores. |
A Dual-Mind Framework for Strategic and Expressive Negotiation Agent (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to negotiation dialogue focus on only one aspect, ignoring the synergistic effect of their combined synergies. |
| Approach: | They propose a dual-mind negotiation agent framework that integrates an intuitive and a deliberative module for slow, expression optimization. |
| Outcome: | The proposed framework achieves state-of-the-art on negotiation datasets showing that it improves negotiation ability. |
LLM-ForcedAligner: A Non-Autoregressive and Accurate LLM-Based Forced Aligner for Multilingual and Long-Form Speech (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for forcing alignment are language-specific and prone to temporal shifts. |
| Approach: | They propose a slot-filling paradigm that uses time indices to predict slot positions. |
| Outcome: | The proposed method reduces accumulated temporal shifts by 69% compared with prior methods. |
MMErroR: A Benchmark for Erroneous Reasoning in Vision-Language Models (2026.acl-long)
Copied to clipboard
Yang Shi, Yifeng Xie, Minzhe Guo, Liangsi Lu, Mingxuan Huang, Jingchao Wang, Zhihong Zhu, Boyan Xu, Zhiqi Huang
| Challenge: | Recent advances in vision-language models have improved performance in multi-modal learning. |
| Approach: | They propose a multi-modal benchmark that embeds a single coherent reasoning error in 1997 samples. |
| Outcome: | The proposed benchmark is based on a set of 1997 samples embedding a single coherent reasoning error. |
A Speaker-Aware Co-Attention Framework for Medical Dialogue Information Extraction (2022.emnlp-main)
Copied to clipboard
Yuan Xia, Zhenhui Shi, Jingbo Zhou, Jiayu Xu, Chao Lu, Yehui Yang, Lei Wang, Haifeng Huang, Xia Zhang, Junwei Liu
| Challenge: | With the development of medical digitization, the extraction and structuring of electronic medical records (EMRs) have become challenging but fundamental tasks. |
| Approach: | They propose a speaker-aware dialogue encoder with multi-task learning which takes the speaker's identity into account and a co-attention fusion network to aggregate the utterance information. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on the public medical dialogue extraction datasets to demonstrate its superiority. |
Enhancing Multimodal Large Language Models for Ancient Chinese Character Evolution Analysis via Glyph-Driven Fine-Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have explored the evolutionary analysis of ancient scripts, with particular attention to the transformation of character forms from oracle bone inscriptions to regular script. |
| Approach: | They propose a benchmark framework that leverages MLLMs to analyze the evolution of ancient Chinese scripts. |
| Outcome: | The proposed framework improves performance on core tasks and character recognition and evolutionary reasoning tasks while limiting performance on other tasks. |
InferenceDynamics: Adaptive LLM Routing through Structured Capability and Knowledge Profiling (2026.acl-long)
Copied to clipboard
Haochen Shi, Tianshi Zheng, Weiqi Wang, Baixuan Xu, Chunyang Li, Chunkit Chan, Tao Fan, Yangqiu Song
| Challenge: | Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs. |
| Approach: | They propose a flexible and scalable multi-dimensional routing framework that models the capability and knowledge of models. |
| Outcome: | The proposed framework can be used to generalize and identify top-performing models for group-level routing using modern benchmarks including MMLU-Pro, GPQA, BigGenBench, and LiveBench. |
SWE-Swiss: A Multi-Task Fine-Tuning and RL Recipe for High-Performance Issue Resolution (2026.findings-acl)
Copied to clipboard
Zhenyu He, Qingping Yang, Wei Shen, Xiaojian Zhong, Kechi Zhang, Chenxin An, Wenlei Shi, Tianle Cai, Di He, Jiaze Chen, Jingjing Xu
| Challenge: | SWE-Swiss-32B demonstrates strong generalization to other common LLM benchmarks. |
| Approach: | They propose a two-phase training recipe that decomposes issue resolution into three core skills: Localization, Repair, and Unit Test Generation. |
| Outcome: | The proposed model achieves a 60.2% score on the SWE-bench Verified benchmark and is in the top-tier performance bracket of much larger models. |
ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling (2026.findings-eacl)
Copied to clipboard
Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Chaojun Xiao, Zhiyuan Liu, Ge Yu, Chenyan Xiong
| Challenge: | Large Language Models (LLMs) exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge. |
| Approach: | They propose a framework that enhances the external knowledge utilization of Large Language Models through a two-stage constructivist cognitive modeling process. |
| Outcome: | The proposed framework achieves a 10% improvement over baseline methods on various question-answering benchmarks. |
GeoRA: Geometry-Aware Low-Rank Adaptation for RLVR (2026.acl-long)
Copied to clipboard
| Challenge: | Existing parameter-efficient methods for RLVR face limitations . low-rank adaptation methods do not account for the distinct optimization dynamics . |
| Approach: | They propose a low-rank adaptation method tailored for RLVR that exploits the anisotropic structure of RL update subspace and extracts its principal directions via Singular Value Decomposition (SVD). |
| Outcome: | Experiments on large reasoning models show that GeoRA outperforms strong low-rank baselines across RLVR settings while showing stronger generalization and less forgetting on out-of-domain tasks. |
AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage (2026.acl-long)
Copied to clipboard
Xuanle Zhao, Zilin Sang, Yuxuan Li, Qi Shi, Weilun Zhao, Shuo Wang, Duzhen Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Efficient reproduction of research papers requires deep domain expertise. |
| Approach: | They propose a framework that systematically mines implicit knowledge from the cited literature to reproduce experimental code in a complete, end-to-end manner. |
| Outcome: | The proposed framework surpasses baselines across all metrics and reproduces experimental code in a complete, end-to-end manner. |
ICL-Bandit: Relevance Labeling in Advertisement Recommendation Systems via LLM (2025.findings-emnlp)
Copied to clipboard
Lu Wang, Chiming Duan, Pu Zhao, Fangkai Yang, Yong Shi, Xuefeng Luo, Bingjing Xu, Weiwei Deng, Qingwei Lin, Dongmei Zhang
| Challenge: | In-context learning (ICL) is a common practice to enhance LLM performance on domain-specific tasks. |
| Approach: | They propose a method that leverages large language models to enhance query-ad relevance labeling . they identify and provide superior demonstrations for ICL, thereby improving labeling performance . |
| Outcome: | The proposed method improves query-ad relevance labeling performance by providing demonstrations. |
Concept rather than Document: Context Compression via AMR-based Conceptual Entropy (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods rely on lexical or token-level features that fragment semantic units and fail to capture conceptually essential content. |
| Approach: | They propose an unsupervised framework leveraging Abstract Meaning Representation to preserve essential information while filtering irrelevant text. |
| Outcome: | The proposed framework outperforms RAG and existing baselines while preserving essential information while filtering irrelevant text. |
DeepRTL2: A Versatile Model for RTL-Related Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Integration of large language models into electronic design automation has been a key driver in eDA. |
| Approach: | They propose a family of large language models that unifies generation- and embedding-based tasks related to RTL. |
| Outcome: | The proposed model achieves state-of-the-art performance across all evaluated tasks. |
PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matching (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) has been used to adapt Large Language Models to a variety of tasks, but it requires substantial computational resources to perform. |
| Approach: | They propose a low-rank adaptive learning approach that leverages LoRA's in-context learning capability through prompt matching via reinforcement learning in resource-constrained environments. |
| Outcome: | The proposed model improves LoRA performance on evaluation metrics and utilises consumer-grade GPU resources. |
GuoFeng: A Benchmark for Zero Pronoun Recovery and Translation (2022.emnlp-main)
Copied to clipboard
Mingzhou Xu, Longyue Wang, Derek F. Wong, Hongye Liu, Linfeng Song, Lidia S. Chao, Shuming Shi, Zhaopeng Tu
| Challenge: | ZPs are often omitted when they can be pragmatically or grammatically inferred from intraand inter-sentential contexts. |
| Approach: | They propose a benchmark testset for target evaluation on Chinese-English ZP translation. |
| Outcome: | The proposed testset covers five genres and identifies current challenges for evaluation. |
ModelScope-Agent: Building Your Customizable Agent System with Open-source Large Language Models (2023.emnlp-demo)
Copied to clipboard
Chenliang Li, He Chen, Ming Yan, Weizhou Shen, Haiyang Xu, Zhikai Wu, Zhicheng Zhang, Wenmeng Zhou, Yingda Chen, Chen Cheng, Hongzhu Shi, Ji Zhang, Fei Huang, Jingren Zhou
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities to comprehend human intentions, engage in reasoning, and design planning-like behavior. |
| Approach: | They propose a framework that equips large language models with tool-use capabilities . they propose LLaMA and Chat-GLM as controllers, and a model-based agent framework . |
| Outcome: | The proposed framework equips open-source LLMs with tool-use capabilities . it provides a user-friendly system library with a customizable engine design . |
KC-ISA: An Implicit Sentiment Analysis Model Combining Knowledge Enhancement and Context Features (2022.coling-1)
Copied to clipboard
| Challenge: | Existing research results on explicit sentiment analysis are limited . implicit sentiment analysis is a process of analyzing text based on whether it contains explicit sentiment words. |
| Approach: | They propose a model that integrates external knowledge and contextual features . they use a knowledge graph to supplement implicit sentiment expression . |
| Outcome: | The proposed model can achieve better results on the SMP2019 implicit sentiment analysis dataset. |
AMR-TST: Abstract Meaning Representation-based Text Style Transfer (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic representation that can enhance natural language generation (NLG) by providing a logical semantic input. |
| Approach: | They propose an AMR-based text style transfer technique that converts source text to an AML graph and generates transferred text based on the AMR graph modified by a TST policy named style rewriting. |
| Outcome: | The proposed method achieves state-of-the-art results compared with baseline models in automatic and human evaluations. |
Scaling Under-Resourced TTS: A Data-Optimized Framework with Advanced Acoustic Modeling for Thai (2025.acl-industry)
Copied to clipboard
| Challenge: | Text-to-speech (TTS) systems are limited by limited data and linguistic complexities. |
| Approach: | They propose a data-optimized framework with an advanced acoustic model to build high-quality TTS systems for low-resource scenarios. |
| Outcome: | The proposed framework enables zero-shot voice cloning and improved performance across diverse client applications, including finance, healthcare, education, and law. |
UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity Mixture-of-Experts (2026.acl-long)
Copied to clipboard
Zhenyu Liu, Yunxin li, Xuanyu Zhang, Qixun Teng, Shenyuan Jiang, Xinyu Chen, Haoyuan Shi, Haolan Chen, Fanbo Meng, Mingjun Zhao, Yu Xu, Yancheng He, Baotian Hu, Haizhou Li, Min Zhang
| Challenge: | Recent advances in unified multimodal models indicate a clear trend towards comprehensive content generation. |
| Approach: | They propose a unified speech and music generation model built upon a novel framework . they propose specialized MoE architectures and curated training strategies to tackle data imbalances . |
| Outcome: | The proposed model achieves state-of-the-art performance on major speech and music generation benchmarks. |
Entity-level Interaction via Heterogeneous Graph for Multimodal Named Entity Recognition (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for name-based entity recognition neglect the integrity of entity semantics and conduct cross-modal interaction at token-level. |
| Approach: | They propose a multimodal named entity recognition model that captures visual information and fuses it into tokens to rid non-entity tokens of visual noise. |
| Outcome: | The proposed model captures entity-related visual information and fuses it into tokens . it eliminates visual noise and makes non-entity tokens easily misidentified as entities . |
EmRel: Joint Representation of Entities and Embedded Relations for Multi-triple Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies only explore entity representations, but propose a novel triple perspective for relation extraction. |
| Approach: | They propose to explicitly introduce relation representation and jointly represent it with entities to identify valid triples. |
| Outcome: | The proposed method is based on ablations and document-level relation extraction and joint entity and relation extraction. |
MIO: A Foundation Model on Multimodal Tokens (2025.emnlp-main)
Copied to clipboard
Zekun Moore Wang, King Zhu, Chunpu Xu, Wangchunshu Zhou, Jiaheng Liu, Yibo Zhang, Jessie Wang, Ning Shi, Siyu Li, Yizhi Li, Haoran Que, Zhaoxiang Zhang, Yuanxing Zhang, Ge Zhang, Ke Xu, Jie Fu, Wenhao Huang
| Challenge: | Existing models lack multimodal understanding capabilities, resulting in closed-source model that does not support multimodal interleaved sequences. |
| Approach: | They propose a foundation model built on multimodal tokens capable of understanding and generating speech, text, images, and videos in an end-to-end, autoregressive manner. |
| Outcome: | The proposed model is able to understand speech, text, images, and videos in an end-to-end, autoregressive manner. |
Mixed-Lingual Pre-training for Cross-lingual Summarization (2020.aacl-main)
Copied to clipboard
| Challenge: | Cross-lingual summarization (CLS) aims at producing a summary in the target language for an article in the source language. |
| Approach: | They propose a mixed-lingual pre-training scheme that leverages both cross-lingual tasks such as translation and monolingual tasks like masked language models. |
| Outcome: | The proposed model improves on the translation and masked language models with no task-specific components and saves memory. |
When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing dialogue models are primarily trained on human-human conversations . thumb ups/downs and gold corrections are often sparse in real-life deployment settings . |
| Approach: | They propose a framework to make use of binary and free-form textual human feedback. |
| Outcome: | The proposed framework improves the final dialogue model by using model-corrected replies. |
An Empirical Study of Many-to-Many Summarization with Large Language Models (2025.acl-long)
Copied to clipboard
Jiaan Wang, Fandong Meng, Zengkui Sun, Yunlong Liang, Yuxuan Cao, Jiarong Xu, Haoxiang Shi, Jie Zhou
| Challenge: | Recent studies have shown that large language models (LLMs) have strong multilingual abilities, giving them the potential to perform M2MS in real applications. |
| Approach: | They propose to use many-to-many summarization (M2MS) to generate a brief summary in any language given a document also in any other language. |
| Outcome: | The proposed model outperforms zero-shot LLMs in terms of automatic evaluations. |
Merge Hijacking: Backdoor Attacks to Model Merging of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing research on model merging focuses on optimizing model performance and minimizing backdoors. |
| Approach: | They propose a backdoor attack targeting model merging in Large Language Models that creates a unified model for multi-domain tasks. |
| Outcome: | The proposed attack is effective across models, merging algorithms, and tasks while maintaining utility across tasks. |
CI-AVSR: A Cantonese Audio-Visual Speech Datasetfor In-car Command Recognition (2022.lrec-1)
Copied to clipboard
Wenliang Dai, Samuel Cahyawijaya, Tiezheng Yu, Elham J. Barezi, Peng Xu, Cheuk Tung Yiu, Rita Frieske, Holy Lovenia, Genta Winata, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | In-car smart assistants should be able to process general as well as car-related commands and perform corresponding actions, which eases driving and improves safety. |
| Approach: | They propose a dataset for in-car command recognition in the cantonese language with both video and audio data. |
| Outcome: | The proposed model can achieve a considerable quality on the clean test set, but the speech recognition quality on noisy data is still inferior. |
Vulnerability of Text-to-Image Models to Prompt Template Stealing: A Differential Evolution Approach (2025.findings-acl)
Copied to clipboard
Yurong Wu, Fangwen Mu, Qiuhong Zhang, Jinjing Zhao, Xinrun Xu, Lingrui Mei, Yang Wu, Lin Shi, Junjie Wang, Zhiming Ding, Yiwei Wang
| Challenge: | Prompt trading has emerged as a significant intellectual property concern in recent years, where vendors entice users by showcasing sample images before selling prompt templates that can generate similar images. |
| Approach: | They propose a prompt-stealing benchmark consisting of 50 templates and 450 images organized into Easy and Hard difficulty levels. |
| Outcome: | The proposed method outperforms baseline methods with an average improvement of over 10%. |
A Simple Contrastive Learning Framework for Interactive Argument Pair Identification via Argument-Context Extraction (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on argument mining uses context-based methods to identify whether two arguments are interactively related. |
| Approach: | They propose a contrastive learning framework to extract valuable information from the context. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the benchmark dataset and visually displays more compact representations. |
ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation (2022.lrec-1)
Copied to clipboard
Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Peng Xu, Yan Xu, Zihan Liu, Rita Frieske, Tiezheng Yu, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | Code-switching is a speech phenomenon occurring when a speaker switches language during a conversation. |
| Approach: | They propose to collect Mandarin Chinese-English code-switching corpus from read speech rather than spontaneous speech to address this phenomenon. |
| Outcome: | ASCEND consists of 10.62 hours of clean speech, collected from 23 bilingual speakers of Chinese and English. |
The Agent’s First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios (2026.findings-acl)
Copied to clipboard
Daocheng Fu, Jianbiao Mei, Rong Wu, Xuemeng Yang, Jia Xu, Ding Wang, Pinlong Cai, Yong Liu, Licheng Wen, Botian Shi
| Challenge: | Existing research mainly focuses on performance upper bounds in static environments, overlooking stochastic real-world deployment. |
| Approach: | They propose a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting. |
| Outcome: | The proposed model evaluates agents in a dynamic evaluation environment that simulates a "trainee" agent continuously exploring a novel setting. |
Unlocking the Power of Large Language Models for Entity Alignment (2024.acl-long)
Copied to clipboard
Xuhui Jiang, Yinghan Shen, Zhichao Shi, Chengjin Xu, Wei Li, Zixuan Li, Jian Guo, Huawei Shen, Yuanzhuo Wang
| Challenge: | Entity Alignment (EA) is a crucial step in unifying data from heterogeneous sources and plays a critical role in data-driven AI applications. |
| Approach: | They propose a framework that incorporates large language models to improve EA. |
| Outcome: | The proposed framework incorporates large language models (LLMs) to improve EA accuracy while preserving efficiency. |
ZiNet: Linking Chinese Characters Spanning Three Thousand Years (2022.findings-acl)
Copied to clipboard
| Challenge: | tens of thousands of ancient characters must be deciphered by experts to interpret unearthed documents. |
| Approach: | They propose a diachronic Chinese knowledge base to help researchers discover glyph similar characters by measuring glyph similarities between ancient Chinese characters. |
| Outcome: | The proposed method shows strong correlations between the scores obtained from the method and from human experts. |
AwarenessBench: Assessing Cognitive Capabilities of Language Models (2026.acl-long)
Copied to clipboard
Xiaojian Li, Rongwu Xu, Tianyun Zhang, Yue Wang, Shuo Chen, Qiner Lyu, Briana Zhang, Peiran Yang, Kyle Xue Chen, Haoyuan Shi, Yu Wang, Wei Xu
| Challenge: | Language models exhibit increasingly consciousness-like behaviors, requiring a baseline to evaluate their cognitive abilities. |
| Approach: | They propose a benchmark to assess the cognitive abilities of language models (LMs) they compare 18 state-of-the-art LMs to human models in metacognition, self-awareness, social awareness and situational awareness . |
| Outcome: | Evaluating 18 state-of-the-art LMs, they find they consistently surpass baselines . but most models fall short in metacognition and self-awareness, the study finds . |
ExtRA: Extracting Prominent Review Aspects from Customer Feedback (D18-1)
Copied to clipboard
| Challenge: | Existing methods for analyzing and summarizing customer reviews are based on a number of prominent review aspects. |
| Approach: | They propose a framework for extracting the most prominent aspects of a given product type from textual reviews. |
| Outcome: | The proposed framework extracts K most prominent aspect terms which do not overlap semantically without supervision. |
Web Fraud Attacks Against LLM-Driven Multi-Agent Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Model (LLM)-driven multi-agent systems (MAS) are rapidly gaining popularity, and its inherent security risks are rapidly becoming a concern. |
| Approach: | They propose a novel attack manipulating unique structures of web links to deceive MAS by using homoglyph deception, sub-directory nesting, and parameter obfuscation. |
| Outcome: | The proposed attacks exploit unique structures of web links to deceive MAS . they exhibit significant destructive potential across different MAS architectures . |
RSDA: Restoring Stale Data Affinity via Dynamic Renovation Strategy for Mitigating Data Scarcity (2026.acl-long)
Copied to clipboard
Yidan Liang, Jia Zhu, Weijie Shi, Hanghui Guo, Yue Cui, Jiawei Shen, Guoqing Ma, Jingjiang Liu, Qingyu Niu, Yilin Wang, Shimin Di, Jiajie Xu
| Challenge: | High-quality data is the cornerstone of advancing large language models, but the supply of premium data is nearing depletion, while vast stale corpora remain underutilized. |
| Approach: | They propose a framework to restore stale data affinity by quantifying the latent value of samples and employing a dynamic renovation strategy selection mechanism to determine the optimal component-level strategy. |
| Outcome: | The proposed framework achieves performance improvements using less than 10% of the data volume, underscoring that the latent potential of stale corpora remains largely untapped. |
PIP: Perturbation-based Iterative Pruning for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are growing in size and complexity, causing significant challenges for their practical deployment in resource-constrained environments. |
| Approach: | They propose a double-view structured pruning method that combines information from two different views to iteratively prune those that struggle to distinguish between them. |
| Outcome: | The proposed method reduces the parameter count by approximately 20% while retaining over 85% of the original model’s accuracy across varied benchmarks. |
ProxyQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models (2024.acl-long)
Copied to clipboard
Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, Linqi Song
| Challenge: | Existing evaluation methods for large language models are labor-intensive and lack efficiency. |
| Approach: | They propose a framework dedicated to assessing long-text generation that includes in-depth human-curated meta-questions spanning various domains . they use a set of proxy-quests with pre-annotated answers to assess the content's quality by incorporating the generated texts as contextual background. |
| Outcome: | The proposed framework assesses the quality of long-text content by matching it with references through human evaluation or automated metrics. |
SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection (2026.acl-industry)
Copied to clipboard
Zhiyong Cao, Dunqiang Liu, Qi Dai, Haojun Xu, Huai Yuen Khor, Hao Wang, Huan He, Yafei Liu, Ke Ma, Ruqian Shi, Sicheng Zhou, Sijia Yao
| Challenge: | High-quality data in training proactive dialogue agents is scarce, despite fine-tuning and reinforcement learning . a recent study has shown that the effectiveness of supervised fine-touring is limited by the lack of high-quality, domain-specific training data. |
| Approach: | They propose a framework for training recruitment proactive dialogue agents using a high-fidelity user simulator and a multi-dimensional evaluation framework based on Chain-of-Intention. |
| Outcome: | The proposed framework outperforms existing simulator-based data selection strategies in a real-world recruitment scenario. |
Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a large computational cost for attention computation in large language models is a major obstacle . |
| Approach: | They propose a convolution-like structure for attention computation using convolution matrices . they then propose an efficient approximation method to approximate the attention matrix . |
| Outcome: | The proposed method achieves nearly linear time complexity in n1+o(1) time. |
Self-imitation Learning for Action Generation in Text-based Games (2023.eacl-main)
Copied to clipboard
| Challenge: | Text-based games are situated systems where the game agents observe textual descriptions, and generate textual commands to interact with the environment. |
| Approach: | They propose a confidence-based self-imitation model to generate action candidates for the RL agent by exploiting past valuable trajectories to adapt a pre-trained language model towards a target game. |
| Outcome: | The proposed model performs well in multiple challenging games. |
Medical Dialogue System: A Survey of Categories, Methods, Evaluation and Challenges (2024.findings-acl)
Copied to clipboard
Xiaoming Shi, Zeming Liu, Li Du, Yuxuan Wang, Hongru Wang, Yuhang Guo, Tong Ruan, Jie Xu, Xiaofan Zhang, Shaoting Zhang
| Challenge: | Existing medical dialogue systems have significant potential to simplify diagnostic procedure and reduce the cost of collecting information from patients. |
| Approach: | They analyze 325 papers from well-known computer science, natural language processing conferences and journals to find out the major challenges of medical dialog systems. |
| Outcome: | The proposed systems have been surveyed in the medical community but have not been evaluated from a technical perspective. |
SConU: Selective Conformal Uncertainty in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks fail to identify outliers that violate the exchangeability assumption, leading to unbounded miscoverage rates and unactionable prediction sets. |
| Approach: | They propose a method that implements significance tests to determine whether a given sample deviates from the uncertainty distribution of the calibration set. |
| Outcome: | The proposed approach facilitates rigorous management of miscoverage rates across single-domain and interdisciplinary contexts, and enhances the efficiency of predictions. |
CheckRLM: Effective Knowledge–Thought Coherence Checking in Retrieval-Augmented Reasoning (2026.acl-long)
Copied to clipboard
Dingling Xu, Ruobing Wang, Qingfei Zhao, Yukun Yan, Zhichun Wang, Daren Zha, Shi Yu, Zhenghao Liu, Shuo Wang, Xu Han, Maosong Sun
| Challenge: | Reasoning Language Models (RLMs) have improved performance on complex tasks by extending the reasoning chain, but they are prone to factual errors, especially in knowledge-intensive tasks. |
| Approach: | They propose a framework that improves the reliability of the reasoning process by timely checking and correcting factual errors. |
| Outcome: | The proposed framework outperforms baselines and shows that it mitigates error accumulation with lower costs. |
KCVR: Knowledge-Centric Video Reconstruction for Structured Pedagogical Summarization via Dynamic Graph Planning (2026.acl-long)
Copied to clipboard
Jingjiang Liu, Jia Zhu, Hanghui Guo, Weijie Shi, Yue Cui, Xiaokang Jin, Yilin Wang, Qingyu Niu, Jiawei Shen, Guoqing Ma, Yidan Liang, Shimin Di, Jiajie Xu
| Challenge: | Existing summarization methods compress content for gist browsing, but they break prerequisite logic in instructional videos. |
| Approach: | They propose a framework that decouples epistemic planning from content generation. |
| Outcome: | The proposed framework outperforms strong end-to-end baselines on Knowledge Progression Consistency and Learning Objective Coverage. |
LogiGraph: Logical Reasoning with Contrastive Learning and Lightweight Graph Networks (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods emphasize contextual semantics while others pay more attention to explicit logical features. Existing models utilize graph convolutional networks (GCN) for node updates, still exhibiting some shortcomings. |
| Approach: | They propose a logical reasoning method with contrastive learning and lightweight graph networks (LogiGraph) they employ conjunction and punctuation marks as two types of edges to construct a dual graph. |
| Outcome: | The proposed method improves the GCN and employs conjunction and punctuation marks as two types of edges to construct a dual graph. |
When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life (2026.findings-acl)
Copied to clipboard
Xinyue Lou, Xu Jinan, Jingyi Yin, Xiaolong Wang, Zhaolu Kang, null Liaoyouwei, Yixuan Wang, Xiangyu Shi, Fengran Mo, SU Yao, Kaiyu Huang
| Challenge: | Safety impact of Multimodal Large Language Models (MLLMs) on human behavior is evaluated in this study. |
| Approach: | They propose a safety-warning-based evaluation framework that encourages models to provide clear and informative safety warnings, rather than generic refusals. |
| Outcome: | The proposed safety-warning-based evaluation framework encourages models to provide clear and informative safety warnings, rather than generic refusals. |
LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning (2025.acl-long)
Copied to clipboard
Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo
| Challenge: | Existing legal judgment prediction methods struggle with logical errors when conducting complex legal reasoning. |
| Approach: | They propose a method which enhances LJP reliability through step-wise verification and correction of the reasoning process. |
| Outcome: | The proposed model significantly improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B. |
TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers’ Guidance (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant strides in problem-solving by incorporating reasoning processes, but this enhanced reasoning capability results in an increased number of output tokens during inference, leading to higher computational costs. |
| Approach: | They propose a method that internalizes explicit reasoning into the model’s habitual behavior through a Teacher-Guided compression strategy inspired by human cognition. |
| Outcome: | The proposed method reduces inference-time costs while maintaining high performance while preserving high quality and diversity of the distillation dataset. |
LCMA-SRT: Language-Conditional Mixture-of-Experts Adapters for Joint Multilingual Speech Recognition and Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing hierarchical transducers suffer from negative transfer and unstable target-language generation, while training separate models for each direction is computationally prohibitive. |
| Approach: | They propose a hierarchical transducer with language-conditional Mixture-of-Experts adapters to improve multilingual joint automatic speech recognition and speech translation. |
| Outcome: | Experiments on Europarl-ST (9 languages, 72 directions) show that LCMA-SRT improves both ASR and ST within a single joint model, reducing average WER and improving BLEU and COMET over strong hierarchical transducer baselines. |
Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (2024.findings-acl)
Copied to clipboard
Ran Xu, Hejie Cui, Yue Yu, Xuan Kan, Wenqi Shi, Yuchen Zhuang, May Dongmei Wang, Wei Jin, Joyce Ho, Carl Yang
| Challenge: | Clinical natural language processing (NLP) is a subfield that requires the extraction, analysis, and interpretation of unstructured clinical text. |
| Approach: | They propose a model which infuses knowledge into clinical text generation with LLMs for clinical NLP tasks. |
| Outcome: | The proposed model improves performance across 8 clinical NLP tasks and 18 datasets by 7.7%-8.7% on average. |
RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Records (2024.acl-short)
Copied to clipboard
| Challenge: | Existing deep learning models for EHRs rely on knowledge from a single source and do not capture the semantic information for medical codes. |
| Approach: | They propose a Retrieval AugMentation pipeline to augment clinical prediction on EHRs . they use multiple knowledge sources to convert them into text and use consistency regularization to capture complementary information from patient visits and summarized knowledge. |
| Outcome: | Experiments on two EHR datasets show that RAM-EHR improves clinical prediction tasks. |
MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding (2024.emnlp-main)
Copied to clipboard
Baixuan Xu, Weiqi Wang, Haochen Shi, Wenxuan Ding, Huihao Jing, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Long Chen, Yangqiu Song
| Challenge: | Existing methods for acquiring large-scale intentions generate product-centric intentions without product images and incur high costs for scalability. |
| Approach: | They propose a multimodal framework that allows Large Vision-Language Models to infer purchase intentions from multimodal product metadata and prioritize human-centric ones. |
| Outcome: | The proposed framework shows that it is robust to different prompts and superior to previous methods. |
The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis (2026.acl-long)
Copied to clipboard
Zihao Wei, Liang Pang, Jiahao Liu, Wenjie Shi, Jingcheng Deng, Shicheng Xu, Zenghao Duan, Jingang Wang, Fei Sun, Huawei Shen, Xueqi Cheng
| Challenge: | Explicit reasoning trajectories increase performance but often trigger overthinking . despite its importance, this study examines how each step of reasoning affects the final outcome . |
| Approach: | They propose a Reasoning Completion Point Detector that detects the RCP by monitoring rank dynamics of termination tokens. |
| Outcome: | The proposed method reduces token usage by up to 44% while preserving accuracy. |
Look Within or Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Parameter-Efficient Fine-Tuning (PEFT) is an alternative to Full-Parameter Fine-tuning, but its effectiveness on complex tasks such as reasoning and instruction-following remains unclear. |
| Approach: | They propose to use PEFT to reduce the number of trainable parameters while freezing the weights of LLMs. |
| Outcome: | The proposed methods perform well on standard tasks, but weaknesses on complex and adversarial settings call for new directions beyond current paradigms. |
DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating large language models face limitations in key aspects such as retrieval triggers and contextual scrutiny of retrieval content. |
| Approach: | They propose a dynamic RAG method that uses cognitive detection and contextual retrieval optimization to determine when retrieval is needed and what to retrieve for LLMs. |
| Outcome: | The proposed method achieves superior performance on all tasks, demonstrating the effectiveness of the proposed method. |
TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidation (2024.acl-long)
Copied to clipboard
| Challenge: | Current methods for Continual Dialogue State Tracking (DST) struggle with catastrophic forgetting and knowledge transfer between tasks. |
| Approach: | They propose a framework for task skill localization and consolidation that enables effective knowledge transfer without relying on memory replay. |
| Outcome: | The proposed framework shows a 7.6% increase in Avg. JGA and 11% rise in BWT metrics over existing state-of-the-art methods. |
AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Code (2026.findings-acl)
Copied to clipboard
ShangZhan Li, Xinyu Yin, Xuanyu Jin, Ye He, Yuxin Zhou, Yuxuan Li, Xu Han, Wanxiang Che, Qi Shi, Ting Liu, Maosong Sun
| Challenge: | Current development practices face a dichotomy between automation and performance. |
| Approach: | They propose a framework to empower LLMs with the capability of automated explicit vectorization. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the SSE and AVX subsets of SimdBench. |
UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle (2024.findings-naacl)
Copied to clipboard
Yutao Mou, Kexiang Wang, Jianhe Lin, Dehong Ma, Jun Fan, Daiting Shi, Zhicong Cheng, Gu Simiu, Dawei Yin, Weiran Xu
| Challenge: | Existing paradigms for pre-training and fine-tuning have limitations . knowledge rekindle aims to break through performance upper bounds of experts without introducing additional annotated data. |
| Approach: | They propose a new paradigm for pre-training and fine-tuning that aims to re-incorporate the fine- tuned expert model into the training cycle and break through performance upper bounds of experts. |
| Outcome: | The proposed model breaks through performance upper bounds of experts without additional annotated data. |
mABC: Multi-Agent Blockchain-inspired Collaboration for Root Cause Analysis in Micro-Services Architecture (2024.findings-emnlp)
Copied to clipboard
Wei Zhang, Hongcheng Guo, Jian Yang, Zhoujin Tian, Yi Zhang, Yan Chaoran, Zhoujun Li, Tongliang Li, Xu Shi, Liangfan Zheng, Bo Zhang
| Challenge: | Root cause analysis (RCA) in Micro-services architectures with escalating complexity is challenging due to fault propagation and circular dependencies among nodes. |
| Approach: | They propose a framework where multiple agents follow Agent Workflow and collaborate in blockchain-inspired voting to ensure the reliability of root cause analysis. |
| Outcome: | The proposed framework reduces the number of steps and standardizes task processing through Agent Workflow. |
Mitigating Legal Hallucinations via Symbolic Constraints and Analogical Precedents (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for finetuning and retrieval-augmented generation suffer from hallucination risk and semantic drift. |
| Approach: | They propose a framework for a dual-retriever based on the legal syllogism and the nature of different legal data. |
| Outcome: | The proposed framework mitigates hallucinations while improving explainability of legal reasoning. |
RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentation (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) enhances large language models by integrating external knowledge retrieved at inference time. |
| Approach: | They evaluate RAG systems using MassiveDS, a large-scale datastore with mixture of knowledge. |
| Outcome: | The proposed approach improves performance on knowledge-intensive NLP tasks. |
Aligning Cross-Lingual Entities with Multi-Aspect Information (D19-1)
Copied to clipboard
| Challenge: | Existing knowledge graphs that represent entities in different languages are not covered by existing systems. |
| Approach: | They propose two ways to embed entities from multilingual knowledge graphs into the same vector space, where equivalent entities are close to each other. |
| Outcome: | The proposed method significantly outperforms existing systems on two benchmark datasets. |
Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling (2026.findings-acl)
Copied to clipboard
Zhenghua Wang, Yiran Ding, Changze Lv, Yixin Wu, Tianlong Li, Zhibo Xu, Muling Wu, Tianyuan Shi, Shizheng Li, Qi Qian, Xuanjing Huang, Xiaoqing Zheng
| Challenge: | Existing methods to address the "lost-in-the-middle" problem suffer from high latency or suboptimal hand-crafted scaling strategies. |
| Approach: | They propose a layer-specific positional embedding scaling method that assigns distinct scaling factors to each layer. |
| Outcome: | Experiments show that the proposed method mitigates positional attention bias and delivers consistent improvements across multiple long-context benchmarks. |
ACSE: An Ancient Character Semantic-Aware Embedding for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on pre-Qin documents are insufficient to understand ancient characters . ancient characters have a low level of digitization and training corpora are extremely scarce . |
| Approach: | They propose a semantic-aware embedding for ancient Chinese characters that integrates glyphs and lexicality into modern Chinese semantic space. |
| Outcome: | The proposed model integrates glyph and lexicality of ancient characters and maps them to the modern Chinese semantic space. |
JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation (2026.findings-acl)
Copied to clipboard
Zhichao Shi, Xuhui Jiang, Chengjin Xu, Cangli Yao, Shengjie Ma, Yinghan Shen, Zixuan Li, Jian Guo, Yuanzhuo Wang
| Challenge: | Current evaluation methods for large language models rely on static benchmarks . limited knowledge coverage and fixed difficulties hinder the targeted optimizations resulting in superficial evaluations of LLMs - a problem that has been addressed by JudgeAgent . |
| Approach: | They propose a knowledge-driven and dynamic evaluation framework for large language models . judgeAgent leverages LLM agents equipped with context graphs to traverse knowledge structures . |
| Outcome: | The proposed framework can achieve comprehensive evaluations and facilitate effective model iterations. |
ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees (2024.findings-emnlp)
Copied to clipboard
Zhiyuan Wang, Jinhao Duan, Lu Cheng, Yue Zhang, Qingni Wang, Xiaoshuang Shi, Kaidi Xu, Heng Tao Shen, Xiaofeng Zhu
| Challenge: | Uncertainty quantification (UQ) in natural language generation tasks remains an open challenge . however, black-box uncertainty measures require investigating with the proliferation of LLMs served via APIs. |
| Approach: | They propose a conformal uncertainty measure and a method to transform heuristic uncertainty notions into rigorous prediction sets. |
| Outcome: | Empirical results show that the proposed method outperforms state-of-the-art methods and can provide reliable guarantees for open-ended NLG tasks. |
MultiSkill: Evaluating Large Multimodal Models for Fine-grained Alignment Skills (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation settings for large multimodal models focus on coarse-grained evaluation without considering skill composition required by specific instructions. |
| Approach: | They propose an evaluation protocol that assesses large multimodal models across multiple fine-grained skills for alignment with human values. |
| Outcome: | The proposed evaluation protocol decomposes coarse-level scoring to fine-grained skill set-level score tailored to each instruction. |
MedEthicEval: Evaluating Large Language Models Based on Chinese Medical Ethics (2025.naacl-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been used in clinical decision support, medical education and patient communication. |
| Approach: | They propose a benchmark to evaluate large language models in the domain of medical ethics and assess their grasp of medical ethical principles and their application across diverse scenarios. |
| Outcome: | The proposed framework assesses the models’ grasp of medical ethics principles and their ability to apply them across diverse scenarios. |
Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Model (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing Large Reasoning Models have demonstrated broad application potential, yet their safety and reliability remain critical concerns. |
| Approach: | They conduct a safety evaluation of 13 MLRMs across 5 benchmarks and examine their safety performance. |
| Outcome: | The proposed model improves safety on jailbreak and safety-awareness benchmarks. |
Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slips (2025.coling-main)
Copied to clipboard
| Challenge: | Using a multi-modal multi-granularity tokenizer, we analyze ancient Chinese scripts . a large proportion of the characters in ancient Chinese are rare or undeciphered . |
| Approach: | They propose a multi-modal multi-granularity tokenizer specifically designed for ancient Chinese scripts. |
| Outcome: | The proposed tokenizer improves on the part-of-speech tagging task on the Chu bamboo slip script. |
CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning (2024.acl-long)
Copied to clipboard
Weiqi Wang, Tianqing Fang, Chunyang Li, Haochen Shi, Wenxuan Ding, Baixuan Xu, Zhaowei Wang, Jiaxin Bai, Xin Liu, Cheng Jiayang, Chunkit Chan, Yangqiu Song
| Challenge: | Existing approaches to generalize commonsense reasoning lack instantiated knowledge and require pre-built concept taxonomies and annotations. |
| Approach: | They propose a framework that iteratively performs contextualized conceptualization and instantiation over commonsense knowledge bases by instructing large language models to generate both types of knowledge with critic filtering. |
| Outcome: | Empirical results show that distilling CANDLE on student models provides benefits across three downstream tasks. |
Making RALM Robust to Irrelevant Contexts via Layer Knowledge Guided Attention (2025.findings-acl)
Copied to clipboard
Weijie Shi, Hao Chen, Jiaming Li, Yao Zhao, Yazhong Zhang, Qijin Chen, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Xiaofang Zhou
| Challenge: | Large language models (LLMs) face factual hallucination and knowledge obsolescence when tackling knowledge-intensive tasks. |
| Approach: | They propose a layer-knowledge guided attention method which harnesses the layer-wise knowledge of large language models to optimize per-layer attention on useful passages. |
| Outcome: | The proposed method outperforms existing methods on RALM benchmarks. |
Dialect-SQL: An Adaptive Framework for Bridging the Dialect Gap in Text-to-SQL (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing Text-to-SQL research focuses on specific database systems, limiting adaptability to different dialects. |
| Approach: | They propose a framework that employs Object Relational Mapping (ORM) code as an intermediate language to bridge this gap. |
| Outcome: | The proposed framework outperforms existing methods that generate SQL queries directly. |
M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for retrieving relevant memories from an external database are coarse-grained and can cause noise and focus on crucial memories. |
| Approach: | They propose a multiple partition paradigm for RAG where each database partition serves as a basic unit for execution. |
| Outcome: | The proposed framework outperforms baseline methods on three language generation tasks on seven datasets. |
Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have focused on replacing the reward model in Reinforcement Learning with Human Feedback (RLHF) methods for Large Language Models (LLMs). |
| Approach: | They propose a self-supervised preference optimization framework that replaces the reward model with a preference loss and alignment loss to improve LLMs' ability to understand human preferences. |
| Outcome: | The proposed framework can be integrated with existing preference optimization methods and significantly boost their performance. |
AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026.acl-long)
Copied to clipboard
Keyu Li, Junhao Shi, Yang Xiao, Mohan Jiang, Jie Sun, Yunze Wu, Dayuan Fu, Shijie Xia, Xiaojie Cai, Tianze Xu, Weiye Si, Wenjie Li, Dequan Wang, Pengfei Liu
| Challenge: | Existing benchmarks focus on single agentic capability, failing to capture long-horizon real-world scenarios. |
| Approach: | They propose a benchmark that evaluates 6 agentic capabilities across 32 real-world scenarios. |
| Outcome: | Experiments show that closed-source models outperform open-source model (48.4% vs 32.1%) integrating models with advanced scaffolds to form autonomous agents is a paradigm shift. |
Benchmarking Multi-National Value Alignment for Large Language Models (2025.findings-acl)
Copied to clipboard
Chengyi Ju, Weijie Shi, Chengzhong Liu, Jiaming Ji, Jipeng Zhang, Ruiyuan Zhang, Jiajie Xu, Yaodong Yang, Sirui Han, Yike Guo
| Challenge: | Existing studies on large language models focus on ethical reviews, failing to capture the diversity of national values. |
| Approach: | They propose a national value extraction pipeline to efficiently construct value assessment datasets and a model-based model with instruction tagging to process raw data sources. |
| Outcome: | The proposed benchmark evaluates the alignment of LLMs with the values of five major nations: China, the United States, the UK, France, and Germany. |
Similarity = Value? Consultation Value-Assessment and Alignment for Personalized Search (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods rely on semantic similarity to align historical consultations with current queries due to the absence of ‘value’ labels, but this lacks exploration of needs in user consultations. |
| Approach: | They propose a consultation value assessment framework that evaluates historical consultations from three novel perspectives: (1) Scenario Scope Value, (2) Posterior Action Value, and (3) Time Decay Value. |
| Outcome: | The proposed model outperforms baselines on public and commercial datasets on both retrieval and ranking tasks. |
MM-ChatAlign: A Novel Multimodal Reasoning Framework based on Large Language Models for Entity Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing MMEA methods rely on knowledge representation learning (KRL) to measure the similarity of entity embeddings. |
| Approach: | They propose a framework that utilizes the visual reasoning abilities of MLLMs for multimodal entity alignment. |
| Outcome: | The proposed framework integrates the visual reasoning abilities of MLLMs for multimodal entity alignment. |
AI2Agent: An End-to-End Framework for Deploying AI Projects as Autonomous Agents (2025.acl-demo)
Copied to clipboard
Jiaxiang Chen, Jingwei Shi, Lei Gan, Jiale Zhang, Qingyu Zhang, Dongqian Zhang, Pang Xin, Zhucong Li, Xu Yinghui
| Challenge: | a new framework automates deployment and debugging of AI projects . complexity of environment configurations, dependency conflicts, and debuggering issues hinder scalability and adoption. |
| Approach: | They propose an end-to-end framework that automates AI project deployment . they conducted experiments on 30 AI deployment cases to evaluate its effectiveness . |
| Outcome: | The proposed framework reduces deployment time and improves success rates by reducing human intervention. |
LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)
Copied to clipboard
Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size. |
| Approach: | They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding. |
| Outcome: | The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models. |
GenLink: Generation-Driven Schema-Linking via Multi-Model Learning for Text-to-SQL (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results on BIRD and Spider benchmarks validate the effectiveness of GenLink. |
| Approach: | They propose a generation-driven schema-linking framework based on multi-model learning . experimental results validate the effectiveness of GenLink . |
| Outcome: | Experimental results show that GenLink improves schema-linking recall rate and cross-domain adaptability. |
On Synthetic Data for Back Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on back translation (BT) focus on beam search or random sampling . a new method to generate synthetic data with a backward model is proposed to improve BT performance. |
| Approach: | They propose a method to generate synthetic data to trade off quality and importance factors . back translation (BT) is one of the most significant technologies in NMT research fields . |
| Outcome: | The proposed method outperforms the baseline methods on WMT14 DE-EN, EN-DE, and RU-EN benchmark tasks. |
DataArc-SynData-Toolkit: A Unified Closed-Loop Framework for Multi-Path, Multimodal, and Multilingual Data Synthesis (2026.acl-demo)
Copied to clipboard
Zhichao Shi, Cehao Yang, Hao Zhou, Xiaojun Wu, Huajie Li, Xuhui Jiang, Chengjin Xu, Yuanzhuo Wang, Jian Guo
| Challenge: | Existing synthetic data tools are limited by convoluted workflows, fragmented data standards, and limited scalability across modalities. |
| Approach: | They develop an open-source framework that aims to reduce the technical barrier to synthetic data generation and subsequent model training. |
| Outcome: | The proposed framework achieves an optimal balance between generation efficiency and data quality. |
WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback (2026.acl-long)
Copied to clipboard
Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville
| Challenge: | Traditional alignment methods rely on human annotations and are subjective and misalignment with real-world user preferences. |
| Approach: | They propose a framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically. |
| Outcome: | The proposed framework identifies and classifies user feedback to LLM responses between conversation turns and creates examples of preferred and dispreferred responses according to user preferences. |
Gen-SQL: Efficient Text-to-SQL By Bridging Natural Language Question And Database Schema With Pseudo-Schema (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies have shifted paradigms and leveraged Large Language Models (LLMs) to tackle the challenging task of Text-to-SQL. |
| Approach: | They propose a framework that leverages large language models to generate SQL queries . they exploit prior knowledge from the LLM to enhance embedding-based retriever . |
| Outcome: | The proposed method improves embedding-based retriever and reduces cost. |
Graph Neural News Recommendation with Unsupervised Preference Disentanglement (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to learn informative user and news representations fail to consider high-order connectivity underlying the user-news interactions. |
| Approach: | They propose a novel Graph Neural News Recommendation model with Unsupervised Preference Disentanglement which can encode high-order relationships into user and news representations by information propagation along the graph. |
| Outcome: | The proposed model can encode high-order relationships into user and news representations by information propagation along the graph and disentangle latent preference factors by a neighborhood routing algorithm. |
RATION: Entropy-Driven Task-Adaptive Visual Attention Allocation Framework for Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior studies have focused on strengthening multimodal reasoning by improving representation alignment or increasing computation, but these methods do not characterize the differences in visual demands across tasks. |
| Approach: | They propose an entropy-driven task-adaptive visual attention allocation framework that uses visual attention entropic as a control signal to dynamically allocate attention according to task demands. |
| Outcome: | The proposed framework achieves consistent performance gains across diverse reasoning tasks, datasets, and models, providing a clear direction toward more reliable multimodal reasoning. |
ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models (2025.coling-main)
Copied to clipboard
Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, Maosong Sun
| Challenge: | Activation sparsity is a promising paradigm for accelerating model inference . few large language models achieve high activation spar and comparable performance . |
| Approach: | They propose a method to achieve activation sparsity and acceleration in large language models . they introduce ReLU activation and adopt progressive sparse regularization . |
| Outcome: | The proposed method achieves high activation sparsity and comparable model performance. |
RLShield: Dynamic Jailbreak Detection for LLMs via Reinforced Adaptive Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to detect jailbreak prompts rely on static model components or fixed decision thresholds. |
| Approach: | They propose a dynamic jailbreak detection framework that employs reinforcement learning for adaptive threshold selection. |
| Outcome: | Experimental results show that the framework outperforms baselines in detection performance while maintaining high computational efficiency. |