Papers by Ye Liu
Enhancing Hierarchical Text Classification through Knowledge Graph Integration (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to hierarchical text classification are limited by lack of domain knowledge, which leads to mistakes in a variety of situations. |
| Approach: | They propose a Knowledge-enabled Hierarchical Text Classification model which integrates knowledge graphs into HTC to address the knowledge limitations of traditional methods. |
| Outcome: | The proposed model integrates knowledge graphs into the hierarchical text classification process, addressing the knowledge limitations of traditional methods. |
ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for code execution reasoning are limited by the difficulty of the training data. |
| Approach: | They propose a model that uses reinforcement learning to reward correct answers from execution traces. |
| Outcome: | The proposed model improves pass@1 by up to 5.9% on code generation tasks over strong baselines. |
Optimizing Code Retrieval: High-Quality and Scalable Dataset Annotation through Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for code retrieval struggle to balance scalability and annotation quality. |
| Approach: | They propose a method that integrates functions called within the repository and information on third-party APIs to enhance the annotation context. |
| Outcome: | The proposed method improves the annotation context by incorporating functions called within the repository and information on third-party API functionalities. |
ParaSuite: Boosting LLM Reasoning via Paradox Resolution (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for paradox research focus on checking basic logical consistency and not reflective reasoning. |
| Approach: | They propose a pipeline dedicated to paradox research that automates data synthesis, evaluation, and training. |
| Outcome: | The proposed pipeline improves paradoxical and general STEM reasoning. |
CLEME2.0: Towards Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction (2025.acl-long)
Copied to clipboard
Jingheng Ye, Zishan Xu, Yinghui Li, Linlin Song, Qingyu Zhou, Hai-Tao Zheng, Ying Shen, Wenhao Jiang, Hong-Gee Kim, Ruitong Liu, Xin Su, Zifei Shan
| Challenge: | Existing studies have focused on the interpretability of Grammatical Error Correction (GEC) evaluation metrics, but the interpretabilty of these metrics has been neglected. |
| Approach: | They propose a reference-based metric that describes four aspects of GEC systems: hit-correction, wrong-corrections, under-correcties, and over-corrects. |
| Outcome: | The proposed metric reveals critical qualities and locates drawbacks of GEC systems. |
Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Numerous architectures and pretraining methods have been proposed for context-dependent text-to-SQL, but the size of the datasets used has been limited due to the high cost of annotating multi-turn dialogue and SQL pairs. |
| Approach: | They propose to augment training datasets using self-play which leverages contextual information to synthesize new interactions to adapt the model to new databases. |
| Outcome: | The proposed model improves accuracy on SParC and CoSQL, two widely used cross-domain text-to-SQl datasets. |
CP-BCS: Binary Code Summarization Guided by Control Flow Graph and Pseudo Code (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current work on understanding assembly code is oriented towards generating function names, which involve numerous abbreviations that make them confusing. |
| Approach: | They propose a control flow graph and pseudo code guided binary code summarization framework to learn the comprehensive binary function execution behavior and logic semantics. |
| Outcome: | The proposed framework improves the efficiency of reverse engineering on 3 different binary optimization levels for 3 different computer architectures. |
GQLBench: A Large-Scale Cross-Domain, Cross-Dialect Benchmark for NL2GQL (2026.acl-long)
Copied to clipboard
| Challenge: | despite growing interest in NL2GQL, benchmarking progress has been constrained by the lack of resources that are simultaneously large-scale, cross-domain, and cross-dialect. |
| Approach: | They propose a framework that integrates NL2SQL-to-NL2GQL conversion with graph-native data generation. |
| Outcome: | The proposed framework supports execution-based evaluation on Cypher and ISO-GQL, covering hundreds of graph databases and over 20k natural language questions for each dialect. |
Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification (2025.acl-long)
Copied to clipboard
Chengwu Liu, Ye Yuan, Yichun Yin, Yan Xu, Xin Xu, Zaoyu Chen, Yasheng Wang, Lifeng Shang, Qun Liu, Ming Zhang
| Challenge: | Chain-of-Thought prompting is a de facto method to elicit reasoning capabilities from large language models (LLMs). |
| Approach: | They propose a step-aware formal verification framework Safe to address hallucinations in CoT prompting . they propose 'formal step' as a benchmark for step correctness theorem proving with 30,809 formal statements. |
| Outcome: | The proposed framework shows significant performance improvement while offering interpretable and verifiable evidence. |
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has focused on pushing weight-only quantization to extremely low-bit due to numerical representation limitations. |
| Approach: | They propose a vector-based quantization approach that pushes LLMs to extremely low-bit . they propose scalar-based weight quantization that reduces memory requirements and optimizes storage costs . |
| Outcome: | The proposed method reduces model quantization perplexity by 0.01-0.34 on LLaMA-2, 0.38-0.68 on mistral-7B, 4.41-7.34, on llaMA-3 on QA tasks on average. |
On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation (2021.acl-long)
Copied to clipboard
Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jiawei Low, Lidong Bing, Luo Si
| Challenge: | Existing studies have shown that adapter-based tuning is more parameter-efficient than fine-tuning. |
| Approach: | They propose to add adapter modules to a pretrained language model and update the parameters of adapter module when learning on a downstream task. |
| Outcome: | The proposed method outperforms fine-tuning on low-resource and cross-lingual tasks and settings. |
HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference (2025.findings-acl)
Copied to clipboard
Ping Gong, Jiawei Yi, Shengnan Wang, Juncheng Zhang, Zewen Jin, Ouxiang Zhou, Ruibo Liu, Guanbin Xu, Youhui Bai, Bowen Ye, Kun Yuan, Tong Yang, Gong Zhang, Renhai Chen, Feng Wu, Cheng Li
| Challenge: | Existing top-k attention methods struggle to strike a balance between efficiency and accuracy. |
| Approach: | They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention. |
| Outcome: | The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy. |
AROMA: Augmented Reasoning Over a Multimodal Architecture for Virtual Cell Genetic Perturbation Modeling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for virtual cell genetic perturbation modeling suffer from unconstrained reasoning, uninterpretable predictions, and retrieval signals that are weakly aligned with regulatory topology. |
| Approach: | They propose an Augmented Reasoning Over a Multimodal Architecture for virtual cell genetic perturbation modeling. |
| Outcome: | The proposed model outperforms existing methods across multiple cell lines and remains robust under zero-shot evaluation on unseen cells. |
FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)
Copied to clipboard
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev
| Challenge: | Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity. |
| Approach: | They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models. |
| Outcome: | The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models. |
HCLD: A Hierarchical Framework for Zero-shot Cross-lingual Dialogue System (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to train task-oriented dialogue systems in monolingual datasets are expensive to build. |
| Approach: | They propose a hierarchical framework to classify intents in high-level and slot filling in low-level . they incorporate sentence-level alignment among different languages to enhance intent detection . |
| Outcome: | The proposed framework achieves the performance on a public task-oriented dialog dataset. |
Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to augment pre-trained large language models require extensive computational efforts and massive data volumes, challenging the widespread accessibility of LLM research. |
| Approach: | They propose a post-pretraining strategy of selectively enhancing shallow layers while pruning less effective deep ones to augment pretrained large language models. |
| Outcome: | The proposed approach improves performance on the corpus of code & math and a legal corpus and is widely applicable. |
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
On Learning to Summarize with Large Language Models as References (2024.naacl-long)
Copied to clipboard
Yixin Liu, Kejian Shi, Katherine He, Longtian Ye, Alexander Fabbri, Pengfei Liu, Dragomir Radev, Arman Cohan
| Challenge: | Recent studies have found that summaries generated by large language models (LLMs) are favored by human annotators when compared to reference summary from widely used summarization datasets. |
| Approach: | They propose to use large language models (LLMs) as reference learning settings for smaller text summarization models to investigate whether their performance can be substantially improved. |
| Outcome: | The proposed model outperforms standard supervised fine-tuning and human evaluations while retaining human-level performance. |
Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search Trajectories (2026.acl-long)
Copied to clipboard
| Challenge: | MCTS methods retain only the single highest-reward trajectory, discarding comparative signals present in the many explored paths. |
| Approach: | They propose a framework that transforms supervision extraction into a synthesis procedure. |
| Outcome: | The proposed framework matches or exceeds baselines on 60K CRPS-synthesized examples on out-of-domain benchmarks. |
GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization (2024.emnlp-main)
Copied to clipboard
Yangfan Ye, Xiachong Feng, Xiaocheng Feng, Weitao Ma, Libo Qin, Dongliang Xu, Qing Yang, Hongtao Liu, Bing Qin
| Challenge: | Current studies focus on single-language or single-document tasks for news summarization . lack of a benchmark inhibits researchers from adequately studying this invaluable problem. |
| Approach: | They propose a novel task that unifies Multi-lingual, Cross-lingual and Multi-document Summarization into one task. |
| Outcome: | The proposed task encapsulates the real-world requirements all-in-one and is validated by extensive analysis. |
Leveraging Entity Information for Cross-Modality Correlation Learning: The Entity-Guided Multimodal Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Summarization with Multimodal Output (MSMO) is a new approach to produce a multimodal summary that integrates both text and relevant images. |
| Approach: | They propose an Entity-Guided Multimodal Summarization model that integrates both text and relevant images to produce a multimodal summary. |
| Outcome: | The proposed model integrates text-image and entity-image information and refines image selection through knowledge distillation from a pre-trained vision-language model. |
RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024.emnlp-demo)
Copied to clipboard
Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, Xiaoyin Che, Zhiyuan Liu, Maosong Sun
| Challenge: | Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation. |
| Approach: | They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation. |
| Outcome: | The proposed framework generates high-quality documentation for the entire project. |
LoopCoder: Scaling Code Intelligence via Looped Language Models (2026.findings-acl)
Copied to clipboard
Jian Yang, Wei Zhang, Shuyue Guo, Yizhi LI, Linzheng Chai, Zhengmao Ye, Shukai Liu, Yuyang Song, Jiajun Wu, Che Liu, Tianyu Zheng, Siwei Wu, Leo L, Xudong Ma, Chuan Hao, Ran Tao, Yan Xing, Jianzhou Wang, Mingjie Tang, Aishan Liu, Zhoujun Li, Xianglong Liu, Weifeng Lv, Bryan Dai
| Challenge: | Large language models have mastered syntax-level code generation, but complex algorithmic reasoning remains a challenge. |
| Approach: | They propose a recurrent inductive bias that aligns with the recursive nature of programming logic. |
| Outcome: | The proposed model achieves comparable performance to standard dense models with more parameters. |
Coreferential Reasoning Learning for Language Representation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing language representation models cannot explicitly handle coreference, which is essential to the coherent understanding of the whole discourse. |
| Approach: | They propose a language representation model that captures coreferential relations in context. |
| Outcome: | The proposed model can achieve significant improvements on downstream NLP tasks while maintaining comparable performance to baseline models on other common NLP task. |
PRISM: Probabilistic Reward Model with Inherent Structural Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluators compress diverse human judgments into a single scalar, leading to brittle alignment and reward hacking. |
| Approach: | They propose a Gaussian-based reinterpretation of reward evaluation as a conditional distribution and a mixture of Gaussians to capture conflicting preference dimensions. |
| Outcome: | The proposed model outperforms scalar baselines in accuracy and generalization. |
Grafting Pre-trained Models for Multimodal Headline Generation (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to generate video headlines with pre-trained language models are labor intensive and impractical. |
| Approach: | They propose to graft the encoder from the pre-trained video-language model on the generative pre-trainer model and propose a consensus fusion mechanism for the integration of different components. |
| Outcome: | The proposed model achieves strong results on a brand-new dataset collected from real-world applications. |
Make-A-Voice: Revisiting Voice Large Language Models as Scalable Multilingual and Multitask Learners (2024.acl-long)
Copied to clipboard
Rongjie Huang, Chunlei Zhang, Yongqi Wang, Dongchao Yang, Jinchuan Tian, Zhenhui Ye, Luping Liu, Zehan Wang, Ziyue Jiang, Xuankai Chang, Jiatong Shi, Chao Weng, Zhou Zhao, Dong Yu
| Challenge: | Large language models (LLMs) have been used for general-purpose interfaces across multiple tasks and languages. |
| Approach: | They propose to use large language models as a general-purpose interface across multiple tasks and languages. |
| Outcome: | The proposed model performs better on 200K hours of 6-language data for voice generation applications. |
Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented LLM Personalization (2026.acl-long)
Copied to clipboard
Linfeng Du, Ye Yuan, Zichen Zhao, Fuyuan Lyu, Emiliano Penaloza, Xiuying Chen, Zipeng Sun, Jikun Kang, Laurent Charlin, Xue Liu, Haolun Wu
| Challenge: | Existing approaches for personalizing large language models require modifying parameters. |
| Approach: | They propose a lightweight approach to personalizing large language models via retrieval augmentation . relevance serves as an unreliable proxy for utility, they argue . |
| Outcome: | The proposed framework outperforms strong heuristic and retrieval-augmented baselines on nine personalization tasks. |
Rehearsal-free Continual Language Learning via Efficient Parameter Isolation (2023.acl-long)
Copied to clipboard
Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, Wenqiu Zeng
| Challenge: | Existing methods for learning continual tasks do not cache history data, which makes the problem more challenging. |
| Approach: | They propose a method that allocates a small portion of private parameters and learns them with a shared pre-trained model. |
| Outcome: | The proposed method is comparable to existing methods and comparable to those using historical data. |
ExplainaBoard: An Explainable Leaderboard for NLP (2021.acl-demo)
Copied to clipboard
Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaichen Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Graham Neubig
| Challenge: | Using leaderboards, researchers can track the performance of various systems on various NLP tasks. |
| Approach: | They propose a new conceptualization and implementation of NLP evaluation using a leaderboard. |
| Outcome: | The ExplainaBoard is an evaluation tool for natural language processing (NLP) it covers more than 400 systems, 50 datasets, 40 languages, and 12 tasks. |
Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive models have boosted the efficiency of neural machine translation, but their performance is significantly worse than that of autoregressive counterparts. |
| Approach: | They propose to incorporate syntactic and semantic structures among natural languages into a non-autoregressive Transformer for the task of neural machine translation. |
| Outcome: | The proposed model achieves faster speed and keeps translation quality compared with other models. |
NOVA-63: Native Omni-lingual Versatile Assessments of 63 Disciplines (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual benchmarks show severe drawbacks, such as overly translated content, the absence of difficulty control, and disciplinary imbalance, making the benchmarking process unreliable and showing low convincingness. |
| Approach: | They propose a multilingual benchmark that integrates LLM-assisted formatting, expert quality verification, and multi-level difficulty screening to provide a comprehensive, difficult multilingual assessment. |
| Outcome: | The proposed benchmark features 93,536 questions sourced from native speakers across 14 languages and 63 academic disciplines. |
DARM: Distribution-Aware Reward Modeling by Alleviating Biases from Low Preference-Context Dependency Data (2026.acl-long)
Copied to clipboard
Shaofan Liu, Guoqiang Zhang, Shihan Dou, Huiyuan Zheng, Yiming Zhou, Junjie Ye, Shaowen Wang, Shichun Liu, Jiazheng Zhang, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing methods for training reward models are vulnerable to context neglect and degraded accuracy. |
| Approach: | They propose distribution-aware reward modeling that augments the RM objective with a conditional mutual information regularizer that maximizes context and the predicted reward conditioned on the response. |
| Outcome: | The proposed model improves performance in RLHF and improves accuracy in other settings. |
FACTS: Table Summarization via Offline Template Generation with Agentic Workflows (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for query-focused table summarization struggle with complex reasoning and token-limit issues. |
| Approach: | They propose a Fast, Accurate, and Privacy-Compliant table summarization approach via Offline Template Generation. |
| Outcome: | The proposed method outperforms baseline methods on widely-used benchmarks. |
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling (2024.acl-long)
Copied to clipboard
Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yu-Gang Jiang, Xipeng Qiu
| Challenge: | Existing language models that use discrete representations for unified processing of various modalities are limited to text generation and do not include multimodal output. |
| Approach: | They propose a multimodal language model that utilizes discrete representations for unified processing of various modalities. |
| Outcome: | The proposed model can be trained stably without any alterations to existing models or training paradigms. |
PsyScam: A Benchmark for Psychological Techniques in Real-World Scams (2025.findings-emnlp)
Copied to clipboard
| Challenge: | PTs are employed by scammers to manipulate victims and cause lasting psychological trauma. |
| Approach: | They propose a benchmark to capture the PTs employed in real-worldscam reports and investigate how LLMs can be utilized to generate variants of scams based on the pts and the contexts provided by thesescams. |
| Outcome: | The proposed model can generate variants of scams based on the PTs employed in real-world scam reports and the contexts provided by these scams. |
Toward Structured Knowledge Reasoning: Contrastive Retrieval-Augmented Generation on Experience (2025.findings-acl)
Copied to clipboard
Jiawei Gu, Ziting Xian, Yuanzhen Xie, Ye Liu, Enjie Liu, Ruichao Zhong, Mochi Gao, Yunzhi Tan, Bo Hu, Zang Li
| Challenge: | Large language models struggle to infer implicit relationships embedded in tabular formats . authors introduce a framework that builds experience memory representations and enhances generalization through contrastive In-Context Learning (ICL). |
| Approach: | They propose a framework that builds experience memory representations and enhances generalization through contrastive In-Context Learning to simulate human-like knowledge transfer. |
| Outcome: | Experiments on Text-to-SQL and TableQA show CoRE significantly improves performance . it achieves gains of 3.44% and 4.24%, with up to 17.2% on challenging tasks . |
TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities (2024.emnlp-main)
Copied to clipboard
Ming Zhang, Caishuang Huang, Yilong Wu, Shichun Liu, Huiyuan Zheng, Yurui Dong, Yujiong Shen, Shihan Dou, Jun Zhao, Junjie Ye, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Current datasets cater to user-led systems and are limited to predefined specific scenarios and slots. |
| Approach: | They propose to use a Chinese dialogue dataset to train a model that authentically simulates human-computer dialogues in 30 popular life service scenarios. |
| Outcome: | The proposed model achieves a joint accuracy of 75.09% in out-of-domain evaluations . it also achieves notable abilities in slot filling and questioning . |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (2024.findings-emnlp)
Copied to clipboard
Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan
| Challenge: | Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales. |
| Approach: | They propose to annotate a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans. |
| Outcome: | The proposed model outperforms existing methods on understanding the capabilities of LLMs in logical reasoning by 10% or more. |
kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning (2024.naacl-long)
Copied to clipboard
Wenting Zhao, Ye Liu, Yao Wan, Yibo Wang, Qingyang Wu, Zhongfen Deng, Jiangshu Du, Shuaiqi Liu, Yunlong Xu, Philip Yu
| Challenge: | Recent advances in task-oriented parsing involve formulating the task as a sequence-to-sequence problem, relying on a wealth of labeled data. |
| Approach: | They propose a task-oriented parsing framework that integrates nearest-neighbor learning with a nearest-nearest approach. |
| Outcome: | The proposed model can be used to synthesize computer programs based on a natural-language prompt without additional data or specialized prompts. |
MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering (2025.findings-acl)
Copied to clipboard
Jingqun Tang, Qi Liu, Yongjie Ye, Jinghui Lu, Shu Wei, An-Lan Wang, Chunhui Lin, Hao Feng, Zhen Zhao, Yanjie Wang, Yuliang Liu, Hao Liu, Xiang Bai, Can Huang
| Challenge: | Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool. |
| Approach: | They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images . |
| Outcome: | The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images. |
QSpell 250K: A Large-Scale, Practical Dataset for Chinese Search Query Spell Correction (2025.naacl-industry)
Copied to clipboard
| Challenge: | Chinese Search Query Spell Correction is a task designed to identify and correct typographical errors within queries. |
| Approach: | They propose a large-scale benchmark specifically developed for Chinese Query Spell Correction. |
| Outcome: | The proposed benchmark covers a broad range of topics, including formal entities, everyday colloquialisms and idiomatic expressions. |
KELE: Residual Knowledge Erasure for Enhanced Multi-hop Reasoning in Knowledge Editing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge editing techniques show limitations when applied to multi-hop reasoning . residual single-hop knowledge causes edited models to revert to original answers . |
| Approach: | They propose a knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE) they propose an erasure function for residual knowledge and an injection function for new knowledge . |
| Outcome: | The proposed method significantly improves multi-hop reasoning capability of edited models. |
MusKGC: A Flexible Multi-source Knowledge Enhancement Framework for Open-World Knowledge Graph Completion (2025.emnlp-main)
Copied to clipboard
| Challenge: | Open-world knowledge graph completion (KGC) aims to infer novel facts by enriching existing graphs with external knowledge sources while maintaining semantic consistency under the open-world assumption (OWA). |
| Approach: | They propose a multi-source knowledge enhancement framework based on an open-world assumption (OWA) that integrates external knowledge sources and a new evaluation strategy to validate new facts. |
| Outcome: | The proposed model achieves SOTA performance across benchmarks and the evaluation strategy effectively assesses new facts under OWA. |
RMSSinger: Realistic-Music-Score based Singing Voice Synthesis (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for singing voice synthesis are limited to fine-grained music scores . manual adjustment destroys regularity of note durations, making fine-grain music scores "crushed" |
| Approach: | They propose a method to synthesize singing voices given realistic music scores . they use real-music-score-based Singing Voice Synthesis to generate high-quality voices . |
| Outcome: | The proposed method eliminates manual annotation and simplifies phoneme-level mel-note alignment. |
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing VideoQA models struggle to adapt to new questions or tasks posed by newly available content. |
| Approach: | They propose a continual learning framework that fine-tunes a large language model for a sequence of tasks and integrates specific question constraint prompting, knowledge acquisition prompting and visual temporal awareness prompting. |
| Outcome: | The proposed model achieves 55.14% accuracy on both NExT-QA and DramaQA datasets and 71.24% accuracy for DramaQA. |
Towards More Fine-grained and Reliable NLP Performance Prediction (2021.eacl-main)
Copied to clipboard
| Challenge: | Performance prediction is a task of estimating a system’s performance without performing experiments. |
| Approach: | They propose to understand reliability of performance prediction models from two angles: confidence intervals and calibration. |
| Outcome: | The proposed methods demonstrate the feasibility of fine-grained performance prediction and the necessity to perform reliability analysis for performance prediction methods in the future. |
AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios (2025.naacl-long)
Copied to clipboard
Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, Zhongyu Wei
| Challenge: | Large language models are increasingly employed to empower autonomous agents to simulate human behavior. |
| Approach: | They propose to evaluate LLM-driven agents through multi-turn interactions using a bottom-up approach to create diverse social scenarios constructed from extensive scripts. |
| Outcome: | The proposed model evaluates LLM-driven agents through multi-turn interactions emphasizing goal completion and implicit reasoning. |
GenPilot: A Multi-Agent System for Test-Time Prompt Optimization in Image Generation (2025.findings-emnlp)
Copied to clipboard
Wen Ye, Zhaocheng Liu, Gui Yuwei, Tingyu Yuan, Yunyue Su, Bowen Fang, Chaoyang Zhao, Qiang Liu, Liang Wang
| Challenge: | Existing methods for text-to-image synthesis lack systematic error analysis and refinement strategies, resulting in limited reliability and effectiveness. |
| Approach: | They propose a plug-and-play multi-agent system called GenPilot that integrates error analysis, clustering-based adaptive exploration, fine-grained verification and a memory module for iterative optimization. |
| Outcome: | The proposed method improves text consistency and structural coherence on images with a plug-and-play system. |
ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback (2026.findings-acl)
Copied to clipboard
| Challenge: | Unlike chatbots, autonomous agents act directly on external environments, making tool invocation safety critical for reliable deployment. |
| Approach: | They develop a benchmark for step-level tool invocation safety detection in LLM agents and a guardrail model that proactively detects unsafe tool invoking actions before execution using multi-task reinforcement learning. |
| Outcome: | The proposed model reduces harmful tool invocations of ReAct-style agents by 65% on average and improves benign task completion by 10% under prompt injection attacks. |
LLM-driven Instruction Following: Progresses and Concerns (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | a tutorial on task instruction is aimed at researchers and practitioners interested in NLP generalization . labeled examples are unlikely to be available in large numbers or do not exist . |
| Approach: | This tutorial will examine the progress of natural language processing (NLP) using labeled examples. authors propose that task instructions act as a novel resource for supervision. |
| Outcome: | This tutorial aims to answer questions about instruction-driven NLP . it focuses on the use of task instructions in a low-shot scenario . |
Multi-Agent Collaboration via Cross-Team Orchestration (2025.findings-acl)
Copied to clipboard
Zhuoyun Du, Chen Qian, Wei Liu, Zihao Xie, YiFei Wang, Rennai Qiu, Yufan Dang, Weize Chen, Cheng Yang, Ye Tian, Xuantang Xiong, Lei Han
| Challenge: | Large Language Models (LLMs) have significantly impacted various domains, especially through organized LLM-driven autonomous agents. |
| Approach: | They propose a framework that enables orchestrated teams to jointly propose various task-oriented solutions and interact with their insights in a self-independence while cross-team collaboration environment for superior solutions generation. |
| Outcome: | Experiments show that the framework can generate better software quality compared to state-of-the-art frameworks. |
A Novel Matching Paradigm: Unified Generative and Discriminative LLM with Prompt Compression for Relevance Learning (2026.acl-industry)
Copied to clipboard
Guoliang Zhao, Zixin Cui, Chao Ye, Dengwu He, Fei Huang, Yubo Liu, Shuanglong Li, Tzungren Kuo, Bin Ding, Shuang Zhang, null KunhongZhu, Zhi Guo, Liu Lin
| Challenge: | Existing approaches to matching use Large Language Models as feature extractors, underutilizing their full modeling capabilities. |
| Approach: | They propose a matching paradigm that integrates two-tower, single-towing, and generative tasks within a unified LLM framework via attention-mask partitioning. |
| Outcome: | The proposed model achieves superior performance and strong practical value in an industrial search engine. |
Traffic Light or Light Traffic? Investigating Phrasal Semantics in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Phrases are fundamental linguistic units through which humans convey semantics. |
| Approach: | They assess the capacity of API-based large language models to comprehend phrase semantics . they use three human-annotated datasets to analyze their results . |
| Outcome: | The proposed model outperforms embedding-based methods in phrase semantic reasoning tasks . the proposed model does not show significant advantage over fine-tuned methods . |
Unlocking Large Audio-Language Models for Interactive Language Learning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Computer-Assisted Pronunciation Training (CAPT) systems provide unintuitive feedback that lacks actionable guidance. |
| Approach: | They propose to use audio-language models to provide more user-friendly feedback for pronunciation training. |
| Outcome: | The proposed model outperforms baselines on mispronunciation detection and suggestion generation. |
Beyond Single-View Detection: A Dual-Space Reasoning Framework for Interpretable Harmful Meme Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for identifying harmful memes rely on modal alignment or black-box classifiers . BPDMoE-Hate provides visual explanations for viewpoint selection and hierarchical structuring . |
| Approach: | They propose a framework that conceptualizes harmful meme detection as a process of "viewpoint decoupling and hierarchical fusion" they propose BPDMoE-Hate, which generates adversarial binary perspectives via VLMs and incorporates an adaptive viewpoint gating to facilitate viewpoint selection. |
| Outcome: | The proposed framework surpasses existing methods in performance and provides visual explanations for viewpoint selection and hierarchical structuring. |
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)
Copied to clipboard
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen
| Challenge: | Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed . |
| Approach: | They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities. |
| Outcome: | The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech . |
RHGN: Relation-gated Heterogeneous Graph Network for Entity Alignment in Knowledge Graphs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for entity alignment fail to account for heterogeneity among KGs and distinction between KG entities and relations. |
| Approach: | They propose a Relation-gated Heterogeneous Graph Network (RHGN) that uses a relation-gate based convolutional layer to distinguish relations and entities in the KG. |
| Outcome: | Extensive experiments on four datasets show that the proposed method is superior to state-of-the-art methods. |
TAMA: Target-Aware Multilingual Abuse Detection by Cascaded Conditional Multi-Task Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models for protecting public figures from online abuse ignore who is targeted and how. |
| Approach: | They propose a target-aware multi-task framework that conditions downstream predictions on upstream beliefs via three lightweight modules: Cross-Task Feature Fusion (CTF), Task-Adaptive Gating (TAG), and Label-Guided Span Detection (LGSD). |
| Outcome: | The proposed framework yields higher average F1 than single-task training and standard multi-task learning. |
Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity (2026.acl-long)
Copied to clipboard
Feiteng Fang, Dingwei Chen, Xiang Huang, Ting-En Lin, Yuchuan Wu, Xiong Liu, Jing Ye, Ziqiang Liu, Haonan Zhang, Liang Zhu, Hamid Alinejad-Rokny, Min Yang, Yongbin Li
| Challenge: | Existing approaches to reward modeling in reinforcement learning tasks are limited when dealing with ambiguous preferences. |
| Approach: | They propose to use AAM to dynamically calibrate preference margins using the Bradley-Terry model's internal parameter knowledge to improve reward modeling in subjective tasks. |
| Outcome: | The proposed approach improves reward modeling by dynamically calibrating preference margins using the model’s internal parameter knowledge. |
Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention Learning (2025.acl-long)
Copied to clipboard
Lei Huang, Xiaocheng Feng, Weitao Ma, Yuchun Fan, Xiachong Feng, Yuxuan Gu, Yangfan Ye, Liang Zhao, Weihong Zhong, Baoxin Wang, Dayong Wu, Guoping Hu, Lingpeng Kong, Tong Xiao, Ting Liu, Bing Qin
| Challenge: | Large language models (LLMs) suffer from severe hallucination issues due to the knowledge misalignment between the pre-training stage and the supervised fine-tuning stage. |
| Approach: | They propose a training objective with an abstention mechanism that selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token. |
| Outcome: | The proposed model selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token. |
HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks (2026.findings-acl)
Copied to clipboard
Yiming Zeng, Jinghan Cao, Zexin Li, Wanhao Yu, Zhankai Ye, Dawei Xiang, Ting Hua, Xin Liu, Shangqian Gao, Tingting Yu
| Challenge: | Existing approaches treat instruction-based text editing as a generic text generation problem. Existing methods either over-edit or fail to apply modifications consistently. |
| Approach: | They propose a framework that processes each editing request to best align with it. |
| Outcome: | The proposed framework achieves 9% improvement over the state-of-the-art model. |
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)
Copied to clipboard
Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Mengmeng Yang, Zheng Fu, Jinyu Miao, Yining Shi, He Zhe Lim, Li Liu, Tianbao Zhou, Hongyi Wang, Huang Yu, Yifei Hu, Guang Li, Guang Chen, Hao Ye, Lijun Sun, Diange Yang
| Challenge: | Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning. |
| Approach: | AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks. |
| Outcome: | Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% . |
Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent efforts have identified adversarial suffixes capable of jailbreaking LLMs . however, GCG struggles with computational inefficiency, limiting further studies . |
| Approach: | They propose a two-stage transfer learning framework which decouples the search process into behavior-agnostic pre-searching and behavior-relevant post-search. |
| Outcome: | The proposed approach outperforms baseline on Llama2-chat-7b with ASRs of 43.9 (+ 22.2) and 39.0 (+ 19.5) on valid and test sets. |
AIPO: Adaptive Information Guided Token-Level Reinforcement Learning for Large Language Model Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing RLVR methods focus on all generated tokens rather than on which tokens contribute to reasoning. |
| Approach: | They propose to use a Random–Fourier approximation of the Hilbert–Schmidt Independence Criterion to focus updates on decisive tokens discovered on the fly to improve the efficiency of mutual-information estimation. |
| Outcome: | The proposed approach yields +20% accuracy over strong RLVR baselines while updating merely 10% of tokens, demonstrating superior efficiency and effectiveness. |
How Transliterations Improve Crosslingual Alignment (2025.coling-main)
Copied to clipboard
Yihong Liu, Mingyang Wang, Amir Hossein Kargaran, Ayyoob ImaniGooghari, Orgest Xhelili, Haotian Ye, Chunlan Ma, François Yvon, Hinrich Schütze
| Challenge: | Recent studies show that post-aligning multilingual pretrained language models improve crosslingual alignment, but it is unclear how and why this is achieved. |
| Approach: | They propose to explicitly evaluate crosslingual alignment by adding transliterations to models using original and transliterated data. |
| Outcome: | The proposed approach improves crosslingual alignment even for random sentences. |
A Fast and High-quality Text-to-Speech Method with Compressed Auxiliary Corpus and Limited Target Speaker Corpus (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to generate high-quality speech with limited target speaker corpus require extensive training data. |
| Approach: | They propose an auxiliary corpus compression algorithm that reduces the training cost while the naturalness of synthesized speech is not significantly degraded. |
| Outcome: | The proposed method significantly reduces training costs while maintaining the naturalness of synthesized speech. |
A Crosslingual Investigation of Conceptualization in 1335 Languages (2023.acl-long)
Copied to clipboard
Yihong Liu, Haotian Ye, Leonie Weissweiler, Philipp Wicke, Renhao Pei, Robert Zangenfeind, Hinrich Schütze
| Challenge: | Conceptualizer is a method that creates a bipartite directed alignment graph between source language concepts and sets of target language strings. |
| Approach: | They propose a method that creates a bipartite directed alignment graph between source language concepts and sets of target language strings. |
| Outcome: | The proposed method has good alignment accuracy across all languages and on 32 Swadesh concepts. |
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)
Copied to clipboard
| Challenge: | OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch . |
| Approach: | OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment. |
| Outcome: | OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment . |
Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection (2026.findings-acl)
Copied to clipboard
Zhiwei Liu, Yupeng Cao, Yuechen Jiang, Mohsinul Kabir, Polydoros Giannouris, Chen Xu, Ziyang Xu, Tianlei Zhu, Md. Tariquzzaman, Triantafillos Papadopoulos, Yan Wang, Lingfei Qian, Xueqing Peng, Zhuohan Xie, Ye Yuan, Saeed Almheiri, Abdulrazzaq Alnajjar, Ming-Bin Chen, Harry Stuart, Paul Thompson, Prayag Tiwari, Alejandro Lopez-Lira, Xue Liu, Jimin Huang, Sophia Ananiadou
| Challenge: | Existing research on LLM biases has focused on direct questioning or general-purpose settings . pronounced behavioral biase despite their growing deployment in financial analysis, forecasting, and decision support. |
| Approach: | They propose a benchmark to evaluate behavioral biases of large language models in MFMD . they use a multilingual financial misinformation dataset to integrate these with misinformation claims . |
| Outcome: | The proposed benchmark evaluates behavioral biases of large language models across economic scenarios. |
DESED: Dialogue-based Explanation for Sentence-level Event Detection (2022.coling-1)
Copied to clipboard
Yinyi Wei, Shuaipeng Liu, Jianwei Lv, Xiangyu Xi, Hailei Yan, Wei Ye, Tong Mo, Fan Yang, Guanglu Wan
| Challenge: | Existing methods for sentence-level event detection depend on manual annotations or domain expertise to design sophisticated templates and rules. |
| Approach: | They propose a dialogue-based explanation paradigm to enhance sentence semantics for event detection. |
| Outcome: | The proposed method can be applied to two event detection datasets. |
CAGenMol: Condition-Aware Diffusion Language Model for Goal-Directed Molecular Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to optimize target-directed molecular generation fail to reconcile conflicting objectives without compromising structural validity. |
| Approach: | They propose a condition-aware discrete diffusion framework that allows for conditional denoising guided by heterogeneous structural and property signals. |
| Outcome: | The proposed framework improves on structure-conditioned, property-conditioned and dual-conditioned benchmarks in binding affinity, drug-likeness, and success rate. |
Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning Large Language Models are slow and lack of performance. |
| Approach: | They propose a Zeroth-Order optimization framework that uses forward passes to fine-tune Large Language Models. |
| Outcome: | The proposed framework achieves 1.7 to 3.0 wall-clock acceleration on LLaMA and OPT models. |
Multimodal Aspect-Based Sentiment Analysis under Conditional Relation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to analyze social media sentiments rely on image-based aspects. |
| Approach: | They propose a multi-task framework to extract aspect terms from text-image pairs and identify their sentiments. |
| Outcome: | The proposed framework outperforms existing methods on a text-image dataset. |
TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models (2023.emnlp-main)
Copied to clipboard
Jing Xiong, Jianhao Shen, Ye Yuan, Haiming Wang, Yichun Yin, Zhengying Liu, Lin Li, Zhijiang Guo, Qingxing Cao, Yinya Huang, Chuanyang Zheng, Xiaodan Liang, Ming Zhang, Qun Liu
| Challenge: | Automated theorem proving (ATP) benchmarks focus on symbolic inference but rarely involve understanding complex number combination reasoning. |
| Approach: | They propose a benchmark that requires a model to reduce a trigonometric expression with step-by-step proof and evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
| Outcome: | The proposed benchmark evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
MADS: Multi-Agent Dialogue Simulation for Diverse Persuasion Data Generation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Recent studies show that LLM-based agents exhibit superior moral and emotional language performance compared to humans, raising expectations for their deployment in persuasive tasks. |
| Approach: | They propose a framework for generating persuasive multi-turn dialogues via agent self-play using user agents designed to simulate diverse persona-driven behaviors, a Dialog Agent executing task-oriented persuasion strategies and an Optimization Agent evaluating and refining dialogue outcomes. |
| Outcome: | The proposed framework significantly improved the persuasion capacity of small LLMs, increasing the organic traffic conversion rate by 22.4% (from 1.83% to 2.24%) . |
MisinfoBench: A Multi-Dimensional Benchmark for Evaluating LLMs’ Resilience to Misinformation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks assess factual accuracy in isolated queries but fail to evaluate LLMs’ resilience to misinformation in interactive settings. |
| Approach: | MisinfoBench is a benchmark designed to assess LLMs’ ability to discern, resist, and reject misinformation. |
| Outcome: | MisinfoBench assesses large language models’ ability to discern, resist, and reject misinformation in interactive settings. |
A Bounding Box is Worth One Token - Interleaving Layout and Text in a Large Language Model for Document Understanding (2025.findings-acl)
Copied to clipboard
Jinghui Lu, Haiyang Yu, Yanjie Wang, Yongjie Ye, Jingqun Tang, Ziwei Yang, Binghong Wu, Qi Liu, Hao Feng, Han Wang, Hao Liu, Can Huang
| Challenge: | Existing methods for integrating spatial layouts with text have limitations . existing methods produce overly long text sequences or lack autoregressive traits of LLMs . |
| Approach: | They introduce Interleaving Layout and Text in a Large Language Model (LayTextLLM) they use OCR-derived text and spatial layouts to integrate with LLMs for document understanding . |
| Outcome: | The proposed model shows an increase in performance in KIE and VQA tasks. |
Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval (2024.lrec-main)
Copied to clipboard
Haowei Liu, Yaya Shi, Haiyang Xu, Chunfeng Yuan, Qinghao Ye, Chenliang Li, Ming Yan, Ji Zhang, Fei Huang, Bing Li, Weiming Hu
| Challenge: | Existing methods for video-text retrieval capture fine-grained semantic concepts . however, they lack the ability to capture finer-grain concepts such as objects and actions. |
| Approach: | They propose a dual-encoder architecture for fast video-text retrieval that learns lexicon representations to capture fine-grained semantics. |
| Outcome: | The proposed framework outperforms existing methods with 4.8% and 8.2% improvement on MSR-VTT and DiDeMo respectively. |
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI (2024.findings-eacl)
Copied to clipboard
Jianguo Zhang, Kun Qian, Zhiwei Liu, Shelby Heinecke, Rui Meng, Ye Liu, Zhou Yu, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | DialogStudio is the largest and most diverse collection of dialogue datasets . existing datasets lack diversity and comprehensiveness, authors say . |
| Approach: | They introduce DialogStudio: the largest and most diverse collection of dialogue datasets . DialogStuio aggregates more than 80 diverse dialogue dataset . |
| Outcome: | a new dataset is created to improve the quality and diversity of dialogue datasets . DialogStudio is the largest and most diverse collection of dialogue data . |
Going “Deeper”: Structured Sememe Prediction via Transformer with Tree Attention (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies ignore hierarchical structures of sememes in sememe-based semantic description systems. |
| Approach: | They propose a structured sememe prediction problem to predict a sememes tree with hierarchical structures rather than a set of sememas. |
| Outcome: | The proposed model outperforms baseline models and shows its effectiveness . it predicts a sememe tree with hierarchical structures rather than a set of sememes . |
Improving Embedding-based Large-scale Retrieval via Label Enhancement (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for large-scale retrieval are trained with 0-1 hard labels that indicate whether a query is relevant to a document, ignoring rich information of the relevance degree. |
| Approach: | They propose to introduce label enhancement for the first time to characterize query-document relevance degree by embedding label distribution into contextual embeddables. |
| Outcome: | The proposed method significantly outperforms existing retrieval models and its counterparts equipped with two alternative methods on English and Chinese large-scale retrieval tasks. |
SimVBG: Simulating Individual Values by Backstory Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have strong human-like capabilities, but rarely simulating individualized human values. |
| Approach: | They propose a framework that simulates individual values based on individual backstories . they use structured data on an individual to transform their backstoried information to a backstory . |
| Outcome: | The proposed framework improves top-1 accuracy by more than 10% over retrieval-augmented generation methods. |
Lunar-Bench: Towards Evaluating Task-Oriented Reasoning of LLMs in Lunar Exploration Scenarios (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on static, context-independent reasoning tasks and fail to capture constraints and dependencies of lunar missions. |
| Approach: | They propose a benchmark to assess the task-oriented reasoning and decision-making performance of large language models through 3,000 tasks derived from mission procedures and documentation. |
| Outcome: | The proposed model achieves 47.8% accuracy compared with 65.1% for human experts on 36 representative missions. |
Knowledge Image Matters: Improving Knowledge-Based Visual Reasoning with Multi-Image Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Knowledge-based visual reasoning (KB-VR) is a challenging task, as it requires machines not only to understand concepts and relationships of visual scenes, but also to associate them with external world knowledge to perform chain of reasoning on open-world questions. |
| Approach: | They propose a visual knowledge card (VKC) that integrates internal visual knowledge and external world knowledge produced by a knowledge generator into an image. |
| Outcome: | The proposed model achieves new state-of-the-art results compared to previous top-performing models on three popular KB-VR benchmarks. |
EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
Jiamin Su, Yibo Yan, Fangteng Fu, Zhang Han, Jingheng Ye, Xiang Liu, Jiahao Huo, Huiyu Zhou, Xuming Hu
| Challenge: | Automated Essay Scoring (AES) systems face three major challenges: reliance on handcrafted features that limit generalizability, difficulty in capturing fine-grained traits like coherence and argumentation, and inability to handle multimodal contexts. |
| Approach: | They propose a multimodal benchmark to evaluate AES capabilities across lexical-, sentence-, and discourse-level traits without manual feature engineering. |
| Outcome: | The proposed system can evaluate AES capabilities across lexical-, sentence-, and discourse-level traits without manual feature engineering. |
ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)
Copied to clipboard
Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong
| Challenge: | Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs). |
| Approach: | They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models. |
| Outcome: | The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks. |
OneNet: A Fine-Tuning Free Framework for Few-Shot Entity Linking via Large Language Model Prompting (2024.emnlp-main)
Copied to clipboard
| Challenge: | Entity Linking (EL) is the process of associating ambiguous textual mentions to specific entities in a knowledge base. |
| Approach: | They propose a framework that utilizes the few-shot learning capabilities of Large Language Models without the need for fine-tuning to improve the accuracy of EL. |
| Outcome: | The framework outperforms current state-of-the-art methods in a few-shot entity linking task. |
CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation (2025.findings-acl)
Copied to clipboard
Guangya Yu, Yanhao Li, Zongying Jiang, Yuxiong Jin, Li Dai, Yupian Lin, Ruihui Hou, Weiyan Zhang, Yongqi Fan, Qi Ye, Jingping Liu, Tong Ruan
| Challenge: | Medical quality control indicators are essential to assess the qualifications of healthcare institutions for medical services. |
| Approach: | They propose a Chinese electronic medical records-based dataset for MQCIC and propose CF-IR method that disentangles clinical fact verification and inferential rule reasoning actions. |
| Outcome: | The proposed method outperforms Chain-of-Thought methods on 20 representative LLMs, covering general and medical models. |
NeuroSym-Cal: Bridging the Reasoning-Execution Gap in Code Generation via Hierarchical Calibration (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing calibration methods rely on the assumption that consensus implies correctness . Existing methods fail under systematic errors, leading to miscalibrated high-confidence predictions. |
| Approach: | They propose a hierarchical calibration framework that measures confidence at two levels . they propose sensitivity analysis to measure local curvature of deductive process . |
| Outcome: | The proposed framework de-saturates overconfident errors and improves selective generation performance on OOD benchmarks. |
Blind Spot Navigation in Large Language Model Reasoning with Thought Space Explorer (2026.findings-eacl)
Copied to clipboard
Jinghan Zhang, Fengran Mo, Tharindu Cyril Weerasooriya, Xinyue Ye, Dongjie Wang, Yanjie Fu, Kunpeng Liu
| Challenge: | Existing studies show that large language models have strong reasoning capabilities through chain-structured methods. |
| Approach: | They propose a framework for navigating and expanding thought structures to overcome blind spots in LLM reasoning. |
| Outcome: | The proposed framework overcomes blind spots in large language models by expanding thought structures . the proposed framework improves accuracy of the final answer and intermediate reasoning steps . |
Semantics-enhanced Cross-modal Masked Image Modeling for Vision-Language Pre-training (2024.lrec-main)
Copied to clipboard
Haowei Liu, Yaya Shi, Haiyang Xu, Chunfeng Yuan, Qinghao Ye, Chenliang Li, Ming Yan, Ji Zhang, Fei Huang, Bing Li, Weiming Hu
| Challenge: | Existing methods for vision-language pre-training lack high-level semantics and text is not sufficiently involved in masked modeling. |
| Approach: | They propose a semantics-enhanced cross-modal MIM framework for vision-language representation learning that harvests high-level semantics from global image features via self-supervised agreement learning and transfers them to local patch encodings by sharing the encode space. |
| Outcome: | The proposed model achieves state-of-the-art or competitive performance on multiple vision-language tasks. |
CodeTool: Enhancing Programmatic Tool Invocation of LLMs via Process Supervision (2025.acl-long)
Copied to clipboard
YifeiLu YifeiLu, Fanghua Ye, Jian Li, Qiang Gao, Cheng Liu, Haibo Luo, Nan Du, Xiaolong Li, Feiliang Ren
| Challenge: | Existing approaches to tool invocation are often unnecessarily long and require lengthy reasoning paths. |
| Approach: | They propose a framework for stepwise code generation that improves LLM tool invocation . they incorporate two distinct process rewards: the On-the-spot and the Latent Reward . |
| Outcome: | The proposed framework improves LLM tool invocation by leveraging the concise nature of code. |
Selective Knowledge Distillation: Fusing LLM Semantic Strengths with DNN Efficiency for Binary Code Similarity Detection (2026.acl-long)
Copied to clipboard
| Challenge: | BinSKD is a binary code similarity detection technique that can be used in bug detection, patch analysis, and malware detection. |
| Approach: | They propose to leverage an LLM-based BCSD method as the teacher model and transfer its knowledge of high-level program semantics to various DNN-based student models. |
| Outcome: | The proposed method yields Recall@1 improvements of 14.5%–91.2% for DNN-based BCSD methods and enables HermesSim to match the teacher’s performance with orders-of-magnitude efficiency. |
SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)
Copied to clipboard
Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li
| Challenge: | Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy. |
| Approach: | They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration. |
| Outcome: | The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent . |
Learning to Extract Structured Entities Using Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language Models (LMs) play a pivotal role in extracting structured information from unstructured text. |
| Approach: | They propose to reformulate the task to be entity-centric, enabling the use of diverse metrics that can provide more insights from various perspectives. |
| Outcome: | The proposed model outperforms baselines and human evaluations on the extracted entities. |
Dense Hierarchical Retrieval for Open-domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA) current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. |
| Approach: | They propose a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic specific to each passage. |
| Outcome: | The proposed framework significantly outperforms the original dense passage retriever and helps an end-to-end QA system outperfect the strong baselines on multiple open-domain QA benchmarks. |
Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey (2024.acl-long)
Copied to clipboard
Jiawei Li, Yizhe Yang, Yu Bai, Xiaofeng Zhou, Yinghao Li, Huashan Sun, Yuhang Liu, Xingpeng Si, Yuhao Ye, Yixiao Wu, 林一冠 林一冠, Bin Xu, Ren Bowen, Chong Feng, Yang Gao, Heyan Huang
| Challenge: | Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications. |
| Approach: | They propose a reliable strategy for domains to choose more robust LLMs for real-world applications. |
| Outcome: | The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications. |
Landmark-Guided Cross-Speaker Lip Reading with Mutual Information Regularization (2024.lrec-main)
Copied to clipboard
| Challenge: | Lip reading is a process of interpreting silent speech from visual lip movements . but lip reading in cross-speaker scenarios poses a challenging problem due to inter-speech variability . |
| Approach: | They propose to exploit lip landmark-guided visual clues instead of mouth-cropped images as input features. |
| Outcome: | Experimental results show that the proposed approach reduces speaker-specific appearance characteristics in cross-speaker scenarios. |
Meet Changes with Constancy: Learning Invariance in Multi-Source Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to multi-source neural machine translation neglect inconsistencies between sources of information. |
| Approach: | They propose a source invariance network to learn invariant information of parallel sources . they propose to integrate such network with multi-encoder based multi-source NMT methods . |
| Outcome: | The proposed approach achieves clear gains in translation quality and captures implicit invariance between different sources. |
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions (2020.acl-main)
Copied to clipboard
| Challenge: | State-of-the-art NLP models can be fooled by human-unaware transformations such as synonymous word substitution. |
| Approach: | They propose a method that constructs a stochastic ensemble by applying random word substitutions on the input sentences and leverages the statistical properties to provably certify the robustness. |
| Outcome: | The proposed method outperforms state-of-the-art methods on IMDB and Amazon text classification tasks with practically meaningful certified accuracy. |
VKIE: The Application of Key Information Extraction on Video Text (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for extracting structured information from videos are coarse-grained at segment level and unable to capture finegrained information at the entity level. |
| Approach: | They propose a task for extracting hierarchical key information from visual texts on videos . they decouple the task into four subtasks and propose two implementation solutions . |
| Outcome: | The proposed solutions achieve remarkable performance and efficient inference speed on a well-defined dataset. |
T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback (2025.acl-long)
Copied to clipboard
Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao
| Challenge: | Text-to-audio (T2A) models still struggle to satisfy human preferences for prompt-following and acoustic quality when generating complex multi-event audio. |
| Approach: | They propose to use AI feedback learning to enhance basic capabilities of text-to-audio models . they use a large audio preference dataset to evaluate the model's capabilities . |
| Outcome: | The proposed model improves in simple and complex scenarios with AI feedback learning. |
DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) with web search capabilities show significant potential for deep research. |
| Approach: | They introduce a framework for end-to-end training of LLM-based deep research agents . they implement a specialized multi-agent architecture where browsing agents extract relevant information from various webpage structures. |
| Outcome: | The proposed framework improves on open-domain research tasks by 28.9 points over prompt engineering and 7.2 points over RAG-based RL agents. |
A Strong and Robust Baseline for Text-Image Matching (P19-2)
Copied to clipboard
| Challenge: | Text-image matching is one of the most popular methods for training text-image embeddings. |
| Approach: | They propose to use a kNN-margin loss that utilizes hard negatives and is robust to noise . they advocate using Inverted Softmax and Cross-modal Local Scaling during inference . |
| Outcome: | The proposed loss function is robust to noise and pseudo negatives are tolerable . the proposed loss functions improve scores of all metrics by a large margin . |
TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data (2025.coling-main)
Copied to clipboard
| Challenge: | Existing mPLMs that handle non-transliterated data are not sufficient to train crosslingual models. |
| Approach: | They propose a framework to transliterate related languages into a common script by exploiting existing mPLMs and their tokenizer without any training. |
| Outcome: | The proposed framework can create strong baselines for data that is transliterated into a common script by exploiting an existing mPLM and its tokenizer without any training. |
FedMABench: Benchmarking Mobile GUI Agents on Decentralized Heterogeneous User Data (2025.emnlp-main)
Copied to clipboard
| Challenge: | Mobile GUI agents have attracted tremendous research participation recently. traditional approaches to mobile agent training rely on centralized data collection. |
| Approach: | They propose a benchmark for federated training and evaluation of mobile GUI agents . they find that federation algorithms consistently outperform local training . |
| Outcome: | The first benchmark for federated training and evaluation of mobile GUI agents is released . it features 6 datasets with 30+ subsets, 8 federation algorithms, 10+ base models, and over 800 apps across 5 categories . |
Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Few-shot table-to-text generation is a task of composing fluent and faithful sentences to convey table content using limited data. |
| Approach: | They propose a novel approach to generate faithful table-to-text sentences using limited data . they aim to exploit table structure and natural linguistic information to generate accurate sentences . |
| Outcome: | The proposed approach generates higher qualified sentences when compared with state-of-the-art models on humans, songs, and books. |
Structure-aware Domain Knowledge Injection for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Structure-aware Continual Pre-Training (SCPT) and Structure-Aware Supervised Fine-Tuning (SSFT) are two-stage strategies for knowledge injection and alignment that reduces the training corpus needs to 5% while achieving 100% of traditional knowledge injection performance. |
| Approach: | They propose a method to efficiently transform foundation Large Language Models into domain specialists by using two-stage strategies: Structure-aware Continual Pre-Training and Structure-Aware Supervised Fine-Tuning. |
| Outcome: | The proposed method significantly reduces the training corpus needs to a mere 5% while achieving 100% of traditional knowledge injection performance. |
SwapMoE: Serving Off-the-shelf MoE-based Large Language Models with Tunable Memory Budget (2024.acl-long)
Copied to clipboard
| Challenge: | Mixture of experts (MoE) is a popular technique to improve capacity of Large Language Models (LLMs) but memory-constrained devices are a major concern in edge AI training and serving. |
| Approach: | They propose a framework for efficient serving of MoE-based large language models with tunable memory budgets. |
| Outcome: | Experiments show that SwapMoE can reduce memory consumption while maintaining reasonable accuracy. |
MinosEval: Distinguishing Factoid and Non-Factoid for Tailored Open-Ended QA Evaluation with LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods overlook the distinction between factoid and non-factoidic questions. |
| Approach: | They propose a method that distinguishes open-ended questions and ranks candidate answers . they propose QA requires longer answer statements and nuanced reasoning processes . |
| Outcome: | The proposed method better aligns with human annotations and offers more interpretable results. |
Shadow-Activated Backdoor Attacks on Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
Ziyi Yin, Muchao Ye, Yuanpu Cao, Jiaqi Wang, Aofei Chang, Han Liu, Jinghui Chen, Ting Wang, Fenglong Ma
| Challenge: | Existing backdoor attacks on Multimodal Large Language Models are less applicable to open-ended conversations with users. |
| Approach: | They propose a shadow-activated backdoor attack scenario where attackers inject malicious content into the responses of MLLMs when the responses explicitly relate to the shadowed object. |
| Outcome: | The proposed framework achieves the desired behaviors by constructing a poisoned dataset and implementing an attention-regularized tuning strategy. |
Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to generate source code summaries are coarse-grained and noise-filled . however, they do not capture contextual code semantics and are often outdated in continuous software iteration. |
| Approach: | They propose a fine-grained Token-level retrieval-augmented mechanism on the decoder side to enhance performance of neural models. |
| Outcome: | The proposed method produces more low-frequency tokens and is interpretable. |
GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion (2025.findings-acl)
Copied to clipboard
Kangyang Luo, Yuzhuo Bai, Cheng Gao, Shuzheng Si, Zhu Liu, Yingli Shen, Zhitong Wang, Cunliang Kong, Wenhao Li, Yufei Huang, Ye Tian, Xuantang Xiong, Lei Han, Maosong Sun
| Challenge: | Existing knowledge graphs lack the ability to integrate structural information into LLMs and output predictions deterministically. |
| Approach: | They propose a method which encodes structural information of KGs and merges it with LLMs to enhance KGC performance. |
| Outcome: | The proposed method improves the performance of KG Completion datasets on KGs by integrating structural information with LLMs. |
TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation (2026.findings-acl)
Copied to clipboard
Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu
| Challenge: | Existing studies show that advanced LLMs produce text indistinguishable from human writing. |
| Approach: | They propose a benchmark to assess persona simulation across diverse contexts by decomposing the evaluation into six fundamental capabilities including opinion consistency, memory recall, logical reasoning, persona tone, and syntactic style. |
| Outcome: | The proposed model achieves moderate accuracy but falls short of the basic capabilities needed to simulate personas in real-world contexts. |
SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)
Copied to clipboard
Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang
| Challenge: | Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases. |
| Approach: | They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs. |
| Outcome: | The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs. |
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing studies on DS-based relation extraction (RE) methods focus on handling label noise, but other factors may have been overlooked. |
| Approach: | They propose a method to automatically adjust DS-RE models to a shifted label distribution problem . they find this problem exists in real-world DS datasets and can be overcome . |
| Outcome: | The proposed method achieves consistent performance gains on DS-trained models with an up to 23% relative F1 improvement, which verifies their assumptions. |
Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biases (2025.acl-long)
Copied to clipboard
Rena Wei Gao, Xuetong Wu, Tatsuki Kuribayashi, Mingrui Ye, Siya Qi, Carsten Roever, Yuanxing Liu, Zheng Yuan, Jey Han Lau
| Challenge: | Large Language Models (LLMs) can simulate non-native-like English use observed in human second language (L2) learners interfered with by their native first language (N1) knowledge. |
| Approach: | They use large language models to simulate non-native-like English use observed in human second language (L2) learners, and then compare their outputs to real L2 learner data. |
| Outcome: | The proposed models replicate L1-dependent patterns observed in human second language (L2) learners, with distinct influences from various languages. |
Best Practices for Distilling Large Language Models into BERT for Web Search Ranking (2025.coling-industry)
Copied to clipboard
| Challenge: | Recent studies have highlighted the potential of Large Language Models (LLMs) as zero-shot relevance rankers. |
| Approach: | They propose to use a ranking loss to transfer ranking knowledge from LLMs to smaller models like BERT. |
| Outcome: | The proposed model has been successfully integrated into a commercial web search engine as of February 2024. |
Plug-and-Play Knowledge Injection for Pre-trained Language Models (2023.acl-long)
Copied to clipboard
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
| Challenge: | Existing knowledge injection methods are not suitable for enhancing pre-trained language models with external knowledge bases. |
| Approach: | They propose a plug-and-play knowledge injection method where knowledge bases are injected into frozen existing downstream models by a knowledge plugin. |
| Outcome: | The proposed method improves the performance of knowledge injection on knowledge-driven tasks while keeping model parameters frozen. |
DT-Solver: Automated Theorem Proving with Dynamic-Tree Sampling Guided by Proof-level Value Function (2023.acl-long)
Copied to clipboard
Haiming Wang, Ye Yuan, Zhengying Liu, Jianhao Shen, Yichun Yin, Jing Xiong, Enze Xie, Han Shi, Yujun Li, Lin Li, Jian Yin, Zhenguo Li, Xiaodan Liang
| Challenge: | Recent advances in neural theorem-proving resort to large language models and tree searches. |
| Approach: | They propose a Dynamic-Tree Driven Theorem Solver to accommodate general theoremes by guiding the search procedure with state confidence and proof-level values. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two popular theorem-proving datasets with a 6.65% improvement on average in terms of success rate. |
Toolscaler: Scalable Generative Tool Calling via Structure-Aware Semantic Tokenization (2025.findings-emnlp)
Copied to clipboard
Yunyue Su, Zhang Jinshuai, Bowen Fang, Wen Ye, Jinghao Zhang, Bowen Song, Weiqiang Wang, Qiang Liu, Liang Wang
| Challenge: | Extensive experiments demonstrate the effectiveness of SGTC across various tasks. |
| Approach: | They propose a generative tool invocation framework that introduces structure-aware semantic tokenization to encode tools as discrete code sequences. |
| Outcome: | The proposed framework reduces the size of the representation space and underutilizes collaborative signals among tools in downstream tasks. |
On the Step Length Confounding in LLM Reasoning Data Selection (2026.findings-acl)
Copied to clipboard
Bing Wang, Rui Miao, Chen Shen, Shaotian Yan, Kaiyuan Liu, Ximing Li, Xiaosong Yuan, Sinan Fan, Jun Zhang, Jieping Ye
| Challenge: | Existing pipelines generate long reasoning data from more capable Large Language Models (LLMs) and apply manually heuristic or naturalness-based selection methods to filter high-quality samples. |
| Approach: | They propose to use supervised fine-tuning to generate long reasoning data from more capable Large Language Models and apply manually heuristic or naturalness-based selection methods to filter high-quality samples. |
| Outcome: | Experiments on four LLMs and five evaluation benchmarks show that the proposed approach is effective in mitigating step length confounding problem. |
A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models (2022.acl-short)
Copied to clipboard
| Challenge: | Existing pre-trained language models cannot recall factual knowledge of entities exhibited in large-scale corpora, especially those rare entities. |
| Approach: | They propose to build a pluggable Entity Lookup Table (PELT) on demand by aggregating the entity’s output representations of multiple occurrences in the corpora. |
| Outcome: | The proposed model can transfer entity knowledge from out-of-domain corpora into PLMs with different architectures. |
Knowledge Graph Enhanced Large Language Model Editing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for editing large language models struggle to track and incorporate changes in knowledge associated with edits, which limits the generalization ability of post-edit LLMs in processing edited knowledge. |
| Approach: | They propose a model editing method that leverages knowledge graphs to enhance LLM editing by capturing changes in associated knowledge by constructing an external graph. |
| Outcome: | The proposed method improves the generalization ability of LLMs in processing edited knowledge. |
Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches on semantic parsing suffer from exponential growth of logical form candidates and can hardly generalize to unseen data. |
| Approach: | They propose a unified semantic parser for question answering on KB and DB . they define the primitive as the essential element in their framework . |
| Outcome: | The proposed framework can predict logical forms by altering and composing top-ranked primitives with different operations. |
TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | The world’s more than 7000 languages are written in at least 293 scripts, which poses a difficulty for multilingual pretrained language models in learning crosslingual knowledge through lexical overlap. |
| Approach: | They propose a framework that optimizes the Transliteration Contrastive Modeling objective to fine-tune an mPLM by contrasting sentences in its training data and transliterations in a unified script. |
| Outcome: | The proposed model outperforms Glot500-m on zero-shot crosslingual transfer tasks while retaining uniformity across scripts. |
CORI: CJKV Benchmark with Romanization Integration - a Step towards Cross-lingual Transfer beyond Textual Scripts (2024.lrec-main)
Copied to clipboard
| Challenge: | Naively assuming English as a source language may hinder cross-lingual transfer . despite recent advances in cross-linguistic research, most studies have restricted themselves to two major assumptions . |
| Approach: | They propose to integrate Romanized transcription beyond textual scripts to capture contact between these languages . they propose to use a benchmark dataset to further encourage in-depth studies of language contact . |
| Outcome: | The proposed method allows for enhanced cross-lingual representations and effective zero-shot cross-linguistic transfer. |
LLM Safety From Within: Detecting Harmful Content with Internal Representations (2026.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art guard models rely on terminal-layer representations and overlook safety-relevant features encoded across internal layers. |
| Approach: | They propose a lightweight guard model that harnesses safety neurons from LLM internals without modifying the underlying model. |
| Outcome: | The proposed model outperforms open-source guard models across multiple benchmarks while using 250 fewer trainable parameters. |
Label Smoothing for Text Mining (2022.coling-1)
Copied to clipboard
| Challenge: | Existing text mining models are trained with 0-1 hard label that indicates whether an instance belongs to a class, ignoring rich information of the relevance degree. |
| Approach: | They propose a keyword-based method to automatically generate soft labels from hard labels . they exploit relevance between labels and instances to incorporate them into models . |
| Outcome: | The proposed method improves models under balanced and unbalanced conditions. |
Prompt Optimization for Relation Extraction using Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods rely heavily on large-scale annotated datasets limiting their applicability in domain-specific and low-resource scenarios. |
| Approach: | They propose a reinforcement learning-based automated prompt optimization framework for domain relation extraction that optimizes prompt quality through interaction with a black-box LLM. |
| Outcome: | The proposed framework outperforms existing prompt-based methods and supervised baselines on multiple extraction datasets across medical, financial, legal, and news domains. |
LangSAMP: Language-Script Aware Multilingual Pretraining (2025.acl-long)
Copied to clipboard
| Challenge: | Recent multilingual pretrained language models often avoid using language embeddings, which places a significant burden on token representations to encode all language-specific information. |
| Approach: | They propose a method that incorporates both language and script embeddings into the output of Transformer blocks before passing the final representations to the language modeling head for prediction. |
| Outcome: | The proposed method outperforms the baseline model in zero-shot crosslingual transfer across diverse downstream tasks. |
Can LLMs Convert Graphs to Text-Attributed Graphs? (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to model graph-structured data are limited by the availability of text-attributed graph data. |
| Approach: | They propose a method to convert existing graphs into text-attributed graphs using large language models. |
| Outcome: | The proposed method outperforms existing approaches that manually design node features on text-free graphs. |
CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to role-playing language models rely on prompt engineering or supervised fine-tuning to emulate character behaviors but neglect the underlying cognitive mechanisms driving these behaviors. |
| Approach: | They propose a novel RPLA adopting a cognize-then-respond reasoning paradigm that leverages dual cognition for more contextually grounded and psychologically coherent responses. |
| Outcome: | The proposed RPLA outperforms baselines and generalizes effectively across diverse role-playing tasks. |
Advancing Test-Time Adaptation in Wild Acoustic Test Settings (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing wild vision TTA methods fail to handle speech data due to the unique characteristics of high-entropy speech frames, which are unreliably filtered out even when containing crucial semantic content. |
| Approach: | They propose a method for acoustic foundation models to perform confidence-based adaptation in wild acustic test settings. |
| Outcome: | The proposed method outperforms baselines under Gaussian noise, environmental sounds, accent variations, and sung speech in the wild. |
A Unified Positive-Unlabeled Learning Framework for Document-Level Relation Extraction with Different Levels of Labeling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing document-level relation extraction methods focus on fully supervised scenarios but in real-world, incomplete labeling is a common problem because the number of entity pairs grows quadratically with the number. |
| Approach: | They propose a positive-unlabeled learning framework for document-level relation extraction (RE) that uses shift and squared ranking loss positive- unlabeles (SSR-PU) learning to solve incomplete labeling problem. |
| Outcome: | The proposed framework outperforms state-of-the-art methods under fully supervised and extremely unlabeled conditions and achieves 14 F1 points over the baseline with incomplete labeling. |
PyramidCodec: Hierarchical Codec for Long-form Music Generation in Audio Domain (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate long music are inefficient and lack of structured representation. |
| Approach: | They propose a hierarchical discrete representation of audio for long audio-domain music generation using residual vector quantization on different levels of features. |
| Outcome: | The proposed method achieves competitive performance in terms of reconstruction quality and token per second (TPS) the proposed method facilitates training a language model that can generate well-structured long-form music for up to 3 minutes. |
LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement (2025.acl-long)
Copied to clipboard
Boyi Kang, Xinfa Zhu, Zihan Zhang, Zhen Ye, Mingshuai Liu, Ziqian Wang, Yike Zhu, Guobin Ma, Jun Chen, Longshuai Xiao, Chao Weng, Wei Xue, Lei Xie
| Challenge: | Recent advances in language models have demonstrated strong capabilities in semantic understanding and contextual modeling. |
| Approach: | They propose a LLaMA-based language model that incentivizes generalization capabilities for speech enhancement. |
| Outcome: | The proposed language model outperforms prior task-specific discriminative and generative models in acoustic enhancement tasks. |
MUSIED: A Benchmark for Event Detection from Multi-Source Heterogeneous Informal Texts (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent efforts to classify unstructured texts into specific types have been limited in practical scenarios. |
| Approach: | They propose to use Chinese text conversations and phone conversations to expand event detection to the scenarios involving informal and heterogeneous texts. |
| Outcome: | The proposed dataset is based on user reviews, text conversations, and phone conversations in a leading e-commerce platform for food service. |
UniRAG: Unified Query Understanding Method for Retrieval Augmented Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing query augmentation methods face knowledge update lag and hallucinations in large language models (LLMs) Existing methods face two key challenges: (1) separation of query augmented and encoding tasks, which hinders information sharing and introduces cumulative errors; (2) difficulty of selecting optimal augmentation strategy for different scenarios. |
| Approach: | They propose a unified framework for query understanding in RAG that integrates internal and external knowledge to enhance query augmentation and encoding tasks. |
| Outcome: | The proposed framework outperforms traditional query augmentation methods in five knowledge-intensive benchmark tasks in both closed and open domain question answering. |
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions (2026.acl-long)
Copied to clipboard
Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai, Xiao Luo
| Challenge: | Existing research on reinforcement learning for LLMs under data scarcity has not been unified. |
| Approach: | They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric. |
| Outcome: | The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area. |
Tiny Scales, Great Challenges: The Limits of Multimodal LLMs in Scale Recognition (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on a single type of quantity or a specific format, lacking a comprehensive evaluation of scale recognition capabilities. |
| Approach: | They propose a visual scale recognition benchmark built using images from COCO, Open Images, and Flickr to evaluate scale recognition capabilities of multimodal large language models. |
| Outcome: | The proposed model achieves 42.60% accuracy, lower than the 97.40% of humans. |
HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for summarizing semantic graph structure from raw text are cumbersome and inefficient for long-text documents. |
| Approach: | They propose a Transformer-based pre-trained model with multi-granularity sparse attentions for long-text extractive summarization. |
| Outcome: | The proposed model performs state-of-the-art on single- and multi-document summarization tasks while using less memory and fewer parameters. |
Bridging the Sensory Gap: Visual Injection for Taxonomy Completion (2026.acl-long)
Copied to clipboard
| Challenge: | Existing text-only methods suffer from a "Sensory Gap" in integrating new concepts into existing hierarchies. |
| Approach: | They propose a framework leveraging Visual Injection for Taxonomy Completion that maps synthesized images into intrinsic pseudo-tokens and decouples magnitude from selection to prevent visual signals from being drowned out. |
| Outcome: | Experiments on three datasets show that VITC achieves state-of-the-art performance . it delivers an average absolute gain of over 19% in Hit@1. |
Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language models (LLMs) have the capability to engage In-context Learning (ICL) however, this particular learning paradigm suffers from high instability stemming from factors such as input distribution, order and prompt formats. |
| Approach: | They propose to quantify the information gain obtained in prediction after observing a given example candidate and to sample those with maximum IG. |
| Outcome: | The proposed method can yield an average relative improvement of 14.3% across six classification tasks using three LLMs. |
DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | DiMo-GUI is a training-free framework for GUI grounding that splits input into textual elements and iconic elements, allowing the model to reason over each modality independently using general-purpose vision-language models. |
| Approach: | They propose a training-free framework for GUI grounding that leverages two core strategies: dynamic visual grounding and modality-aware optimization. |
| Outcome: | The proposed framework splits the input into textual elements and iconic elements, allowing the model to reason over each modality independently using general-purpose vision-language models. |
RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Services (2025.emnlp-industry)
Copied to clipboard
Fei Zhao, Chonggang Lu, null Wangyue, Zheyong Xie, Ziyan Liu, Haofu Qian, Jianzhao Huang, Fangcheng Shi, Zijie Meng, Hongcheng Guo, Mingqian He, Xinze Lyu, Zheyu Ye, Weiting Liu, Boyang Wang, Shaosheng Cao
| Challenge: | Social networking services (SNS) have experienced rapid growth, which has proposed significant challenges for platform content management and interaction quality improvement. |
| Approach: | They propose a domain-specific LLM to break the performance bottleneck of single-task baselines and establish a comprehensive foundation for social networking services. |
| Outcome: | The proposed model achieves an average improvement of 14.02% across 8 major tasks and 7.56% in bilingual evaluation benchmark, compared with baseline models. |
ICA-Proto: Iterative Cross Alignment Prototypical Network for Incremental Few-Shot Relation Classification (2023.findings-eacl)
Copied to clipboard
Wangjie Jiang, Zhihao Ye, Bang Liu, Ruihui Zhao, Jianguang Zheng, Mengyao Li, Zhiyong Li, Yujiu Yang, Yefeng Zheng
| Challenge: | Existing methods for relation classification suffer from the scarcity of manually annotated data. |
| Approach: | They propose a novel relation classification model that incorporates query representation into the encoding of novel prototypes and utilizes iteratively to achieve more interaction. |
| Outcome: | The proposed model outperforms the state-of-the-art model on two benchmark datasets. |
Discover and Prove: An Open-source Agentic Framework for Hard Mode Automated Theorem Proving in Lean 4 (2026.acl-long)
Copied to clipboard
Chengwu Liu, Yichun Yin, Ye Yuan, Jiaxuan Xie, Botao Li, Siqi Li, Jianhao Shen, Yan Xu, Lifeng Shang, Ming Zhang
| Challenge: | Existing approaches to solving mathematical problems fall into two broad categories: informal methods and formal methods. |
| Approach: | They propose to use LLM natural-language reasoning to discover answers . they introduce Discover And Prove framework that rewrites Hard Mode statements into Easy Mode ones for existing ATP provers. |
| Outcome: | The proposed framework can be used to prove hard mode statements on ATP benchmarks. |
DopplerBAS: Binaural Audio Synthesis Addressing Doppler Effect (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for binaural audio synthesis are limited in phase estimation, which is crucial for spatial hearing. |
| Approach: | They propose a method to explicitly address the Doppler effect of the moving speaker . it calculates the radial relative velocity of the speaker in spherical coordinates . |
| Outcome: | The proposed method improves the representative WarpNet and BinauralGrad backbones in phase error metric and reaches a new state of the art (SOTA) it is compared with the current method which is limited in phase estimation . |
DebugBench: Evaluating Debugging Capability of Large Language Models (2024.findings-acl)
Copied to clipboard
Runchu Tian, Yining Ye, Yujia Qin, Xin Cong, Yankai Lin, Yinxu Pan, Yesai Wu, Hui Haotian, Liu Weichuan, Zhiyuan Liu, Maosong Sun
| Challenge: | Large language models (LLMs) have demonstrated exceptional coding capabilities, but their debugging capabilities remain relatively unexplored. |
| Approach: | They propose a debugging benchmark consisting of 4,253 LLMs with four major bug categories and 18 minor types in C++, Java, and Python. |
| Outcome: | The proposed benchmark covers four major bug categories and 18 minor types in C++, Java, and Python. |
AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Code (2026.findings-acl)
Copied to clipboard
ShangZhan Li, Xinyu Yin, Xuanyu Jin, Ye He, Yuxin Zhou, Yuxuan Li, Xu Han, Wanxiang Che, Qi Shi, Ting Liu, Maosong Sun
| Challenge: | Current development practices face a dichotomy between automation and performance. |
| Approach: | They propose a framework to empower LLMs with the capability of automated explicit vectorization. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the SSE and AVX subsets of SimdBench. |
Position: LLMs Can be Good Tutors in English Education (2025.emnlp-main)
Copied to clipboard
Jingheng Ye, Shen Wang, Deqing Zou, Yibo Yan, Kun Wang, Hai-Tao Zheng, Ruitong Liu, Zenglin Xu, Irwin King, Philip S. Yu, Qingsong Wen
| Challenge: | Recent efforts to integrate large language models into English education lack adaptability to language learning. |
| Approach: | They argue that large language models can be effective tutors in English education . they encourage interdisciplinary research to explore these roles, fostering innovation and risks . |
| Outcome: | The proposed models can play three critical roles: 1) as data enhancers, 2) as task predictors, 3) as agents, enabling personalized and inclusive education. |
CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-Training (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for expressive text-to-speech only implicitly learn prosody with masked token reconstruction tasks. |
| Approach: | They propose a cross-modal contrastive pre-training framework that learns from prosody variance of the same text token under different contexts. |
| Outcome: | The proposed framework can learn from prosody variance of a text token under different contexts. |
An Inversion Attack Against Obfuscated Embedding Matrix in Language Model Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored transforming user inputs to obfuscated embedded vectors, so that the data will not be eavesdropped by service providers. |
| Approach: | They propose to transform user inputs to obfuscated embedded vectors so that the data will not be eavesdropped by service providers. |
| Outcome: | The proposed inversion attack can recover user input 100% from the obfuscated vectors without a solid and deliberate security design and analysis . |
From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research on PTQ spans three primary directions. |
| Approach: | They conduct a systematic analysis of post-training quantization failures using PTQ . they show that targeted repair can mitigate Signal Degradation but remains ineffective for Computation Collapse . |
| Outcome: | The proposed method mitigates Signal Degradation but remains ineffective for Computation Collapse. |
Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing scaling laws focus on general performance, overlooking crucial fine-grained factors and how quantization differentially impacts diverse knowledge capabilities. |
| Approach: | They propose a framework that unifies model size, bit-width, and fine-grained factors into memorization, application, and reasoning. |
| Outcome: | The proposed framework shows strong fit and cross-architecture consistency on 293 different PTQ configurations. |
Benchmarking Large Language Models on Communicative Medical Coaching: A Dataset and a Novel System (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing applications of natural language processing (NLP) focus on patient-centered services, but the potential of NLP to benefit inexperienced doctors remains unexplored. |
| Approach: | They propose a human-AI cooperative framework to assist medical learners in practicing communication skills during patient consultations. |
| Outcome: | The proposed framework enables medical learners to practice communication skills during patient consultations while a coach agent provides immediate, structured feedback. |
Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation (2025.acl-long)
Copied to clipboard
Shuo Tang, Xianghe Pang, Zexi Liu, Bohan Tang, Rui Ye, Tian Jin, Xiaowen Dong, Yanfeng Wang, Siheng Chen
| Challenge: | Extensive experiments demonstrate that our framework effectively generates both general and domain-specific data. |
| Approach: | They propose a multi-agent simulator that automatically generates diverse text-based scenarios, capturing a wide range of real-world human needs. |
| Outcome: | Experiments show that the proposed model outperforms Meta’s Llama-3-8B-Instruct model on AlpacaEval 2 and Arena-Hard benchmarks with just 20K instruction-response pairs. |
A Novel Framework Based on Medical Concept Driven Attention for Explainable Medical Code Prediction via External Knowledge (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict medical codes from clinical notes lack interpretability due to lengthy and noisy clinical notes. |
| Approach: | They propose a framework based on medical concept driven attention to integrate external knowledge for explainable medical code prediction from clinical notes. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on a benchmark dataset showing that it is more accurate than existing methods. |
POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversion (2025.emnlp-main)
Copied to clipboard
Yuan Liu, Zhongyin Zhao, Le Tian, Haicheng Wang, Xubing Ye, Yangxiu You, Zilin Yu, Chuhan Wu, Zhou Xiao, Yang Yu, Jie Zhou
| Challenge: | Existing approaches to training document conversion models with manual annotation are costly and time-consuming, and training student models by distilling outputs from teacher models can significantly limit their performance in real-world applications. |
| Approach: | They propose a fully automated framework for constructing high-quality document extraction datasets and models capable of handling diverse document formats and layouts. |
| Outcome: | The proposed model outperforms existing models and improves on annotated documents. |
Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards (2026.acl-industry)
Copied to clipboard
| Challenge: | a large language model (LLM) is used as a business development agent for persuasive price negotiation in online travel agencies. |
| Approach: | They propose a reward-enhancing policy optimization method that integrates three complementary reward sources-a preference-trained reward model and an LLM-as-a-judge. |
| Outcome: | The proposed method improves average dialogue rating to 4.63 (+0.33 over GRPO) and raises share of conversations with at least one excellent response to 66.67% (+23.34 pp over grepo). |
HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Autoregressive decoding limits the inference throughput of Large Language Models due to its sequential dependency. |
| Approach: | They propose a framework that allocates verification effort in proportion to candidate uncertainty. |
| Outcome: | Speculative decoding achieves an average speedup over state-of-the-art methods . a small subset of high-confidence predictions accounts for most successful verifications . |
HPE: Answering Complex Questions over Text by Hybrid Question Parsing and Execution (2023.findings-emnlp)
Copied to clipboard
| Challenge: | End-to-end neural networks excel at answering natural language questions but fail on complex ones . a proposed framework for question parsing and execution on textual QA is designed to combine the strengths of neural and symbolic methods. |
| Approach: | They propose a framework for question parsing and execution on textual QA . they parse questions into an intermediate representation and use deterministic rules to translate them . |
| Outcome: | The proposed framework outperforms existing methods in supervised, few-shot, and zero-shot settings while preserving its underlying reasoning process. |
QA‐LIGN: Aligning LLMs through Constitutionally Decomposed QA (2025.findings-emnlp)
Copied to clipboard
Jacob Dineen, Aswin Rrv, Qin Liu, Zhikun Xu, Xiao Ye, Ming Shen, Zhaonan Li, Shijie Lu, Chitta Baral, Muhao Chen, Ben Zhou
| Challenge: | QA-LIGN decomposes monolithic rewards into interpretable principle-specific evaluations . scalar rewards obscure which objectives drive the training signal . |
| Approach: | a new method decomposes monolithic rewards into interpretable principle-specific evaluations . QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate . |
| Outcome: | QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate . the results outperform DPO and GRPO with state-of-the-art reward models given equivalent training . |
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation (2023.acl-long)
Copied to clipboard
Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao
| Challenge: | Existing models for speech-to-speech translation suffer from distinct degradation in noisy environments and fail to translate visual speech. |
| Approach: | They propose a text-based audio-visual speech-to-speech translation model that integrates visual information with audio-only data to improve system robustness. |
| Outcome: | The proposed model outperforms models trained on audio-only corpus in two languages . it also improves with low-resource audio-visual data, compared with baselines . |
Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization (2026.acl-long)
Copied to clipboard
Weixu Zhang, Ye Yuan, Changjiang Han, Yuxing Tian, Zipeng Sun, Linfeng Du, Jikun Kang, Hong Kang, Xue Liu, Haolun Wu
| Challenge: | Large Language Models exhibit strong implicit personalization ability, but most approaches treat this behavior as a black box. |
| Approach: | They propose a mechanistic interpretation perspective and propose 'sparse' set of Preference Heads . they compute a Preference Contribution Score for each attention head and compare their predictions . |
| Outcome: | The proposed framework computes a Preference Contribution Score (PCS) for each attention head and measures its causal impact on user aligned outputs. |
Beyond Surface Features: Advancing Medical Vision-Language Alignment via Dynamic Evidence-Guided Preference Optimization (2026.acl-long)
Copied to clipboard
Zixuan Huang, Zhihong Zhu, Xiaolong Liu, Yanchao Hao, Manman Zhang, Zheng Wei, Bowen Xing, Xian Wu, Ye Li, Fen Miao, Yefeng Zheng
| Challenge: | Existing preference-based methods for medical large vision-Language Models face limitations in medical settings . existing methods are limited by overfitting to superficial cues and pseudo convergence of the preference signal. |
| Approach: | They propose a framework that enables evidence-aware and adaptive preference learning for Med-LVLMs. |
| Outcome: | The proposed framework improves evidence-aware and adaptive preference learning for Med-LVLMs. |
iPET: An Interactive Emotional Companion Dialogue System with LLM-Powered Virtual Pet World Simulation (2025.acl-demo)
Copied to clipboard
Zheyong Xie, Shaosheng Cao, Zuozhu Liu, Zheyu Ye, Zihan Niu, Chonggang Lu, Tong Xu, Enhong Chen, Zhe Xu, Yao Hu, Wei Lu
| Challenge: | Existing approaches to role-playing emotional companion products lack sustained personalization and contextual adaptability, limiting their effectiveness in real-world settings. |
| Approach: | They propose a virtual pet agent that can enhance user engagement through rich, dynamic pet behaviors and interactions tailored to individual preferences. |
| Outcome: | The proposed system has been deployed in a real-world, non-commercial product for 200 days and has demonstrated its effectiveness in practical applications. |
Are LLMs Rational Investors? A Study on the Financial Bias in LLMs (2025.findings-acl)
Copied to clipboard
Yuhang Zhou, Yuchen Ni, Zhiheng Xi, Zhangyue Yin, Yu He, Gan Yunhui, Xiang Liu, Zhang Jian, Sen Liu, Xipeng Qiu, Yixin Cao, Guangnan Ye, Hongfeng Chai
| Challenge: | Existing studies on biases within specific domains, such as finance, remain limited. |
| Approach: | They propose a framework to detect, detect, analyze and mitigate financial biases in large language models. |
| Outcome: | The proposed framework reduces bias by 68% for the most biased model, according to key metrics. |
Disentangled Code Representation Learning for Multiple Programming Languages (2021.findings-acl)
Copied to clipboard
| Challenge: | Developing effective distributed representations of source code is challenging . current code embedding approaches that represent the semantic and syntax of code are less interpretable . |
| Approach: | They propose a disentangled code representation learning approach to separate the semantic from the syntax of source code under a multi-programming-language setting. |
| Outcome: | The proposed approach achieves better interpretability and generalizability over existing methods. |
TED-TTS: Training-Free Intra-Utterance Emotion and Duration Control for Text-to-Speech Synthesis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing controllable Text-to-Speech methods limited to inter-utterance-level control . utterance expressiveness remains a challenge in building human-like TTS synthesis systems . |
| Approach: | They propose a training-free controllable framework for pretrained zero-shot TTS to enable intra-utterance emotion and duration expression. |
| Outcome: | The proposed framework achieves state-of-the-art intra-utterance consistency while maintaining baseline-level speech quality. |
R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQL (2024.findings-emnlp)
Copied to clipboard
Yuhang Zhou, Yu He, Siyu Tian, Yuchen Ni, Zhangyue Yin, Xiang Liu, Chuanjun Ji, Sen Liu, Xipeng Qiu, Guangnan Ye, Hongfeng Chai
| Challenge: | Adapting existing approaches for converting natural language to SQL encounters hurdles due to distinct nature of GQL compared to SQL. |
| Approach: | They propose a method that integrates both small and large Foundation Models for ranking, rewriting, and refining tasks. |
| Outcome: | The proposed approach integrates both small and large Foundation Models for ranking, rewriting, and refining tasks while capitalizing on the superior generalization and query generation prowess of larger models for the final transformation of natural language queries into GQL formats. |
Polymorphic Universal Transformer (2026.acl-long)
Copied to clipboard
Yilong Chen, Zitian Gao, Yihao Xiao, Jason Klein Liu, Xinyu Yang, Yifan Luo, Haoming Luo, Zhengmao Ye, Tingwen Liu, Ran Tao, Bryan Dai
| Challenge: | Compute Distribution Skew is a pathological phenomenon in ultra-deep recurrent models . it causes over-smoothing, representation rank collapse, and degraded reasoning performance. |
| Approach: | They propose a dynamic architecture that redefines recursive computation by decoupling parameter count from depth. |
| Outcome: | The proposed model significantly improves representation rank and reasoning robustness while reducing computation by 64.7%. |
LexRel: Benchmarking Legal Relation Extraction for Chinese Civil Cases (2026.acl-long)
Copied to clipboard
Yida Cai, Ranjuexiao Hu, Huiyuan Xie, Chenyang Li, Yun Liu, Yuxiao Ye, Zhenghao Liu, Weixing Shen, Zhiyuan Liu
| Challenge: | Legal relations are an important analytical framework for dispute resolution in civil cases. |
| Approach: | They propose a comprehensive schema for legal relations in civil cases with hierarchical taxonomy and definitions of arguments. |
| Outcome: | The proposed schema shows that existing LLMs lack the ability to identify civil legal relations and performance improves on downstream tasks. |
Data to Defense: The Role of Curation in Aligning Large Language Models Against Safety Compromise (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have identified a vulnerability in large language models (LLMs) during customization. |
| Approach: | They propose an adaptive data curation approach that allows any text to be curated to enhance its effectiveness in counteracting harmful samples during customization. |
| Outcome: | The proposed approach reduces compromising effects and generates 100% safe responses. |
Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on colexification patterns relies on annotated word lists, limiting scalability and usefulness in NLP. |
| Approach: | They propose two methods to train multilingual graphs from colexification patterns using an unannotated parallel corpus. |
| Outcome: | The proposed methods achieve high recall on CLICS and transfer learning in multilingual graphs. |
Concise and Organized Perception Facilitates Reasoning in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Extensive experimental results on several popular logical benchmarks (ProofWriter, PrOntoQA, PrONtoQA-OOD, and FOLIO) and mathematical benchmark (DI-GSM) show that COP significantly outperforms previous state-of-the-art methods. |
| Approach: | They propose a reasoning approach called Concise and Organized Perception (COP) that carefully analyzes the given statements to identify the most pertinent information while eliminating redundancy efficiently. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on several popular logical benchmarks and mathematical benchmarks. |
MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation (2025.naacl-long)
Copied to clipboard
Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao, Bohan Wu, Ye Yuan, Haozhe Zhao, Zhihui Guo, Yichi Zhang, Jingyang Yuan, Wei Ju, Luchen Liu, Tianyu Liu, Baobao Chang, Ming Zhang
| Challenge: | Large Multimodal Models (LMMs) exhibit impressive cross-modal understanding and reasoning abilities, but many benchmarks suffer from systematic biases. |
| Approach: | They propose a benchmark to avoid Type-I errors by creating one perception question and one knowledge anchor question through a meticulous annotation process. |
| Outcome: | The proposed benchmark avoids Type-I errors while maintaining reliability of MCQ evaluations. |
Improving Speech Translation by Fusing Speech and Text (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In speech translation, multimodal data to address limitations of individual modalities has shown significant effectiveness. |
| Approach: | They propose a cross-modal model which supports three input modalities for speech, text and fused speech-text. |
| Outcome: | The proposed model achieves an average of 34.0 BLEU on MuST-C, GigaST and newstest benchmark. |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)
Copied to clipboard
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
BatonVoice: An Operationalist Framework for Enhancing Controllable Speech Synthesis with Linguistic Intelligence from LLMs (2026.acl-long)
Copied to clipboard
Yue Wang, Ruotian Ma, Xingyu Chen, Zhengliang Shi, Morunliu Yang, Wanshun Chen, Huang Liu, Jiadi Yao, Xin He, Qu Yang, Qingxuan Jiang, Fanghua Ye, Juntao Li, Zhaopeng Tu, Xiaolong Li, Liefeng Bo, Min Zhang
| Challenge: | Existing approaches often fail to leverage the linguistic intelligence of Large Language Models (LLMs) Existing models lack the ability to follow text instructions for controllable Text-to-Speech (TTS). |
| Approach: | They propose a framework where an LLM acts as a conductor, understanding user instructions and generating a textual plan - explicit vocal features. |
| Outcome: | The proposed model outperforms open- and closed-source models in speech synthesis and achieves zero-shot cross-lingual generalization. |
LinguaGame: A Linguistically Grounded Game-Theoretic Paradigm for Multi-Agent Dialogue Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have enabled Multi-Agent Systems (MASs) where agents interact through natural language to solve complex tasks or simulate multi-party dialogues. |
| Approach: | They propose a linguistically-grounded game-theoretic paradigm for multi-agent dialogue generation that uses a training-free equilibrium approximation algorithm to model dialogue over communicative intents and strategies. |
| Outcome: | The proposed framework improves agents’ communication efficiency by helping them convey their intended meaning more effectively through language. |
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)
Copied to clipboard
Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains. |
| Approach: | They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries. |
| Outcome: | The proposed system outperforms baselines in the open domain task-solving benchmark. |
CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is popular in reasoning tasks, but its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored. |
| Approach: | They propose a Coarse-to-Fine Chain-of-Thought approach that breaks down NLU tasks into multiple reasoning steps where LLMs can learn to acquire essential concepts. |
| Outcome: | The proposed approach is effective in assisting the LLMs adapt to multi-grained NLU tasks under zero-shot and few-shot multi-domain settings. |
Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for open attribute value extraction for emerging entities are noisy or incomplete, even missing. |
| Approach: | They propose a knowledge-guided reinforcement learning framework for open attribute value extraction for emerging entities. |
| Outcome: | The proposed framework outperforms baselines by 16.5 - 27.8%. |
QuadrupletBERT: An Efficient Model For Embedding-Based Large-Scale Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for large-scale query-document retrieval are expensive and require sparse handcrafted features. |
| Approach: | They propose a quadrupletBERT model for effective and efficient retrieval using pre-trained language models like BERT. |
| Outcome: | The proposed model improves retrieval phase and leverages distances between simple negative and hard negative instances to obtain better embeddings. |
DocRED: A Large-Scale Document-Level Relation Extraction Dataset (P19-1)
Copied to clipboard
Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, Maosong Sun
| Challenge: | Existing relation extraction methods focus on extracting intra-sentence relations for single entities. |
| Approach: | They propose a relation extraction dataset from Wikipedia and Wikidata with three features . document-level relation extraction is a task to identify relational facts between entities . |
| Outcome: | The proposed dataset is the largest human-annotated dataset for document-level RE from plain text. |