Papers by Shi Yu
Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing research focuses on task-oriented or open-domain dialogue systems with influence skills. |
| Approach: | They propose to define and introduce a category of social influence dialogue systems that influence users’ cognitive and emotional responses. |
| Outcome: | The proposed system is task-oriented or goal-oriented, but it is not open-domain. |
Towards Self-Evolving Agents: Enabling Autonomy through Interactive Experience Refinement (2026.findings-acl)
Copied to clipboard
Cheng Yang, Xuemeng Yang, Licheng Wen, Daocheng Fu, Jianbiao Mei, Rong Wu, Pinlong Cai, Yufan Shen, Nianchen Deng, Jia Xu, Botian Shi, Yu Qiao, Haifeng Li
| Challenge: | Large Language Models struggle with complex, multi-step operational tasks because they remain static during inference and cannot learn from past experience. |
| Approach: | They propose a framework that organizes cross-domain insights to facilitate orchestration of long-horizon workflows. |
| Outcome: | The proposed framework outperforms existing methods on the TAC productivity benchmark and shows strong cross-task transferability. |
Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback (2025.acl-long)
Copied to clipboard
Jiakang Yuan, Xiangchao Yan, Bo Zhang, Tao Chen, Botian Shi, Wanli Ouyang, Yu Qiao, Lei Bai, Bowen Zhou
| Challenge: | Recent studies show that AI-assisted research methods can improve research efficiency . a closed-loop framework is used to enhance the automation level of scientific research . |
| Approach: | They propose a closed-loop LLM-driven framework to enhance the automation level of scientific research. |
| Outcome: | The proposed framework improves the efficiency of scientific research by improving data analysis, accelerating computation, and fostering novel idea generation. |
Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset (2022.lrec-1)
Copied to clipboard
Tiezheng Yu, Rita Frieske, Peng Xu, Samuel Cahyawijaya, Cheuk Tung Yiu, Holy Lovenia, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | In this paper, we address the problem of data scarcity for the Hong Kong Cantonese language . due to the popularization of deep learning, ASR technology has led to a significant improvement in recognizing many languages. |
| Approach: | They propose to use a dataset to analyze the data available for the Hong Kong Cantonese language . they use zh-HK as a source and a state-of-the-art ASR model to build a powerful model . |
| Outcome: | The proposed model improves on the biggest existing dataset, Common Voice zh-HK. |
Sign Languages and the Online World Online Dictionaries & Lexicostatistics (L18-1)
Copied to clipboard
| Challenge: | Several online dictionaries documenting the lexicon of a variety of sign languages are available . methodological issues must be addressed regarding how these resources are used for research purposes. |
| Approach: | They propose a web-based tool for annotating the articulatory features of signs . they compare handshapes for four Asian SLs and handshape for the entire sample . |
| Outcome: | The proposed tool compares handshapes and handsights of Asian SLs with European, American, and Brazilian SL samples. |
Hierarchical Memory Organization for Wikipedia Generation (2025.acl-long)
Copied to clipboard
Eugene J. Yu, Dawei Zhu, Yifan Song, Xiangyu Wong, Jiebin Zhang, Wenxuan Shi, Xiaoguang Li, Qun Liu, Sujian Li
| Challenge: | Existing methods for generating Wikipedia articles do not utilize memory directly for outline generation. |
| Approach: | They propose a method to generate Wikipedia articles autonomously by leveraging a hierarchical memory architecture. |
| Outcome: | The proposed framework outperforms baseline methods in producing informative and reliable articles. |
Resource-Limited Joint Multimodal Sentiment Reasoning and Classification via Chain-of-Thought Enhancement and Distillation (2026.findings-acl)
Copied to clipboard
| Challenge: | Current approaches for Multimodal Sentiment Analysis (MSA) rely on parameter-heavy LLMs for classification, overlooking multimodal sentiment reasoning generation in resource-limited environments. |
| Approach: | They propose a multimodal sentiment reasoning distillation model that employs a teacher-assistant-student paradigm to address deployment constraints in resource-limited environments. |
| Outcome: | The proposed model performs well on a resource-limited JMSRC task with only 3B parameters and shows generalization and interpretability. |
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)
Copied to clipboard
Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy. |
| Approach: | They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses . |
| Outcome: | The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples. |
SEGMENT+: Long Text Processing with Short-Context Language Models (2024.emnlp-main)
Copied to clipboard
Wei Shi, Shuang Li, Kerun Yu, Jinglei Chen, Zujie Liang, Xinhui Wu, Yuxi Qian, Feng Wei, Bo Zheng, Jiaqing Liang, Jiangjie Chen, Yanghua Xiao
| Challenge: | Existing frameworks that increase context window do not guarantee robust performance across long input tasks. |
| Approach: | They propose a framework that enables language models to handle extended inputs within limited context windows efficiently. |
| Outcome: | The framework improves performance on long-document question-answering and Needle-in-a-Haystack tasks. |
Large Language Models in Bioinformatics: A Survey (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are revolutionizing bioinformatics, enabling advanced analysis of DNA, RNA, proteins, and single-cell data. |
| Approach: | They examine the evolution of Large Language Models (LLMs) in bioinformatics and precision medicine by focusing on genomic sequence modeling, RNA structure prediction, protein function inference, and single-cell transcriptomics. |
| Outcome: | The proposed models are capable of predicting RNA structure and function and predicting single-cell transcriptomics. |
Selective Differential Privacy for Language Modeling (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to protect sensitive data from leaking are over-pessimistic and undifferentiated. |
| Approach: | They propose a new privacy notion, selective differential privacy, to provide rigorous privacy guarantees on the sensitive portion of the data to improve model utility. |
| Outcome: | The proposed privacy-preserving mechanism achieves better utility while remaining safe under various privacy attacks compared to baselines. |
TexSmart: A System for Enhanced Natural Language Understanding (2021.acl-demo)
Copied to clipboard
Lemao Liu, Haisong Zhang, Haiyun Jiang, Yangming Li, Enbo Zhao, Kun Xu, Linfeng Song, Suncong Zheng, Botong Zhou, Dick Zhu, Xiao Feng, Tao Chen, Tao Yang, Dong Yu, Feng Zhang, ZhanHui Kang, Shuming Shi
| Challenge: | TexSmart supports fine-grained named entity recognition (NER) Large-scale fine-granular entity types are expected to provide richer semantic information for downstream NLP applications. |
| Approach: | They introduce TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities. |
| Outcome: | The proposed system supports fine-grained named entity recognition (NER) and enhanced semantic analysis functions. |
Sentiment Adaptive End-to-End Dialog Systems (P18-1)
Copied to clipboard
| Challenge: | Existing methods to train dialog systems only consider semantic inputs and under-utilize other user information. |
| Approach: | They propose to include user sentiment in the end-to-end learning framework to make dialog systems more user-adaptive and effective. |
| Outcome: | The proposed system improves on a bus information search task with sentiment information. |
MedAdapter: Efficient Test-Time Adaptation of Large Language Models Towards Medical Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have improved generation and reasoning capabilities compared to traditional BERT-sized models due to massive number of parameters and extensive pre-training on vast textual corpora. |
| Approach: | They propose a unified post-hoc adapter for test-time adaptation of large language models . they propose to fine-tune only a small BERT-sized adapter to rank candidate LLMs . |
| Outcome: | The proposed adapter improves performance on four biomedical tasks without requiring computational resources or sharing data with third parties. |
Make-A-Voice: Revisiting Voice Large Language Models as Scalable Multilingual and Multitask Learners (2024.acl-long)
Copied to clipboard
Rongjie Huang, Chunlei Zhang, Yongqi Wang, Dongchao Yang, Jinchuan Tian, Zhenhui Ye, Luping Liu, Zehan Wang, Ziyue Jiang, Xuankai Chang, Jiatong Shi, Chao Weng, Zhou Zhao, Dong Yu
| Challenge: | Large language models (LLMs) have been used for general-purpose interfaces across multiple tasks and languages. |
| Approach: | They propose to use large language models as a general-purpose interface across multiple tasks and languages. |
| Outcome: | The proposed model performs better on 200K hours of 6-language data for voice generation applications. |
ESCP: Enhancing Emotion Recognition in Conversation with Speech and Contextual Prefixes (2024.lrec-main)
Copied to clipboard
| Challenge: | Emotion Recognition in Conversation (ERC) aims to analyze the speaker’s emotional state in a conversation. |
| Approach: | They propose to combine a directed acyclic graph and contextual prefixes to model historical utterances in a conversation and incorporate a contextual prefixed containing sentiment and semantics of historical . |
| Outcome: | The proposed model achieves state-of-the-art (SOTA) performance on several public benchmarks. |
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers (2024.emnlp-main)
Copied to clipboard
Ran Xu, Wenqi Shi, Yue Yu, Yuchen Zhuang, Yanqiao Zhu, May Dongmei Wang, Joyce Ho, Chao Zhang, Carl Yang
| Challenge: | Developing effective biomedical retrieval models is important for excelling at knowledge-intensive biomedically tasks but still challenging due to the lack of sufficient publicly annotated biomedic data and computational resources. |
| Approach: | They propose a series of dense retrievers for enhancing biomedical retrieval via unsupervised pre-training on large biomedically corpora, followed by instruction fine-tuning on a combination of labeled datasets and synthetic pairs. |
| Outcome: | Experiments on 5 biomedical tasks across 11 datasets confirm the performance of the retrieval model on various biomedically demanding tasks. |
Program Transfer for Answering Complex Questions over Knowledge Bases (2022.acl-long)
Copied to clipboard
| Challenge: | Program induction for complex questions over knowledge bases relies on a large number of parallel question-program pairs for the given KB, but the gold program annotations are usually lacking, making learning difficult. |
| Approach: | They propose an approach to leverage program annotations on rich KBs as external supervision signals to aid program induction for low-resourced KB. |
| Outcome: | The proposed approach outperforms SOTA methods on ComplexWebQuestions and WebQuestionSP. |
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization (2023.acl-long)
Copied to clipboard
Pengcheng He, Baolin Peng, Song Wang, Yang Liu, Ruochen Xu, Hany Hassan, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang
| Challenge: | Z-Code++ is a pre-trained language model optimized for abstractive text summarization. |
| Approach: | They propose a pre-trained language model optimized for abstractive text summarization that uses a two-phase pre-training technique to improve model's performance. |
| Outcome: | The proposed model outperforms the competing models on low-resource summarization tasks in zero-shot and few-shot settings. |
Efficient Multi-Agent System Training with Data Influence-Oriented Tree Search (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Model (LLM) based multi-agent systems (MAS) have high potential for tackling complex tasks through collaborative intelligence. |
| Approach: | They propose a framework that incorporates influence scores to guide tree search and data selection in data synthesis. |
| Outcome: | The proposed framework incorporates influence scores to guide tree search and data selection in data synthesis. |
EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Records (2024.emnlp-main)
Copied to clipboard
Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce Ho, Carl Yang, May Dongmei Wang
| Challenge: | EHRAgent enables clinicians to interact with EHRs using natural language . reliance on rule-based conversion systems often necessitates additional training or effort from data engineers. |
| Approach: | They propose a large language model agent that generates and executes code in natural language to facilitate clinicians in directly interacting with EHRs. |
| Outcome: | The proposed agent outperforms the strongest baseline by up to 29.6% in success rate on three real-world EHR datasets. |
INSPIRED: Toward Sociable Recommendation Dialog Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on recommendation dialog systems lack a study on communication strategies used by human speakers for making successful and persuasive recommendations. |
| Approach: | They propose to annotate a dataset of human-human movie recommendation dialogs with sociable recommendation strategies. |
| Outcome: | The proposed model outperforms the baseline model in automatic and human evaluation. |
BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solving (2025.acl-long)
Copied to clipboard
Teng Wang, Wing Yin Yu, Zhenqi He, Zehua Liu, HaileiGong HaileiGong, Han Wu, Xiongwei Han, Wei Shi, Ruifeng She, Fangzhou Zhu, Tao Zhong
| Challenge: | Existing datasets in operations research domain lack detailed annotations of the modeling process, focusing only on objective values. |
| Approach: | They propose an annotation-based tree-of-thought tree-based reasoning algorithm that integrates reinforcement learning into a tree- of-though. |
| Outcome: | The proposed algorithm outperforms state-of-the-art methods on StructuredOR, NL4OPT, and MAMO-ComplexLP datasets. |
PRAL: A Tailored Pre-Training Model for Task-Oriented Dialog Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to building task-oriented dialog systems require a substantial amount of annotations and thus are labor-intensive. |
| Approach: | They propose a Pre-trainedRole Alternating Language model (PRAL) that is explicitly designed for task-oriented dialog tasks. |
| Outcome: | The proposed model outperforms or is on par with state-of-the-art models on task-oriented dialog tasks. |
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)
Copied to clipboard
Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Xuemei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang
| Challenge: | i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data. |
| Approach: | They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data. |
| Outcome: | i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks. |
OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safety (2024.acl-demos)
Copied to clipboard
Chuang Liu, Linhao Yu, Jiaxuan Li, Renren Jin, Yufei Huang, Ling Shi, Junhui Zhang, Xinmeng Ji, Tingting Cui, Liutao Liutao, Jinwang Song, Hongying Zan, Sun Li, Deyi Xiong
| Challenge: | a rapid development of Chinese large language models poses big challenges for efficient LLM evaluation. |
| Approach: | They propose an evaluation testbed that benchmarks Chinese LLMs across capability, alignment and safety. |
| Outcome: | The evaluation platform OpenEval benchmarks Chinese LLMs across capability, alignment and safety. |
Computer Assisted Translation with Neural Quality Estimation and Automatic Post-Editing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Using neural machine translation to approximate human parity is difficult due to the lack of parallel training corpora. |
| Approach: | They propose an end-to-end deep learning framework for quality estimation and automatic post-editing of machine translation output. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the English–German dataset and human translators can significantly expedite their post-editing processing with the model. |
HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static compression methods suffer from coarse-grained caching and high I/O overhead. |
| Approach: | They propose a training-free dynamic compression framework that uses a sparse attention mechanism to categorize attention heads based on stability and similarity. |
| Outcome: | The proposed framework achieves state-of-the-art performance on long-context benchmarks and accelerates decoding by up to 3 compared to the original model with a 224K context. |
KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)
Copied to clipboard
Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals. |
| Approach: | They propose a self-supervised framework that enhances RAG systems through efficient model adaptation. |
| Outcome: | The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models. |
Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models (2026.acl-long)
Copied to clipboard
Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim
| Challenge: | Incomplete learning is widespread and heterogeneous in large language models . authors identify five recurrent sources of incomplete learning: missing prerequisite knowledge, conflicts between SFT supervision and pre-training knowledge, internal inconsistencies within SFT data, left-side forgetting during sequential fine-tuning, and insufficient optimization for rare or complex patterns. |
| Approach: | They propose a diagnostic-first framework that maps incomplete learning to causes . they identify five recurrent sources of incomplete learning: missing prerequisite knowledge, conflicts between supervision and pre-training knowledge, internal inconsistencies, left-side forgetting during sequential fine-tuning, and insufficient optimization for rare or complex patterns. |
| Outcome: | The proposed framework maps incomplete learning to causes using observable training and inference signals. |
Cross-lingual Text-to-SQL Semantic Parsing with Representation Mixup (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that Rex can benefit from cross-lingual training and improve the effectiveness of semantic parsers. |
| Approach: | They propose a Representation Mixup Framework for effectively exploiting translations in the cross-lingual Text-to-SQL task. |
| Outcome: | The proposed framework can benefit from cross-lingual training and improve the effectiveness of semantic parsers, achieving state-of-the-art performance. |
Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring (2026.acl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) are vulnerable to a growing array of multimodal jailbreak attacks, necessitating a generalizable defense that is efficient for practical deployment. |
| Approach: | They propose a framework that uses a lightweight projection to separate benign and malicious inputs in safety-critical layers. |
| Outcome: | The proposed framework enables a simple yet powerful contrastive score that differentiates true malicious intent from mere distribution shift. |
Benchmarking and Learning Real-World Customer Service Dialogue (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks and training pipelines for industrial intelligent customer service (ICS) focus on task completion and tool correctness. |
| Approach: | They propose a benchmark-to-optimization loop that bridges offline gains to deployment . they propose OlaMind, which distills reusable reasoning patterns from expert dialogues . |
| Outcome: | The proposed benchmark surpasses GPT-5.2 and Gemini 3 Pro on OlaBench . it delivers an average +23.67% issue resolution and -6.6% human transfer rate versus baseline . |
Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoning (2026.findings-acl)
Copied to clipboard
Haiyang Yu, Yuchuan Wu, Fan Shi, Jinghui Lu, Ke Niu, Xiaodong Ge, Minghan Zhuo, Jingqun Tang, Bin Li
| Challenge: | Existing document benchmarks focus on English printed texts or simplified Chinese . current vision-language models struggle with visual complexity and poor adaptability . |
| Approach: | They propose a benchmark to evaluate Chinese ancient documents' visual/linguistic complexity . ancient documents are valuable cultural heritage, but they face challenges in digitization and understanding . |
| Outcome: | the first benchmark for Chinese ancient documents evaluates VLMs from OCR to knowledge reasoning . ancient documents carry thousands of years of Chinese history and culture . traditional methods only scan images, while current models struggle with visual complexity . |
JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing KG-augmented models for commonsense question answering ignore the effectively fusing and reasoning over question context representations and the KG representations. |
| Approach: | They propose a novel model which combines a logical reasoning and a dynamic pruning mechanism to solve these limitations. |
| Outcome: | The proposed model improves existing models and performs interpretable reasoning on the CommonsenseQA and OpenBookQA datasets. |
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)
Copied to clipboard
Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Mengmeng Yang, Zheng Fu, Jinyu Miao, Yining Shi, He Zhe Lim, Li Liu, Tianbao Zhou, Hongyi Wang, Huang Yu, Yifei Hu, Guang Li, Guang Chen, Hao Ye, Lijun Sun, Diange Yang
| Challenge: | Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning. |
| Approach: | AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks. |
| Outcome: | Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% . |
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to language-based environment manipulation are difficult to generalize across environments. |
| Approach: | They propose a general framework for language-based environment manipulation tasks that can deal with various environments using the same generative language model. |
| Outcome: | The proposed framework achieves new state-of-the-art results on four of the tasks and the execution-guided pre-training strategy brings remarkable improvements on all experimental tasks. |
Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Persuasion dialogue systems have long-standing problems of dialogue repetition and inconsistency which could impact user experience and impede the persuaded outcome. |
| Approach: | They propose to refine a language model baseline without user simulators and distill sentence-level information about repetition, inconsistency, and task relevance through rewards. |
| Outcome: | The proposed model outperforms state-of-the-art models on automatic metrics and human evaluation results on a donation persuasion task and generates more diverse, consistent and persuasive conversations according to user feedback. |
Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language (2025.acl-long)
Copied to clipboard
Bo Zeng, Chenyang Lyu, Sinuo Liu, Mingyan Zeng, Minghao Wu, Xuanfan Ni, Tianqi Shi, Yu Zhao, Yefeng Liu, Chenyu Zhu, Ruizhe Li, Jiahui Geng, Qing Li, Yu Tong, Longyue Wang, Weihua Luo, Kaifu Zhang
| Challenge: | Existing datasets for instruction-following are monolingual and centered on English . existing data are unable to capture linguistic and cultural subtle differences . |
| Approach: | They propose an extension of IFEval to a localized multilingual version called Marco-Bench-MIF . their benchmark addresses linguistic constraints and cultural references via translation and verification . |
| Outcome: | The proposed extension of IFEval to a localized multilingual version covers 30 languages with varying levels of localization. |
Skeletons Matter: Dynamic Data Augmentation for Text-to-Query (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on a single query language, resulting in limited generalizability . a new task paradigm is proposed to unify semantic parsing tasks across different query languages . |
| Approach: | They propose a task paradigm that unifies parsing tasks across query languages . they identify query skeletons as a shared optimization target of Text-to-Query tasks . |
| Outcome: | The proposed method achieves state-of-the-art performance using only a small amount of synthesized data. |
CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)
Copied to clipboard
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan
| Challenge: | Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages . |
| Approach: | They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models . |
| Outcome: | The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English . |
Faceted Hierarchy: A New Graph Type to Organize Scientific Concepts and a Construction Method (D19-53)
Copied to clipboard
| Challenge: | faceted concept hierarchy is a structure of parent-child relationships . concepts are expected to be organized in a hierarchical structure for student learning . |
| Approach: | They propose a faceted concept hierarchy that aims to build facets from scientific literature. |
| Outcome: | The proposed hierarchy is more complete than "type-of" relations, and resolves conflicts by maintaining the acyclic structure of a hierarchy. |
PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)
Copied to clipboard
Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei
| Challenge: | Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning. |
| Approach: | They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios. |
| Outcome: | The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics. |
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning (2025.acl-long)
Copied to clipboard
Yujie Feng, Xujia Wang, Zexin Lu, Shenghong Fu, Guangyuan Shi, Yongxin Xu, Yasha Wang, Philip S. Yu, Xu Chu, Xiao-Ming Wu
| Challenge: | Continual learning (CL) is crucial for large language models without costly retraining. |
| Approach: | They propose a framework for recurrent knowledge identification and fusion that enables dynamic estimation of parameter importance distributions to enhance knowledge transfer. |
| Outcome: | The proposed framework mitigates catastrophic forgetting and enhances knowledge transfer. |
Decoding LLM Personality Measurement: Forced-Choice vs. Likert (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research has focused on investigating the psychological characteristics of Large Language Models (LLMs), emphasizing the importance of comprehending their behavioral traits. |
| Approach: | They evaluated six Large Language Models: Llama-3.1-8B, GLM-4-9B, Claude-3.5-sonnet, and Deepseek-V3 and used the forced-choice test to assess their personality traits. |
| Outcome: | The forced-choice test is more reliable and more accurate than the likert scale and forced-CHOICE test results for LLMs' Big Five personality scores. |
SAD: A Large-Scale Strategic Argumentative Dialogue Dataset (2026.acl-long)
Copied to clipboard
YongKang Liu, Jiayang Yu, Mingyang Wang, Yiqun Zhang, Ercong Nie, Shi Feng, Daling Wang, Kaisong Song, Hinrich Schuetze
| Challenge: | Argumentation is a key part of human reasoning and decision-making . existing argumentative corpora focus on single-turn settings, but multi-turn dialogues are often realized as multi-turned dialogues . |
| Approach: | They present a dataset for strategic multi-turn argumentation dialogues . they annotate each utterance with five strategy types, allowing multiple strategies per utterrance . |
| Outcome: | The proposed dataset shows that explicit prompting improves fluency, stylistic coherence and persuasiveness. |
Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty (2026.findings-acl)
Copied to clipboard
Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim
| Challenge: | Existing approaches to generating reward models rely on voting-based mechanisms to evaluate CoT outputs. |
| Approach: | They propose an efficient generative reward modeling framework grounded in model-internal uncertainty. |
| Outcome: | The proposed framework reduces inference cost while improving answer accuracy. |
Training LLMs to be Better Text Embedders through Bidirectional Reconstruction (2025.emnlp-main)
Copied to clipboard
Chang Su, Dengliang Shi, Siyuan Huang, Jintao Du, Changhua Meng, Yu Cheng, Weiqiang Wang, Zhouhan Lin
| Challenge: | Existing text embedding approaches often leverage the embeddment of the final token, typically a reserved special token such as ‘[EOS]‘. |
| Approach: | They propose to add a new training stage before contrastive learning to enrich the semantics of the final token embedding. |
| Outcome: | The proposed training stage improves performance on the Massive Text Embedding Benchmark (MTEB), achieving new state-of-the-art results across different LLM base models and scales. |
MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoning (2026.acl-long)
Copied to clipboard
Weikang Shi, Aldrich Yu, Rongyao Fang, Houxing Ren, Ke Wang, Aojun Zhou, Changyao Tian, Xinyu Fu, Yuxuan Hu, Zimu Lu, Linjiang Huang, Si Liu, Rui Liu, Hongsheng Li
| Challenge: | Existing approaches to visual chain-of-thought are limited by external tools or fail to generate high-fidelity diagrams. |
| Approach: | They propose a framework to enable large multimodal models with VCoT capabilities . they pre-train a model on a 15.2M-pair corpus and teach it how to leverage visual aids . |
| Outcome: | The proposed framework unlocks complex, human-like visual reasoning in large language models . it pre-trains the model on a 15.2M-pair corpus and fine-tunes it on MathCanvas-Instruct . |
Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions (D19-1)
Copied to clipboard
Rui Zhang, Tao Yu, Heyang Er, Sungrok Shim, Eric Xue, Xi Victoria Lin, Tianze Shi, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | Generating SQL queries from user utterances is an important task to help end users acquire information from databases. |
| Approach: | They propose a context-dependent text-to-SQL generation task that edits previous queries . they use an utterance-table encoder and a table-aware decoder to incorporate context . |
| Outcome: | The proposed model is flexible to change individual tokens and robust to error propagation. |
Edge-Enhanced Graph Convolution Networks for Event Detection with Syntactic Relation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Event detection (ED) is a key subtask of information extraction. |
| Approach: | They propose an architecture that exploits syntactic structure and typed dependency label information to perform event detection. |
| Outcome: | The proposed architecture exploits syntactic structure and typed dependency label information to perform ED. |
DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | . - (EN) |
| Approach: | . - (EN) |
| Outcome: | . - (EN) |
MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)
Copied to clipboard
Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun
| Challenge: | Scientific data visualization is an essential process in research, but its use of large language models remains unexplored. |
| Approach: | They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks. |
| Outcome: | The proposed framework improves performance of commercial and open-source models. |
Design Challenges in Low-resource Cross-lingual Entity Linking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing techniques for grounding mentions of entities in a foreign language do not rise to the challenges introduced by text in low-resource languages (LRL) and fail to generalize to text not taken from Wikipedia, on which they are usually trained. |
| Approach: | They propose a cross-lingual XEL technique that uses search engines to locate and search for foreign language entries in Wikipedia. |
| Outcome: | The proposed system shows an increase of 25% in gold candidate recall and 13% in end-to-end linking accuracy over state-of-the-art baselines. |
EvoR: Evolving Retrieval for Code Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pipelines for retrieval-augmented code generation (RACG) use static knowledge bases with a single source, limiting adaptation capabilities of Large Language Models (LLMs) Extensive experiments demonstrate that EVOR achieves two to four times of execution accuracy compared to other methods such as Reflexion. |
| Approach: | They propose a retrieval-augmented code generation pipeline that employs the synchronous evolution of queries and diverse knowledge bases. |
| Outcome: | The proposed pipeline achieves two to four times of execution accuracy compared to other methods. |
How to Build User Simulators to Train RL-based Dialog Systems (D19-1)
Copied to clipboard
| Challenge: | RL-based dialog systems require interaction with the environment and obtaining real human users to interact with the system is time-consuming and labor-intensive. |
| Approach: | They propose a method to standardize user simulator building to compare dialog system quality using the same set of user simulators. |
| Outcome: | The proposed method can be used by the community to compare dialog system quality using the same set of user simulators fairly. |
Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data (2023.findings-acl)
Copied to clipboard
| Challenge: | Structure Aware Dense Retrieval (SANTA) model encodes user queries and structured data in one universal embedding space for retrieving structured data. |
| Approach: | They propose to use structured data and unstructured data to encode queries and structured data in one universal embedding space for retrieving structured data. |
| Outcome: | The proposed model achieves state-of-the-art on code search and product search and conducts convincing results in the zero-shot setting. |
Structured Attention for Unsupervised Dialogue Structure Induction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using structured attention, a model can learn dialogue structure in unsupervised fashion. |
| Approach: | They propose to incorporate structured attention layers into a Variational Recurrent Neural Network model with discrete latent states to learn dialogue structure in an unsupervised fashion. |
| Outcome: | The proposed model learns semantic structures similar to templates used to generate a dialogue corpus on two-party datasets and on multi-party dialogues, disentangling dialogues without human annotation. |
Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models (2025.tacl-1)
Copied to clipboard
| Challenge: | a recent study revisits six core challenges that have influenced the evolution of Neural Machine Translation (NMT) domain mismatch, amount of parallel data, rare word prediction, translation of long sentences and sub-optimal beam search remain challenges in LLMs. |
| Approach: | They revisit core challenges that have acted as benchmarks for progress in NMT . they propose to revisit these challenges and offer insights into their relevance . |
| Outcome: | The proposed models significantly improve translation of sentences containing approximately 80 words, even translating documents up to 512 words. |
Craw4LLM: Efficient Web Crawling for LLM Pretraining (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work discards over 90% of the raw data collected from web crawls, highlighting the inefficiency of current web crawlers in collecting LLM pretraining data. |
| Approach: | They propose a web crawling method that leverages the preference of LLMs as the priority score of the web crawler’s scheduler to obtain high-quality pretraining data. |
| Outcome: | The proposed method achieves high-quality pretraining data on a web graph containing 900 million webpages from a commercial search engine's index with just 21% URLs crawled. |
MdEval: Massively Multilingual Code Debugging (2026.findings-acl)
Copied to clipboard
Shukai Liu, Linzheng Chai, Jian Yang, Jiajun Shi, He Zhu, Liran Wang, Jin Ke, Wei Zhang, Hualei Zhu, Shuyue Guo, Tao Sun, Jiaheng Liu, Yunlong Duan, Yu Hao, Liqun Yang, Guanglin Niu, Ge Zhang, Zhoujun Li
| Challenge: | Existing benchmarks primarily focus on Python and are limited in terms of language diversity. |
| Approach: | They propose a multilingual debugging benchmark that includes 3.9K test samples of 20 programming languages and introduces the debug instruction corpora MdEval-Instruct by injecting bugs into the correct multilingual queries and solutions. |
| Outcome: | The proposed benchmark includes 3.9K test samples of 20 programming languages and covers the automated program repair task, bug localization task, and bug identification task. |
Document-Level Machine Translation with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) such as ChatGPT can produce coherent, cohesive, relevant, and fluent answers for various natural language processing tasks. |
| Approach: | They examine the impact of different prompts on document-level translation quality and discourse phenomena using figures and lines, which are invisible to GPT-4. |
| Outcome: | The proposed models outperform commercial MT systems and advanced document-level MT methods on a number of benchmarks and show potential to become a new paradigm for document- level translation. |
BSCodec: A Band-Split Neural Codec for High-Quality Universal Audio Reconstruction (2026.findings-eacl)
Copied to clipboard
| Challenge: | Neural audio codecs have enabled high-fidelity reconstruction of speech, music and sound . however, speech-optimized codec systems suffer degradation on music or sound if they ignore spectral differences . |
| Approach: | They propose a neural audio codec that splits the spectral dimension into separate bands and compresses each band independently. |
| Outcome: | Experimental results show that BSCodec achieves better reconstruction quality on music and sound compared to existing codecs. |
ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling (2026.findings-eacl)
Copied to clipboard
Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Chaojun Xiao, Zhiyuan Liu, Ge Yu, Chenyan Xiong
| Challenge: | Large Language Models (LLMs) exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge. |
| Approach: | They propose a framework that enhances the external knowledge utilization of Large Language Models through a two-stage constructivist cognitive modeling process. |
| Outcome: | The proposed framework achieves a 10% improvement over baseline methods on various question-answering benchmarks. |
Understanding User Resistance Strategies in Persuasive Conversations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Persuasive dialog systems have various usages, such as donation persuation and physical exercise persulasion. |
| Approach: | They adopt a preliminary framework on persuasion resistance in psychology and build a fine-grained resistance strategy annotation scheme to analyze the persuitee's resistance strategies. |
| Outcome: | The proposed system can understand and address user resistance strategies appropriately. |
Re-TASK: Revisiting LLM Tasks from Capability, Skill, and Knowledge Perspectives (2025.findings-acl)
Copied to clipboard
Zhihu Wang, Shiwan Zhao, Yu Wang, Heyuan Huang, Sitao Xie, Yubo Zhang, Jiaxin Shi, Zhixing Wang, Hongyan Li, Junchi Yan
| Challenge: | Existing approaches to solving complex tasks with large language models (LLMs) fail to decompose tasks accurately or execute subtasks effectively. |
| Approach: | They propose a Chain-of-Learning (CoL) paradigm that highlights task dependencies on specific capability items, further broken down into their constituent knowledge and skill components. |
| Outcome: | The proposed model improves Yi-1.5-9B and Llama3-Chinese-8B for legal tasks by 45.00% and 24.50% on different domains. |
TUNA: Comprehensive Fine-grained Temporal Understanding Evaluation on Dense Dynamic Videos (2025.acl-long)
Copied to clipboard
Fanheng Kong, Jingyuan Zhang, Hongzhi Zhang, Shi Feng, Daling Wang, Linhao Yu, Xingguang Ji, Yu Tian, V. W., Fuzheng Zhang
| Challenge: | Existing benchmarks for video understanding often focus on specific aspects, overlooking the holistic nature of video content. |
| Approach: | They propose a temporal-oriented benchmark for fine-grained understanding on dense dynamic videos with two complementary tasks: captioning and QA. |
| Outcome: | The proposed model performs well on diverse video scenarios and dynamic videos, with interpretable and robust evaluation criteria. |
Logic-Consistency Text Generation from Semantic Parses (2021.findings-acl)
Copied to clipboard
| Challenge: | Text generation from semantic parses is challenging due to the complexity of the inner logic and the lack of automatic evaluation metrics for logic consistency. |
| Approach: | They propose a framework for logic consistent text generation from semantic parses that employs iterative training procedures and quality control. |
| Outcome: | The proposed framework enhances logic consistency and human evaluation on two benchmark datasets. |
MIC: A Multi-task Interactive Curation Tool (2022.emnlp-demos)
Copied to clipboard
| Challenge: | MIC is a human-machine collaborative curation tool for multiple NLP tasks. |
| Approach: | They propose to use a multi-task interactive curation tool for multiple NLP tasks to help solve pain points in real NLP. |
| Outcome: | The proposed tool supports multiple projects with multiple users and allows for collaborative annotations. |
SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)
Copied to clipboard
Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang
| Challenge: | Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases. |
| Approach: | They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs. |
| Outcome: | The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs. |
Fusion-in-T5: Unifying Variant Signals for Simple and Effective Document Ranking with Attention Fusion (2024.lrec-main)
Copied to clipboard
| Challenge: | Current document ranking pipelines involve multiple ranking layers to integrate different information step-by-step. |
| Approach: | They propose a novel re-ranker Fusion-in-T5 which integrates text matching information, ranking features, and global document information into one single unified model via templated-based input and global attention. |
| Outcome: | The proposed model significantly improves ranking performance over complex cascade pipelines. |
UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity Mixture-of-Experts (2026.acl-long)
Copied to clipboard
Zhenyu Liu, Yunxin li, Xuanyu Zhang, Qixun Teng, Shenyuan Jiang, Xinyu Chen, Haoyuan Shi, Haolan Chen, Fanbo Meng, Mingjun Zhao, Yu Xu, Yancheng He, Baotian Hu, Haizhou Li, Min Zhang
| Challenge: | Recent advances in unified multimodal models indicate a clear trend towards comprehensive content generation. |
| Approach: | They propose a unified speech and music generation model built upon a novel framework . they propose specialized MoE architectures and curated training strategies to tackle data imbalances . |
| Outcome: | The proposed model achieves state-of-the-art performance on major speech and music generation benchmarks. |
TSGP: Two-Stage Generative Prompting for Unsupervised Commonsense Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on acquiring relevant knowledge by retrieving external knowledge bases and fine-tuning pre-trained models. |
| Approach: | They propose a two-stage prompt-based unsupervised commonsense question answering framework that leverages implicit knowledge stored in PrLMs to generate knowledge for questions with unlimited types and possible candidate answers independent of specified choices. |
| Outcome: | The proposed framework significantly improves the reasoning ability of language models in unsupervised settings. |
Improved Visual Story Generation with Adaptive Context Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent work shows that diffusion models generate images of high resolution and semantic consistency to text prompts. |
| Approach: | They propose a method that uses adaptive context modeling to improve leading system . they evaluate their method on pororoSV and FlintstonesSV datasets . |
| Outcome: | The proposed method achieves state-of-the-art FID scores on pororo and Flintstones datasets. |
One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2023.findings-acl)
Copied to clipboard
Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, Tao Yu
| Challenge: | a new method for embedding text is developed for tasks that require specialized encoders . INSTRUCTOR is a single embedder that can generate text embeddables tailored to different tasks and domains based on instruction finetuning . |
| Approach: | They introduce a new method for computing text embeddings given task instructions . they first annotate instructions for 330 diverse tasks and train INSTRUCTOR on this multitask mixture . |
| Outcome: | The proposed method improves on 70 embedding evaluation tasks with fewer parameters than the previous best model. |
Mixed-Lingual Pre-training for Cross-lingual Summarization (2020.aacl-main)
Copied to clipboard
| Challenge: | Cross-lingual summarization (CLS) aims at producing a summary in the target language for an article in the source language. |
| Approach: | They propose a mixed-lingual pre-training scheme that leverages both cross-lingual tasks such as translation and monolingual tasks like masked language models. |
| Outcome: | The proposed model improves on the translation and masked language models with no task-specific components and saves memory. |
AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026.findings-acl)
Copied to clipboard
Wentao Shi, Yu Wang, Yuyang Zhao, Yuxin Chen, Fuli Feng, Xueyuan Hao, Xi Su, Qi GU, Hui Su, Xunliang Cai, Xiangnan He
| Challenge: | Existing approaches to verify agent behaviors in complex environments rely on rule-based verifiers or LLM-as-a-Judge models. |
| Approach: | They propose a benchmark to evaluate Agent-as-a-Judge across three domains . the benchmark covers search, data systems, and graphical user interfaces - with 155 tasks and 516 trajectories . |
| Outcome: | The proposed benchmark outperforms existing benchmarks in search, data systems, and GUI domains while revealing open challenges in agent-based verification. |
ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework (2026.acl-long)
Copied to clipboard
Kai Qin, Liangxin Liu, Yu Liang, Longzheng Wang, null Wangyan, Zhang Yueyang, Long Xia, Zhiyuan Sun, Houde Liu, Daiting Shi
| Challenge: | Existing methods for generating reward models focus on outcome-level supervision, neglecting analytical process quality, which constrains their potential. |
| Approach: | They propose a novel reward model that leverages self-reflection to assess analytical quality and enhance preference modeling. |
| Outcome: | The proposed model improves performance on four benchmarks and significantly mitigates positional bias. |
Seamlessly Integrating Factual Information and Social Content with Persuasive Dialogue (2022.aacl-main)
Copied to clipboard
| Challenge: | Persuasive dialogue systems are designed for chatbots to communicate with and influence users with specific goals. |
| Approach: | They propose a modular dialogue system framework that integrates factual information and social content into persuasive dialogues. |
| Outcome: | The proposed framework is generalizable to any dialogue tasks that have mixed social and task contents. |
ReCUT: Balancing Reasoning Length and Accuracy in LLMs via Stepwise Trails and Preference Optimization (2025.findings-emnlp)
Copied to clipboard
Zhensheng Jin, Xinze Li, Yifan Ji, Chunyi Peng, Zhenghao Liu, Qi Shi, Yukun Yan, Shuo Wang, Furong Peng, Ge Yu
| Challenge: | Existing methods to train LLMs suffer from overthinking, leading to lengthy reasoning traces . Existing approaches to train large language models suffer from this problem . |
| Approach: | They propose a method to combine multiple reasoning chains for training LLMs . they use stepwise exploration and long-short switched sampling to evaluate reasoning paths . |
| Outcome: | The proposed method reduces reasoning lengths by approximately 30-50% . it also maintains or improves reasoning accuracy compared to baselines . |
AutoReply: Detecting Nonsense in Dialogue with Discriminative Replies (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for dialogue detection make many errors in their own messages . a dataset of long dialogues richly grounded in the game state contains many errors . |
| Approach: | They propose to use an annotated dialogue dataset to generate automatic responses for dialogue models. |
| Outcome: | The proposed model outperforms handcrafted replies and performs on par with supervised learning approaches. |
LEGOEval: An Open-Source Toolkit for Dialogue System Evaluation via Crowdsourcing (2021.acl-demo)
Copied to clipboard
| Challenge: | Currently, researchers use automatic metrics and human evaluation to evaluate dialogue systems. |
| Approach: | They propose to use a Python API to easily evaluate dialogue systems using Amazon Mechanical Turk. |
| Outcome: | The open-source toolkit provides a fast, consistent method for reproducing human evaluation results. |
Gardener: An Agentic AI System for Single-Cell RNA Sequence Analysis (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing large language models encode workflow progress as conversational state and rely on cloud-centric execution, which hinders traceability and auditability. |
| Approach: | Gardener is an open-source desktop application for macOS and windows under the Apache License 2.0. |
| Outcome: | Gardener is released as an open-source desktop application for macOS and Windows under the Apache License 2.0. |
CI-AVSR: A Cantonese Audio-Visual Speech Datasetfor In-car Command Recognition (2022.lrec-1)
Copied to clipboard
Wenliang Dai, Samuel Cahyawijaya, Tiezheng Yu, Elham J. Barezi, Peng Xu, Cheuk Tung Yiu, Rita Frieske, Holy Lovenia, Genta Winata, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | In-car smart assistants should be able to process general as well as car-related commands and perform corresponding actions, which eases driving and improves safety. |
| Approach: | They propose a dataset for in-car command recognition in the cantonese language with both video and audio data. |
| Outcome: | The proposed model can achieve a considerable quality on the clean test set, but the speech recognition quality on noisy data is still inferior. |
Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study (2025.coling-main)
Copied to clipboard
Bowen Li, Wenhan Wu, Ziwei Tang, Lin Shi, John Yang, Jinyang Li, Shunyu Yao, Chen Qian, Binyuan Hui, Qicheng Zhang, Zhiyin Yu, He Du, Ping Yang, Dahua Lin, Chao Peng, Kai Chen
| Challenge: | Existing benchmarks focused on simplified or isolated aspects of coding, ignoring the full spectrum of programming challenges. |
| Approach: | They propose a case study that examines the performance of large language models across the entire software development lifecycle with four programming languages, multiple domains, and carefully designed and verified metrics for each task. |
| Outcome: | The proposed model performs across the entire software development lifecycle, including design, environment setup, implementation, acceptance testing, and unit testing. |
Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In (2023.acl-long)
Copied to clipboard
| Challenge: | Prior work on retrieval augmentation fine-tuned the retriever and the LM, making them closely coupled. |
| Approach: | They propose a generic retrieval plug-in that can be used to fine-tune retrieval augmentation and a LM to learn a user's preferences. |
| Outcome: | The proposed retriever improves the generalization of large language models on the MMLU and PopQA datasets by learning LM’s preferences from a known source LM . |
Named Entity Recognition through Deep Representation Learning and Weak Supervision (2021.findings-acl)
Copied to clipboard
| Challenge: | Weakly supervised named entity recognition (NER) uses noisy labels to estimate the true labels of a dataset. |
| Approach: | They propose a model to learn optimal assignments of latent NER tags using observed tokens and weak labels provided by labeling functions. |
| Outcome: | The proposed model improves the quality of weak labels on four public datasets. |
Making Better Use of Training Corpus: Retrieval-based Aspect Sentiment Triplet Extraction via Label Interpolation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to extract sentimental triplets are infeasible and counterproductive . aspect Sentiment Triplets Extraction (ASTE) task is an emerging sub-task of Aspect-based Sentimence Analysis . |
| Approach: | They propose a retrieval-based approach to the Aspect Sentiment Triplet Extraction task . they retrieve semantic similar triplets from the training corpus and interpolate their label information . |
| Outcome: | The proposed approach establishes a new state-of-the-art on the Aspect Sentiment Triplet Extraction task. |
ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation (2022.lrec-1)
Copied to clipboard
Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Peng Xu, Yan Xu, Zihan Liu, Rita Frieske, Tiezheng Yu, Wenliang Dai, Elham J. Barezi, Qifeng Chen, Xiaojuan Ma, Bertram Shi, Pascale Fung
| Challenge: | Code-switching is a speech phenomenon occurring when a speaker switches language during a conversation. |
| Approach: | They propose to collect Mandarin Chinese-English code-switching corpus from read speech rather than spontaneous speech to address this phenomenon. |
| Outcome: | ASCEND consists of 10.62 hours of clean speech, collected from 23 bilingual speakers of Chinese and English. |
RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts (2025.acl-long)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) models enable Large Language Models to access external knowledge. |
| Approach: | They propose a knowledge refinement method that incorporates reranking signals to generate CoT-based summarization based on query and retrieval documents. |
| Outcome: | RankCoT generates CoT-based summarization based on query and all retrieval documents . Rank CoT incorporates a self-reflection mechanism that refines the outputs . |
A Model-agnostic Data Manipulation Method for Persona-based Dialogue Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models for introducing explicit personas are expensive due to their expensive collection costs. |
| Approach: | They propose a data manipulation method which is model-agnostic to be packed with any persona-based dialogue generation model to improve their performance. |
| Outcome: | The proposed method is model-agnostic to be packed with any persona-based dialogue generation model to improve their performance. |
AwarenessBench: Assessing Cognitive Capabilities of Language Models (2026.acl-long)
Copied to clipboard
Xiaojian Li, Rongwu Xu, Tianyun Zhang, Yue Wang, Shuo Chen, Qiner Lyu, Briana Zhang, Peiran Yang, Kyle Xue Chen, Haoyuan Shi, Yu Wang, Wei Xu
| Challenge: | Language models exhibit increasingly consciousness-like behaviors, requiring a baseline to evaluate their cognitive abilities. |
| Approach: | They propose a benchmark to assess the cognitive abilities of language models (LMs) they compare 18 state-of-the-art LMs to human models in metacognition, self-awareness, social awareness and situational awareness . |
| Outcome: | Evaluating 18 state-of-the-art LMs, they find they consistently surpass baselines . but most models fall short in metacognition and self-awareness, the study finds . |
ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training (2026.acl-long)
Copied to clipboard
Yu Liang, Liangxin Liu, Longzheng Wang, null Wangyan, Zhang Yueyang, Long Xia, Zhiyuan Sun, Daiting Shi
| Challenge: | ConsistRM is a self-training framework that enables effective and stable GRM training without human annotations. |
| Approach: | They propose a self-training framework that enables effective and stable GRM training without human annotations. |
| Outcome: | The proposed framework outperforms vanilla Reinforcement Fine-Tuning (RFT) by 1.5% on five benchmark datasets. |
SSMLoRA: Enhancing Low-Rank Adaptation with State Space Model (2025.naacl-long)
Copied to clipboard
| Challenge: | Fine-tuning requires substantial computational resources and is prone to overfitting when applied to small datasets. |
| Approach: | They propose a parameter-efficient fine-tuning method that integrates a State Space Model (SSM) to interconnect low-rank matrices. |
| Outcome: | The proposed method achieves comparable performance to LoRA on the general language understanding evaluation (GLUE) benchmark while using only half the parameters. |
Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification (2026.acl-long)
Copied to clipboard
Zenghao Duan, Zhiyi Yin, Zhichao Shi, Liang Pang, Shaoling Jing, Zihe Huang, Jiayi Wu, Yu Yan, Jingcheng Deng, Huawei Shen, Xueqi Cheng
| Challenge: | Large language models (LLMs) exhibit exceptional performance but pose inherent risks of generating toxic content. |
| Approach: | They propose a method that removes toxic subspaces from FFN parameters . they propose to use a lightweight method to eliminate toxic subespaces . |
| Outcome: | The proposed method achieves SOTA detoxification while preserving general capabilities without large-scale retraining. |
Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good (P19-1)
Copied to clipboard
| Challenge: | Persuasion agents are a form of communication that can be used to change people's opinions and actions for social good. |
| Approach: | They designed an online persuasion task where one participant was asked to persult the other to donate to a specific charity. |
| Outcome: | The proposed system could change people's opinions and actions for social good. |
Retrieval-Augmented Few-shot Text Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for retrieval-augmented text classification are successful in the few-shot scenario with limited retrieval space. |
| Approach: | They propose to use EM-L and R-L to provide task-specific guidance to retrieval metric . they also propose to incorporate retrieved memory alongside parameters for better generalization . |
| Outcome: | The proposed methods perform better on the few-shot scenario with limited retrieval space. |
RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation (2025.findings-emnlp)
Copied to clipboard
Tianjiao Li, Mengran Yu, Chenyu Shi, Yanjun Zhao, Xiaojing Liu, Qi Zhang, Xuanjing Huang, Qiang Zhang, Jiayin Wang
| Challenge: | Using reinforcement learning from human feedback, large language models perform poorly when applied to colloquial subtitle translation tasks. |
| Approach: | They propose an adversarial training framework that iteratively updates the offline reward model and the online LLM to improve training outcomes. |
| Outcome: | The proposed training framework significantly improves upon translation baselines. |
CheckRLM: Effective Knowledge–Thought Coherence Checking in Retrieval-Augmented Reasoning (2026.acl-long)
Copied to clipboard
Dingling Xu, Ruobing Wang, Qingfei Zhao, Yukun Yan, Zhichun Wang, Daren Zha, Shi Yu, Zhenghao Liu, Shuo Wang, Xu Han, Maosong Sun
| Challenge: | Reasoning Language Models (RLMs) have improved performance on complex tasks by extending the reasoning chain, but they are prone to factual errors, especially in knowledge-intensive tasks. |
| Approach: | They propose a framework that improves the reliability of the reasoning process by timely checking and correcting factual errors. |
| Outcome: | The proposed framework outperforms baselines and shows that it mitigates error accumulation with lower costs. |
ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing dense retrieval with query augmentation ignore the alignment between generation and ranking objectives. |
| Approach: | They propose a unified LLM-augmented dense retrieval framework that jointly optimizes both the LLM and the retriever. |
| Outcome: | Experimental results show that ExpandR outperforms strong baselines, achieving more than 5% improvement in retrieval performance. |
Explanation Graph Generation via Generative Pre-training over Synthetic Graphs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for explanation graph generation are limited due to the large number of datasets available. |
| Approach: | They propose a text-to-graph generative task to pre-train a model to bridge the text-graph gap. |
| Outcome: | The proposed framework surpasses all baseline systems with remarkable margins on ExplaGraphs and CommonsenseQA. |
Controllable Mixed-Initiative Dialogue Generation through Prompting (2023.acl-short)
Copied to clipboard
| Challenge: | Mixed initiative dialogue systems allow all interacting agents to initiate actions to control the interaction. |
| Approach: | They propose to prompt large language models as a drop-in replacement for fine-tuning on conditional generation. |
| Outcome: | The proposed prompts improve fine-tuning and ground truth responses . the results show that generated responses are high . |
Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (2024.findings-acl)
Copied to clipboard
Ran Xu, Hejie Cui, Yue Yu, Xuan Kan, Wenqi Shi, Yuchen Zhuang, May Dongmei Wang, Wei Jin, Joyce Ho, Carl Yang
| Challenge: | Clinical natural language processing (NLP) is a subfield that requires the extraction, analysis, and interpretation of unstructured clinical text. |
| Approach: | They propose a model which infuses knowledge into clinical text generation with LLMs for clinical NLP tasks. |
| Outcome: | The proposed model improves performance across 8 clinical NLP tasks and 18 datasets by 7.7%-8.7% on average. |
RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Records (2024.acl-short)
Copied to clipboard
| Challenge: | Existing deep learning models for EHRs rely on knowledge from a single source and do not capture the semantic information for medical codes. |
| Approach: | They propose a Retrieval AugMentation pipeline to augment clinical prediction on EHRs . they use multiple knowledge sources to convert them into text and use consistency regularization to capture complementary information from patient visits and summarized knowledge. |
| Outcome: | Experiments on two EHR datasets show that RAM-EHR improves clinical prediction tasks. |
MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding (2024.emnlp-main)
Copied to clipboard
Baixuan Xu, Weiqi Wang, Haochen Shi, Wenxuan Ding, Huihao Jing, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Long Chen, Yangqiu Song
| Challenge: | Existing methods for acquiring large-scale intentions generate product-centric intentions without product images and incur high costs for scalability. |
| Approach: | They propose a multimodal framework that allows Large Vision-Language Models to infer purchase intentions from multimodal product metadata and prioritize human-centric ones. |
| Outcome: | The proposed framework shows that it is robust to different prompts and superior to previous methods. |
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022.emnlp-main)
Copied to clipboard
Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu
| Challenge: | Structured knowledge grounding (SKG) uses structured knowledge to complete user requests . since inputs and outputs of SKG tasks are heterogeneous, they have been studied separately . |
| Approach: | They propose a framework that unifies 21 SKG tasks into a text-to-text format . they use unifiedSKG to benchmark T5 with different sizes . |
| Outcome: | The proposed framework unifies 21 SKG tasks into a text-to-text format . it achieves state-of-the-art performance on almost all of the 21 tasks, the authors show . |
Speech Aerodynamics Database, Tools and Visualisation (2022.lrec-1)
Copied to clipboard
| Challenge: | Aerodynamic processes underlie the characteristics of the acoustic signal of speech sounds. |
| Approach: | a database of aerodynamic processes underlies the characteristics of the acoustic signal of speech sounds . a project was undertaken to obtain data with simultaneous recording of speech acustic signals . |
| Outcome: | the database was designed during an ARC project . it contains recordings of 2 English, 1 Amharic, and 7 French speakers . |
Logic-level Evidence Retrieval and Graph-based Verification Network for Table-based Fact Verification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods leverage programs that contain rich logical information to enhance the verification process. |
| Approach: | They propose a table-based fact verification task as an evidence retrieval framework . they retrieve logic-level program-like evidence from the given table and a statement as supplementary evidence for the table . |
| Outcome: | The proposed method is able to retrieve logic-level program-like evidence from a table and a statement as supplementary evidence for the table. |
Semantic Role Labeling Guided Out-of-distribution Detection (2024.lrec-main)
Copied to clipboard
Jinan Zou, Maihao Guo, Yu Tian, Yuhao Lin, Haiyao Cao, Lingqiao Liu, Ehsan Abbasnejad, Javen Qinfeng Shi
| Challenge: | Existing methods for identifying domain-shifted instances are prone to OOD and adversarial inputs. |
| Approach: | They propose an unsupervised method that separates, extracts, and learns the semantic role labeling guided out-of-distribution Detection (SRLOOD) they propose a self-supervised approach to enhance global-local feature learning by predicting SRL extracted role. |
| Outcome: | The proposed method achieves SOTA performance on four OOD benchmarks. |
DPEPO: Diverse Parallel Exploration Policy Optimization for LLM-based Agents (2026.acl-long)
Copied to clipboard
JunShuo Zhang, Chengrui Huang, Feng Guo, Zihan Li, Ke Shi, Menghua Jiang, Jiguo Yu, Shuo Shang, Shen Gao
| Challenge: | Existing approaches to large language model (LLM) agents that follow the sequential "reason-then-act" paradigm suffer from limited exploration and incomplete environmental understanding as they interact with only a single environment per step. |
| Approach: | They propose a paradigm that enables an agent to interact with multiple environments simultaneously and share cross-trajectory experiences. |
| Outcome: | The proposed paradigm achieves state-of-the-art (SOTA) success rates while maintaining comparable efficiency to strong sequential baselines. |
D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chat (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing human-machine dialogue systems are not able to provide diagnostic information for depression diagnosis due to stigma associated with mental illness. |
| Approach: | They propose to construct a Chinese Dialogue Dataset for depression-diagnosis-oriented chat based on clinical depression diagnostic criteria. |
| Outcome: | The proposed system can be used to diagnose depression using a Chinese Dialogue Dataset. |
Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Quality Estimation (QE) is an essential role in applications of Machine Translation (MT). |
| Approach: | They propose to fuse uncertainty quantification into a pre-trained cross-lingual language model to predict the translation quality. |
| Outcome: | The proposed method achieves state-of-the-art on the datasets of WMT 2020 QE shared task. |
GUI Agents: A Survey (2025.findings-acl)
Copied to clipboard
Dang Nguyen, Jian Chen, Yu Wang, Gang Wu, Namyong Park, Zhengmian Hu, Hanjia Lyu, Junda Wu, Ryan Aponte, Yu Xia, Xintong Li, Jing Shi, Hongjie Chen, Viet Dac Lai, Zhouhang Xie, Sungchul Kim, Ruiyi Zhang, Tong Yu, Mehrab Tanjim, Nesreen K. Ahmed, Puneet Mathur, Seunghyun Yoon, Lina Yao, Branislav Kveton, Jihyung Kil, Thien Huu Nguyen, Trung Bui, Tianyi Zhou, Ryan A. Rossi, Franck Dernoncourt
| Challenge: | Large Foundation Models (LFMs) have transformed the landscape of AI research and day-to-day life. |
| Approach: | They propose a framework that delineates GUI agents' perception, reasoning, planning, and acting capabilities. |
| Outcome: | The proposed framework delineates their perception, reasoning, planning, and acting capabilities. |
RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentation (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) enhances large language models by integrating external knowledge retrieved at inference time. |
| Approach: | They evaluate RAG systems using MassiveDS, a large-scale datastore with mixture of knowledge. |
| Outcome: | The proposed approach improves performance on knowledge-intensive NLP tasks. |
SOM-NCSCM : An Efficient Neural Chinese Sentence Compression Model Enhanced with Self-Organizing Map (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sentence Compression (SC) is an important natural language processing task . it aims to shorten sentences while preserving the original meanings of the words . improvements on Chinese SC models are still lacking due to several difficulties . |
| Approach: | They propose a neural Chinese SC model enhanced with a Self-Organizing Map from Chinese colloquial sentences from a real-life question answering system. |
| Outcome: | The proposed model achieves a promising F1 score of 89.655 and BLEU4 score of 70.116 . it improves the performance of the whole neural Chinese SC model in a valid manner . |
Multi-Modal Multi-Granularity Tokenizer for Chu Bamboo Slips (2025.coling-main)
Copied to clipboard
| Challenge: | Using a multi-modal multi-granularity tokenizer, we analyze ancient Chinese scripts . a large proportion of the characters in ancient Chinese are rare or undeciphered . |
| Approach: | They propose a multi-modal multi-granularity tokenizer specifically designed for ancient Chinese scripts. |
| Outcome: | The proposed tokenizer improves on the part-of-speech tagging task on the Chu bamboo slip script. |
Self-Adjust Softmax (2025.emnlp-main)
Copied to clipboard
Chuanyang Zheng, Yihang Gao, Guoxuan Chen, Han Shi, Jing Xiong, Xiaozhe Ren, Chao Huang, Zhenguo Li, Yu Li
| Challenge: | Usually, tokens with larger attention scores are important for the final prediction. |
| Approach: | They propose to modify softmax(z) to z softmax and its normalized variant to improve the Transformer attention mechanism by making minor adjustments to the softmax function. |
| Outcome: | The proposed model provides enhanced gradient properties compared to the vanilla softmax function. |
When Hearst Is not Enough: Improving Hypernymy Detection from Corpus with Distributional Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | a taxonomy is a semantic hierarchy of words or concepts organized w.r.t. their hypernymy relationships. |
| Approach: | They propose a framework for hypernymy detection using large textual corpora . they quantify the non-negligible existence of specific sparsity cases . |
| Outcome: | The proposed framework quantifies the non-negligible existence of specific sparsity cases on several benchmark datasets. |
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases (D19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, Dragomir Radev
| Challenge: | CoSQL is a corpus for building cross-domain, general-purpose database querying dialogue systems. |
| Approach: | They present a corpus for building cross-domain, general-purpose database querying dialogue systems . they use a Wizard-of-Oz collection of 3k turns plus 10k+ annotated SQL queries . |
| Outcome: | The proposed corpus is based on a Wizard-of-Oz dataset of 3k dialogues querying 200 complex DBs spanning 138 domains. |
Unsupervised Dialog Structure Learning (N19-1)
Copied to clipboard
| Challenge: | Current dialog systems require human experts to design the dialog structure, which is time consuming and sometimes insufficient to satisfy various customer needs. |
| Approach: | They propose to extract dialog structure using a modified VRNN model with discrete latent vectors. |
| Outcome: | The proposed model outperforms existing models on the ability to predict unseen data and is faster and more effective in a reinforcement learning setting. |
Marco-o1 v2: Towards Widening The Distillation Bottleneck for Reasoning Models (2025.acl-long)
Copied to clipboard
Huifeng Yin, Yu Zhao, Minghao Wu, Xuanfan Ni, Bo Zeng, Huaiyu.wh Huaiyu.wh, Tianqi Shi, Liangying Shao, Chenyang Lyu, Longyue Wang, Weihua Luo, Kaifu Zhang
| Challenge: | Recent efforts to distill large reasoning models into smaller lightweight models have shown competitive performances. |
| Approach: | They propose to distill long Chain-of-Thought data to improve SFT and RL methods by constructing data from scratch using Monte Carlo Tree Search. |
| Outcome: | The proposed method significantly improves reasoning performance on various benchmarks such as math (GSM8K, MATH, AIME). |
Similarity = Value? Consultation Value-Assessment and Alignment for Personalized Search (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods rely on semantic similarity to align historical consultations with current queries due to the absence of ‘value’ labels, but this lacks exploration of needs in user consultations. |
| Approach: | They propose a consultation value assessment framework that evaluates historical consultations from three novel perspectives: (1) Scenario Scope Value, (2) Posterior Action Value, and (3) Time Decay Value. |
| Outcome: | The proposed model outperforms baselines on public and commercial datasets on both retrieval and ranking tasks. |
CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies (2024.findings-emnlp)
Copied to clipboard
| Challenge: | CultureBank is a knowledge base built upon users’ self-narratives with 12K cultural descriptors sourced from TikTok and 11K from Reddit. |
| Approach: | They construct a pipeline to construct cultural knowledge bases from different online communities on a massive scale. |
| Outcome: | The proposed pipeline improves cultural awareness of language models by evaluating them on two cultural tasks in a zero-shot setting. |
MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods to predict the start and end positions of answer spans generate two probability vectors. |
| Approach: | They propose a method that extends the probability vector to a probability matrix. |
| Outcome: | The proposed method improves on SQuAD 1.1 and three other question answering benchmarks. |
LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)
Copied to clipboard
Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size. |
| Approach: | They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding. |
| Outcome: | The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models. |
Learn to Combine Linguistic and Symbolic Information for Table-based Fact Verification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification lack attention to combine linguistic and symbolic information. |
| Approach: | They propose a graph-based reasoning approach that learns to combine linguistic and symbolic information effectively. |
| Outcome: | The proposed method can combine linguistic and symbolic information effectively. |
Speed Up Your Code: Progressive Code Acceleration Through Bidirectional Tree Editing (2025.acl-long)
Copied to clipboard
Longhui Zhang, Jiahao Wang, Meishan Zhang, GaoXiong Cao, Ensheng Shi, Mayuchi Mayuchi, Jun Yu, Honghai Liu, Jing Li, Min Zhang
| Challenge: | Existing training methods, such as direct instruction fine-tuning, overlook hierarchical relationships among acceleration patterns. |
| Approach: | They propose a new training paradigm that uses bidirectional tree editing and progressive code acceleration learning to improve LLMs’ CA capabilities. |
| Outcome: | The proposed training paradigm outperforms prompt-enhanced GPT-4 and current training-based methods on average across five programming languages. |
Enhancing Large Language Model for Knowledge Graph Completion via Structure-Aware Alignment-Tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge graph completion methods ignore inconsistent representation spaces between natural language and graph structures, leading to duplicate works and time-consuming processes. |
| Approach: | They propose a framework that enhances LLMs for KGC via structure-aware alignment-tuning to align graph embeddings with the natural language space through multi-task contrastive learning. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two KGC tasks across four benchmark datasets. |
Safety Alignment via Constrained Knowledge Unlearning (2025.acl-long)
Copied to clipboard
Zesheng Shi, Yucheng Zhou, Jing Li, Yuxin Jin, Yu Li, Daojing He, Fangming Liu, Saleh Alharbi, Jun Yu, Min Zhang
| Challenge: | Existing defense mechanisms have not fully deleted harmful knowledge in large language models (LLMs) Existing methods to address safety alignment have not completely deleted harmful information in LLMs. |
| Approach: | They propose a safety alignment strategy that uses scoring neurons to identify useful knowledge in LLMs and pruning the gradients of neurons in U to preserve beneficial information. |
| Outcome: | The proposed method significantly improves model safety while maintaining utility compared to existing methods. |
DAPE V2: Process Attention Score as Feature Map for Length Extrapolation (2025.acl-long)
Copied to clipboard
Chuanyang Zheng, Yihang Gao, Han Shi, Jing Xiong, Jiankai Sun, Jingyao Li, Minbin Huang, Xiaozhe Ren, Michael Ng, Xin Jiang, Zhenguo Li, Yu Li
| Challenge: | Extensive experiments demonstrate that treating attention as a feature map and applying convolution as . a processing method significantly enhances Transformer performance. |
| Approach: | They propose to use the convolution operator to mimic the processing methods in computer vision to treat attention as a feature map and apply it to neighboring attention scores across different heads. |
| Outcome: | The proposed model can be adapted to various attention-related models and achieves high performance. |
UniCreative: Unifying Long-form Logic and Short-form Sparkle via Reference-Free Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Xiaolong Wei, Zerun Zhu, Simin Niu, Xingyu Zhang, Peiying Yu, Changxuan Xiao, Yuchen Li, Jicheng Yang, Zhejun Zhao, Chong Meng, Long Xia, Daiting Shi
| Challenge: | Existing alignment paradigms for creative writing use static reward signals and supervised data. |
| Approach: | They propose a constraint-aware reward model that synthesizes query-specific criteria to provide fine-grained preference judgments. |
| Outcome: | The proposed framework aligns models with human preferences across content quality and structural paradigms without supervised fine-tuning and ground-truth references. |
Forward-Backward Reasoning in Large Language Models for Mathematical Verification (2024.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on six standard mathematical data sets and three LLMs show that FOBAR achieves state-of-the-art performance. |
| Approach: | They propose to combine forward and backward reasoning to verify candidate answers . they propose to use a template to mask a number and ask the LLM to answer a backward question . |
| Outcome: | Experiments on mathematical data show that proposed backward reasoning outperforms Self-Consistency. |
BERT-BC: A Unified Alignment and Interaction Model over Hierarchical BERT for Response Selection (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent performance boosting for dialogue response selection task achieved by Cross-Encoder based models is limited and the learned models have poor generalization capability in realistic scenarios. |
| Approach: | They propose a model that combines the representation-based Bi-Encoder and interaction-based Cross-Encoding to achieve better semantic representation. |
| Outcome: | The proposed model can achieve state-of-the-art performance on three benchmark datasets for multi-turn response selection. |
Concise Math Reasoning via Difficulty-Aware Distillation (2026.findings-acl)
Copied to clipboard
Yifan Wu, Jingze Shi, Bingheng Wu, Jiayi Zhang, Xiaotian Lin, Yizhang Zhu, Zhaoyang Yu, Bang Liu, Chenglin Wu, Nan Tang, Yuyu Luo
| Challenge: | Human experts tackle difficult math problems by identifying and executing a few pivotal steps rather than listing every intermediate thought. |
| Approach: | They propose a method for producing training data that mirrors concise human reasoning by rewriting a problem's solution to retain only the essential steps. |
| Outcome: | The proposed method outperforms models trained on 800k long CoT and cuts training and inference costs. |
Just Fine-tune Twice: Selective Differential Privacy for Large Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to protect language models from privacy leakage suffer from limited user control and low utility . et al., 2018: a novel framework that achieves SDP for state-of-the-art large transformer-based models. |
| Approach: | They propose a framework that applies differential privacy to large language models . they use redacted in-domain data to fine-tune the model with original in- domain data . |
| Outcome: | The proposed framework achieves strong utility compared to baselines. |