Papers by Ye Yu
Improving Knowledge Graph Completion with Generative Hard Negative Mining (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph completion (KGC) use generative methods with a self-information-enhanced training strategy to generate high-quality negatives. |
| Approach: | They propose to leverage a sequence-to-sequence architecture to generate high-quality hard negatives from the same decoding distributions as the anchor. |
| Outcome: | The proposed method produces high-quality negatives with good hardness and diversity on three KGC benchmarks. |
ParaSuite: Boosting LLM Reasoning via Paradox Resolution (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for paradox research focus on checking basic logical consistency and not reflective reasoning. |
| Approach: | They propose a pipeline dedicated to paradox research that automates data synthesis, evaluation, and training. |
| Outcome: | The proposed pipeline improves paradoxical and general STEM reasoning. |
Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Numerous architectures and pretraining methods have been proposed for context-dependent text-to-SQL, but the size of the datasets used has been limited due to the high cost of annotating multi-turn dialogue and SQL pairs. |
| Approach: | They propose to augment training datasets using self-play which leverages contextual information to synthesize new interactions to adapt the model to new databases. |
| Outcome: | The proposed model improves accuracy on SParC and CoSQL, two widely used cross-domain text-to-SQl datasets. |
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning (2026.findings-acl)
Copied to clipboard
Yibo Yan, Shen Wang, Jiahao Huo, Jingheng Ye, Zhendong Chu, Xuming Hu, Philip S. Yu, Carla P Gomes, Bart Selman, Qingsong Wen
| Challenge: | Current scientific reasoning models struggle with generalization across domains and fall short of multimodal perception. |
| Approach: | They propose to use multimodal large language models to integrate text, images, and other modalities to enhance scientific reasoning. |
| Outcome: | The proposed models can integrate text, images, and other modalities and improve reasoning across disciplines. |
Noise-robust Cross-modal Interactive Learning with Text2Image Mask for Multi-modal Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on multi-modal neural machine translation focus on visual information, but text and image may not match exactly, and visual noise is often ignored. |
| Approach: | They propose a noise-robust multi-modal interactive fusion approach with cross-modal relation-aware mask mechanism for MNMT. |
| Outcome: | The proposed model achieves state-of-the-art scores in all En-De, En-Fr and En-Cs translation tasks. |
MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models (2023.emnlp-demo)
Copied to clipboard
| Challenge: | MusicAgent integrates numerous music-related tools and an autonomous workflow to address user requirements. |
| Approach: | a new system is built to integrate music-related tools and an autonomous workflow . the system is based on large language models (LLMs) that can be used to organize and decompose requests . |
| Outcome: | the proposed system integrates numerous music-related tools and an autonomous workflow to address user requirements. |
CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)
Copied to clipboard
| Challenge: | a number of tools are used to perform complex tasks, but the tool utilization process can cause errors. |
| Approach: | They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks. |
| Outcome: | The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios. |
SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing closed-source LLMs have a performance gap in text-to-SQL reasoning tasks. |
| Approach: | They propose a SQL-based approach to synthesize reliable data to enhance text-to-SQL reasoning in LLMs. |
| Outcome: | The proposed model achieves state-of-the-art accuracy on the widely recognized Spider and BIRD benchmarks, significantly narrowing the performance gap with closed-source methods. |
FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)
Copied to clipboard
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev
| Challenge: | Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity. |
| Approach: | They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models. |
| Outcome: | The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models. |
Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to augment pre-trained large language models require extensive computational efforts and massive data volumes, challenging the widespread accessibility of LLM research. |
| Approach: | They propose a post-pretraining strategy of selectively enhancing shallow layers while pruning less effective deep ones to augment pretrained large language models. |
| Outcome: | The proposed approach improves performance on the corpus of code & math and a legal corpus and is widely applicable. |
Self-Consistency Boosts Calibration for Math Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing solutions for math reasoning tasks use semantic parsing or AST decoding, but performance can degrade dramatically even with slight changes to the questions. |
| Approach: | They propose three calibration methods based on self-consistency for math reasoning tasks. |
| Outcome: | The proposed methods bridge model confidence and accuracy better than existing methods based on p(True) or logit. |
Optimizing Question Semantic Space for Dynamic Retrieval-Augmented Multi-hop Question Answering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval-augmented generation methods are insufficient for multi-hop question answering . however, they tend to generate hallucinations due to semantic mismatching . |
| Approach: | They propose to optimize question semantic space for dynamic retrieval-augmented multi-hop question answering by optimizing the semantic embeddings. |
| Outcome: | The proposed method outperforms existing RAG methods in both in- and out-of-domain settings. |
Make-A-Voice: Revisiting Voice Large Language Models as Scalable Multilingual and Multitask Learners (2024.acl-long)
Copied to clipboard
Rongjie Huang, Chunlei Zhang, Yongqi Wang, Dongchao Yang, Jinchuan Tian, Zhenhui Ye, Luping Liu, Zehan Wang, Ziyue Jiang, Xuankai Chang, Jiatong Shi, Chao Weng, Zhou Zhao, Dong Yu
| Challenge: | Large language models (LLMs) have been used for general-purpose interfaces across multiple tasks and languages. |
| Approach: | They propose to use large language models as a general-purpose interface across multiple tasks and languages. |
| Outcome: | The proposed model performs better on 200K hours of 6-language data for voice generation applications. |
A Unified Temporal Knowledge Graph Reasoning Model Towards Interpolation and Extrapolation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for temporal knowledge graphs de-emphasize temporal correlations between facts sequences and ignore inferring clues from missing facts. |
| Approach: | They propose a Temporal PAth-based reasoning model that is robust to ambiguous temporal data. |
| Outcome: | The proposed model outperforms SOTA methods on the link prediction task. |
Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive models have boosted the efficiency of neural machine translation, but their performance is significantly worse than that of autoregressive counterparts. |
| Approach: | They propose to incorporate syntactic and semantic structures among natural languages into a non-autoregressive Transformer for the task of neural machine translation. |
| Outcome: | The proposed model achieves faster speed and keeps translation quality compared with other models. |
NOVA-63: Native Omni-lingual Versatile Assessments of 63 Disciplines (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual benchmarks show severe drawbacks, such as overly translated content, the absence of difficulty control, and disciplinary imbalance, making the benchmarking process unreliable and showing low convincingness. |
| Approach: | They propose a multilingual benchmark that integrates LLM-assisted formatting, expert quality verification, and multi-level difficulty screening to provide a comprehensive, difficult multilingual assessment. |
| Outcome: | The proposed benchmark features 93,536 questions sourced from native speakers across 14 languages and 63 academic disciplines. |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (2024.findings-emnlp)
Copied to clipboard
Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan
| Challenge: | Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales. |
| Approach: | They propose to annotate a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans. |
| Outcome: | The proposed model outperforms existing methods on understanding the capabilities of LLMs in logical reasoning by 10% or more. |
kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning (2024.naacl-long)
Copied to clipboard
Wenting Zhao, Ye Liu, Yao Wan, Yibo Wang, Qingyang Wu, Zhongfen Deng, Jiangshu Du, Shuaiqi Liu, Yunlong Xu, Philip Yu
| Challenge: | Recent advances in task-oriented parsing involve formulating the task as a sequence-to-sequence problem, relying on a wealth of labeled data. |
| Approach: | They propose a task-oriented parsing framework that integrates nearest-neighbor learning with a nearest-nearest approach. |
| Outcome: | The proposed model can be used to synthesize computer programs based on a natural-language prompt without additional data or specialized prompts. |
Exploring Mathematical Extrapolation of Large Language Models with Synthetic Data (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown excellent capabilities in language understanding, text generation and many other tasks, but struggle in complex multi-step reasoning problems such as mathematical reasoning. |
| Approach: | They propose to fine tune an open-llama-3B model to perform well on multi-step reasoning tasks via synthetic data. |
| Outcome: | The proposed model can reach a zero-shot pass@1 at 0.44 on the in-domain dataset and demonstrates certain generalization capabilities on the out-of-domain data. |
KidsArtBench: Multi-Dimensional Children’s Art Evaluation with Attribute-Aware MLLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) show impressive capabilities across visual–language tasks, but their capacity to evaluate artistic expression remains limited. |
| Approach: | They propose an attribute-specific multi-LoRA approach where each attribute corresponds to a distinct evaluation dimension in the scoring rubric. |
| Outcome: | The proposed approach increases correlation from 0.468 to 0.653 on Qwen2.5-VL-7B, with the largest gains on perceptual dimensions and narrowed gaps on higher-order attributes. |
AOT*: Efficient Synthesis Planning via LLM-Empowered AND-OR Tree Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Experimental evaluation shows that AOT* achieves competitive solve rates using 3-5 fewer iterations than existing LLM-based approaches. |
| Approach: | They propose a framework that integrates LLM-generated chemical synthesis pathways with systematic AND-OR tree search. |
| Outcome: | Experimental results show that AOT* improves search efficiency and solves faster than existing approaches. |
Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to supervised relational triple extraction require huge amounts of labeled data. |
| Approach: | They propose a multi-prototype embedding network model to extract the composition of relational triples from unstructured text. |
| Outcome: | The proposed method improves the performance of the few-shot relational triple extraction problem. |
Experience is the Teacher: Reusing Atomic Thoughts from LLMs to Improve Medical Dialogue (2026.findings-acl)
Copied to clipboard
Guangya Yu, Hui Luo, Qi Ye, Ruihui Hou, Weiyan Zhang, Mingxi Shang, Xuanwu Li, ChunMing Wang, Tong Ruan
| Challenge: | Recent large reasoning models (LLMs) lack dynamic and diverse thinking capabilities . reusing atomic thoughts provides a practical pathway toward dynamic reasoning . |
| Approach: | They propose a framework that extracts atomic thoughts from teacher models and reuses them to guide reasoning and generate responses. |
| Outcome: | The proposed framework extracts atomic thoughts from teacher models and reuses them to guide reasoning and generate responses. |
HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks (2026.findings-acl)
Copied to clipboard
Yiming Zeng, Jinghan Cao, Zexin Li, Wanhao Yu, Zhankai Ye, Dawei Xiang, Ting Hua, Xin Liu, Shangqian Gao, Tingting Yu
| Challenge: | Existing approaches treat instruction-based text editing as a generic text generation problem. Existing methods either over-edit or fail to apply modifications consistently. |
| Approach: | They propose a framework that processes each editing request to best align with it. |
| Outcome: | The proposed framework achieves 9% improvement over the state-of-the-art model. |
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)
Copied to clipboard
Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Mengmeng Yang, Zheng Fu, Jinyu Miao, Yining Shi, He Zhe Lim, Li Liu, Tianbao Zhou, Hongyi Wang, Huang Yu, Yifei Hu, Guang Li, Guang Chen, Hao Ye, Lijun Sun, Diange Yang
| Challenge: | Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning. |
| Approach: | AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks. |
| Outcome: | Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% . |
RBPtool: A Deep Language Model Framework for Multi-Resolution RBP-RNA Binding Prediction and RNA Molecule Design (2025.emnlp-main)
Copied to clipboard
Jiyue Jiang, Yitao Xu, Zikang Wang, Yihan Ye, Yanruisheng Shao, Yuheng Shan, Jiuming Wang, Xiaodan Fan, Jiao Yuan, Yu Li
| Challenge: | RNA-binding proteins play key roles in post-transcriptional gene regulation . existing methods focus on shallow sequence features or coarse structural representations . large language models allow for precise modeling and biologically informed de novo RNA design . |
| Approach: | They extend RPI15223 into a multi-resolution, structure-level RBP-RNA dataset and introduce RBPtool, a framework that fuses sequence and structural information. |
| Outcome: | The proposed framework achieves state-of-the-art performance on public benchmarks and the RPI15223 dataset while supporting fine-grained level predictions. |
AIPO: Adaptive Information Guided Token-Level Reinforcement Learning for Large Language Model Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing RLVR methods focus on all generated tokens rather than on which tokens contribute to reasoning. |
| Approach: | They propose to use a Random–Fourier approximation of the Hilbert–Schmidt Independence Criterion to focus updates on decisive tokens discovered on the fly to improve the efficiency of mutual-information estimation. |
| Outcome: | The proposed approach yields +20% accuracy over strong RLVR baselines while updating merely 10% of tokens, demonstrating superior efficiency and effectiveness. |
LLM Agents for Education: Advances and Applications (2025.findings-emnlp)
Copied to clipboard
Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jingheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S. Yu, Qingsong Wen
| Challenge: | Large Language Model (LLM) agents are transforming education by automating complex tasks and enhancing both teaching and learning processes. |
| Approach: | This survey analyzes recent advances in applying Large Language Model agents to educational settings . it highlights ethical issues, hallucination and overreliance, and integration with existing ecosystems . |
| Outcome: | The authors analyze the technologies enabling LLM agents and highlight key challenges in deploying them in educational settings. |
MADS: Multi-Agent Dialogue Simulation for Diverse Persuasion Data Generation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Recent studies show that LLM-based agents exhibit superior moral and emotional language performance compared to humans, raising expectations for their deployment in persuasive tasks. |
| Approach: | They propose a framework for generating persuasive multi-turn dialogues via agent self-play using user agents designed to simulate diverse persona-driven behaviors, a Dialog Agent executing task-oriented persuasion strategies and an Optimization Agent evaluating and refining dialogue outcomes. |
| Outcome: | The proposed framework significantly improved the persuasion capacity of small LLMs, increasing the organic traffic conversion rate by 22.4% (from 1.83% to 2.24%) . |
A Bounding Box is Worth One Token - Interleaving Layout and Text in a Large Language Model for Document Understanding (2025.findings-acl)
Copied to clipboard
Jinghui Lu, Haiyang Yu, Yanjie Wang, Yongjie Ye, Jingqun Tang, Ziwei Yang, Binghong Wu, Qi Liu, Hao Feng, Han Wang, Hao Liu, Can Huang
| Challenge: | Existing methods for integrating spatial layouts with text have limitations . existing methods produce overly long text sequences or lack autoregressive traits of LLMs . |
| Approach: | They introduce Interleaving Layout and Text in a Large Language Model (LayTextLLM) they use OCR-derived text and spatial layouts to integrate with LLMs for document understanding . |
| Outcome: | The proposed model shows an increase in performance in KIE and VQA tasks. |
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI (2024.findings-eacl)
Copied to clipboard
Jianguo Zhang, Kun Qian, Zhiwei Liu, Shelby Heinecke, Rui Meng, Ye Liu, Zhou Yu, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | DialogStudio is the largest and most diverse collection of dialogue datasets . existing datasets lack diversity and comprehensiveness, authors say . |
| Approach: | They introduce DialogStudio: the largest and most diverse collection of dialogue datasets . DialogStuio aggregates more than 80 diverse dialogue dataset . |
| Outcome: | a new dataset is created to improve the quality and diversity of dialogue datasets . DialogStudio is the largest and most diverse collection of dialogue data . |
ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)
Copied to clipboard
Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong
| Challenge: | Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs). |
| Approach: | They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models. |
| Outcome: | The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks. |
Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction (2025.acl-industry)
Copied to clipboard
Yinghui Li, Shang Qin, Jingheng Ye, Haojing Huang, Yangning Li, Shu-Yu Guo, Libo Qin, Xuming Hu, Wenhao Jiang, Hai-Tao Zheng, Philip S. Yu
| Challenge: | Recent studies have shown that Large Language Models’ performance as correctors on Chinese Grammatical Error Correction (CGEC) remains unsatisfactory due to the challenging nature of the task. |
| Approach: | They propose a training framework EXAM that uses LLMs as explainers to enhance CGEC small models and a novel evaluation method SEE that utilizes LLM as evaluators to bring more reasonable evaluations. |
| Outcome: | The proposed methods improve the performance of LLMs on Chinese Grammatical Error Correction (CGEC) task. |
CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation (2025.findings-acl)
Copied to clipboard
Guangya Yu, Yanhao Li, Zongying Jiang, Yuxiong Jin, Li Dai, Yupian Lin, Ruihui Hou, Weiyan Zhang, Yongqi Fan, Qi Ye, Jingping Liu, Tong Ruan
| Challenge: | Medical quality control indicators are essential to assess the qualifications of healthcare institutions for medical services. |
| Approach: | They propose a Chinese electronic medical records-based dataset for MQCIC and propose CF-IR method that disentangles clinical fact verification and inferential rule reasoning actions. |
| Outcome: | The proposed method outperforms Chain-of-Thought methods on 20 representative LLMs, covering general and medical models. |
SpecCache: Speculative KV Cache Reuse for Efficient RAG Serving (2026.acl-long)
Copied to clipboard
Zijian Wen, Tao Zhang, Shuangwu Chen, Shenghao Ye, Yu Guo, Qirui Chen, Jingxian Shuai, Yunpeng Hou, Huasen He, null Jianyang
| Challenge: | Retrieval-Augmented Generation (RAG) improves LLMs but faces high prefill latency during long contexts. |
| Approach: | They propose a method that uses deep-layer hidden-state norms to guide token selection . they propose to use deep-layered hidden-status norms as a proxy to guide the token selection. |
| Outcome: | The proposed SpecCache outperforms state-of-the-art (SOTA) benchmarks. |
SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)
Copied to clipboard
Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li
| Challenge: | Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy. |
| Approach: | They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration. |
| Outcome: | The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent . |
Dense Hierarchical Retrieval for Open-domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA) current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. |
| Approach: | They propose a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic specific to each passage. |
| Outcome: | The proposed framework significantly outperforms the original dense passage retriever and helps an end-to-end QA system outperfect the strong baselines on multiple open-domain QA benchmarks. |
DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population (2022.emnlp-demos)
Copied to clipboard
Ningyu Zhang, Xin Xu, Liankuan Tao, Haiyang Yu, Hongbin Ye, Shuofei Qiao, Xin Xie, Xiang Chen, Zhoubo Li, Lei Li
| Challenge: | Existing knowledge extraction tools are not complete due to emerging entities and relations in real-world applications. |
| Approach: | They propose an open-source knowledge extraction toolkit DeepKE that supports low-resource, document-level and multimodal scenarios in the knowledge base population. |
| Outcome: | The proposed toolkit supports low-resource, document-level and multimodal scenarios in the knowledge base population. |
Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey (2024.acl-long)
Copied to clipboard
Jiawei Li, Yizhe Yang, Yu Bai, Xiaofeng Zhou, Yinghao Li, Huashan Sun, Yuhang Liu, Xingpeng Si, Yuhao Ye, Yixiao Wu, 林一冠 林一冠, Bin Xu, Ren Bowen, Chong Feng, Yang Gao, Heyan Huang
| Challenge: | Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications. |
| Approach: | They propose a reliable strategy for domains to choose more robust LLMs for real-world applications. |
| Outcome: | The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications. |
Improving LLM Generations via Fine-Grained Self-Endorsement (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have demonstrated remarkable capabilities but can still fail frequently on knowledge-intensive tasks. |
| Approach: | They propose a self-endorsement framework that leverages fine-grained fact-level comparisons across multiple sampled responses. |
| Outcome: | The proposed framework can improve factuality of generations with simple prompts across scales of LLMs. |
Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models (2025.tacl-1)
Copied to clipboard
| Challenge: | a recent study revisits six core challenges that have influenced the evolution of Neural Machine Translation (NMT) domain mismatch, amount of parallel data, rare word prediction, translation of long sentences and sub-optimal beam search remain challenges in LLMs. |
| Approach: | They revisit core challenges that have acted as benchmarks for progress in NMT . they propose to revisit these challenges and offer insights into their relevance . |
| Outcome: | The proposed models significantly improve translation of sentences containing approximately 80 words, even translating documents up to 512 words. |
Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correction (2024.findings-acl)
Copied to clipboard
| Challenge: | Chinese Spelling Correction (CSC) lacks large-scale high-quality corpora due to labor-intensive labeling of spelling errors in real-life writing or typing scenarios. |
| Approach: | They propose to use OCR/ASR-based generation to refine Chinese Spelling Correction models on random replacement-based corpora and filter them based on prediction confidence. |
| Outcome: | The proposed model outperforms existing models on three widely-used benchmarks while significantly alleviating over-correction. |
FedMABench: Benchmarking Mobile GUI Agents on Decentralized Heterogeneous User Data (2025.emnlp-main)
Copied to clipboard
| Challenge: | Mobile GUI agents have attracted tremendous research participation recently. traditional approaches to mobile agent training rely on centralized data collection. |
| Approach: | They propose a benchmark for federated training and evaluation of mobile GUI agents . they find that federation algorithms consistently outperform local training . |
| Outcome: | The first benchmark for federated training and evaluation of mobile GUI agents is released . it features 6 datasets with 30+ subsets, 8 federation algorithms, 10+ base models, and over 800 apps across 5 categories . |
Attend, Memorize and Generate: Towards Faithful Table-to-Text Generation in Few Shots (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Few-shot table-to-text generation is a task of composing fluent and faithful sentences to convey table content using limited data. |
| Approach: | They propose a novel approach to generate faithful table-to-text sentences using limited data . they aim to exploit table structure and natural linguistic information to generate accurate sentences . |
| Outcome: | The proposed approach generates higher qualified sentences when compared with state-of-the-art models on humans, songs, and books. |
Don’t Get Lost in the Trees: Streamlining LLM Reasoning by Overcoming Tree Search Exploration Pitfalls (2025.acl-long)
Copied to clipboard
Ante Wang, Linfeng Song, Ye Tian, Dian Yu, Haitao Mi, Xiangyu Duan, Zhaopeng Tu, Jinsong Su, Dong Yu
| Challenge: | Recent advances in tree search algorithms guided by verifiers have significantly enhanced the reasoning capabilities of large language models (LLMs), but at the cost of increased computational resources. |
| Approach: | They propose an e ffici ent tree sear ch framework that is a plug-and-play system compatible with various tree search algorithms. |
| Outcome: | The proposed framework reduces computational costs and prioritizes resource allocation to harder tasks (Levels 3-4) over simpler ones (Level 1-2), addressing both over-exploration in basic problems and under-exploation in complex cases. |
Beyond Self-Report: Bridging the Intention-Behavior Gap in Critical Thinking Assessment via Interpretable Multi-Agent System (2026.acl-long)
Copied to clipboard
Zekun Li, Jifan Yu, Haoxuan Li, Ye He, Daniel Zhang-Li, Shangqing Tu, Joy Jia Yin Lim, Yikun Jiang, Jiaxin Yuan, Yu Zhang
| Challenge: | Accurate assessment of critical thinking is limited by the Intention Behavior Gap in psychology . evaluators that measure self-reported competence are limited by multiagent architectures . |
| Approach: | They propose a framework that operationalizes cognitive assessment into an interpretable multi-agent workflow with Assessment Chain-of-Thought. |
| Outcome: | The proposed framework aligns better with human expert ratings than gold-standard inventories on large-scale simulations and human participants. |
SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)
Copied to clipboard
Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang
| Challenge: | Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases. |
| Approach: | They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs. |
| Outcome: | The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs. |
DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenes (2025.findings-emnlp)
Copied to clipboard
Zhaowei Wang, Hongming Zhang, Tianqing Fang, Ye Tian, Yue Yang, Kaixin Ma, Xiaoman Pan, Yangqiu Song, Dong Yu
| Challenge: | Large Vision-Language Models (LVLMs) have achieved significant progress in tasks like visual question answering and document understanding. |
| Approach: | They introduce DivScene, a large-scale dataset with 4,614 houses across 81 scene types and 5,707 kinds of target objects. |
| Outcome: | The proposed dataset provides a much greater diversity of target objects and scene types than existing datasets, enabling a comprehensive task evaluation. |
PURE: Aligning LLM via Pluggable Query Reformulation for Enhanced Helpfulness (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) depend on vast amounts of text data sourced from the Internet for their training. |
| Approach: | They propose a new alignment paradigm that reformulates risky queries into highly relevant yet harmless ones before feeding them into LLMs. |
| Outcome: | The proposed approach eliminates the high costs of training base LLMs and achieves a promising balance of harmlessness and helpfulness. |
FastSeq: Make Sequence Generation Faster (2021.acl-demo)
Copied to clipboard
Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang
| Challenge: | Transformer-based models have made tremendous impact in natural language generation, but inference speed is still a bottleneck due to large model size and intensive computing involved in auto-regressive decoding process. |
| Approach: | They propose an attention cache optimization, an efficient algorithm for detecting repeated n-grams, and an asynchronous generation pipeline with parallel I/O to accelerate sequence generation without loss of accuracy. |
| Outcome: | The proposed framework can accelerate the sequence generation by 4x to 9x with a simple one-line code change for a set of widely used and diverse models. |
Transforming Visual Scene Graphs to Image Captions (2023.acl-long)
Copied to clipboard
Xu Yang, Jiawei Peng, Zihua Wang, Haiyang Xu, Qinghao Ye, Chenliang Li, Songfang Huang, Fei Huang, Zhangzikang Li, Yu Zhang
| Challenge: | Existing approaches to generate captions using image captioning are based on multi-head attention (MHA) |
| Approach: | They propose to transform scene graphs into more descriptive captions by using multi-head attention to build a Graph Neural Network (GNN) . they construct a Mixture-of-Expert (MOE)-based decoder where each expert is built on MHA for discriminating the graph embeddings to generate different kinds of words. |
| Outcome: | The proposed framework can generate captions from multiple visual features and objects . it is based on a mixture-of-expert (MOE)-based decoder based upon MHA . |
CORI: CJKV Benchmark with Romanization Integration - a Step towards Cross-lingual Transfer beyond Textual Scripts (2024.lrec-main)
Copied to clipboard
| Challenge: | Naively assuming English as a source language may hinder cross-lingual transfer . despite recent advances in cross-linguistic research, most studies have restricted themselves to two major assumptions . |
| Approach: | They propose to integrate Romanized transcription beyond textual scripts to capture contact between these languages . they propose to use a benchmark dataset to further encourage in-depth studies of language contact . |
| Outcome: | The proposed method allows for enhanced cross-lingual representations and effective zero-shot cross-linguistic transfer. |
Rethinking Table Pruning in TableQA: From Sequential Revisions to Gold Trajectory-Supervised Parallel Search (2026.acl-long)
Copied to clipboard
Yu Guo, Shenghao Ye, Shuangwu Chen, Zijian Wen, Tao Zhang, Bai Qirui, Dong Jin, Yunpeng Hou, Huasen He, null Jianyang, Xiaobin Tan
| Challenge: | Existing pruning methods rely on sequential revisions and unreliable critique signals . Existing methods fail to detect the loss of answer-critical data . |
| Approach: | They propose a table pruning framework which transforms table pruning to gold trajectory-supervised parallel search. |
| Outcome: | The proposed framework outperforms the strongest baseline pruning framework by 3.2% on various tabular reasoning tasks. |
Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open-domain question answering (QA) models employ a retriever-reader pipeline . however, state-of-the-art readers fail to capture complex relationships between entities . |
| Approach: | They propose a knowledge graph enhanced passage reader that captures entities in questions and retrieved passages. |
| Outcome: | The proposed knowledge graph enhanced passage reader improves on open-domain QA benchmarks by up to 2.2 exact match scores. |
SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus on single-step reasoning, ignoring logical dependencies between steps. |
| Approach: | They propose a method that maximizes a structure-based return to facilitate structured reasoning and explanation. |
| Outcome: | The proposed method outperforms state-of-the-art methods on EntailmentBank and STREET benchmarks. |
UniRAG: Unified Query Understanding Method for Retrieval Augmented Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing query augmentation methods face knowledge update lag and hallucinations in large language models (LLMs) Existing methods face two key challenges: (1) separation of query augmented and encoding tasks, which hinders information sharing and introduces cumulative errors; (2) difficulty of selecting optimal augmentation strategy for different scenarios. |
| Approach: | They propose a unified framework for query understanding in RAG that integrates internal and external knowledge to enhance query augmentation and encoding tasks. |
| Outcome: | The proposed framework outperforms traditional query augmentation methods in five knowledge-intensive benchmark tasks in both closed and open domain question answering. |
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios. |
| Approach: | They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset. |
| Outcome: | The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation. |
A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions (2026.acl-long)
Copied to clipboard
Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu, Yatao Bian, Ming Zhang, Wei Ye, Tieke He, Jie Yang, Guanjie Zheng, Zhonghai Wu, Bo Zhang, Lei Bai, Xiao Luo
| Challenge: | Existing research on reinforcement learning for LLMs under data scarcity has not been unified. |
| Approach: | They propose a top-up hierarchical framework built around three complementary perspectives: data-centric, training-centric and framework-centric. |
| Outcome: | The proposed framework provides a clear conceptual foundation for understanding the design space of data-efficient RL for large language models and to guide researchers working in this emerging area. |
HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for summarizing semantic graph structure from raw text are cumbersome and inefficient for long-text documents. |
| Approach: | They propose a Transformer-based pre-trained model with multi-granularity sparse attentions for long-text extractive summarization. |
| Outcome: | The proposed model performs state-of-the-art on single- and multi-document summarization tasks while using less memory and fewer parameters. |
Joint Optimization of Training Data and Policy in RLHF (2026.findings-acl)
Copied to clipboard
Zhuohao Yu, Jiali Zeng, Weizheng Gu, Mengyuan Sun, Yidong Wang, Fandong Meng, Jie Zhou, Shikun Zhang, Wei Ye
| Challenge: | JODP optimizes policies on fixed training inputs, limiting the diversity of learning signals. |
| Approach: | They propose a framework where policy generates improved variants of training problems to enhance its own learning. |
| Outcome: | The proposed framework improves on safety alignment tasks by allowing 4B models to reach 8B model performance with less than 1% additional computational overhead. |
Generating Data for Symbolic Language with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) bring performance and complexity, but they incur a large computational cost in practice. |
| Approach: | They propose a task-based model which uses large language models to generate symbolic language data by an informative prompt and agreement-based verifier. |
| Outcome: | The proposed model can generate symbolic language data with a few human demonstrations and saves a considerable amount of inference effort. |
Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for Ensembling (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in a wide range of tasks and contexts. |
| Approach: | They propose to use a token-level ensembling method to exploit the probability information at each generation step and to avoid early incorrect tokens. |
| Outcome: | The proposed method breaks the existing community performance ceiling and improves on several benchmarks. |
Position: LLMs Can be Good Tutors in English Education (2025.emnlp-main)
Copied to clipboard
Jingheng Ye, Shen Wang, Deqing Zou, Yibo Yan, Kun Wang, Hai-Tao Zheng, Ruitong Liu, Zenglin Xu, Irwin King, Philip S. Yu, Qingsong Wen
| Challenge: | Recent efforts to integrate large language models into English education lack adaptability to language learning. |
| Approach: | They argue that large language models can be effective tutors in English education . they encourage interdisciplinary research to explore these roles, fostering innovation and risks . |
| Outcome: | The proposed models can play three critical roles: 1) as data enhancers, 2) as task predictors, 3) as agents, enabling personalized and inclusive education. |
An Inversion Attack Against Obfuscated Embedding Matrix in Language Model Inference (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored transforming user inputs to obfuscated embedded vectors, so that the data will not be eavesdropped by service providers. |
| Approach: | They propose to transform user inputs to obfuscated embedded vectors so that the data will not be eavesdropped by service providers. |
| Outcome: | The proposed inversion attack can recover user input 100% from the obfuscated vectors without a solid and deliberate security design and analysis . |
Takin-VC: Expressive Zero-Shot Voice Conversion via Adaptive Hybrid Content Encoding and Enhanced Timbre Modeling (2025.acl-long)
Copied to clipboard
Yang Yuguang, Yu Pan, Jixun Yao, Xiang Zhang, Jianhao Ye, Hongbin Zhou, Lei Xie, Lei Ma, Jianjun Zhao
| Challenge: | Expressive zero-shot voice conversion (VC) aims to modify source timbre to match unseen speaker . existing zero- shot VC systems struggle to reproduce paralinguistic information in highly expressive speech . |
| Approach: | They propose a framework for expressive zero-shot voice conversion that uses hybrid content encoding and memory-augmented context-aware timbre modeling. |
| Outcome: | The proposed framework surpasses state-of-the-art VC systems in speech naturalness, speaker similarity, and speaker similarness. |
From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research on PTQ spans three primary directions. |
| Approach: | They conduct a systematic analysis of post-training quantization failures using PTQ . they show that targeted repair can mitigate Signal Degradation but remains ineffective for Computation Collapse . |
| Outcome: | The proposed method mitigates Signal Degradation but remains ineffective for Computation Collapse. |
Rule By Example: Harnessing Logical Rules for Explainable Hate Speech Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to content moderation are based on rule-based heuristics, but they lack the flexibility and robustness needed to moderate harmful content. |
| Approach: | They propose a novel contrastive learning approach for learning from logical rules for content moderation using only a few data examples. |
| Outcome: | The proposed approach outperforms state-of-the-art deep learning classifiers while providing more explainable predictions. |
Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing scaling laws focus on general performance, overlooking crucial fine-grained factors and how quantization differentially impacts diverse knowledge capabilities. |
| Approach: | They propose a framework that unifies model size, bit-width, and fine-grained factors into memorization, application, and reasoning. |
| Outcome: | The proposed framework shows strong fit and cross-architecture consistency on 293 different PTQ configurations. |
POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversion (2025.emnlp-main)
Copied to clipboard
Yuan Liu, Zhongyin Zhao, Le Tian, Haicheng Wang, Xubing Ye, Yangxiu You, Zilin Yu, Chuhan Wu, Zhou Xiao, Yang Yu, Jie Zhou
| Challenge: | Existing approaches to training document conversion models with manual annotation are costly and time-consuming, and training student models by distilling outputs from teacher models can significantly limit their performance in real-world applications. |
| Approach: | They propose a fully automated framework for constructing high-quality document extraction datasets and models capable of handling diverse document formats and layouts. |
| Outcome: | The proposed model outperforms existing models and improves on annotated documents. |
Towards Mitigating Modality Bias in Vision-Language Models for Temporal Action Localization (2026.acl-long)
Copied to clipboard
| Challenge: | Existing vision-language models overemphasize linguistic priors, leading to modality bias. |
| Approach: | They propose a vision-language aggregation framework that mitigates modality bias in TAL by preserving vision as the dominant signal while adaptively exploiting language only when beneficial. |
| Outcome: | Experiments on THUMOS14 show that the proposed model outperforms state-of-the-art models by up to 3.2% mAP. |
Behavioral Consistency Validation for LLM Agents: An Analysis of Trading-Style Switching through Stock-Market Simulation (2026.findings-acl)
Copied to clipboard
Zeping Li, Guancheng Wan, Keyang Chen, Yu Chen, Yiwen Zhao, Philip Torr, Guangnan Ye, Zhenfei Yin, Hongfeng Chai
| Challenge: | Recent studies have applied Large Language Models (LLMs) as agents in financial stock market simulations to test if micro-level behaviors aggregate into macro-level phenomena. |
| Approach: | They propose four alignment metrics and use Mann–Whitney U tests to compare agents’ style-switching behavior with financial theory. |
| Outcome: | The proposed model is only partially consistent with financial theory. |
Are LLMs Rational Investors? A Study on the Financial Bias in LLMs (2025.findings-acl)
Copied to clipboard
Yuhang Zhou, Yuchen Ni, Zhiheng Xi, Zhangyue Yin, Yu He, Gan Yunhui, Xiang Liu, Zhang Jian, Sen Liu, Xipeng Qiu, Yixin Cao, Guangnan Ye, Hongfeng Chai
| Challenge: | Existing studies on biases within specific domains, such as finance, remain limited. |
| Approach: | They propose a framework to detect, detect, analyze and mitigate financial biases in large language models. |
| Outcome: | The proposed framework reduces bias by 68% for the most biased model, according to key metrics. |
R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQL (2024.findings-emnlp)
Copied to clipboard
Yuhang Zhou, Yu He, Siyu Tian, Yuchen Ni, Zhangyue Yin, Xiang Liu, Chuanjun Ji, Sen Liu, Xipeng Qiu, Guangnan Ye, Hongfeng Chai
| Challenge: | Adapting existing approaches for converting natural language to SQL encounters hurdles due to distinct nature of GQL compared to SQL. |
| Approach: | They propose a method that integrates both small and large Foundation Models for ranking, rewriting, and refining tasks. |
| Outcome: | The proposed approach integrates both small and large Foundation Models for ranking, rewriting, and refining tasks while capitalizing on the superior generalization and query generation prowess of larger models for the final transformation of natural language queries into GQL formats. |
OpenICL: An Open-Source Framework for In-context Learning (2023.acl-demo)
Copied to clipboard
| Challenge: | In-context Learning (ICL) is a new paradigm for large language model evaluation. |
| Approach: | They propose an open-source toolkit for ICL and LLM evaluation. |
| Outcome: | The proposed framework is highly flexible and flexible and can be easily combined with other tools to suit users' needs. |
KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models (2024.acl-long)
Copied to clipboard
Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Wei Ye, Jindong Wang, Xing Xie, Yue Zhang, Shikun Zhang
| Challenge: | Existing methods to detect contaminated texts focus on quantifying contamination status instead of accurately gauging model performance. |
| Approach: | They propose a Knowledge-grounded Interactive Evaluation framework which incorporates an LLM-powered “interactor” role for the first time to accomplish a dynamic contamination-resilient evaluation. |
| Outcome: | The proposed framework is based on a question in a standard LLM benchmark and can be used to evaluate models in real-world conversations. |
FreeEval: A Modular Framework for Trustworthy and Efficient Evaluation of Large Language Models (2024.emnlp-demo)
Copied to clipboard
Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Zhengran Zeng, Wei Ye, Jindong Wang, Yue Zhang, Shikun Zhang
| Challenge: | Large language models (LLMs) have revolutionized natural language processing with impressive performance across various tasks. |
| Approach: | They propose a framework for automated evaluations of large language models . they open-source their code at https://github.com/WisdomShell/FreeEval . |
| Outcome: | The framework is open-source and can be used to develop and validate new evaluation methods. |
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)
Copied to clipboard
Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains. |
| Approach: | They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries. |
| Outcome: | The proposed system outperforms baselines in the open domain task-solving benchmark. |
CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is popular in reasoning tasks, but its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored. |
| Approach: | They propose a Coarse-to-Fine Chain-of-Thought approach that breaks down NLU tasks into multiple reasoning steps where LLMs can learn to acquire essential concepts. |
| Outcome: | The proposed approach is effective in assisting the LLMs adapt to multi-grained NLU tasks under zero-shot and few-shot multi-domain settings. |
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on dataset-generation-based zero-shot learning has shown promising results by training a task-specific model with a dataset synthesized from large pre-trained language models (PLMs). |
| Approach: | They propose a progressive zero-shot dataset generation framework which leverages feedback from the task-specific model to guide the generation of new training data via in-context examples. |
| Outcome: | The proposed framework achieves on-par or superior performance with only 1% synthetic dataset size, when compared to baseline methods without in-context feedback. |
Energy-based Unknown Intent Detection with Data Manipulation (2021.findings-acl)
Copied to clipboard
| Challenge: | Unknown intent detection aims to identify the out-of-distribution (OOD) utterance whose intent has never appeared in the training set. |
| Approach: | They propose a framework to generate high-quality OOD utterances with importance weighTs (GOT) their framework is fine-tuned to detect out-of-distribution utterrances . |
| Outcome: | The proposed framework can achieve state-of-the-art results on two benchmark datasets. |
Now You Hear Me: Audio Narrative Attacks Against Large Audio–Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing jailbreaks against large audio-language models fall into two categories . early work converted text-based prompts into synthetic speech, while subsequent work introduced minor acoustic variations such as accent shifts, phonetic spellings, or stress patterns. |
| Approach: | They propose a text-to-audio jailbreak that embeds disallowed directives within a narrative-style audio stream. |
| Outcome: | The proposed attack exploits structural and acoustic properties of a text-to-audio model . it achieves 98.26% success rate, significantly exceeding baselines for text-based models . |
When TableQA Meets Noise: A Dual Denoising Framework for Complex Questions and Large-scale Tables (2026.acl-long)
Copied to clipboard
Shenghao Ye, Yu Guo, Dong Jin, Yuxiang Wang, Yikai Shen, Yunpeng Hou, Shuangwu Chen, null Jianyang, Xiaofeng Jiang
| Challenge: | Extensive research shows that noisy data significantly degrades the performance of table reasoning in real-world applications. |
| Approach: | They propose a dual denoising framework for complex questions and large-scale tables that uses Tree-guided table pruning to remove irrelevant data step by step. |
| Outcome: | The proposed framework achieves outstanding performance on TableQA tasks with complex questions and large-scale tables. |