Papers by Shuo Zhang
MERaLiON-AudioLLM: Advancing Speech and Language Understanding for Singapore (2025.acl-demo)
Copied to clipboard
Yingxu He, Zhuohan Liu, Geyu Lin, Shuo Sun, Bin Wang, Wenyu Zhang, Xunlong Zou, Nancy F. Chen, AiTi Aw
| Challenge: | MERaLiON-AudioLLM is the first general-purpose audio-based large language model for multitask learning. |
| Approach: | They introduce MERaLiON-AudioLLM, a general-purpose audio-based large language model for multitask learning with a focus on Singlish understanding. |
| Outcome: | The proposed model exhibits strong generalization across a diverse set of tasks . it is a leading solution for region-specific AI applications. |
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)
Copied to clipboard
Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy. |
| Approach: | They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses . |
| Outcome: | The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples. |
OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory (2026.acl-long)
Copied to clipboard
| Challenge: | Existing LLMs are limited by text-context budgets, resulting in token-expensive storage of raw trajectories . Optical Context Retrieval Memory (OCR-Memory) renders historical tra-jectorios into images annotated with unique visual identifiers. |
| Approach: | They propose a framework that leverages the visual modality as a high-density representation of agent experience. |
| Outcome: | Optical Context Retrieval Memory (OCRM) renders historical trajectories into images annotated with unique visual identifiers. |
MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators (2022.acl-long)
Copied to clipboard
| Challenge: | Prompting has been shown to be a promising approach for applying pre-trained language models to perform downstream tasks. |
| Approach: | They propose a method that divides the translation process into three stages using pre-trained language models. |
| Outcome: | The proposed method significantly improves translation performance of pre-trained language models on three translation tasks. |
Generate-then-Ground in Retrieval-Augmented Generation for Multi-hop Question Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to solve multi-hop question are constrained by the retriever and the noise in the retrieved documents. |
| Approach: | They propose a framework that integrates parametric knowledge of large language models with external documents to solve a multi-hop question. |
| Outcome: | The proposed framework is based on the parametric knowledge of LLMs and external documents to solve a multi-hop question. |
LiveCANNBench: Benchmark SWE AI Coding for Ascend CANN (2026.findings-acl)
Copied to clipboard
Sijie Wang, Kai Zhao, Wee Peng Tay, Shuo Zhang, Chengwen Liu, Quanjiang Guo, Ren Junhao, Xin Li, Heng Lian, Jingdi Lei, Rui She, Huacan Wang, Ronghao Chen
| Challenge: | Recent advances in agents have enabled multi-file, multi-language, and dependency-aware AI coding. |
| Approach: | They propose an SWE-level benchmark for AI coding in the Huawei Ascend CANN software stack. |
| Outcome: | The proposed benchmark is constructed from real-world CANN repositories and consists of over 400 task instances spanning multiple file, multi-language, and execution-aware coding challenges. |
Uncertainty-Aware Cross-Lingual Transfer with Pseudo Partial Labels (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to train pre-trained language models for zero-shot cross-lingual tasks are noisy and lack confidence. |
| Approach: | They propose an uncertainty-aware cross-lingual transfer framework with pseudo-partial-label to maximize the utilization of unlabeled data by reducing noise. |
| Outcome: | The proposed framework outperforms baselines on named entity recognition and natural language inference tasks on 40 languages. |
TART: Improved Few-shot Text Classification Using Task-Adaptive Reference Transformation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fewshot text classification depend on inter-class variance . Existing approaches suffer from MLADA, which performs poorly on tasks with high inter- class variance whereas it fails to distinguish samples from tasks with low inter-group variance. |
| Approach: | They propose a task-adaptive reference transformation network to transform class prototypes to per-class fixed reference points in task-adapted metric spaces. |
| Outcome: | The proposed method surpasses state-of-the-art methods in 1-shot and 5-shot classifications on the 20 Newsgroups dataset. |
AudioBench: A Universal Benchmark for Audio Large Language Models (2025.naacl-long)
Copied to clipboard
Bin Wang, Xunlong Zou, Geyu Lin, Shuo Sun, Zhuohan Liu, Wenyu Zhang, Zhengyuan Liu, AiTi Aw, Nancy F. Chen
| Challenge: | Existing evaluation regimes for audio large language models do not cover the breadth of their possible use cases. |
| Approach: | They propose to use AudioBench to evaluate audio large language models . they found that no single model excels consistently across all tasks . |
| Outcome: | The proposed evaluation targets speech understanding, audio scene understanding, and voice understanding (paralinguistic) . no single model excels consistently across all tasks, the paper found . |
Enhancing Tabular Reasoning with Pattern Exploiting Training (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing methods based on pre-trained language models have shown superior performance over tabular tasks despite showing inherent problems such as not using the right evidence and inconsistent predictions across inputs. |
| Approach: | They utilize Pattern-Exploiting Training (PET) on pre-trained language models to strengthen tabular reasoning models’ pre-existing knowledge and reasoning abilities. |
| Outcome: | The proposed model exhibits superior understanding of knowledge facts and tabular reasoning compared to baseline models. |
Realistic Data Augmentation Framework for Enhancing Tabular Reasoning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to constructing training data for Natural Language Inference (NLI) tasks are expensive and time consuming. |
| Approach: | They propose a semi-automated framework for data augmentation for tabular inference . framework generates hypothesis templates transferable to similar tables . authors say framework could generate human-like tabular examples . |
| Outcome: | The proposed framework generates human-like tabular inference examples . it is based on human-written constraints and premise paraphrasing . |
Enhancing RAG Efficiency with Adaptive Context Compression (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods apply fixed compression rates, over-compressing simple queries or under-compressed complex ones. |
| Approach: | a new framework uses a hierarchical compressor and a context selector to optimize inference efficiency . a framework that dynamically adjusts compression rates based on input complexity optimizes inference without loss of accuracy. |
| Outcome: | Adaptive Context Compression for RAG outperforms fixed-rate methods on Wikipedia and five QA datasets . |
Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that tabular reasoning models use spurious correlations and focus on false evidence or ignore it altogether. |
| Approach: | They propose a task where models need to extract evidence and then inference labels . they crowdsource evidence row labels and develop unsupervised evidence extraction strategies . |
| Outcome: | The proposed approach outperforms baseline models on the inference task using only the automatically extracted evidence as the premise. |
LLM×MapReduce-V3: Enabling Interactive In-Depth Survey Generation through a MCP-Driven Hierarchically Modular Agent System (2025.emnlp-demos)
Copied to clipboard
Yu Chao, Siyu Lin, Xiaorong Wang, Zhu Zhang, Zihan Zhou, Haoyu Wang, Shuo Wang, Jie Zhou, Zhiyuan Liu, Maosong Sun
| Challenge: | Generating high-quality long-form survey articles poses significant challenges to AI Agent systems. |
| Approach: | They propose a hierarchically modular agent system for long-form survey generation . they use atomic models to implement skeleton initialization, digest construction, and skelet refinement . human evaluations demonstrate system surpasses representative baselines . |
| Outcome: | The proposed system surpasses representative baselines in both content depth and length, highlighting the strength of MCP-based modular planning. |
Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT - A Text-to-SQL Parsing Comparison (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a number of open-source large language models claim to be performing better than commercial ones . however, these models fall short of the performance achieved by closed-source models like GPT-3.5 . |
| Approach: | They evaluate six popular large language models against each other to evaluate their performance . authors say open-source models are not as effective as those built by commercial models . |
| Outcome: | a new set of models claim to match or surpass the language understanding abilities of commercial models . the results show that the models performed far below the performance of closed-source models compared to open-source ones . |
Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Existing research addresses ambiguous visual questions by rephrasing questions, but it fails to address the inherently interactive nature of user interactions with visual language models (VLMs). Existing studies focus on re-phrase questions, and lack of a benchmark to assess VLMs’ capacity for resolving ambiguities through interaction. |
| Approach: | They propose a visual question answering task that provides a natural language answer to a question based on a given image and an automated pipeline to generate ambiguity-clarification question pairs. |
| Outcome: | The proposed benchmark targets three common categories of ambiguity in visual question answering (VQA) context and encompasses various VQA scenarios. |
DAEA: Enhancing Entity Alignment in Real-World Knowledge Graphs Through Multi-Source Domain Adaptation (2025.coling-main)
Copied to clipboard
| Challenge: | Entity Alignment (EA) is a critical task in Knowledge Graph (KG) integration. |
| Approach: | They propose a novel approach that leverages the data characteristics of synthetic benchmarks to improve performance in real-world datasets. |
| Outcome: | The proposed approach outperforms state-of-the-art models on real-world datasets and achieves a 29.94% improvement in Hits@1 on DOREMUS and 5.64% improvement on AGROLD. |
QiMeng-PRepair: Precise Code Repair via Edit-Aware Reward Optimization (2026.acl-long)
Copied to clipboard
Changxin Ke, Rui Zhang, Jiaming Guo, Yuanbo Wen, Li Ding, Shuo Wang, Xuyuan Zhu, Xiong Peng, Di Huang, Zidong Du, Xing Hu, Qi Guo, Yunji Chen
| Challenge: | Existing approaches to program repair are based on correctness alone. |
| Approach: | They propose a framework that mitigates over-editing and improves repair accuracy by generating buggy programs and re-edits. |
| Outcome: | The proposed framework improves repair precision by 31.4% under fix1@1, a metric that considers repair correctness and extent, and significantly increases decoding throughput when combined with speculative editing. |
TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance (2021.acl-long)
Copied to clipboard
Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, Tat-Seng Chua
| Challenge: | Existing QA systems focus on unstructured text, structured knowledge base, or semi-structured tables. |
| Approach: | They propose a large-scale question answering model based on financial reports . numerical reasoning is usually required to infer the answer . |
| Outcome: | The proposed model achieves 58.0% inF1, an 11.1% increase over the baseline model, but still lags behind the best human model. |
Generative Bridging Network for Neural Sequence Prediction (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to improve the likelihood of sequence prediction models are based on MLE and teacher forcing. |
| Approach: | They propose a Generative Bridging Network (GBN) that extends the point-wise ground truth to a bridge distribution conditioned on it and optimizes their KL-divergence. |
| Outcome: | The proposed bridge module can improve on two recognized sequence prediction tasks and minimize learning burden. |
From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations (2025.emnlp-main)
Copied to clipboard
Benlu Wang, Iris Xia, Yifan Zhang, Junda Wang, null Feiyun Ouyang, Shuo Han, Arman Cohan, Hong Yu, Zonghai Yao
| Challenge: | Existing benchmarks assess only the final answer with a wide numerical tolerance, overlooking systematic reasoning failures and potentially causing serious clinical misjudgments. |
| Approach: | They propose a new step-by-step evaluation pipeline that assesses formula selection, entity extraction, and arithmetic computation. |
| Outcome: | The proposed method improves the accuracy of large language models on medical benchmarks from 16.35% to 53.19%. |
LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models have improved their capacity to handle long text inputs, but current models still exhibit unsatisfactory performance in long-form generation. |
| Approach: | They propose a method to enhance long-form text generation through step-level supervision by leveraging Monte Carlo Tree Search to collect stepwise preference pairs and employ a global memory pool to maintain factual accuracy. |
| Outcome: | The proposed method improves performance on long-form generation benchmarks while maintaining lossless performance on several general benchmarks. |
Probabilistic Aggregation and Targeted Embedding Optimization for Collective Moral Reasoning in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have impressive moral reasoning abilities, yet they often diverge when confronted with complex, multi-factor moral dilemmas. |
| Approach: | They propose a framework that synthesizes multiple LLMs’ moral judgments into a collectively formulated moral judgment, realigning models that deviate significantly from this consensus. |
| Outcome: | The proposed framework synthesizes multiple LLMs’ moral judgments into a collectively formulated moral judgment, realigning models that deviate significantly from this consensus. |
SPARK: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for training large language models waste computation budget on trivial steps while failing to guarantee sample quality. |
| Approach: | They propose a framework that selectively branches at critical decision states for resource-efficient exploration. |
| Outcome: | The proposed framework activates adaptive branching exploration at critical decision states to probe promising trajectories, thereby achieving precise resource allocation that prioritizes sampling quality over blind coverage. |
RADE: Reference-Assisted Dialogue Evaluation for Open-Domain Dialogue (2023.acl-long)
Copied to clipboard
| Challenge: | Evaluating open-domain dialogue systems is challenging because of the one-to-many problem. |
| Approach: | They propose a reference-based dialogue evaluation approach that leverages the pre-created utterance as reference other than the gold response to relieve the one-to-many problem. |
| Outcome: | The proposed method outperforms state-of-the-art evaluation methods on three datasets and two existing benchmarks. |
PACE: Prefix-Protected and Difficulty-Aware Compression for Efficient Reasoning (2026.findings-acl)
Copied to clipboard
Ruixiang Feng, Yuntao Wen, Silin Zhou, Ke Shi, Yifan Wang, Ran Le, Zhenwei An, Zongchao Chen, Chen Yang, Guangyue Peng, Yiming Jia, Dongsheng Wang, Tao Zhang, Lisi Chen, Yang Song, Shen Gao, Shuo Shang
| Challenge: | Existing LRMs often suffer from "overthinking" and excessively long reasoning traces . a dual-level framework for length compression of LRM is proposed . |
| Approach: | They propose a framework for prefix-protected and difficulty-aware compression under hierarchical supervision. |
| Outcome: | The proposed framework reduces token usage while improving accuracy on math benchmarks. |
AfriCLIRMatrix: Enabling Cross-Lingual Information Retrieval for African Languages (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for cross-lingual information retrieval are limited in many languages, especially those spoken in Africa. |
| Approach: | They propose to build a test collection for cross-lingual information retrieval in 15 diverse African languages. |
| Outcome: | AfriCLIRMatrix contains 6 million queries in English and 23 million relevance judgments automatically mined from Wikipedia inter-language links, covering many more African languages than any existing information retrieval test collection. |
Plum: Prompt Learning using Metaheuristics (2024.findings-acl)
Copied to clipboard
Rui Pan, Shuo Xing, Shizhe Diao, Wenhe Sun, Xiang Liu, KaShun Shum, Jipeng Zhang, Renjie Pi, Tong Zhang
| Challenge: | Recent advances in prompt learning have led to a need for general prompt optimization methods. |
| Approach: | They propose a branch of discrete non-convex optimization methods with over 100 options as a promising approach to prompt learning. |
| Outcome: | The proposed methods can be used to discover more human-understandable prompts that were previously unknown in reasoning and image generation tasks. |
Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric (2025.acl-long)
Copied to clipboard
Yuming Yang, Yang Nan, Junjie Ye, Shihan Dou, Xiao Wang, Shuo Li, Huijie Lv, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing studies have explored various diversity-aware data selection methods to construct high-quality datasets and enhance model performance. |
| Approach: | They propose to use data diversity to measure instruction tuning of large language models. |
| Outcome: | The proposed diversity metric outperforms existing methods on simulated and real-world data and shows that it captures diversity variations and achieves a 0.97 correlation with instruction tuning. |
Spec-o3: A Tool-Augmented Vision-Language Agent for Rare Celestial Object Candidate Vetting via Automated Spectral Inspection (2026.acl-long)
Copied to clipboard
| Challenge: | Spec-o3 is a tool-augmented vision-language agent that performs astronomer-aligned spectral inspection. |
| Approach: | They propose a tool-augmented vision-language agent that performs astronomer-aligned spectral inspection via interleaved multimodal chain-of-thought reasoning. |
| Outcome: | Spec-o3 outperforms traditional visual inspection methods on rare-object inspection tasks. |
Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation (2026.findings-acl)
Copied to clipboard
Rui Qi, Fengran Mo, Yufeng Chen, Xue Zhang, Shuo Wang, Hongliang Li, Xu Jinan, Meng Jiang, Jian-Yun Nie, Kaiyu Huang
| Challenge: | Existing approaches to multilingual retrieval-augmented generation (MRAG) use a single-turn retrieval and subsequent optimization to acquire and integrate beneficial external knowledge from multilingual collections. |
| Approach: | They propose a multilingual search-augmented reinforcement learning framework that integrates a language-coupled Group Relative Policy Optimization into the policy and reward models. |
| Outcome: | The proposed framework achieves competitive performance and is appropriate for various practical scenarios such as constrained training data and retrieval over collections encompassing a large number of languages. |
AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage (2026.acl-long)
Copied to clipboard
Xuanle Zhao, Zilin Sang, Yuxuan Li, Qi Shi, Weilun Zhao, Shuo Wang, Duzhen Zhang, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Efficient reproduction of research papers requires deep domain expertise. |
| Approach: | They propose a framework that systematically mines implicit knowledge from the cited literature to reproduce experimental code in a complete, end-to-end manner. |
| Outcome: | The proposed framework surpasses baselines across all metrics and reproduces experimental code in a complete, end-to-end manner. |
Multimodal Pragmatic Jailbreak on Text-to-image Models (2025.acl-long)
Copied to clipboard
Tong Liu, Zhixin Lai, Jiawen Wang, Gengyuan Zhang, Shuo Chen, Philip Torr, Vera Demberg, Volker Tresp, Jindong Gu
| Challenge: | Existing jailbreaks for diffusion-based text-to-image models generate unsafe content . experimental results show that all tested models suffer from unsafe generation . |
| Approach: | They propose a jailbreak that triggers diffusion-based text-to-image models to generate the image with visual text, resulting in unsafe content. |
| Outcome: | The proposed model generates image with visual text, but the model is unsafe under such jailbreak. |
MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria (2025.naacl-long)
Copied to clipboard
Wentao Ge, Shunian Chen, Hardy Chen, Nuo Chen, Junying Chen, Zhihong Chen, Wenya Xie, Shuo Yan, ChenghaoZhu ChenghaoZhu, Ziyue Lin, Dingjie Song, Xidong Wang, Anningzhe Gao, Zhang Zhiyi, Jianquan Li, Xiang Wan, Benyou Wang
| Challenge: | Existing evaluation methodologies for multimodal large language models are limited in evaluating objective queries without considering real-world user experiences. |
| Approach: | They propose to evaluate multimodal large language models with per-sample criteria using potent MLLM as the judge. |
| Outcome: | The proposed evaluation paradigm shows that it can be used to evaluate multimodal large language models with per-sample criteria. |
Retrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purification (2026.acl-long)
Copied to clipboard
| Challenge: | Recent audio-visual question answering methods lack effective mechanisms for handling missing modalities, leading to performance degradation in real-world scenarios with data interruptions. |
| Approach: | They propose a framework that shifts the paradigm of missing modality handling to retrieval-based recovery . they leverage cross-modal retrieval via unified semantic embeddings to acquire missing domain-specific knowledge. |
| Outcome: | The proposed framework improves AVQA and enhances robustness in modal-incomplete scenarios. |
RiTeK: A Dataset for Large Language Models Complex Reasoning over Textual Knowledge Graphs in Medicine (2026.findings-acl)
Copied to clipboard
Jiatan Huang, Mingchen Li, Zonghai Yao, Dawei Li, Yuxin Zhang, Zhichao Yang, Yongkang Xiao, Feiyun Ouyang, Xiaohan Li, Shuo Han, Hong yu
| Challenge: | Existing methods for retrieving medical textual knowledge Graphs struggle to perform well, a study finds . existing methods struggle to provide accurate answers to complex questions, he says . |
| Approach: | They synthesize user queries integrating diverse topological structures, relational information, and complex textual descriptions. |
| Outcome: | a new dataset for medical textual knowledge graphs shows that existing methods struggle to perform well . main bottlenecks lie in the scarcity of existing medical TKGs and the limited expressiveness of their topological structures . |
Collaborative Beam Search: Enhancing LLM Reasoning via Collective Consensus (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the reasoning capabilities of large language models (LLMs) depend on domain-specific external verifiers or self-evaluation which is brittle and prompt-sensitive. |
| Approach: | They propose a framework that harnesses the collective intelligence of multiple large language models across both generation and verification stages. |
| Outcome: | The proposed framework outperforms singlemodel scaling and multi-model ensemble baselines on six tasks by over 4 percentage points in average accuracy. |
Visual Question Decomposition on Multimodal Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for question decomposition focus on unimodal language models, but question decomposing capability of Multimodal Large Language Models (MLLMs) has yet to be explored. |
| Approach: | They propose a finetuning dataset and a training objective for selective decomposition to enhance the model's question decomposing capability. |
| Outcome: | The proposed dataset shows that existing models struggle to produce high-quality sub-questions. |
An Exploratory Study on Model Compression for Text-to-SQL (2023.findings-acl)
Copied to clipboard
| Challenge: | Text-to-SQL translates user queries into SQL statements that can retrieve relevant answers from relational databases. |
| Approach: | They propose to apply model compression techniques to sketch-based and sequence-to-sequence Text-toSQL models. |
| Outcome: | The proposed models have higher inference efficiency and respond better to model compression than sequence-to-sequence models. |
TempTabQA: Temporal Question Answering for Semi-Structured Tables (2023.emnlp-main)
Copied to clipboard
| Challenge: | Semi-structured data often include temporal information about entities, either implicitly or explicitly. |
| Approach: | They present a dataset that includes 11,454 question-answer pairs from Wikipedia Infobox tables spanning more than 90 distinct domains. |
| Outcome: | The proposed dataset can be used as a benchmark to improve models for temporal reasoning on semi-structured tables. |
Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reasoning-oriented LLMs lack a blind self-thinking paradigm . current models fail to recognize when their reasoning is underinformed or based on ambiguous user instructions . |
| Approach: | They propose a new reasoning paradigm that transforms LLMs into proactive inquirers that interleave reasoning with clarification. |
| Outcome: | The proposed model outperforms baseline models on mathematical reasoning, code generation, and document editing. |
InfoSync: Information Synchronization across Multilingual Semi-structured Tables (2023.findings-acl)
Copied to clipboard
| Challenge: | Information Synchronization of semi-structured data across languages is challenging . culture differences, topic preferences, and editing inconsistency lead to information mismatches . |
| Approach: | They propose a method for tabular synchronization that uses information from Wikipedia tables in one language with tables in another language. |
| Outcome: | The proposed method achieves an acceptance rate of 77.28% on Wikipedia . english articles across the web are more timely updated than other languages . |
Distance between Relevant Information Pieces Causes Bias in Long-Context LLMs (2025.findings-acl)
Copied to clipboard
Runchu Tian, Yanghao Li, Yuepeng Fu, Siyang Deng, Qinyu Luo, Cheng Qian, Shuo Wang, Xin Cong, Zhong Zhang, Yesai Wu, Yankai Lin, Huadong Wang, Xiaojiang Liu
| Challenge: | Positional biases in large language models hinder their ability to process long inputs. |
| Approach: | They propose a benchmark to assess positional bias in large language models involving multiple pieces of relevant information. |
| Outcome: | The proposed benchmark assesses the performance of long-context language models by examining their models with different input lengths and tasks. |
AwarenessBench: Assessing Cognitive Capabilities of Language Models (2026.acl-long)
Copied to clipboard
Xiaojian Li, Rongwu Xu, Tianyun Zhang, Yue Wang, Shuo Chen, Qiner Lyu, Briana Zhang, Peiran Yang, Kyle Xue Chen, Haoyuan Shi, Yu Wang, Wei Xu
| Challenge: | Language models exhibit increasingly consciousness-like behaviors, requiring a baseline to evaluate their cognitive abilities. |
| Approach: | They propose a benchmark to assess the cognitive abilities of language models (LMs) they compare 18 state-of-the-art LMs to human models in metacognition, self-awareness, social awareness and situational awareness . |
| Outcome: | Evaluating 18 state-of-the-art LMs, they find they consistently surpass baselines . but most models fall short in metacognition and self-awareness, the study finds . |
Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown strong potential for tabular data generation by modeling textualized feature-value pairs. |
| Approach: | They propose a method that explicitly integrates sparse dependency graphs into LLMs’ attention mechanism. |
| Outcome: | The proposed method outperforms existing LLM-based approaches by up to 12% on complex datasets while achieving competitive results with state-of-the-art approaches in synthetic data quality. |
Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Data (2025.findings-emnlp)
Copied to clipboard
Qiongqiong Wang, Hardik Bhupendra Sailor, Tianchi Liu, Wenyu Zhang, Muhammad Huzaifah, Nattadaporn Lertcheva, Shuo Sun, Nancy F. Chen, Jinyang Wu, AiTi Aw
| Challenge: | Recent speech-LLMs have shown impressive performance in tasks like transcription and translation, yet they remain limited in understanding the paralinguistic aspects of speech crucial for social and emotional intelligence. |
| Approach: | They propose a benchmark for evaluating speech-LLMs on contextual paralinguistic reasoning . the benchmark includes curated question answering datasets requiring both linguistic and empathetic understanding . |
| Outcome: | The proposed benchmark reveals a key gap in existing evaluations and offers insights into building more context-aware and emotionally intelligent LLMs. |
Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research indicates that large language models (LLMs) have demonstrated remark-able capabilities in various programming-related domains, such as code generation and code refinement. |
| Approach: | They propose a framework that combines exploration with refinement to reduce test-time computation overhead. |
| Outcome: | The proposed framework outperforms SOTA and AgentCoder on humanEval and MBPP benchmarks while reducing test-time computation overhead and scalability. |
Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages (2024.acl-long)
Copied to clipboard
| Challenge: | Contemporary large language models (LLMs) are pre-trained on multilingual corpora, but their performance lags behind in most languages compared to a few resource-rich languages. |
| Approach: | They propose a method that leverages the internal capabilities of large language models on resource-rich languages to enhance multilingual performance. |
| Outcome: | The proposed method improves multilingual performance while minimizing impact on original performance in resource-rich languages. |
GenomeQA: Benchmarking General Large Language Models for Genome Sequence Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on specialized DNA models trained for sequence prediction or evaluate biological knowledge using text-only questions. |
| Approach: | They propose a benchmark to evaluate general-purpose LLMs on sequence-based genome inference tasks. |
| Outcome: | The proposed benchmark outperforms baseline models on sequence-based genome inference tasks. |
RAPO: An Adaptive Ranking Paradigm for Bilingual Lexicon Induction (2022.emnlp-main)
Copied to clipboard
Zhoujin Tian, Chaozhuo Li, Shuo Ren, Zhiqiang Zuo, Zengxuan Wen, Xinyue Hu, Xiao Han, Haizhen Huang, Denvy Deng, Qi Zhang, Xing Xie
| Challenge: | Existing approaches focus on minimizing distances between words in aligned pairs, while suffering from low discriminative capability to distinguish the relative orders between positive and negative candidates. |
| Approach: | They propose a ranking-oriented induction model to learn personalized mapping function for each word. |
| Outcome: | The proposed model can learn personalized mapping function for each word on public datasets including rich-resource and low-resourced languages. |
RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Models (2025.acl-long)
Copied to clipboard
Hieu Tran, Zonghai Yao, Zhichao Yang, Junda Wang, Yifan Zhang, Shuo Han, null Feiyun Ouyang, Hong Yu
| Challenge: | Existing work aims to improve reasoning accuracy and factual integrity across large language models for knowledge-intensive tasks such as medical and commonsense reasoning. |
| Approach: | They propose a versatile extension to the mutual reasoning framework (rStar) that enhances reasoning accuracy and factual integrity across large language models. |
| Outcome: | The proposed extension to the mutual reasoning framework improves reasoning accuracy and factual integrity across large language models for complex, knowledge-intensive tasks. |
DPEPO: Diverse Parallel Exploration Policy Optimization for LLM-based Agents (2026.acl-long)
Copied to clipboard
JunShuo Zhang, Chengrui Huang, Feng Guo, Zihan Li, Ke Shi, Menghua Jiang, Jiguo Yu, Shuo Shang, Shen Gao
| Challenge: | Existing approaches to large language model (LLM) agents that follow the sequential "reason-then-act" paradigm suffer from limited exploration and incomplete environmental understanding as they interact with only a single environment per step. |
| Approach: | They propose a paradigm that enables an agent to interact with multiple environments simultaneously and share cross-trajectory experiences. |
| Outcome: | The proposed paradigm achieves state-of-the-art (SOTA) success rates while maintaining comparable efficiency to strong sequential baselines. |
TemplateGEC: Improving Grammatical Error Correction with Detection Template (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for grammatical error correction (GEC) have been developed. |
| Approach: | They propose a method which integrates the detection labels from a Seq2Edit model to construct a template as the input. |
| Outcome: | The proposed method can perform human-in-the-loop error correction tasks. |
The Knowledge Alignment Problem: Bridging Human and External Knowledge for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models often ignore external knowledge to generate accurate answers . despite correct groundings, they can rely on wrong grounding or biases to hallucinate . |
| Approach: | They propose a framework that integrates human and human user clarifications to improve knowledge alignment. |
| Outcome: | The proposed framework improves model performance and mitigates hallucination by producing user-centered clarifications. |
LMR-BENCH: Evaluating LLM Agent’s Ability on Reproducing Language Modeling Research (2025.emnlp-main)
Copied to clipboard
Shuo Yan, Ruochen Li, Ziming Luo, Zimu Wang, Daoyang Li, Liqiang Jing, Kaiyu He, Peilin Wu, Juntong Ni, George Michalopoulos, Yue Zhang, Ziyang Zhang, Mian Zhang, Zhiyu Chen, Xinya Du
| Challenge: | Large language model (LLM) agents have demonstrated remarkable potential in advancing scientific discovery, but their capability in reproducing code from research papers remains underexplored. |
| Approach: | They propose to evaluate LLM agents' ability to reproduce scientific research papers by analyzing code reproduction tasks from 23 research papers published in top-tier NLP venues. |
| Outcome: | The proposed benchmark systematically evaluates the capability of large language model (LLM) agents on code reproduction from Language Modeling Research. |
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way (2023.emnlp-demo)
Copied to clipboard
Kai Lv, Shuo Zhang, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu
| Challenge: | Large language models (LLMs) are increasingly pivotal in a wide range of tasks . however, the resources required for training these models necessitate efficient solutions . |
| Approach: | They propose a library that facilitates collaborative training of large language models . they use 3D parallelism, parameter-efficient fine-tuning methods and optimizers . |
| Outcome: | The proposed library has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios. |
Explainable Recommendation with Personalized Review Retrieval and Aspect Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Recent years have witnessed a growing interest in the development of explainable recommendation models. |
| Approach: | They propose a model that combines prediction and generation tasks to produce more persuasive explanations by obtaining additional information from the training sets. |
| Outcome: | The proposed model outperforms state-of-the-art models on three datasets and shows that it is more persuasive than previous models. |
ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development (2026.findings-acl)
Copied to clipboard
Jie Yang, Honglin Guo, Li Ji, Jiazheng Zhou, Rui Zheng, Zhikai Lei, Shuo Zhang, Zhiheng Xi, Shichun Liu, Yuxin Wang, Bo Wang, Yining Zheng, Tao Gui, Xipeng Qiu
| Challenge: | Large Language Models (LLMs) have redefined the role of AI in software engineering . current benchmarks focus on localized code generation, but neglect dynamic, full-process requirements of real-world engineering. |
| Approach: | They propose a benchmark to evaluate agentic backend coding within a realistic, executable workflow. |
| Outcome: | The ABC-Bench benchmark evaluates agentic backend coding within a realistic, executable workflow. |
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing (2022.acl-long)
Copied to clipboard
Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei
| Challenge: | Existing work shows that pre-trained models can improve in various natural language processing tasks. |
| Approach: | They propose a unified-modal encoder-decoder framework that pre-trains speech-text representations using large-scale unlabeled speech and text data. |
| Outcome: | The proposed framework is superior to existing models on speech-to-text processing tasks. |
Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels (2025.emnlp-main)
Copied to clipboard
Junjie Ye, Yuming Yang, Yang Nan, Shuo Li, Qi Zhang, Tao Gui, Xuanjing Huang, Peng Wang, Zhongchao Shi, Jianping Fan
| Challenge: | Large language models (LLMs) acquire substantial world knowledge during pretraining, which is further shaped by post-training techniques such as supervised fine-tuning (SFT). |
| Approach: | They evaluate closed-book question answering (CBQA) performance across five LLMs from the LLaMA-2 and LLama-3 families and examine the impact of supervised fine-tuning on model knowledge. |
| Outcome: | The proposed model performance is 14% worse than models fine-tuned on 1,920 samples and 12% worse on 240 samples. |
IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction (2025.findings-acl)
Copied to clipboard
| Challenge: | RULEARN is a benchmark to assess the rule-learning abilities of large language models (LLMs) in interactive environments. |
| Approach: | They propose a framework that integrates the process of **I**nduction, **De**duction, and **A**bduction. |
| Outcome: | The proposed framework improves on the baseline and human-like rule learning in real-world scenarios. |
More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectives (2025.acl-long)
Copied to clipboard
Xiaoqing Zhang, Ang Lv, Yuhan Liu, Flood Sung, Wei Liu, Jian Luan, Shuo Shang, Xiuying Chen, Rui Yan
| Challenge: | Large language models excel at few-shot in-context learning but performance plateaus as ICL demonstrations increase from a few to many. |
| Approach: | They propose a novel optimization method that optimizes the negative log-likelihood objective and reweights the model to achieve many-shot performance. |
| Outcome: | The proposed method achieves significant performance improvements across a large-scale dataset. |
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages (2024.emnlp-main)
Copied to clipboard
Holy Lovenia, Rahmad Mahendra, Salsabil Akbar, Lester James Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jonibek Mansurov, Joseph Marvin Imperial, Onno Kampman, Joel Moniz, Muhammad Habibi, Frederikus Hudi, Jann Montalan, Ryan Hadiwijaya, Joanito Lopo, William Nixon, Börje Karlsson, James Jaya, Ryandito Diandaru, Yuze Gao, Patrick Irawan, Bin Wang, Jan Christian Blaise Cruz, Chenxi Whitehouse, Ivan Parmonangan, Maria Khelli, Wenyu Zhang, Lucky Susanto, Reynard Ryanda, Sonny Hermawan, Dan Velasco, Muhammad Kautsar, Willy Hendria, Yasmin Moslem, Noah Flynn, Muhammad Adilazuarda, Haochen Li, Johanes Lee, R. Damanhuri, Shuo Sun, Muhammad Qorib, Amirbek Djanibekov, Wei Qi Leong, Quyet V. Do, Niklas Muennighoff, Tanrada Pansuwan, Ilham Firdausi Putra, Yan Xu, Tai Chia, Ayu Purwarianti, Sebastian Ruder, William Tjhi, Peerat Limkonchotiwat, Alham Aji, Sedrick Keh, Genta Winata, Ruochen Zhang, Fajri Koto, Zheng Xin Yong, Samuel Cahyawijaya
| Challenge: | Southeast Asia (SEA) is home to over 1,300 indigenous languages and 671 million people . prevailing AI models suffer from a significant lack of representation of texts, images, and audio datasets from SEA . |
| Approach: | They propose to provide a resource center that provides standardized corpora in nearly 1,000 SEA languages across three modalities. |
| Outcome: | a new benchmark assesses the quality of AI models on 36 SEA languages across 13 tasks . the results highlight the importance of SEA as a culturally diverse region . |
Towards More Accurate Uncertainty Estimation In Text Classification (2020.emnlp-main)
Copied to clipboard
Jianfeng He, Xuchao Zhang, Shuo Lei, Zhiqian Chen, Fanglan Chen, Abdulaziz Alhamadani, Bei Xiao, ChangTien Lu
| Challenge: | Existing models of uncertainty score depend on winning score, which is the maximum probability in a semantic vector. |
| Approach: | They propose to generate accurate uncertainty score by improving the confidence of winning scores by reducing the effect of overconfidence of winning score and considering the impact of different categories simultaneously. |
| Outcome: | The proposed model reduces the effect of overconfidence of winning score and considers impact of different categories of uncertainty simultaneously. |
Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations (2025.findings-emnlp)
Copied to clipboard
Shuo Li, Jiajun Sun, Guodong Zheng, Xiaoran Fan, Yujiong Shen, Yi Lu, Zhiheng Xi, Yuming Yang, Wenming Tan, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Multimodal large language models have demonstrated remarkable performance in visual-language tasks, but their authenticity is often compromised by object hallucinations. |
| Approach: | They propose a multi-frequency perturbation method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference. |
| Outcome: | The proposed method significantly mitigates object hallucinations across various model architectures. |
Doubling Your Data in Minutes: Ultra-fast Tabular Data Generation via LLM-Induced Dependency Graphs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Tabular data is critical across diverse domains, yet high-quality tabular datasets remain scarce due to privacy concerns and the cost of collection. |
| Approach: | They propose a lightweight generative framework that captures sparse dependencies via an LLM-induced graph. |
| Outcome: | The proposed framework reduces constraint violations by 4% and accelerates generation by nearly 9,500 over diffusion-based methods. |
KnowMe-Bench: Benchmarking Person Understanding for Lifelong Digital Companions (2026.acl-long)
Copied to clipboard
Tingyu Wu, Zhisheng Chen, Ziyan Weng, Shuhe Wang, Shuo Zhang, Sen Hu, Silin Wu, Qizhen Lan, Huacan Wang, Ronghao Chen
| Challenge: | Existing long-horizon memory benchmarks use multi-turn dialogues or synthetic user histories . despite rapid progress on long-term memory evaluation, there are gaps in existing benchmarks . |
| Approach: | They propose a long-form autobiographical narrative benchmark that reconstructs each narrative into a flashback-aware, time-anchored stream and evaluates models with evidence-linked questions. |
| Outcome: | The proposed benchmarks build from long-form autobiographical narratives . they show that retrieval-augmented systems improve factual accuracy while errors persist on temporally grounded explanations and higher-level inferences. |
ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent Web (2026.acl-long)
Copied to clipboard
Zhiyuan Yao, Zishan Xu, Yifu Guo, Zhiguang Han, Cheng Yang, Shuo Zhang, Weinan Zhang, Xingshan Zeng, Weiwen Liu
| Challenge: | Existing routers that use hardcoded tools are limited by scalability and generality bottlenecks. |
| Approach: | They propose a pipeline for training history-aware routers to empower precise navigation in large-scale ecosystems. |
| Outcome: | The proposed pipeline can train routers with dynamic context understanding to create the plug-and-play Light Routing Agent. |
SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent approaches to generate tabular data are limited by their static dependences and lack of fidelity. |
| Approach: | They propose a novel LLM-based generation framework that enforces sparse and dynamic dependency guidance. |
| Outcome: | The proposed framework boosts F1 scores by 10% and reduces policy violations by one point. |
Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in text-only "slow thinking" reasoning have prompted efforts to transfer this capability to vision-language models (VLMs). |
| Approach: | They propose a VRM Reflection-V which enhances visual reflection based on reasoning data for cold-start and reward design for reinforcement learning. |
| Outcome: | The proposed model improves visual reflection for cold-start and reward design for reinforcement learning (RL) it maintains a stronger and more consistent reliance on visual information during visual reasoning, indicating effective enhancement in visual reflection capabilities. |