Papers by Ming Zhao
DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive (AR) decoding in large language models is latency-bounded by strictly sequential token generation. |
| Approach: | They propose a diffusion-based drafter that proposes multi-token candidates and then verifies them in parallel by the target model. |
| Outcome: | The proposed drafter generates multi-token proposals in a single forward pass while remaining compatible with standard AR verifiers. |
Shortcuts Arising from Contrast: Towards Effective and Lightweight Clean-Label Attacks in Prompt-Based Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prompt-based learning paradigms are vulnerable to backdoor attacks, requiring false activations and false data augmentation. |
| Approach: | They propose a method that uses triggers to create stronger shortcuts by leveraging activation values and data selection strategies to create the shortcuts. |
| Outcome: | The proposed method is based on the concept that a backdoor acts as a shortcut and can achieve high effectiveness and stealthiness at low poisoning rates. |
FLAIR: Steering LLM Mathematical Problem Solving based on A Fuzzy-Logic-AssIsted Reasoner (2026.acl-long)
Copied to clipboard
Hao Wu, Hongru Sun, Wanqing Li, Xinguo Yu, Hao Ming, Xiao Luo, Wenbin Zhang, Jiahong Zhao, Yi Guo, Jie Yang
| Challenge: | Existing approaches to mathematical reasoning rely on static heuristics or pre-determined reasoning strategies. |
| Approach: | They propose an adaptive framework that integrates fuzzy theory into LLM-based mathematical reasoning. |
| Outcome: | The proposed framework outperforms state-of-the-art models while offering effective and interpretable diagnostics of intermediate problem-solving states. |
Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning Through Trap Problems (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current LLMs lack systematic compositionality, and therefore cannot serve as reliable cognitive models. |
| Approach: | They propose to introduce logical traps into the original problems of MATH and GSM8K to investigate the compositionality of large language models in mathematical reasoning. |
| Outcome: | The proposed model can generate infinite combinations from finite learned components. |
LEMMA: Learning from Errors for MatheMatical Advancement in LLMs (2025.findings-acl)
Copied to clipboard
Zhuoshi Pan, Yu Li, Honglin Lin, Qizhi Pei, Zinan Tang, Wei Wu, Chenlin Ming, H. Vicky Zhao, Conghui He, Lijun Wu
| Challenge: | Existing approaches focus on improving the quality of correct training data, neglecting the value contained in error data, thereby hindering the model’s reflective ability. |
| Approach: | They propose to improve LLM's reasoning ability by learning from error data and a grounded mistake augmentation method to collect representative errors. |
| Outcome: | The proposed model achieves significant performance improvements over other strong models with less than 90k data. |
Model Merging for Knowledge Editing (2025.acl-industry)
Copied to clipboard
Zichuan Fu, Xian Wu, Guojing Li, Yingying Zhang, Yefeng Zheng, Tianshi Ming, Yejing Wang, Wanyu Wang, Xiangyu Zhao
| Challenge: | Existing knowledge editing approaches struggle with sequential editing scenarios and harm the general capabilities of the model. |
| Approach: | They propose a framework that combines robust supervised fine-tuning and model merging for knowledge editing to combine supervised and supervised learning. |
| Outcome: | The proposed approach outperforms existing methods in sequential editing while preserving the original performance of the model. |
Instance Regularization for Discriminative Language Model Pre-training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have optimized independent strategies of ennoising or denosing . Existing methods treat training instances equally throughout the training process . |
| Approach: | They propose to use ennoising and denoising to train discriminative pre-trained language models . they propose to model the complexity of restoring the original sentences from corrupted ones . |
| Outcome: | Experimental results show that the proposed method improves pre-training efficiency, effectiveness, and robustness. |
AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science (2025.findings-emnlp)
Copied to clipboard
An Luo, Xun Xian, Jin Du, Fangqiao Tian, Ganghua Wang, Ming Zhong, Shengchun Zhao, Xuan Bi, Zirui Liu, Jiawei Zhou, Jayanth Srinivasa, Ashish Kundu, Charles Fleming, Mingyi Hong, Jie Ding
| Challenge: | Large language models (LLMs) have advanced the automation of data science workflows, yet it remains unclear whether they can critically leverage external domain knowledge as human data scientists do in practice. |
| Approach: | They propose a benchmark to evaluate how large language models handle external domain knowledge in tabular prediction tasks. |
| Outcome: | The proposed model evaluates whether it can critically leverage external domain knowledge as human data scientists do in practice. |
Neural Attention-Aware Hierarchical Topic Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural topic models (NTMs) use deep neural networks to learn topic information. |
| Approach: | They propose a variational autoencoder model that reconstructs sentence and document word counts using bag-of-words embeddings and pre-trained semantic embedders. |
| Outcome: | The proposed model lowers reconstruction errors at sentence and document levels and finds more coherent topics from real-world datasets. |
Reaction Miner: An Integrated System for Chemical Reaction Extraction from Textual Data (2023.emnlp-demo)
Copied to clipboard
Ming Zhong, Siru Ouyang, Yizhu Jiao, Priyanka Kargupta, Leo Luo, Yanzhen Shen, Bobby Zhou, Xianrui Zhong, Xuan Liu, Hongxiang Li, Jinfeng Xiao, Minhao Jiang, Vivian Hu, Xuan Wang, Heng Ji, Martin Burke, Huimin Zhao, Jiawei Han
| Challenge: | Reaction Miner is a system designed to extract chemical reactions from raw scientific PDFs. |
| Approach: | They propose a system that extracts chemical reactions directly from raw scientific PDFs. |
| Outcome: | The proposed system can extract chemical reactions from raw scientific PDFs. |
TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities (2024.emnlp-main)
Copied to clipboard
Ming Zhang, Caishuang Huang, Yilong Wu, Shichun Liu, Huiyuan Zheng, Yurui Dong, Yujiong Shen, Shihan Dou, Jun Zhao, Junjie Ye, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Current datasets cater to user-led systems and are limited to predefined specific scenarios and slots. |
| Approach: | They propose to use a Chinese dialogue dataset to train a model that authentically simulates human-computer dialogues in 30 popular life service scenarios. |
| Outcome: | The proposed model achieves a joint accuracy of 75.09% in out-of-domain evaluations . it also achieves notable abilities in slot filling and questioning . |
RAG+: Enhancing Retrieval-Augmented Generation with Application-Aware Reasoning (2025.emnlp-main)
Copied to clipboard
Yu Wang, Shiwan Zhao, Zhihu Wang, Ming Fan, Xicheng Zhang, Yubo Zhang, Zhengfan Wang, Heyuan Huang, Ting Liu
| Challenge: | Existing RAG paradigms often overlook the cognitive step of applying knowledge, leaving a gap between retrieved facts and task-specific reasoning. |
| Approach: | They introduce a module extension that integrates application-aware reasoning into the RAG pipeline. |
| Outcome: | Experiments show that RAG+ outperforms standard RAG variants and achieves gains of 3–5% in complex scenarios. |
LRQuant: Learnable and Robust Post-Training Quantization for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for post-training quantization (PTQ) are limited by the complexity of the quantization parameter and performance degradations when tested on unseen datasets. |
| Approach: | They propose a learnable smooth-based PTQ framework that allows for rapid adaptation during testing. |
| Outcome: | The proposed framework improves performance on unseen datasets and reduces memory constraints. |
A Survey of RAG-Reasoning Systems in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Yangning Li, Weizhi Zhang, Yuyao Yang, Wei-Chieh Huang, Yaozu Wu, Junyu Luo, Yuanchen Bei, Henry Peng Zou, Xiao Luo, Yusheng Zhao, Chunkit Chan, Yankai Chen, Zhongfen Deng, Yinghui Li, Hai-Tao Zheng, Dongyuan Li, Renhe Jiang, Ming Zhang, Yangqiu Song, Philip S. Yu
| Challenge: | a survey of RAG-based reasoning-based approaches shows that it is not effective for multi-step inferences. |
| Approach: | They map how advanced reasoning optimizes each stage of RAG . they show how retrieved knowledge supply missing premises and expand context for complex inference . |
| Outcome: | The proposed frameworks achieve state-of-the-art across knowledge-intensive benchmarks. |
I²B-LPO: Latent Policy Optimization via Iterative Information Bottleneck (2026.acl-long)
Copied to clipboard
Huilin Deng, Hongchen Luo, Yue Zhu, Long Li, Zhuoyue Chen, Xinghao Zhao, Ming LI, Chuyang Zhao, Jihai Zhang, MengChang Wang, Yang Cao, Yu Kang
| Challenge: | Existing methods for large language model reasoning suffer from exploration collapse due to the semantic homogeneity of random rollouts. |
| Approach: | They propose to use latent policy optimization via iterative information bottleneck to optimize reasoning trajectories by diversifying reasoning . |
| Outcome: | Empirical results show that the proposed method achieves state-of-the-art performance with margins of up to 5.3% in accuracy and 7.4% in diversity metrics. |
SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods fail to fully exploit the knowledge embedded in models from previous tasks . Existing techniques fail to exploit the information embedded in previous tasks, resulting in a large number of replay samples to achieve good results. |
| Approach: | They propose a method that uses attention weights to extract knowledge from previous tasks . they use a data replay strategy to extract the knowledge from the previous tasks. |
| Outcome: | The proposed method achieves comparable or even better performance with only 1/10 of replayed data used by other methods. |
Instruct and Extract: Instruction Tuning for On-Demand Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models with instruction-following capabilities are not suitable for long-tail ad hoc extraction use cases for non-expert users. |
| Approach: | They propose a task that follows instructions to extract the desired content from the associated text and present it in a structured tabular format. |
| Outcome: | The proposed paradigm outperforms existing open-source models of similar size in terms of information extraction. |
DISCO Balances the Scales: Adaptive Domain- and Difficulty-Aware Reinforcement Learning on Imbalanced Data (2025.findings-emnlp)
Copied to clipboard
Yuhang Zhou, Jing Zhu, Shengyi Qian, Zhuokai Zhao, Xiyao Wang, Xiaoyu Liu, Ming Li, Paiheng Xu, Wei Ai, Furong Huang
| Challenge: | Large Language Models (LLMs) are increasingly aligned with human preferences through Reinforcement Learning from Human Feedback (RLHF). |
| Approach: | a new study proposes a domain-informed self-consistency policy optimization extension to GRPO that addresses inter-group imbalance. |
| Outcome: | a new extension of GRPO addresses inter-group imbalance with two key innovations . the proposed method outperforms existing GR PO variants by 5% on Qwen3 models . |
Embracing Large Language Models in Traffic Flow Forecasting (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict future traffic flows capture spatio-temporal dependencies, but they fail to adapt to test-time environmental changes. |
| Approach: | They propose to use large language models to help traffic flow forecasting by capturing spatio-temporal dependencies and using a large language model to select the most likely result. |
| Outcome: | The proposed method is based on large language models (LLMs) and an LLM-based selector. |
Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuning (2026.acl-long)
Copied to clipboard
Yanrui Du, Fenglei Fan, Sendong Zhao, Jiawei Cao, Ming Ma, Danyang Zhao, Shuren Qi, Ting Liu, Bing Qin
| Challenge: | Instruction Fine-Tuning (IFT) has emerged as a critical technique for customizing Large Language Models (LLMs) however, recent studies have revealed that IFT can compromise the built-in security mechanisms of LLMs, posing significant security risks. |
| Approach: | They propose a method that shifts learning burden onto security-robust parameters and propose 'warm-up' phase that preferentially trains Mods_Rob to learn low-level features with minimal security risk. |
| Outcome: | The proposed method reduces security risks without sacrificing performance gains across knowledge-intensive datasets. |
Visual Attention Reasoning via Hierarchical Search and Self-Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) often hallucinate due to fragile, linear reasoning and weak visual grounding. |
| Approach: | They propose a framework that reformulates reasoning as a hierarchical search with self-verification and replaces linear Chain-of-Thought with a tree-search policy capable of backtracking to correct logical errors. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on hallucination and safety benchmarks. |
PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for sub 2-bit quantization introduce an extra 1-bit or more per weight. |
| Approach: | They propose a sub 2-bit post-training quantization method that enables weight quantization to 1.61-bit for the first time. |
| Outcome: | The proposed method reduces the upper bound of quantization error to 1.61-bit for the first time. |
DoCIA: An Online Document-Level Context Incorporation Agent for Speech Translation (2025.findings-acl)
Copied to clipboard
Xinglin Lyu, Wei Tang, Yuang Li, Xiaofeng Zhao, Ming Zhu, Junhui Li, Yunfei Lu, Min Zhang, Daimeng Wei, Hao Yang, Min Zhang
| Challenge: | Document-level context is crucial for speech translation due to noise from ASR . incorporating document-level contextual information into ST remains a challenge . |
| Approach: | They develop an online framework that integrates document-level context into machine translation . they use document-based modules to integrate document- level context into ST . |
| Outcome: | The proposed framework outperforms baselines in sentence and discourse metrics . it can correct ASR transcription errors and improve translation performance . |
Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Current instruction tuning relies on teacher models or human intervention to generate and refine the instructions and responses for training, which are costly, non-sustainable, and may lack diversity. |
| Approach: | They propose a human/model-free compositional data synthesis method that can create rich and diverse augmentations from existing instruction tuning data to enhance large language models. |
| Outcome: | The proposed method improves performance over benchmarks and reduces training costs by 80% compared with original instruction tuning. |
Compressing Sentence Representation for Semantic Retrieval via Homomorphic Projective Distillation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models produce large sentence embeddings, resulting in performance gap between large and small models. |
| Approach: | They propose a method that augments a small Transformer encoder model with learnable projection layers to produce compact sentences while mimicking a large pre-trained language model to retain the sentence representation quality. |
| Outcome: | The proposed method achieves 2.7-4.5 points performance gain on STS and SR tasks while maintaining the quality of the pre-trained language models. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches involve models iterating and improving their previous responses based on internal reflection ability or external feedback. |
| Approach: | They propose a reflection framework that leverages meta-thoughts and self-consistency to enhance the iterative reflection capability of Large LanguageModels. |
| Outcome: | The proposed framework achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability. |
Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Earlier studies of instruction tuning on Large Language Models focus on creating large, varied, and high-quality datasets with responses curated by human experts. |
| Approach: | They propose to use a smaller and weaker model to fine tune a larger and stronger model . they find it can largely speed up the data filtering and improve performance . |
| Outcome: | The proposed model can filter instruction data faster and better on benchmarks. |
Multifaceted Evaluation of Audio-Visual Capability for MLLMs: Effectiveness, Efficiency, Generalizability and Robustness (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-modal large language models have been used for processing and understanding information from diverse modalities. |
| Approach: | They propose to evaluate the audio-visual capabilities of multi-modal large language models . they focus on effectiveness, efficiency, generalizability, and robustness . |
| Outcome: | The proposed models exhibit strong zero-shot and few-shot generalization abilities . their success relies heavily on the vision modality, which impairs performance when visual input is corrupted or missing. |
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention (2025.acl-long)
Copied to clipboard
Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Yuxing Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, Wangding Zeng
| Challenge: | Long-context modeling is crucial for next-generation language models, but high computational cost of standard attention mechanisms poses significant computational challenges. |
| Approach: | They propose a natively trained Sparse Attention mechanism that integrates algorithms with hardware-aligned optimizations to achieve efficient long-context modeling. |
| Outcome: | The proposed model maintains or exceeds Full Attention models across general benchmarks, long-context tasks, and instruction-based reasoning. |
EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on ERC focus on context modeling but ignore representation of contextual emotional tendency. |
| Approach: | They propose to use Emoformer to extract multi-modal emotion vectors from different modalities and fuse them with sentence vector to be an emotion capsule. |
| Outcome: | The proposed model outperforms the state-of-the-art models on two benchmark datasets. |
Less Is More: Domain Adaptation with Lottery Ticket for Reading Comprehension (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing domain adaptation paradigms for reading comprehension require large amounts of annotation data to achieve the desired task performance. |
| Approach: | They propose a few-shot domain adaptation paradigm for reading comprehension . they introduce self-attention attribution to weigh parameters and refine the lottery subnetwork . |
| Outcome: | The proposed model outperforms the full model fine-tuning adaptation on four out of five domains with a small amount of data available for adaptation. |
PARA: Parameter-Efficient Fine-tuning with Prompt-Aware Representation Adjustment (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning excel in the context of single-backbone multi-tenant applications. |
| Approach: | They propose to integrate a lightweight vector generator within each Transformer layer to improve prompt-aware representation adjustment. |
| Outcome: | The proposed method surpasses current benchmarks in terms of performance despite having a similar number of adjustable parameters. |
A Critical Analysis of Document Out-of-Distribution Detection (2023.findings-emnlp)
Copied to clipboard
Jiuxiang Gu, Yifei Ming, Yi Zhou, Jason Kuen, Vlad Morariu, Handong Zhao, Ruiyi Zhang, Nikolaos Barmpalios, Anqi Liu, Yixuan Li, Tong Sun, Ani Nenkova
| Challenge: | Existing document understanding models focus on single-modal inputs such as images or texts. |
| Approach: | They propose to use a spatial-aware adapter to adapt transformer-based language models to document domain to exploit multi-modal information. |
| Outcome: | The proposed model significantly improves the OOD detection performance compared to using a standard language model and to competitive baselines. |
Neural Document Summarization by Jointly Learning to Score and Select Sentences (P18-1)
Copied to clipboard
| Challenge: | Sentence scoring and sentence selection are two main steps in extractive document summarization systems. |
| Approach: | They propose an end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences. |
| Outcome: | The proposed framework outperforms the state-of-the-art summarization models on the CNN/Daily Mail dataset. |
Learning to Collaborate for Question Answering and Asking (N18-1)
Copied to clipboard
| Challenge: | Question answering (QA) and question generation (QG) are closely related tasks. |
| Approach: | They propose a training algorithm that generalizes both Generative Adversarial Network and Generating Domain-Adaptive Nets under the question answering scenario. |
| Outcome: | The proposed training algorithm generalizes both Generative Adversarial Network (GAN) and Generating Domain-Adaptive Nets (GDAN) under the question answering scenario. |
On the Evaluation of Vision-and-Language Navigation Instructions (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing instruction generators have not been evaluated using human wayfinders . BLEU, ROUGE, METEOR and CIDEr are ineffective for evaluating grounded navigation instructions. |
| Approach: | They propose an instruction-trajectory compatibility model that operates without reference instructions to improve wayfinding performance. |
| Outcome: | The proposed model shows the highest correlation with human wayfinding outcomes when scoring individual instructions. |
MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation (2025.naacl-long)
Copied to clipboard
Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao, Bohan Wu, Ye Yuan, Haozhe Zhao, Zhihui Guo, Yichi Zhang, Jingyang Yuan, Wei Ju, Luchen Liu, Tianyu Liu, Baobao Chang, Ming Zhang
| Challenge: | Large Multimodal Models (LMMs) exhibit impressive cross-modal understanding and reasoning abilities, but many benchmarks suffer from systematic biases. |
| Approach: | They propose a benchmark to avoid Type-I errors by creating one perception question and one knowledge anchor question through a meticulous annotation process. |
| Outcome: | The proposed benchmark avoids Type-I errors while maintaining reliability of MCQ evaluations. |
Infrared-LLaVA: Enhancing Understanding of Infrared Images in Multi-Modal Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for infrared modeling ignore supervisory signals of infra-modality-specific attributes, which may lead to biased understanding of in-frarea images. |
| Approach: | They propose a multi-agent generation system which transfers knowledge from visible images to generate infrared image-text pairs and infra-instructional data. |
| Outcome: | The proposed system generates infrared image-text pairs and infra-response data and is able to answer common infreas tasks with the proposed model. |