Papers by Dacheng Tao

Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing (2026.findings-acl)

Challenge:	a new method for unlearning large language models is proposed to improve the performance of large language model models.
Approach:	They propose a probability perturbation-based unlearning paradigm that allows models to forget implicit knowledge in large language models with a focus on generalisation.
Outcome:	The proposed model improves unlearning vanilla target data while forgetting implicit knowledge.

Copied to clipboard

Yuchen Wu, Liang Ding, Li Shen, Dacheng Tao

Zero-shot Sharpness-Aware Quantization for Pre-trained Language Models (2023.emnlp-main)

Challenge:	Current methods for modifying parameters to integrate new knowledge are not accurate enough.
Approach:	They propose an SFT+RL framework that instills process-level faithfulness by a stage-aware Reward mechanism and a Stage-assisted Reward Mechanism.
Outcome:	The proposed framework instills process-level faithfulness while boosting final accuracy.

Copied to clipboard

Miaoxi Zhu, Qihuang Zhong, Li Shen, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

Revisiting Knowledge Distillation for Autoregressive Language Models (2024.acl-long)

Challenge:	Existing zero-shot quantization methods are based on overfitting problem in adversarial learning process, leading to sub-optimal performance.
Approach:	They propose a zero-shot sharpness-aware quantization framework for the quantization of various PLMs by optimizing a minimax problem.
Outcome:	The proposed framework can achieve significant performance gains on discriminative and generative PLMs.

Copied to clipboard

Qihuang Zhong, Liang Ding, Li Shen, Juhua Liu, Bo Du, Dacheng Tao

Improving Neural Machine Translation by Bidirectional Training (2021.emnlp-main)

Challenge:	Autoregressive language models (LMs) are expensive and memory intensive, preventing the development of industrial applications.
Approach:	They propose an adaptive teaching approach to improve the KD of autoregressive language models by distilling knowledge into a small student model.
Outcome:	The proposed method can achieve consistent and significant performance gains across all model types and sizes.

Copied to clipboard

Liang Ding, Di Wu, Dacheng Tao

Self-Powered LLM Modality Expansion for Large Speech-Text Models (2024.emnlp-main)

Challenge:	Experimental results show that bidirectional training pushes the SOTA neural machine translation performance significantly higher.
Approach:	They propose a bidirectional training strategy that updates model parameters at the early stage and tunes it normally.
Outcome:	The proposed approach pushes the SOTA neural machine translation performance significantly higher on 15 translation tasks on 8 language pairs.

Copied to clipboard

Tengfei Yu, Xuebo Liu, Zhiyi Hou, Liang Ding, Dacheng Tao, Min Zhang

Multimodality

Challenge:	Large language models exhibit remarkable performance across diverse tasks . however, these methods require significant resource demands and tend to overfit specific tasks.
Approach:	They propose a self-powered LSM that leverages augmented automatic speech recognition data generated by the model itself for more effective instruction tuning.
Outcome:	The proposed model mitigates speech anchor bias and improves the fusion of speech and text modalities in large language models.

Progressive Multi-Granularity Training for Non-Autoregressive Translation (2021.findings-acl)

Copied to clipboard

Liang Ding, Longyue Wang, Xuebo Liu, Derek F. Wong, Dacheng Tao, Zhaopeng Tu

Dynamic Parallel Tree Search for Efficient LLM Reasoning (2025.acl-long)

Challenge:	Non-autoregressive translation models are weak at learning high-mode knowledge, argues a new study . despite the improved learning difficulty, there are still complicated word orders and structures in the synthetic sentences, making the NAT performance sub-optimal.
Approach:	They propose to train non-autoregressive translation models to learn fine-grained lower-mode knowledge . they break down sentence-level examples into three types and increase granularities .
Outcome:	The proposed method improves phrase translation accuracy and model reordering ability against strong NAT baselines.

Copied to clipboard

Yifu Ding, Wentao Jiang, Shunyu Liu, Yongcheng Jing, Jinyang Guo, Yingjie Wang, Jing Zhang, Zengmao Wang, Ziwei Liu, Bo Du, Xianglong Liu, Dacheng Tao

Intention Analysis Makes LLMs A Good Jailbreak Defender (2025.coling-main)

Challenge:	Recent methods focus on search accuracy while overlooking computational efficiency.
Approach:	They propose a parallelism framework that dynamically optimizes reasoning path in inference.
Outcome:	The proposed framework improves efficiency by 2-4 on average while maintaining or even surpassing existing reasoning algorithms in accuracy.

Copied to clipboard

Yuqi Zhang, Liang Ding, Lefei Zhang, Dacheng Tao

DB-LLM: Accurate Dual-Binarization for Efficient LLMs (2024.findings-acl)

Challenge:	Existing methods to align large language models with human values overlook the intrinsic nature of jailbreaks, which limits their effectiveness in complex scenarios.
Approach:	They propose a simple yet highly effective defense strategy, i.e., Intention Analysis (IA). They show that IA suppresses LLM’s tendency to follow jailbreak prompts, thereby enhancing safety.
Outcome:	The proposed strategy reduces harmfulness of LLMs and outperforms GPT-3.5 in attack success rate.

Copied to clipboard

Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding, Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, Dacheng Tao

Revisiting Token Dropping Strategy in Efficient BERT Pretraining (2023.acl-long)

Challenge:	Existing methods for ultra-low bit quantization cause severe accuracy drops . a novel Dual-Binarization method is proposed for efficient Large Language Models .
Approach:	They propose a Dual-Binarization method that takes 2-bit-width and binarization into account . they propose DB-LLM, which uses a 2-bit binarized weighted model to represent weights efficiently .
Outcome:	The proposed method surpasses the current State-of-the-Art in ultra-low bit quantization and achieves 20% reduction in computational consumption compared to the SOTA method under the same bit-width.

Copied to clipboard

Qihuang Zhong, Liang Ding, Juhua Liu, Xuebo Liu, Min Zhang, Bo Du, Dacheng Tao

Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models (2025.findings-emnlp)

Challenge:	Token dropping is a recently-proposed strategy to speed up the pretraining of masked language models, such as BERT.
Approach:	They propose a semantic-consistent learning method to improve token dropping by skipping the computation of a subset of input tokens at several middle layers.
Outcome:	The proposed method achieves consistent and significant performance gains across all tasks and model sizes.

Copied to clipboard

Zonghao Ying, Deyue Zhang, Zonglei Jing, Yisong Xiao, Quanchen Zou, Aishan Liu, Siyuan Liang, Xiangzheng Zhang, Xianglong Liu, Dacheng Tao

TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack (2022.emnlp-main)

Challenge:	Existing methods for implementing multi-turn jailbreaks struggle to balance semantic coherence with attack effectiveness, resulting in benign semantic drift or ineffective detection evasion.
Approach:	They propose a framework that reformulates harmful queries into benign reasoning tasks and leverages LLMs’ strong reasoning capabilities to compromise safety alignment.
Outcome:	The proposed framework achieves state-of-the-art attack effectiveness in complex conversational scenarios, with average ASRs increasing by up to 96%.

Copied to clipboard

Yu Cao, Dianqi Li, Meng Fang, Tianyi Zhou, Jun Gao, Yibing Zhan, Dacheng Tao

Revisiting Catastrophic Forgetting in Large Language Model Tuning (2024.findings-emnlp)

Challenge:	Existing adversarial models rely on keyword matching and ignore relevant contextual relations for answer prediction.
Approach:	They propose to use keyword matching to attack model with two biases that rely on a perturbed answer sentence and a distracting answer sentence to misguide model.
Outcome:	The proposed method produces fluent and grammatical adversarial contexts while maintaining gold answers.

Copied to clipboard

Hongyu Li, Liang Ding, Meng Fang, Dacheng Tao

Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking (2023.acl-long)

Challenge:	Catastrophic Forgetting (CF) compromises the effectiveness of large language models during fine-tuning, yet the underlying causes of CF remain largely unexplored.
Approach:	They propose a method to flatten the model loss landscape to mitigate CF by flattening the loss landscape.
Outcome:	The proposed method complements existing anti-forgetting strategies, further enhancing the resistance of LLMs to CF.

Copied to clipboard

Qingyue Wang, Liang Ding, Yanan Cao, Yibing Zhan, Zheng Lin, Shi Wang, Dacheng Tao, Li Guo

Reasoning-Guided Exploration for Online DPO (2026.findings-acl)

Challenge:	Existing methods to enhance the zeroshot generalization of DST fail to effectively decouple semantics of samples, limiting the zero-shot performance of the system.
Approach:	They propose a new learning schema that explicitly disentangles the semantics of seen data and leverages the performance and robustness with the mixture-of-experts mechanism.
Outcome:	The proposed model achieves state-of-the-art on multiWOZ2.1 with 10M trainable parameters and is robust to the mixture-of experts mechanism.

Copied to clipboard

Zetian Hu, Shunyu Liu, Ting-En Lin, Fei Huang, Yongbin Li, Dacheng Tao

SEAD: A Surrogate-free Label-only Membership Inference Attack against Pre-trained LLMs with Semantic-Aware Density (2026.findings-acl)

Challenge:	Recent work has aimed to enhance reasoning capabilities of language models, but these methods are limited to domains with objectively verifiable answers.
Approach:	They propose a self-play framework to improve reasoning on general-domain data.
Outcome:	Experiments show that the proposed framework improves reasoning performance on general-domain data while maintaining competitive performance on verifiable academic benchmarks.

Copied to clipboard

Biao Yi, Jiahao Li, Yiming Li, Yu He, Baolei Zhang, Zheli Liu, Dacheng Tao

RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents (2026.acl-long)

Challenge:	Existing membership inference attacks require access to complete logits, but such access is often unavailable in real-world deployments where only the generated text is exposed.
Approach:	They propose a surrogate-free label-only MIA approach that directly estimates token probabilities through Monte Carlo sampling of the target model.
Outcome:	The proposed approach outperforms existing label-only attacks and serves as a foundational density estimator in the label-exclusive setting.

Copied to clipboard

Jize Wang, Han Wu, Zhiyuan You, Yiming Song, Yijun Wang, Zifei Shan, Yining Li, Songyang Zhang, Xinyi Le, Cailian Chen, Xinping Guan, Dacheng Tao

Token-Level Self-Evolution Training for Sequence-to-Sequence Learning (2023.acl-short)

Challenge:	Existing methods for mixing-of-agents (MoA) lack model selection criteria and struggle with large model pools.
Approach:	They propose a mixture-of-agents framework with dynamic routing that uses a lightweight scorer to perform initial screening and refines the model scores through self- and cross-assessment.
Outcome:	The proposed framework outperforms existing methods for large model pools and tasks . it reduces cost by 89.8% and latency by 63.6% in the large-scale model pool.

Copied to clipboard

Keqin Peng, Liang Ding, Qihuang Zhong, Yuanxin Ouyang, Wenge Rong, Zhang Xiong, Dacheng Tao

Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (2023.emnlp-main)

Challenge:	Adaptive training approaches do not consider the variation of learning difficulty in different training steps, making the learning deterministic and sub-optimal.
Approach:	They propose a dynamic token-level self-evolution training method that reweighs the training losses of different target tokens based on priors.
Outcome:	Empirically, the proposed method yields significant improvements on three translation tasks.

Copied to clipboard

Shwai He, Run-Ze Fan, Liang Ding, Li Shen, Tianyi Zhou, Dacheng Tao

Uncertainty Aware Learning for Language Model Alignment (2024.acl-long)

Challenge:	Extensive experiments show that MEO significantly improves computational efficiency . compared to dense networks, sparsely activated networks only employ a few parameters for each input .
Approach:	They propose a method that merges multiple experts into one to reduce computation costs . they demonstrate that a sparse Mixture of Experts (MoE) can reduce the cost by activating a small subset of parameters for each input .
Outcome:	The proposed approach reduces the computational cost to that of a single expert by 83.3% compared to 82.6% in vanilla MoE.

Copied to clipboard

Yikun Wang, Rui Zheng, Liang Ding, Qi Zhang, Dahua Lin, Dacheng Tao

SAFER: A Controllable Safeguard for LLMs against Backdoor Attacks (2026.findings-acl)

Challenge:	Existing alignment strategies that focus on diverse and high-quality data often overlook the intrinsic uncertainty of tasks, learning all data samples equally.
Approach:	They propose to introduce the sample uncertainty into the alignment of different task scenarios by a simple fashion by setting the label smoothing value of training according to the uncertainty of individual samples.
Outcome:	The proposed model outperforms standard supervised fine-tuning on high-entropy tasks and complex low-entropic tasks.

Copied to clipboard

Zirui Hu, Zheng Zhang, Yingjie Wang, Dacheng Tao

PAD-Net: An Efficient Framework for Dynamic Networks (2023.acl-long)

Challenge:	Existing inference-time defenses lack explicit control over false acceptance rate (FAR) existing inference time defenses aim to mitigate poisoned inputs but lack explicit FAR control .
Approach:	They propose a framework that provides explicit control over false acceptance rate without prior knowledge of backdoor samples.
Outcome:	The proposed framework outperforms existing inference-time defenses on three benchmark datasets . it provides explicit and provable control over false acceptance rate without prior knowledge of backdoor samples .

Copied to clipboard

Shwai He, Liang Ding, Daize Dong, Boan Liu, Fuqiang Yu, Dacheng Tao

Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding (2025.acl-short)

Challenge:	Dynamic networks can significantly improve the model’s representation power with acceptable computational cost.
Approach:	They propose a partially dynamic network to transform redundant dynamic parameters into static ones and iterative mode partition to partition dynamic and static parameters efficiently.
Outcome:	The proposed network surpasses fully dynamic networks by +0.7% top-1 acc with only 30% dynamic parameters for DY-Conv and +1.9% average score in language understanding with only 50% dynamic parameters.

Copied to clipboard

Keqin Peng, Liang Ding, Yuanxin Ouyang, Meng Fang, Yancheng Yuan, Dacheng Tao

Pretrained Language Models for Dialogue Generation with Multiple Input Sources (2020.findings-emnlp)

Challenge:	Prior research has found that large language models overlook input-label mapping information in ICL, relying more on their pre-trained knowledge.
Approach:	They propose a novel method that contrasts input-label mappings between positive and negative in-context examples to improve model performance.
Outcome:	The proposed method improves performance on 7 natural language understanding tasks without additional training.

Copied to clipboard

Yu Cao, Wei Bi, Meng Fang, Dacheng Tao

Dialogue Systems & Conversational Agents

Challenge:	Large-scale pretrained language models have achieved outstanding performance on natural language understanding tasks.
Approach:	They propose to fuse attention information from multiple input sources to achieve better relevance with dialogue history than simple fusion baselines.
Outcome:	The proposed models deliver higher relevance with dialogue history than baselines.

BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering (N19-1)

Copied to clipboard

Yu Cao, Meng Fang, Dacheng Tao

Structured Data in NLP

Challenge:	Existing datasets for question answering and machine comprehension (MC) are limited to a single paragraph, or even part of it.
Approach:	They propose a bi-directional Attention Entity Graph Convolutional Network (BAG) that leverages relationships between nodes in an entity graph and attention information between a query and the entity graph to generate a prediction.
Outcome:	Experimental results show that the proposed network achieves state-of-the-art accuracy on the QAngaroo WIKIHOP dataset.

TransGEC: Improving Grammatical Error Correction with Translationese (2023.findings-acl)

Copied to clipboard

Tao Fang, Xuebo Liu, Derek F. Wong, Runzhe Zhan, Liang Ding, Lidia S. Chao, Dacheng Tao, Min Zhang

KaFT: Knowledge-aware Fine-tuning for Boosting LLMs’ Domain-specific Question-Answering Performance (2025.findings-acl)

Challenge:	Experimental results show that data augmentation improves accuracy over strong baselines.
Approach:	They propose to use translationese as input for GEC data augmentation to overcome stylistic discrepancies . they propose to obtain human-translated texts with a more similar style to non-native texts .
Outcome:	The proposed method improves correction accuracy over strong baselines on four GEC benchmarks.

Copied to clipboard

Qihuang Zhong, Liang Ding, Xiantao Cai, Juhua Liu, Bo Du, Dacheng Tao

Question Answering

Challenge:	Recent literature reveals that supervised fine-tuning (SFT) is suboptimal for domain-specific question-answering tasks.
Approach:	They propose a query diversification strategy for robust conflict detection and a knowledge-aware fine-tuning approach to effectively boost LLMs’ performance.
Outcome:	The proposed approach improves the model generalization and alleviates the hallucination.

GeometryZero: Advancing Geometry Solving via Group Contrastive Policy Optimization (2026.findings-acl)

Copied to clipboard

Yikun Wang, Yibin Wang, Dianyi Wang, Zimian Peng, Qipeng Guo, Dacheng Tao, Jiaqi Wang

Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA (2025.findings-emnlp)

Challenge:	Existing methods for auxiliary construction training are expensive and underperform . Existing Corresponding Author training methods lack self-correction capabilities in reasoning chains.
Approach:	They propose a reinforcement learning framework that rewards auxiliary construction with geometric reasoning by grouping construction rewards with a Length Reward.
Outcome:	Experiments on Geometry3K and MathVista show that GeometryZero outperforms baselines on auxiliary constructions.

Copied to clipboard

Yuchen Wu, Liang Ding, Li Shen, Dacheng Tao

A Contrastive Cross-Channel Data Augmentation Framework for Aspect-Based Sentiment Analysis (2022.coling-1)

Challenge:	Large language models encode vast amounts of knowledge but remain static once trained, making timely integration of emerging facts prohibitively expensive via full retraining.
Approach:	They introduce a reasoning-chain-based editing framework that steers a pretrained LLM through four structured stages to filter distractors in a single pass.
Outcome:	The proposed framework steers a pretrained LLM through four structured stages to filter distractors in a single pass.

Copied to clipboard

Bing Wang, Liang Ding, Qihuang Zhong, Ximing Li, Dacheng Tao

Sentiment Analysis

Challenge:	Aspect-based sentiment analysis is sensitive to multi-aspect challenges, resulting in multiple aspects in a sentence.
Approach:	They propose a framework that leverages an in-domain generator to construct more multi-aspect samples . they then boost the robustness of ABSA models via contrastive learning on these generated samples ."
Outcome:	The proposed framework outperforms baselines without any augmentations on accuracy and Macro- F1 . the proposed framework can generate more multi-aspect samples and boost the robustness of ABSA models .

On the Complementarity between Pre-Training and Random-Initialization for Resource-Rich Machine Translation (2022.coling-1)

Copied to clipboard

Changtong Zan, Liang Ding, Li Shen, Yu Cao, Weifeng Liu, Dacheng Tao

Revisiting Demonstration Selection Strategies in In-Context Learning (2024.acl-long)

Challenge:	Pre-Training (PT) of text representations has been successfully applied to low-resource Neural Machine Translation (NMT) however, it often fails to achieve notable gains on resource-rich NMT on par with its Random-Initialization (RI) counterpart.
Approach:	They propose to combine pre-training and random-initialization techniques to achieve significant improvements in NMT.
Outcome:	The proposed model fusion algorithm can achieve significant improvements on two resource-rich translation benchmarks.

Copied to clipboard

Keqin Peng, Liang Ding, Yancheng Yuan, Xuebo Liu, Min Zhang, Yuanxin Ouyang, Dacheng Tao

Low-Resource NLP

Challenge:	Large language models (LLMs) have shown an impressive ability to perform a wide range of tasks using in-context learning (ICL).
Approach:	They propose a data- and model-dependent method to select models using in-context learning, TopK + ConE, and propose unified explanations for the effectiveness of previous methods.
Outcome:	The proposed method improves language understanding and generation tasks with different model scales.

Controllable Contamination Detection for Reliable LLM Evaluation with Statistical Guarantees (2026.acl-long)

Copied to clipboard

Zheng Zhang, Qi Liu, Siyuan Liang, Ning Li, Zirui Hu, Weibo Gao, Rui Li, Zhenya Huang, Leszek Rutkowski, Baosheng Yu, Dacheng Tao

Toward Human-Like Evaluation for Natural Language Generation with Error Analysis (2023.acl-long)

Challenge:	Existing training data detectors fail to detect clean samples from contaminated test sets . existing methods fail to identify clean samples due to black-box nature of LLMs .
Approach:	They propose a framework that detects and filters contaminated evaluation data . they propose 'failure detection' to reduce the proportion of contaminated samples mistakenly retained .
Outcome:	The proposed framework reduces false discovery rate (FDR) under valid FDR control while maintaining evaluation consistency.

Copied to clipboard

Qingyu Lu, Liang Ding, Liping Xie, Kanjian Zhang, Derek F. Wong, Dacheng Tao

Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning (2025.coling-main)

Challenge:	Pre-trained language models (PLMs) have been used to evaluate language generation tasks . pretrained error analysis can be used to refine the generated sentence toward higher confidence .
Approach:	They propose to combine pretrained language model based metrics with human-like error analysis to improve sentence confidence.
Outcome:	The proposed method outperforms top-scoring metrics in 19/25 settings.

Copied to clipboard

Yanfang Zhang, Yiliu Sun, Yibing Zhan, Dapeng Tao, Dacheng Tao, Chen Gong

Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation (2022.acl-long)

Challenge:	Recent studies have focused on improving the ability of Large Language Models to perform complex reasoning.
Approach:	They propose a Direct-Indirect Reasoning method that integrates DR and IR as parallel reasoning paths that are merged to derive the final answer.
Outcome:	The proposed method outperforms existing methods on four datasets related to logical reasoning and proof.

Copied to clipboard

Liang Ding, Longyue Wang, Shuming Shi, Dacheng Tao, Zhaopeng Tu

VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search (2025.acl-long)

Challenge:	Knowledge distillation (KD) is the preliminary step for training non-autoregressive translation models, but it can lose important information for translating low-frequency words.
Approach:	They propose a knowledge distillation method which trains NAT student on external monolingual data with AT teacher trained on the original bilingual data.
Outcome:	Extensive experiments on eight WMT benchmarks show that monolingual KD outperforms the standard KD by improving low-frequency word translation without introducing any computational cost.

Copied to clipboard

Yikun Wang, Siyin Wang, Qinyuan Cheng, Zhaoye Fei, Liang Ding, Qipeng Guo, Dacheng Tao, Xipeng Qiu

Multimodality

Challenge:	Existing approaches to large vision-language models fail to capture interleaved nature of human visual-verbal reasoning processes.
Approach:	They propose a framework that integrates visuospatial and linguistic domains to facilitate multimodal slow thinking by enabling progressive visual-textual reasoning.
Outcome:	Experiments show that VisuoThink significantly improves reasoning capabilities even without fine-tuning.

Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks (2023.emnlp-main)

Copied to clipboard

Haoqi Zheng, Qihuang Zhong, Liang Ding, Zhiliang Tian, Xin Niu, Changjian Wang, Dongsheng Li, Dacheng Tao

Low-Resource NLP

Challenge:	Text classification tasks often encounter few-shot scenarios with limited labeled data, and addressing data scarcity is crucial.
Approach:	They propose a self-evolution learning (SE) based mixup approach for data augmentation in text classification which generates more adaptive and model-friendly pseudo samples for the model training.
Outcome:	The proposed approach can generate more adaptive and model-friendly pseudo samples for the model training.

ROSE Doesn’t Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding (2024.findings-acl)

Copied to clipboard

Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction (2024.lrec-main)

Challenge:	Existing methods for aligning LLMs output with expected safety require substantial training efforts and expensive computational resources.
Approach:	They propose a method to directly boost the safety of existing instruction-tuned large language models without additional training.
Outcome:	The proposed method improves safety of instruction-tuned large language models without training and requires expensive computational resources.

Copied to clipboard

Ziyang Xu, Keqin Peng, Liang Ding, Dacheng Tao, Xiliang Lu

O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning (2026.findings-acl)

Challenge:	Recent research shows that pre-trained language models suffer from “prompt bias” in factual knowledge extraction.
Approach:	They propose a representation-based approach to mitigate prompt bias during inference time by querying the model and removing it from its internal representations to generate debiased representations.
Outcome:	The proposed approach corrects the overfitted performance caused by prompt bias and significantly improves prompt retrieval capability.

Copied to clipboard

Haotian Luo, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Xiaochun Cao, Dacheng Tao, Naiqiang Tan, Li Shen

MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluators (2025.coling-main)

Challenge:	Recent long-thought reasoning models adopt extended reasoning processes similar to how humans ponder over complex problems.
Approach:	They propose a model that uses RL-style fine-tuning to reduce inference overhead while maintaining accuracy.
Outcome:	The proposed model reduces inference overhead while maintaining accuracy.

Copied to clipboard

Qingyu Lu, Liang Ding, Kanjian Zhang, Jinxia Zhang, Dacheng Tao

Distillation Traps and Guards: A Calibration Knob for LLM Distillability (2026.acl-long)

Challenge:	Large Language Models (LLMs) have shown significant potential as judges for Machine Translation (MT) quality assessment.
Approach:	They propose a framework that automatically post-edits the original translation based on each error, thereby filtering out non-impactful errors.
Outcome:	The proposed framework improves reliability and quality of error spans against GEMBA-MQM, across eight LLMs in both high- and low-resource languages.

Copied to clipboard

Weixiao Zhan, Yongcheng Jing, Leszek Rutkowski, Dacheng Tao

CASN:Class-Aware Score Network for Textual Adversarial Detection (2023.acl-long)

Challenge:	Knowledge distillation (KD) transfers capabilities from large language models (LLMs) to smaller students, yet it can fail unpredictably and also underpins model leakage risks.
Approach:	They propose a method that allows teachers to control their distillability via reinforcement fine-tuning (RFT) they propose to use tail noise, off-policy instability, and the teacher–student gap to improve KD.
Outcome:	The proposed method outperforms SFT and KD baselines and can be used to protect teachers and students from bottlenecks.

Copied to clipboard

Rong Bao, Rui Zheng, Liang Ding, Qi Zhang, Dacheng Tao

Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs (2025.findings-acl)

Challenge:	Existing approaches to counteract adversarial attacks can be divided into two directions, adversarials defense and adversarially detection.
Approach:	They propose a score-based generative method to implicitly model the data distribution using a log-density distribution and supervised contrastive learning to guide the estimation using label information.
Outcome:	The proposed method improves on three text classification tasks on four advanced attack algorithms.

Copied to clipboard

Yuchen Wu, Liang Ding, Li Shen, Dacheng Tao

SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters (2022.findings-emnlp)

Challenge:	Existing methods to update large language models focus on single-language editing or basic multilingual editing, failing to achieve true cross-linguistic knowledge synchronization.
Approach:	They propose a cross-linguistic knowledge democracy edit technique to improve cross-lingual performance.
Outcome:	The proposed method improves cross-lingual performance while maintaining high accuracy in monolingual settings.

Copied to clipboard

Shwai He, Liang Ding, Daize Dong, Jeremy Zhang, Dacheng Tao

PromptST: Abstract Prompt Learning for End-to-End Speech Translation (2023.emnlp-main)

Challenge:	Pretrain-finetuned models are increasingly complex and require more parameters to match the performance of full fine-tuning.
Approach:	They propose an efficient Adapter Tuning technique that freezes pretrained language models and fine-tunes a few extra modules.
Outcome:	The proposed setting outperforms the standard Adapter Tuning by 80% . the proposed setting is easy to use and has a high sparse ratio .

Copied to clipboard

Tengfei Yu, Liang Ding, Xuebo Liu, Kehai Chen, Meishan Zhang, Dacheng Tao, Min Zhang

Powering Verifiable Learning via Automated Evolutionary Data Synthesis (2026.acl-long)

Challenge:	Experimental results show that PromptST can improve speech-to-text translation by capturing richer linguistic knowledge.
Approach:	They propose a plug-in prompt-enhanced S2T model that captures richer linguistic knowledge . they use a 10GB linguistic probing benchmark to investigate the fusion of speech and text features .
Outcome:	The proposed model can improve on a strong baseline by capturing richer linguistic knowledge.

Copied to clipboard

He Du, Bowen Li, Aijun Yang, Siyang He, Qipeng Guo, Kai Chen, Dacheng Tao

A Model-agnostic Data Manipulation Method for Persona-based Dialogue Generation (2022.acl-long)

Challenge:	Existing approaches to building generalizable verifiable data are task-specific and lack a principled, universal evaluator of verifikatability.
Approach:	They propose a task-agnostic, strategy-guided, executably-checkable data synthesis framework that synthesizes problems, diverse candidate solutions and verification artifacts from a single source.
Outcome:	The proposed framework synthesizes problems, candidates, and verification artifacts from human-annotated and strategy-induced checks and iteratively discovers strategies.

Copied to clipboard

Yu Cao, Wei Bi, Meng Fang, Shuming Shi, Dacheng Tao

Dialogue Systems & Conversational Agents

Challenge:	Existing models for introducing explicit personas are expensive due to their expensive collection costs.
Approach:	They propose a data manipulation method which is model-agnostic to be packed with any persona-based dialogue generation model to improve their performance.
Outcome:	The proposed method is model-agnostic to be packed with any persona-based dialogue generation model to improve their performance.

Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation (2021.acl-long)

Copied to clipboard

Liang Ding, Longyue Wang, Xuebo Liu, Derek F. Wong, Dacheng Tao, Zhaopeng Tu

3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset (2024.lrec-main)

Challenge:	Knowledge distillation (KD) is commonly used to construct synthetic data for training non-autoregressive translation models.
Approach:	They propose to use knowledge distillation to generate training data for non-autoregressive translation models by leveraging pretraining.
Outcome:	The proposed approach achieves 28.2 and 33.9 BLEU points on the WMT14 English-German and WMT16 Romanian-English datasets.

Copied to clipboard

Xinyu Ma, Xuebo Liu, Derek F. Wong, Jun Rao, Bei Li, Liang Ding, Lidia S. Chao, Dacheng Tao, Min Zhang

Speech Sense Disambiguation: Tackling Homophone Ambiguity in End-to-End Speech Translation (2024.acl-long)

Challenge:	Existing studies have shown that visual information in existing MMT datasets is insufficient, causing models to disregard it and overestimate their capabilities.
Approach:	They propose to use 3AM to create an ambiguity-aware multimodal machine translation dataset.
Outcome:	The proposed dataset includes more ambiguity and a greater variety of captions and images than other MMT datasets.

Copied to clipboard

Tengfei Yu, Xuebo Liu, Liang Ding, Kehai Chen, Dacheng Tao, Min Zhang

Supervised Optimism Correction: Be Confident When LLMs Are Sure (2025.findings-acl)

Challenge:	End-to-end speech translation (ST) models require simultaneous crossmodal and crosslingual transformations to be effective.
Approach:	They propose a homophone-aware contrastive learning approach that integrates a speech-text masking strategy to reduce ambiguity.
Outcome:	The proposed approach achieves SOTA results on BLEU scores on different MuST-C and CoVoST ST tasks, underlining its effectiveness in reducing speech sense ambiguity.

Copied to clipboard

Junjie Zhang, Rushuai Yang, Shunyu Liu, Ting-En Lin, Fei Huang, Yi Chen, Yongbin Li, Dacheng Tao

Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training (2023.findings-acl)

Challenge:	Recent advances in Large Language Models (LLMs) have demonstrated remarkable success across diverse tasks such as instruction following, code generation, and medical diagnosis.
Approach:	They propose a supervised fine-tuning-based auxiliary loss for Q-value estimations during supervised refinement.
Outcome:	The proposed method outperforms beam search on GSM8K, MATH, and GAOKAO on reasoning benchmarks.

Copied to clipboard

Yibin Lei, Liang Ding, Yu Cao, Changtong Zan, Andrew Yates, Dacheng Tao

ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models (2025.acl-long)

Challenge:	Dense retrievers have impressive performance, but their demand for abundant training data limits their application scenarios.
Approach:	They propose a method which uses unlabeled data to construct pseudo-positive examples from unlabelled data and then contrastively weighs the contrastive loss of different pairs according to the estimated relevance.
Outcome:	The proposed method beats the SOTA unsupervised Contriever model on BEIR and open-domain QA retrieval benchmarks and is a good few-shot learner.

Copied to clipboard

Xuxu Liu, Siyuan Liang, Mengya Han, Yong Luo, Aishan Liu, Xiantao Cai, Zheng He, Dacheng Tao

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit (2024.emnlp-industry)

Challenge:	Existing backdoor models are limited in coverage of attack, system integrity and backdoor alignment . ELBA-Bench provides over 1300 experiments encompassing 12 attack methods, 18 datasets, and 12 LLMs.
Approach:	They propose a framework that allows attackers to inject backdoor through parameter efficient fine-tuning or without fine-uning techniques.
Outcome:	ELBA-Bench provides over 1300 experiments encompassing 12 attack methods, 18 datasets, and 12 LLMs.

Copied to clipboard

Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Dacheng Tao, Xianglong Liu

Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL (2024.findings-emnlp)

Challenge:	Existing quantization techniques have been categorized as 'simple' and 'highly efficient' however, their configurations vary from each other and cannot be fairly compared .
Approach:	They propose a plug-and-play compression toolkit to explore the impact of quantization.
Outcome:	The proposed toolkit explores the impact of quantization on large language models.

Copied to clipboard

Qihuang Zhong, Kunfeng Chen, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

Programming Languages in NLP

Challenge:	Existing text-to-SQL LLMs are computationally expensive and difficult to deploy in real-world applications.
Approach:	They propose to distill a larger teacher model into a smaller student model by using imperfect data to improve the KD.
Outcome:	The proposed method achieves the best tradeoff between performance and efficiency on 5 text-to-SQL benchmarks.

Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments (2025.findings-emnlp)

Copied to clipboard

Qingyu Lu, Liang Ding, Siyi Cao, Xuebo Liu, Kanjian Zhang, Jinxia Zhang, Dacheng Tao

Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models (2024.findings-acl)

Challenge:	Experiments with 4 different LLMs across 5 embodied environments show significant efficiency improvements, with only minor drops in agent performance.
Approach:	They propose an intrinsic method that injects exit instructions during generation and an extransic system that verifies task completion to determine when to halt an agent’s trial.
Outcome:	The proposed method injects exit instructions during generation and an exit method verifies task completion to determine when to halt an agent’s trial.

Copied to clipboard

Qingyu Lu, Baopu Qiu, Liang Ding, Kanjian Zhang, Tom Kocmi, Dacheng Tao

Self-Evolution Learning for Discriminative Language Model Pretraining (2023.findings-acl)

Challenge:	Recent research shows that large language models (LLMs) perform poorly at segment level.
Approach:	They propose a new prompting method that emulates the commonly accepted human evaluation framework . they will release their code and scripts to facilitate the community .
Outcome:	The proposed method is based on the human evaluation framework MQM and produces explainable and reliable MT evaluations at both the system and segment level.

Copied to clipboard

Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models (2025.naacl-long)

Challenge:	Random masking does not consider the importance of the different words in the sentence meaning, e.g., entity-level masking requires expensive prior knowledge and generally does not use existing model weights.
Approach:	They propose a token masking and learning method that uses a random masking strategy to learn the under-explored tokens.
Outcome:	The proposed method improves linguistic knowledge learning and generalization on 10 tasks.

Copied to clipboard

Ying Nie, Binwei Yan, Tianyu Guo, Hao Liu, Haoyu Wang, Wei He, Binfan Zheng, Weihao Wang, Qiang Li, Weijian Sun, Yunhe Wang, Dacheng Tao

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models (2022.findings-emnlp)

Challenge:	Large language models (LLMs) have achieved remarkable performance on various NLP tasks, yet their potential in more challenging task like finance, has not been fully explored.
Approach:	They propose a benchmark to assess the financial knowledge of large language models (LLMs) in China.
Outcome:	The proposed benchmark is the most comprehensive evaluation benchmark to date for LLMs in finance.

Copied to clipboard

Qihuang Zhong, Liang Ding, Li Shen, Peng Mi, Juhua Liu, Bo Du, Dacheng Tao

Towards Making the Most of ChatGPT for Machine Translation (2023.findings-emnlp)

Challenge:	Existing methods for fine-tuning pretrained language models suffer from poor generalization . however, they add a perturbation to each model parameter equally, which is sub-optimal .
Approach:	They propose a sharpness-aware minimization optimization procedure that introduces a Fisher mask to improve the efficiency of SAM.
Outcome:	The proposed method outperforms the vanilla sharpness-aware minimization method on GLUE and SuperGLUE benchmarks.

Copied to clipboard

Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, Dacheng Tao

Context-Aware Cross-Attention for Non-Autoregressive Translation (2020.coling-main)

Challenge:	Prior studies have shown that ChatGPT achieves comparable results to commercial systems for high-resource languages, but lags behind in complex tasks, e.g., low-resourced and distant-language-pairs translation.
Approach:	They propose task-specific prompts and domain-specific prompts which are based on task information and domain information and a task-specific prompt.
Outcome:	The proposed prompts improve the performance of ChatGPT in complex tasks and generate hallucinations for non-English-centric tasks.

Copied to clipboard

Liang Ding, Longyue Wang, Di Wu, Dacheng Tao, Zhaopeng Tu

The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check (2026.acl-long)

Challenge:	Existing studies have shown that non-autoregressive translation models can predict all tokens independently and simultaneously.
Approach:	They propose to enhance signals of neighbour source tokens into conventional cross-attention to address a locality perception problem in NAT cross- attention.
Outcome:	The proposed approach improves translation quality over strong NAT baselines on representative datasets.

Copied to clipboard

Qingyu Lu, Liang Ding, Kanjian Zhang, Jinxia Zhang, Dacheng Tao

Self-Attention with Cross-Lingual Position Representation (2020.acl-main)

Challenge:	Embodied and Tool-Calling agents are effective in planning and complex reasoning, but require causal, precise, and logically grounded reasoning mechanisms to be viable for agentic tasks.
Approach:	They propose a framework that integrates dLLMs as plug-and-play cognitive cores.
Outcome:	The proposed model breaks the sequential latency bottleneck in agentic interactions.

Copied to clipboard

Liang Ding, Longyue Wang, Dacheng Tao