Papers by Haifeng Zhang
Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty Estimation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent pre-trained language models have achieved remarkable zero-shot performance . we propose a self-learning framework that utilizes unlabeled data of target languages . |
| Approach: | They propose a self-learning framework that utilizes unlabeled data of target languages to select silver labels for cross-lingual transfer tasks. |
| Outcome: | The proposed framework outperforms baseline models on two cross-lingual tasks by 10 F1 on average and 2.5 accuracy on natural language inference (NLI). |
CORD: Bridging the Audio–Text Reasoning Gap via Weighted On-policy Cross-modal Distillation (2026.findings-acl)
Copied to clipboard
Hu Jing, Danxiang Zhu, Xianlong Luo, Dan Zhang, Shuwei He, Yishu Lei, Shikun Feng, Hai-Tao Zheng, Jingzhou HE, Yu Sun, Hua Wu, Haifeng Wang
| Challenge: | Large Audio Language Models (LALMs) exhibit a degradation in knowledge and reasoning capabilities . empirical results show that CORD significantly bridges the audio–text performance gap . |
| Approach: | They propose a framework that performs online cross-modal self-distillation to bridge the acoustic-semantic gap between LALMs and text-based models. |
| Outcome: | The proposed framework bridges the acoustic-semantic gap between LALMs and text-based models . it employs on-policy reverse KL divergence with importance-aware weighting . |
Rethinking Smoothness for Fast and Adaptable Entity Alignment Decoding (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for integrating knowledge graphs rely on entity and relation embeddings . Fig. 1 shows how to decode knowledge graph in under 6 seconds . |
| Approach: | They propose a framework that only utilizes entity embeddings to decode knowledge graphs. |
| Outcome: | The proposed framework reconstructs KG representation by maximizing smoothness of entity embeddings. |
Uncertainty-Aware Routing for Principled Alignment with MoE Dynamics (2026.acl-long)
Copied to clipboard
Yilong Chen, Junyuan Shang, Yuchen Feng, Zhenyu Zhang, Naibin Gu, Ziqi Wang, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang
| Challenge: | Mixture-of-Experts (MoE) is a cornerstone for scaling LLMs, yet its training dynamics remain poorly understood, often leading to sub-optimal specialization. |
| Approach: | They propose to use Helmholtz Free Energy and Router Entropy to study the MoE lifecycle and identify a universal Three-Stage Phase Transition . |
| Outcome: | The proposed model reduces perplexity and improves expert distinctiveness, offering a principled path toward thermodynamically aligned computation. |
VALU: A Benchmark for Video Anomaly Temporal Localization and Understanding at Multiple Semantic Levels (2026.acl-long)
Copied to clipboard
Yixiao He, Menghao Zhang, Haifeng Sun, Jing Wang, Kangheng Lin, Jinghan Wang, Chenye Xu, Pengfei Ren, Qi Qi, Jingyu Wang
| Challenge: | Recent advances in Video Large Language Models (Video-LLMs) enhance the ability of VAU models to describe and interpret anomalies. |
| Approach: | They propose a benchmark that explicitly defines anomalies across five semantic levels and provides detailed temporal boundaries and detailed textual descriptions for each. |
| Outcome: | The proposed benchmark defines anomalies across five semantic levels and provides detailed descriptions for each. |
PLATO-Ad: A Unified Advertisement Text Generation Framework with Multi-Task Prompt Learning (2022.emnlp-industry)
Copied to clipboard
Zeyang Lei, Chao Zhang, Xinchao Xu, Wenquan Wu, Zheng-yu Niu, Hua Wu, Haifeng Wang, Yi Yang, Shuanglong Li
| Challenge: | Online advertisement text generation models have achieved remarkable success in generating high-quality text ads, but some challenges remain, such as low-resource scenarios and training efficiency for multiple ad tasks. |
| Approach: | They propose a unified text ad generation framework with multi-task prompt learning to tackle low-resource ade generation problem and a multi-step prompt learning mechanism to efficiently solve multiple aed generation tasks. |
| Outcome: | The proposed framework outperforms the state-of-the-art on offline and online metrics. |
ClusterAttn: KV Cache Compression under Intrinsic Attention Clustering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for sparse attention apply the same pattern across different attention heads and inputs, but fail to capture the intrinsic attention clustering in large language models. |
| Approach: | They propose a training-free sparse attention method that provides an efficient prompt cache compression scheme under intrinsic attention clustering for efficient LLM inference. |
| Outcome: | The proposed method reduces memory usage by 10%–65% and increases throughput by 2.6–4.8 times with no accuracy loss. |
Learning Adaptive Segmentation Policy for Simultaneous Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that adaptive segmentation policies for simultaneous translation are more accurate than current methods . if translation starts before adequate source content is delivered, the quality of translation degrades . waiting for too much source text increases latency, which would hurt accuracy . |
| Approach: | They propose a new adaptive segmentation policy for simultaneous translation based on human interpreters . it learns to segment the source text by considering possible translations produced by the translation model . |
| Outcome: | Experimental results show that the proposed method achieves better accuracy-latency trade-off over state-of-the-art methods. |
SSS: Editing Factual Knowledge in Language Models towards Semantic Sparse Space (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to modify LMs suffer from sub-optimal locality, where irrelevant neighborhood examples can be adversely influenced. |
| Approach: | They propose to use a model editing method to modify specific examples in LMs to improve locality and reasoning capability by directing the hidden state of edit example towards spaces where semantics are sparse. |
| Outcome: | The proposed method improves locality and reasoning capability on two datasets. |
DuReadervis: A Chinese Dataset for Open-domain Document Visual Question Answering (2022.findings-acl)
Copied to clipboard
| Challenge: | Open-domain question answering is a task that requires answering questions based on a collection of document images. |
| Approach: | They propose to use document images to answer questions using layouts and visual features instead of text. |
| Outcome: | The proposed approach reduces human cost and improves scalability of QA systems by incorporating layouts and visual features. |
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework (P19-1)
Copied to clipboard
Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, Haifeng Wang
| Challenge: | Simultaneous translation is notoriously dif- ficult due to word-order differences. |
| Approach: | They propose a prefix-to-prefix framework that implicitly learns to anticipate in a single translation model. |
| Outcome: | The proposed framework achieves low latency and reasonable qual- ity on 4 directions. |
Unsupervised Concept Representation Learning for Length-Varying Text Similarity (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing document similarity approaches suffer from the information gap caused by context and vocabulary mismatches when comparing varying-length texts. |
| Approach: | They propose an unsupervised concept representation learning approach to address this issue . they propose a concept-based document matching method to leverage recognition of local phrase features . |
| Outcome: | The proposed method achieves a better F1 score than baseline models on real-world data sets. |
Learning Adaptive Segmentation Policy for End-to-End Simultaneous Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to perform simultaneous speech-to-text translation ignore contextual information and suffer from low translation quality. |
| Approach: | They propose an adaptive segmentation policy for simultaneous speech-to-text translation . it learns to segment the source streaming speech into meaningful units . |
| Outcome: | The proposed method achieves a good accuracy-latency trade-off over state-of-the-art methods on English-German and Chinese-English. |
Mind the Gap in Cultural Alignment: Task-Aware Culture Management for Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing cultural alignment approaches fail to align LLMs’ broad cultural values with the specific goals of downstream tasks and suffer from cross-cultural interference. |
| Approach: | They propose a novel pipeline for task-specific cultural alignment that synthesizes task-aware cultural data in line with target task formats. |
| Outcome: | Experiments across five national cultures and ten culture-sensitive tasks show consistent improvements over prompt-based and fine-tuning baselines. |
Proactive Human-Machine Conversation with Explicit Conversation Goal (P19-1)
Copied to clipboard
| Challenge: | Typical human-machine conversation systems only use utterances and responses as training data, which results in uninformative and inappropriate responses. |
| Approach: | They propose a dataset where one acts as a conversation leader and the other as 'follower' they establish baseline results on a 270K utterances and 30k dialogues dataset using state-of-the-art models. |
| Outcome: | The proposed model can generate diverse multi-turn conversations using knowledge from a new dataset . |
Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotation (2026.findings-eacl)
Copied to clipboard
Minhua Lin, Zhengzhang Chen, Yanchi Liu, Xujiang Zhao, Zongyu Wu, Junxiang Wang, Xiang Zhang, Suhang Wang, Haifeng Chen
| Challenge: | Time series data is ubiquitous across various domains, including manufacturing, finance, and healthcare. |
| Approach: | They propose a multi-agent system to generate general and domain-specific annotations for time series data. |
| Outcome: | The proposed system outperforms existing methods on synthetic and real-world datasets. |
Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking (2025.acl-long)
Copied to clipboard
Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yanxi Xie, Jiawei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang
| Challenge: | Large language models face inherent performance bottlenecks under parameter constraints . challenging tokens induce abrupt gradient spikes across layers, exposing stress points . |
| Approach: | They propose an inner thinking transformer that reimagines layer computations as implicit thinking steps. |
| Outcome: | Empirical results show that ITT outperforms Transformer/Loop variants in 11 benchmarks. |
A Speaker-Aware Co-Attention Framework for Medical Dialogue Information Extraction (2022.emnlp-main)
Copied to clipboard
Yuan Xia, Zhenhui Shi, Jingbo Zhou, Jiayu Xu, Chao Lu, Yehui Yang, Lei Wang, Haifeng Huang, Xia Zhang, Junwei Liu
| Challenge: | With the development of medical digitization, the extraction and structuring of electronic medical records (EMRs) have become challenging but fundamental tasks. |
| Approach: | They propose a speaker-aware dialogue encoder with multi-task learning which takes the speaker's identity into account and a co-attention fusion network to aggregate the utterance information. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on the public medical dialogue extraction datasets to demonstrate its superiority. |
Pruning as a Domain-specific LLM Extractor (2024.findings-naacl)
Copied to clipboard
Nan Zhang, Yanchi Liu, Xujiang Zhao, Wei Cheng, Runxue Bao, Rui Zhang, Prasenjit Mitra, Haifeng Chen
| Challenge: | Large Language Models (LLMs) have exhibited remarkable proficiency across a wide array of NLP tasks. |
| Approach: | They propose a method for pruning large language models using general or task-specific weights to extract a compressed, task-agnostic LLM. |
| Outcome: | The proposed method extracts a compressed, domain-specific, and task- agnostic LLM by identifying LLM weights that are pivotal for general capabilities, like linguistic capability and multi-task solving, and domain- specific knowledge. |
One Token Is Enough: Improving Diffusion Language Models with a Sink Token (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Diffusion Language Models lack a structural constraint to stabilize attention sinks. |
| Approach: | They propose a simple but effective extra sink token that is constrained to attend to itself while remaining globally visible to all other tokens. |
| Outcome: | The proposed token is able to stabilize attention sinks and improve model performance. |
MoE Adapter for Large Audio Language Models: Sparsity, Disentanglement, and Gradient-Conflict-Free (2026.findings-acl)
Copied to clipboard
Yishu Lei, Shuwei He, Hu Jing, Dan Zhang, Xianlong Luo, Danxiang Zhu, Shikun Feng, Rui Liu, Jingzhou HE, Yu Sun, Hua Wu, Haifeng Wang
| Challenge: | Existing research on Large Language Models (LLMs) limited to textual input modality . acoustic information is intrinsically heterogeneous, entangling attributes such as speech, music, and environmental context. |
| Approach: | They propose a sparse Mixture-of-Experts architecture to decouple acoustic information by routing audio tokens to specialized experts. |
| Outcome: | The proposed architecture outperforms existing models on audio semantic and paralinguistic tasks while retaining shared experts for global context. |
Open-ended Commonsense Reasoning with Unrestricted Answer Candidates (2023.findings-emnlp)
Copied to clipboard
Chen Ling, Xuchao Zhang, Xujiang Zhao, Yanchi Liu, Wei Cheng, Mika Oishi, Takao Osaki, Katsushi Matsuda, Haifeng Chen, Liang Zhao
| Challenge: | Current approaches to commonsense reasoning are limited due to limited answer scope. |
| Approach: | They propose to solve a commonsense question without a pre-defined answer scope . they leverage pre-trained language models to iteratively retrieve reasoning paths on the external knowledge base . |
| Outcome: | The proposed method achieves better performance on two commonsense benchmark datasets. |
CodeFort: Robust Training for Code Generation Models (2024.findings-emnlp)
Copied to clipboard
Yuhao Zhang, Shiqi Wang, Haifeng Qian, Zijian Wang, Mingyue Shang, Linbo Liu, Sanjay Krishna Gouda, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras
| Challenge: | Existing research efforts to improve code generation models are inadequate . code generation model performance is degraded under small perturbations . |
| Approach: | They propose a framework to improve the robustness of code generation models by generalizing code perturbations to enrich training data and enabling various robust training strategies. |
| Outcome: | The proposed framework increases pass rates and robustness drop rate against code-syntax perturbations. |
Correcting Chinese Spelling Errors with Phonetic Pre-training (2021.findings-acl)
Copied to clipboard
Ruiqing Zhang, Chao Pang, Chuanqiang Zhang, Shuohuan Wang, Zhongjun He, Yu Sun, Hua Wu, Haifeng Wang
| Challenge: | Existing methods for Chinese spelling correction only use pre-trained language model or incorporate phonological information as external knowledge. |
| Approach: | They propose a phonetic Chinese spelling correction model that integrates phonetic features into language model by leveraging pre-training and fine-tuning methods. |
| Outcome: | The proposed model outperforms existing methods on SIGHAN datasets and improves on other datasets. |
D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension (D19-58)
Copied to clipboard
Hongyu Li, Xiyuan Zhang, Yibing Liu, Yiming Zhang, Quan Wang, Xiangyang Zhou, Jing Liu, Hua Wu, Haifeng Wang
| Challenge: | MRC requires machines to understand text and answer questions about the text. |
| Approach: | They propose a simple system Baidu submitted for MRQA 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models. |
| Outcome: | The proposed system is ranked at top 1 of all participants in terms of averaged F1 score. |
Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Experimental results on multilingual similarity search and bitext mining tasks show the effectiveness of our approach. |
| Approach: | They propose a multilingual sentence representation model that aligns different languages in a shared representation space. |
| Outcome: | The proposed model performs better than LASER3 on similarity searches and bitext mining tasks. |
Uncertainty Quantification for In-Context Learning of Large Language Models (2024.naacl-long)
Copied to clipboard
Chen Ling, Xujiang Zhao, Xuchao Zhang, Wei Cheng, Yanchi Liu, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Jie Ji, Guangji Bai, Liang Zhao, Haifeng Chen
| Challenge: | Existing studies on in-context learning have focused on quantifying the uncertainty associated with the model's response, but they neglect the complexity of the LLM and the uniqueness of in-constitut learning. |
| Approach: | They propose a method to quantify the uncertainty associated with in-context learning and propose corresponding estimation method to quantify both types of uncertainties. |
| Outcome: | The proposed method offers an unsupervised way to understand the prediction of in-context learning in a plug-and-play fashion. |
XDailyDialog: A Multilingual Parallel Dialogue Corpus (2023.acl-long)
Copied to clipboard
Zeming Liu, Ping Nie, Jie Cai, Haifeng Wang, Zheng-Yu Niu, Peng Zhang, Mrinmaya Sachan, Kaiping Peng
| Challenge: | Existing datasets for open-domain dialogue modeling limited to a single language . absence of multilingual datasets hinders development of robust open- domain dialog systems . |
| Approach: | They propose a multilingual parallel open-domain dialog dataset to explore multilingual and cross-lingual open- domain dialog. |
| Outcome: | The proposed model can be used to explore multilingual and cross-lingual open-domain dialogs in other languages. |
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve zero-shot translation performance by learning language-agnostic representations and maximizing cross-lingual transfer have been proposed. |
| Approach: | They propose a cross-lingual consistency regularization to bridge the representation gap between different languages and boost zero-shot translation performance. |
| Outcome: | The proposed model improves translation performance on low-resource and high-res benchmarks and closes the sentence representation gap and aligns the representation space. |
DuSQL: A Large-Scale and Pragmatic Chinese Text-to-SQL Dataset (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL parsing methods mainly focus on English, but there is no labeled data available for the language . a larges-scale and pragmatic Chinese dataset is used for cross-domain text- to-Sql task . |
| Approach: | They propose a larges-scale Chinese dataset for a cross-domain text-to-SQL task . they analyze questions from several representative applications and use an effective data construction framework . |
| Outcome: | The proposed dataset contains 200 databases, 813 tables, and 23,797 question/SQL pairs. |
Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to parse text-to-SQL data are lacking labeled data for unseen evaluation databases. |
| Approach: | They propose a framework for enhancing SQL queries by automatically producing large numbers of SQL queries based on an abstract syntax tree grammar. |
| Outcome: | The proposed framework can produce high-quality natural language questions over strong baselines. |
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for speech-to-text translation (ST) have achieved impressive supervised and zero-shot performance. |
| Approach: | They propose to use consistency regularization methods to boost end-to-end (E2E) speech-totext translation (ST) by regularizing the intra-modal consistency instead of the modality gap. |
| Outcome: | The proposed training strategies achieve state-of-the-art (SOTA) performance in most translation directions. |
Sparse Growing Transformer: Training-Time Sparse Depth Allocation via Progressive Attention Looping (2026.findings-acl)
Copied to clipboard
Yao Chen, Yilong Chen, Yinqi Yang, Junyuan Shang, Zhenyu Zhang, Zefeng Zhang, Shuaiyi Nie, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang, Tingwen Liu
| Challenge: | Existing approaches to increasing effective depth of LLMs rely on parameter reuse, extending computation through recursive execution. |
| Approach: | They propose a training-time sparse depth allocation framework that progressively increases depth for a small subset of parameters as training evolves. |
| Outcome: | The proposed model outperforms existing approaches to increasing the effective depth of language models while reducing training FLOPs overhead from approximately 16–20% to only 1–3% relative to a standard Transformer backbone. |
BeamLoRA: Beam-Constraint Low-Rank Adaptation (2025.acl-long)
Copied to clipboard
Naibin Gu, Zhenyu Zhang, Xiyu Liu, Peng Fu, Zheng Lin, Shuohuan Wang, Yu Sun, Hua Wu, Weiping Wang, Haifeng Wang
| Challenge: | Low-Rank Adaptation (LoRA) is one of the most efficient parameter-efficient fine-tuning methods. |
| Approach: | They propose to conceptualize each LoRA module as a beam where each rank corresponds to a potential sub-solution. |
| Outcome: | The proposed method improves performance on three base models and 12 datasets. |
Dual Activation-Weight Sparsity: A Training-Free Framework for Efficient Large Language Model Compression (2026.acl-long)
Copied to clipboard
Luoyang Sun, Guangyan Li, Cheng Deng, Haifeng Zhang, Jian Zhao, Yongqiang Tang, Wensheng Zhang, Jun Wang
| Challenge: | Large language models (LLMs) excel at natural language tasks but face deployment bottlenecks due to computational demands. |
| Approach: | They propose a training-free framework that exploits activation and weight sparsity . they use a three-tier routing strategy that uses magnitude-based pruning . |
| Outcome: | Experiments on Llama and Mistral models show that DAWS outperforms activation-weight sparsity pruning methods. |
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to model coarse-grained linguistic information do not integrate coarse-gram information into pre-training. |
| Approach: | They propose an explicitly n-gram masking method to enhance integration of coarse-grained linguistic information into pre-training. |
| Outcome: | The proposed method outperforms existing models on English and Chinese text corpora and fine-tunes on 19 downstream tasks. |
Data-Efficiently Learn Large Language Model for Universal 3D Scene Perception (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for 3D scene understanding are limited to specific downstream tasks, hindering their practicality in real-world applications. |
| Approach: | They propose a 3D visual perceptual ability and advanced reasoning capabilities for 3D scenes by aligning 3D representations into the feature space of advanced LLMs. |
| Outcome: | The proposed system achieves a 82.2% relative score compared with state-of-the-art methods with limited data. |
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding (2022.findings-emnlp)
Copied to clipboard
Qiming Peng, Yinxu Pan, Wenjin Wang, Bin Luo, Zhenyu Zhang, Zhengjie Huang, Yuhui Cao, Weichong Yin, Yongfeng Chen, Yin Zhang, Shikun Feng, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang
| Challenge: | Existing methods for visually rich document understanding lack layout-centered knowledge . experimental results show that ERNIE-Layout improves layout awareness . |
| Approach: | They propose a document pre-training solution with layout knowledge enhancement in the whole workflow to learn better representations that combine the features from text, layout, and image. |
| Outcome: | The proposed model outperforms existing models on key downstream tasks. |
LOKA: Conflict-Aware LLM Knowledge Update with Adaptive Knowledge Memory (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches that tackle unlearning and learning separately encounter *task conflicts* and *knowledge management issues* when applied to comprehensive knowledge updates. |
| Approach: | They propose a conflict-aware framework for Large language mOdel Knowledge updAtes that integrates updated knowledge across multiple memory units during training and integrates it with original LLM. |
| Outcome: | The proposed framework is based on theoretical analysis and empirical evidence and validates the proposed framework with empirical and theoretical evidence. |
Example Quality Matters: Multi-Aspects Example Augmentation for Private Library Programming (2026.acl-long)
Copied to clipboard
Yuhao Li, Haifeng Sun, Xuesong Zhang, Shu Yao, Haoyu Zheng, Yvchuan Wang, Huazheng Wang, Zirui Zhuang, Qi Qi, Jianxin Liao, Jingyu Wang
| Challenge: | Existing approaches to code generation fail to consider the quality of retrieved examples. |
| Approach: | They propose a retrieval-augmented generation method that combines existing API examples to improve complexity and readability. |
| Outcome: | The proposed method achieves up to 22% accuracy improvement over baseline methods. |