Papers by Zhihua Zhang
MR-P: A Parallel Decoding Algorithm for Iterative Refinement Non-Autoregressive Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Non-autoregressive neural machine translation models remove dependency between tokens in the target sentence and generate all tokens on parallel . |
| Approach: | They propose a non-autoregressive neural machine translation model that decodes with the Mask-Predict algorithm which iteratively refines the output. |
| Outcome: | The proposed algorithm increases the performance of the WMT’14 translation task by 1.39 points. |
Train Once, and Decode As You Like (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to machine translation support autoregressive, semi-autoregressive and refinement-based non-auto-regressives. |
| Approach: | They propose a unified approach for supporting different generation manners of machine translation including autoregressive, semi-autoregressive and refinement-based non-auto-regressives. |
| Outcome: | The proposed approach achieves better or competitive translation performance compared with strong baseline models in all the settings. |
LSTDial: Enhancing Dialogue Generation via Long- and Short-Term Measurement Feedback (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing dialogue systems do not utilize quality dimensions specifically designed for dialogue evaluation to guide the response generation during training. |
| Approach: | They propose a two-stage framework which generates and utilizes conversation evaluation as explicit feedback during training. |
| Outcome: | The proposed framework generates and utilizes conversation evaluation as explicit feedback during training. |
Active Learning Approaches to Enhancing Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a limited human translation budget is required to train neural machine translation models. |
| Approach: | They propose to integrate active learning into neural machine translation techniques . they propose a word frequency based acquisition function and an uncertainty based method . |
| Outcome: | The proposed method outperforms other acquisition functions on a limited human translation budget. |
Con-NAT: Contrastive Non-autoregressive Neural Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation models are autoregressive, which means they predict tokens one by one based on source tokens and previously predicted tokens. |
| Approach: | They propose a conditional masked language model which incorporates contrastive learning into the conditional language model. |
| Outcome: | The proposed model improves on WMT’16 Ro-En translation directions with different data sizes. |
GAP: a Global Adaptive Pruning Method for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing structured pruning methods employ uniform compression rates across network layers, neglecting the varying importance of different network depths. |
| Approach: | They propose a pruning framework that minimizes global capability loss by layer-adaptive pruning rates. |
| Outcome: | The proposed approach achieves comparable performance with state-of-the-art methods at high pruning rates and shows significant advantages at low pruning rates. |
Memory-Efficient Differentiable Transformer Architecture Search (2021.findings-acl)
Copied to clipboard
| Challenge: | Current neural architecture search methods suffer from huge computational cost. |
| Approach: | They propose a reversible recursive backpropagation algorithm that uses the last layer to store the outputs of the network. |
| Outcome: | The proposed algorithm outperforms standard Transformers on three sequence-to-sequence datasets. |
Thinking Clearly, Talking Fast: Concept-Guided Non-Autoregressive Generation for Open-Domain Dialogue Systems (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models with seq2seq framework lack ability to effectively manage concept transitions . lack of concept management strategies might lead to incoherent dialogue due to loosely connected concepts . |
| Approach: | They propose a concept-guided non-autoregressive model for open-domain dialogue generation that learns to identify multiple associated concepts from a conceptual graph and a customized Insertion Transformer to perform concept-directed generation to complete a response. |
| Outcome: | The proposed model outperforms state-of-the-art models in automatic and human evaluations with substantially faster inference speed. |
Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition (2025.acl-long)
Copied to clipboard
Kehua Feng, Keyan Ding, Tan Hongzhi, Kede Ma, Zhihua Wang, Shuangquan Guo, Cheng Yuzhou, Ge Sun, Guozhou Zheng, Qiang Zhang, Huajun Chen
| Challenge: | Existing methods for evaluation of large language models are inefficient and inefficient due to inaccuracy of standard metrics in human perception of text quality and inefficiency in sampling informative test examples. |
| Approach: | They propose a sample-efficient human evaluation method for large language models based on the principle of MAximum Discrepancy (MAD) competition. |
| Outcome: | The proposed method achieves the “golden” ranking of LLMs with a minimum set of input instructions, which in turn reveal their relative strengths and weaknesses. |
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing (2022.acl-long)
Copied to clipboard
Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei
| Challenge: | Existing work shows that pre-trained models can improve in various natural language processing tasks. |
| Approach: | They propose a unified-modal encoder-decoder framework that pre-trains speech-text representations using large-scale unlabeled speech and text data. |
| Outcome: | The proposed framework is superior to existing models on speech-to-text processing tasks. |
Multi-split Reversible Transformers Can Enhance Neural Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Large-scale transformers have been shown to improve neural machine translation performance but training these wider and deeper networks could be extremely memory intensive. |
| Approach: | They propose a multi-split based reversible transformer and a backpropagation algorithm that does not need to store activations for most layers. |
| Outcome: | The proposed model outperforms the vanilla transformer by at least 1.4 BLEU points in three datasets. |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)
Copied to clipboard
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |