Papers by Chao Pang
MoNMT: Modularly Leveraging Monolingual and Bilingual Knowledge for Neural Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models for multi-domain translation tasks only use monolingual data, whereas bilingual data is indispensable for improving the models. |
| Approach: | They propose a modular strategy that facilitates the cooperation of monolingual and bilingual knowledge in translation tasks by avoiding catastrophic forgetting. |
| Outcome: | The proposed model exhibits superior generalization and robustness over the conventional approach. |
Attention-Enhancing Backdoor Attacks Against BERT-based Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks focus on generating stealthy triggers or modifying model weights. |
| Approach: | They propose a Trojan Attention Loss (TAL) which enhances the Trojan behavior by directly manipulating attention patterns. |
| Outcome: | The proposed method improves the effectiveness of the backdoor attacks on different backbone models and tasks. |
LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can process extremely long contexts, requiring efficient inference over extended inputs. |
| Approach: | They propose a model that uses a constant-sized key-value cache to train long-context models. |
| Outcome: | Experimental results show that LongSpec achieves 3.26x speedup over strong Flash Attention baselines and 2.34x wall clock time on four math reasoning tasks. |
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for pretraining cross-lingual models are limited in their size due to the limited amount of parallel corpora. |
| Approach: | They propose a method that encourages the model to align multiple languages with monolingual corpora to overcome the constraint of the parallel corpus size. |
| Outcome: | The proposed method outperforms existing cross-lingual models and delivers new state-of-the-art results in various cross-linguistic downstream tasks. |
Self-Generated Critiques Boost Reward Modeling for Language Models (2025.naacl-long)
Copied to clipboard
Yue Yu, Zhengxing Chen, Aston Zhang, Liang Tan, Chenguang Zhu, Richard Yuanzhe Pang, Yundi Qian, Xuewei Wang, Suchin Gururangan, Chao Zhang, Melanie Kambadur, Dhruv Mahajan, Rui Hou
| Challenge: | Existing reward models produce scalar scores and struggle to incorporate critiques in a natural language format. |
| Approach: | They propose a framework that predicts critiques and rewards using self-generated critiques without extra supervision. |
| Outcome: | The proposed framework improves reward modeling accuracy by 3.7%-7.3% compared to standard reward models and LLM judges. |
Task-Agnostic Detector for Insertion-Based Backdoor Attacks (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for textual backdoor detection are task-specific and less effective beyond sentence classification. |
| Approach: | They propose a task-agnostic method for backdoor detection that leverages final layer logits and an efficient pooling technique. |
| Outcome: | TABDet can jointly learn from diverse task-specific models, demonstrating superior detection efficacy over traditional methods. |
METNet: A Mutual Enhanced Transformation Network for Aspect-based Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for learning complex sentences with multiple aspects are ill-equipped to learn complex sentences . |
| Approach: | They propose a mutual enhanced transformation network for the ABSA task . it improves representation learning of the aspect with contextual semantic features . |
| Outcome: | The proposed model improves representation learning of the aspect with contextual semantic features, giving the aspect more abundant information. |
Correcting Chinese Spelling Errors with Phonetic Pre-training (2021.findings-acl)
Copied to clipboard
Ruiqing Zhang, Chao Pang, Chuanqiang Zhang, Shuohuan Wang, Zhongjun He, Yu Sun, Hua Wu, Haifeng Wang
| Challenge: | Existing methods for Chinese spelling correction only use pre-trained language model or incorporate phonological information as external knowledge. |
| Approach: | They propose a phonetic Chinese spelling correction model that integrates phonetic features into language model by leveraging pre-training and fine-tuning methods. |
| Outcome: | The proposed model outperforms existing methods on SIGHAN datasets and improves on other datasets. |
PEDNet: A Persona Enhanced Dual Alternating Learning Network for Conversational Response Generation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing persona-based dialogue models generate personalized responses using predefined persona information, but they lack personality. |
| Approach: | They propose a persona-based dual Alternating Learning Network that generates personalized responses using predefined persona information. |
| Outcome: | The proposed method produces more personalized responses than baseline methods. |
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | ERNIE-Code is a unified pre-trained language model for 116 NLs and 6 PLs. |
| Approach: | They propose a unified pre-trained language model for 116 NLs and 6 PLs . they employ span-corruption language modeling that learns patterns from monolingual NL or PL . |
| Outcome: | The proposed model outperforms previous multilingual models for NL or NL across end tasks. |