Papers by Pengcheng Yang
Context-Interactive Pre-Training for Document Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document machine translation typically suffers from a lack of document-level bilingual data. |
| Approach: | They propose a document machine translation model that incorporates contextual information into the training signals by capturing cross-sentence dependency within the target document and cross sentence translation to make better use of contextual information. |
| Outcome: | The proposed model outperforms baselines on three benchmark datasets and significantly outperformed previous approaches. |
Learning Sentiment Memories for Sentiment Modification without Parallel Data (D18-1)
Copied to clipboard
| Challenge: | Existing methods for sentiment modification generate input-irrelevant texts due to lack of parallel data. |
| Approach: | They propose a method that automatically extracts appropriate sentiment information from learned sentiment memories according to the specific context. |
| Outcome: | The proposed method significantly improves the content preservation degree and achieves the state-of-the-art performance. |
Cross-Modal Commentator: Automatic Machine Commenting Based on Cross-Modal Information (P19-1)
Copied to clipboard
| Challenge: | Existing work on commenting based on textual content is focused on other modalities, such as graphics and images. |
| Approach: | They propose a task to integrate multiple modalities into automatic commenting . they construct a large-scale dataset and propose 'co-attention' model to capture dependency between textual and visual information. |
| Outcome: | The proposed model can achieve better performance than baselines. |
Towards Fine-grained Text Sentiment Transfer (P19-1)
Copied to clipboard
| Challenge: | Existing methods for fine-grained text sentiment transfer only reverse the sentiment polarity of text, but they lack a robust and parallel learning algorithm. |
| Approach: | They propose a novel fine-grained text sentiment transfer task that revises a sequence to satisfy a given sentiment intensity while preserving the original semantic content. |
| Outcome: | The proposed model outperforms existing methods by a large margin in automatic evaluation and human evaluation. |
Enhancing Topic-to-Essay Generation with External Commonsense Knowledge (P19-1)
Copied to clipboard
| Challenge: | Existing methods for topic-to-essay generation are insufficient for generating novel, diverse, and topic-consistent paragraph-level text with a set of topics. |
| Approach: | They propose to integrate commonsense from external knowledge base into the generator through dynamic memory mechanism and adversarial training to further improve topic-consistency. |
| Outcome: | The proposed task is more novel, diverse, and topic-consistent than existing methods in terms of both automatic and human evaluation. |
Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization (2023.acl-long)
Copied to clipboard
Pengcheng He, Baolin Peng, Song Wang, Yang Liu, Ruochen Xu, Hany Hassan, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang
| Challenge: | Z-Code++ is a pre-trained language model optimized for abstractive text summarization. |
| Approach: | They propose a pre-trained language model optimized for abstractive text summarization that uses a two-phase pre-training technique to improve model's performance. |
| Outcome: | The proposed model outperforms the competing models on low-resource summarization tasks in zero-shot and few-shot settings. |
Automatic Generation of Personalized Comment Based on User Profile (P19-2)
Copied to clipboard
| Challenge: | Experimental results show that our model can generate natural, human-like and personalized comments. |
| Approach: | They propose a model that takes user profile into account when generating comments on social media and integrates it with a gated memory. |
| Outcome: | The proposed model can generate natural, human-like and personalized comments on social media. |
Asking Clarification Questions in Knowledge-Based Question Answering (D19-1)
Copied to clipboard
| Challenge: | Existing clarification datasets with limited annotated examples do not address ambiguous phenomena. |
| Approach: | They propose a dataset that allows users to ask clarification questions using open-domain examples. |
| Outcome: | The proposed model achieves better performance than strong baselines and provides new challenges. |
Towards Comprehensive Description Generation from Factual Attribute-value Tables (P19-1)
Copied to clipboard
| Challenge: | Existing models for comprehensive descriptions for factual attribute-value tables might suffer from missing key attributes and groundless information problems. |
| Approach: | They propose a force attention method to encourage the generator to pay more attention to uncovered attributes to avoid potential key attributes missing. |
| Outcome: | The proposed model outperforms the state-of-the-art baselines on automatic and human evaluation. |
Multi-Granularity Contrasting for Cross-Lingual Pre-Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to pre-training focus on embedding alignment, but they neglect the modeling of bidirectional contexts. |
| Approach: | They propose a framework to learn languageuniversal representations using multi-granularity contrasting framework . they encode semantic equivalents from different languages into similar representations . |
| Outcome: | The proposed framework can achieve significant performance gains in machine translation and cross-lingual language understanding. |
Pun-GAN: Generative Adversarial Network for Pun Generation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for generating pun sentences with word senses lack large-scale corpus for supervised learning . a pun is a clever and amusing use of a word with two meanings (word senses) |
| Approach: | They propose an adversarial generative network for pun generation with a generator and a discriminator to distinguish between generated pun sentences and real sentences with specific word senses. |
| Outcome: | The proposed network generates sentences that are more ambiguous and diverse in both automatic and human evaluation. |
SafeMT: Multi-turn Safety for Multimodal Language Models (2026.acl-long)
Copied to clipboard
Han Zhu, Juntao Dai, Jiaming Ji, Haoran Li, Chengkun Cai, Pengcheng Wen, Chi-Min Chan, Boyuan Chen, Yaodong Yang, Sirui Han, Yike Guo
| Challenge: | Multi-turn dialogues pose a greater risk than single prompts, but existing safety benchmarks do not account for this situation. |
| Approach: | They propose a benchmark that features dialogues of varying lengths generated from harmful queries accompanied by images. |
| Outcome: | The proposed model reduces multi-turn Attack Success Rate (ASR) compared to existing guard models. |
Boosting Policy and Process Reward Models with Monte Carlo Tree Search in Open-Domain QA (2025.findings-acl)
Copied to clipboard
Chi-Min Chan, Chunpu Xu, Junqi Zhu, Jiaming Ji, Donghai Hong, Pengcheng Wen, Chunyang Jiang, Zhen Ye, Yaodong Yang, Wei Xue, Sirui Han, Yike Guo
| Challenge: | Experimental results show that our approach can effectively improve the performance of both the policy model and the reward model. |
| Approach: | They propose to use Monte Carlo Tree Search for both policy model improvement and reward model improvement to bridge it to more subtle open-domain question answering. |
| Outcome: | The proposed approach surpasses existing methods for annotation and training data with fewer data points and achieves better performance in test-time scaling strategies. |
Towards Robust Evidence-Aware Fake News Detection via Improving Semantic Perception (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods lack sufficient semantic perception and are easily blinded by textual expressions. |
| Approach: | They propose a model-agnostic training framework to improve the semantic perception of evidence-aware fake news detection by combining two kinds of data augmentations with synthetic data. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on the extended test set while achieving competitive performance on the original one. |
SGM: Sequence Generation Model for Multi-label Classification (C18-1)
Copied to clipboard
| Challenge: | Existing methods ignore the correlations between labels and different parts of the text can contribute differently for predicting different labels. |
| Approach: | They propose to view the multi-label classification task as a sequence generation problem and apply a decoder-based sequence generation model to solve it. |
| Outcome: | The proposed methods outperform previous work by a substantial margin. |
Specificity-Driven Cascading Approach for Unsupervised Sentiment Modification (D19-1)
Copied to clipboard
| Challenge: | Existing methods for unsupervised sentiment modification lack specific information in text generated without parallel data . specificity-driven cascading approach can improve specificity of generated text and content preservation . |
| Approach: | They propose a specificity-driven cascading approach for unsupervised sentiment modification . the method performs target sentiment addition and content reconstruction independently . |
| Outcome: | The proposed method outperforms competitive systems by a large margin on Yelp and Amazon datasets. |
Key Fact as Pivot: A Two-Stage Model for Low Resource Table-to-Text Generation (P19-1)
Copied to clipboard
| Challenge: | Existing methods for table-to-text generation use encoder-decoder framework, but lack of large parallel data is a problem for many domains. |
| Approach: | They propose a model to separate table-to-text generation into two stages: key fact prediction and surface realization. |
| Outcome: | The proposed model achieves 27.34 BLEU score with only 1,000 parallel data, while the baseline model only achieves 9.71 BLUE score. |
Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification (D18-1)
Copied to clipboard
| Challenge: | a novel model for multi-label text classification is proposed for the task of assigning multiple labels for a given text. |
| Approach: | They propose a novel model for multi-label text classification based on sequence-to-sequence learning and a hybrid attention mechanism that extracts both the word-level and the semantic unit. |
| Outcome: | The proposed model is competitive to the baseline models and more robust to classifying low-frequency labels. |
Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work has treated procedures as shallow structures without modeling the parent-child relation. |
| Approach: | They propose to construct an open-domain hierarchical knowledge-base (KB) of procedures based on wikiHow . they link steps in an article to other articles with similar goals, recursively building the KB . |
| Outcome: | The proposed method significantly outperforms baselines according to automatic evaluation, human judgment, and application to downstream tasks such as instructional video retrieval. |
A Deep Reinforced Sequence-to-Set Model for Multi-Label Classification (P19-1)
Copied to clipboard
| Challenge: | Multi-label classification (MLC) aims to assign multiple labels to each sample. |
| Approach: | They propose a sequence-to-set model that is trained via reinforcement learning and rewards feedback independent of the label order. |
| Outcome: | The proposed model outperforms baseline models and reduces sensitivity to label order. |
Rethinking Denoised Auto-Encoding in Language Pre-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained models such as BERT have achieved success in learning sequence representations, but they tend to learn representations that are covariant with the noise of pre-training. |
| Approach: | They propose to train self-trained models to learn noise invariant sequence representations . they encourage consistency between original sequence and corrupted version via unsupervised instance-wise training signals. |
| Outcome: | The proposed model improves on 11 natural language understanding and cross-modal tasks and achieves 0.6% gain on GLUE benchmarks and 0.8% increment on NLVR2 . |
Lightweight LLM Agent Memory with Small Language Models (2026.acl-long)
Copied to clipboard
Jiaquan Zhang, Chaoning Zhang, Shuxu Chen, Zhenzhen Huang, Pengcheng Zheng, Zhicheng Wang, Ping Guo, Fan Mo, Sung-Ho Bae, Jie Zou, Jiwei Wei, Yang Yang
| Challenge: | Existing external memory systems for LLMs have low online overhead but are unstable in accumulating latency over long interactions. |
| Approach: | They propose a lightweight memory system for better agent memory driven by Small Language Models . lightmem modularizes memory retrieval, writing, and long-term consolidation . they show consistent gains across model scales and high efficiency . |
| Outcome: | The proposed system improves agent memory but has low latency and low online overhead . it separates online processing from offline consolidation to enable efficient memory invocation . the proposed system achieves an average F1 improvement of 2.5 over A-MEM on LoCoMo . |
Learning to Control the Fine-grained Sentiment for Story Ending Generation (P19-1)
Copied to clipboard
| Challenge: | Existing studies focus on controlling the sentiment of story endings. |
| Approach: | They propose a generic and novel framework which controls fine-grained sentiment intensity for automatic story ending generation without manually annotating sentiment labels. |
| Outcome: | The proposed framework can generate story endings which meet the given sentiment intensity better. |
MAAM: A Morphology-Aware Alignment Model for Unsupervised Bilingual Lexicon Induction (P19-1)
Copied to clipboard
| Challenge: | Existing work shows that morphological variation is an intractable challenge for the unsupervised bilingual lexicon induction task. |
| Approach: | They propose a morphology-aware alignment model to alleviate the adverse effect of morphological variation by introducing grammatical information learned by the pre-trained denoising language model. |
| Outcome: | The proposed model outperforms state-of-the-art unsupervised systems and achieves competitive performance compared to supervised methods. |
Decoder-Only LLMs can be Masked Auto-Encoders (2025.acl-short)
Copied to clipboard
Dan Qiao, Yuan Gao, Zheming Yang, Di Yang, Ziheng Wu, Pengcheng Lu, Minghui Qiu, Juntao Li, Min Zhang
| Challenge: | Modern NLP workflows require different models for generation and embedding tasks. |
| Approach: | They propose a method that transforms an LLM into a Uni-Directional Masked Auto-Encoder. |
| Outcome: | The proposed method achieves state-of-the-art under unsupervised conditions with merely 100 training steps. |
LMGQS: A Large-scale Dataset for Query-focused Summarization (2023.findings-emnlp)
Copied to clipboard
Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Pengcheng He, Chenguang Zhu, Michael Zeng
| Challenge: | Lack of large-scale datasets for query-focused summarization hinders model development . lack of data limits the ability of QFS models to train robust neural models . |
| Approach: | They propose to generate a query for each summary sentence in a generic summarization annotation using a pretrained language model. |
| Outcome: | The proposed model achieves state-of-the-art zero-shot and supervised performance on multiple existing QFS benchmarks. |
Automatic Academic Paper Rating Based on Modularized Hierarchical Convolutional Neural Network (P18-2)
Copied to clipboard
| Challenge: | Existing methods to rate academic papers require a lot of feature engineering and can cause inequality. |
| Approach: | They propose to use a novel convolutional neural network to automatically rate academic papers . they propose to build a dataset to automatically determine whether to accept academic papers. |
| Outcome: | The proposed model outperforms baselines by a large margin. |