Papers by Zijun Sun
An Extensible Plug-and-Play Method for Multi-Aspect Controllable Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Multi-aspect controllable text generation has attracted increasing attention . but the mutual interference of multiple prefixes limits its extensibility to training-time unseen combinations. |
| Approach: | They propose to use trainable gates to normalize the intervention of prefixes to restrain the interference. |
| Outcome: | The proposed approach outperforms baselines on constraint accuracy, text quality, and extensibility. |
Curriculum Masking in Vision-Language Pretraining to Maximize Cross Modal Interaction (2024.naacl-long)
Copied to clipboard
| Challenge: | masked language modeling is widely used as a pretraining component in Vision and language (V+L) but performance on benchmarks has not received the attention it deserves. |
| Approach: | They propose a curriculum masking scheme that uses a parallel mask selection agent to mask tokens at a frequency proportional to the level of cross modal interaction necessary to reconstruct them. |
| Outcome: | The proposed method improves relational understanding on a wide range of V+L tasks. |
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information (2021.acl-long)
Copied to clipboard
| Challenge: | ChineseBERT model incorporates glyph and pinyin information of Chinese characters into pretraining . proposed model achieves new performance boost over baseline models with fewer training steps . |
| Approach: | They propose a ChineseBERT model that incorporates glyph and pinyin information into pretraining . the glyph embedding is obtained based on different fonts of a character, and the pinyink embeddment characterizes the pronunciation of Chinese characters. |
| Outcome: | The proposed model achieves new performance boosts over baseline models with fewer training steps. |
HER: Human-like Reasoning and Reinforcement Learning for LLM Role-playing (2026.findings-acl)
Copied to clipboard
Chengyu Du, Xintao Wang, Aili Chen, Weiyuan Li, Rui Xu, Junteng Liu, Zishan Huang, Rong Tian, Zijun Sun, Yuhao Li, Liheng Feng, Deming Ding, Pengyu Zhao, Yanghua Xiao
| Challenge: | Existing models for LLM role-playing lack high-quality datasets with explicit reasoning traces and reliable reward signals aligned with human preferences. |
| Approach: | They propose a unified framework for cognitive-level persona simulation that strictly distinguishes characters’ first-person thinking processes from LLMs’ third-person reasoning. |
| Outcome: | The proposed framework outperforms the Qwen3-32B baseline model and achieves a 30.26% and 14.97% performance on the minimax benchmarks. |
Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages (2024.acl-long)
Copied to clipboard
| Challenge: | Contemporary large language models (LLMs) are pre-trained on multilingual corpora, but their performance lags behind in most languages compared to a few resource-rich languages. |
| Approach: | They propose a method that leverages the internal capabilities of large language models on resource-rich languages to enhance multilingual performance. |
| Outcome: | The proposed method improves multilingual performance while minimizing impact on original performance in resource-rich languages. |
Entity-Relation Extraction as Multi-Turn Question Answering (P19-1)
Copied to clipboard
| Challenge: | Identifying entities and their relations is the prerequisite of extracting structured knowledge from unstructured raw texts. |
| Approach: | They propose a new paradigm for the task of entity-relation extraction . they cast the task as a multi-turn question answering problem . |
| Outcome: | The proposed paradigm significantly outperforms previous best models on the ACE and CoNLL04 datasets. |
Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models focus on local word prediction, and cannot make high level plans on what to generate. |
| Approach: | They propose a pipelined system that summarises, outlines and elaborates on each bullet point to generate the corresponding segment. |
| Outcome: | The proposed system produces long texts with significantly better quality and faster convergence speed. |