Papers by Zihou Zhang
One Token Is Enough: Improving Diffusion Language Models with a Sink Token (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Diffusion Language Models lack a structural constraint to stabilize attention sinks. |
| Approach: | They propose a simple but effective extra sink token that is constrained to attend to itself while remaining globally visible to all other tokens. |
| Outcome: | The proposed token is able to stabilize attention sinks and improve model performance. |