Papers by Kuanning Wang
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing generative masked language models have a shared training objective, i.e., denoising. |
| Approach: | They propose a noise schedule for the forward diffusion process that controls the degree of noise added at each step based on the information of each token. |
| Outcome: | The proposed model improves on existing models in terms of perplexity and BLEU score. |