Papers by Ying Jiangyong
P3LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing autoregressive left-to-right (L2R) models are limited to unidirectional information and constrained on strong local dependencies. |
| Approach: | They propose a probabilistically permuted prophet language model which strengthens the modeling of bidirectional information and long token dependencies for sequence generation. |
| Outcome: | Experiments on GLGE dataset show that P3LM improves on natural language generation tasks. |