Papers by Anastasiia Razdaibiedina
Residual Prompt Tuning: improving prompt tuning with residual reparameterization (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt tuning is one of the most parameter-efficient approaches for parameter-effective tuning of pre-trained language models. |
| Approach: | They propose to reparameterize soft prompt embeddings using a shallow network with a residual connection and use it to tune prompt embeds P. |
| Outcome: | The proposed method outperforms prompt tuning on SuperGLUE, T5-Base and BERT-Bass models and can reduce the prompt length by 10 times without hurting performance. |
MIReAD: Simple Method for Learning High-quality Representations from Scientific Documents (2023.acl-short)
Copied to clipboard
| Challenge: | Pretrained language models can learn rich textual representations, but they cannot provide powerful document-level representations for scientific articles. |
| Approach: | They propose a transformer-based method that learns semantically meaningful representations from scientific papers by fine-tuning transformer models to predict the target journal class based on the abstract. |
| Outcome: | The proposed method outperforms six existing models for representation learning on scientific documents across four evaluation standards. |