Papers with SSCL
Alleviating Over-smoothing for Unsupervised Sentence Representation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to learn better unsupervised sentence representations have been successful . over-smoothing problem in unsupervised sentences reduces the capacity of powerful PLMs . |
| Approach: | They propose a method to solve the over-smoothing problem in unsupervised sentence representations by combining negatives from PLMs intermediate layers. |
| Outcome: | The proposed method improves on different strong baselines on Semantic Textual Similarity and Transfer datasets. |
Self-Supervised Curriculum Learning for Spelling Error Correction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to SEC typically leverage a pre-training then fine-tuning procedure that treats data equally. |
| Approach: | They propose a self-supervised curriculum learning approach to improve model performance and model learning. |
| Outcome: | The proposed approach improves the model training and improves CL measurement. |