Papers with STS-B.
SupCL-Seq: Supervised Contrastive Learning for Downstream Optimized Sequence Representations (2021.findings-emnlp)
Copied to clipboard
| Challenge: | SupCL-Seq extends contrastive learning from computer vision to sequence classification tasks. |
| Approach: | They propose a supervised alternative to Masked Language Modeling (MLM) that extends contrastive learning to sequence optimization in NLP by altering the dropout mask probability in standard Transformer architectures. |
| Outcome: | The proposed method leads to large gains on the GLUE benchmark, including 6% absolute improvement on CoLA, 5.4% on MRPC, 4.7% on RTE and 2.6% on STS-B. |
Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that humans are not supervised by the natural language inference . |
| Approach: | They propose to solve the natural language inference problem via task-agnostic multimodal pretraining. |
| Outcome: | The proposed network outperforms fully-supervised BiLSTM and BiLS+ELMO on plain text inference datasets. |