Multi-level Contrastive Learning for Script-based Character Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Scripts are written text for plays, movies, or broadcasts. |
| Approach: | They propose a multi-level contrastive learning framework to capture characters’ global information in a fine-grained manner. |
| Outcome: | The proposed framework improves on three character understanding sub-tasks by a considerable margin. |
Similar Papers
Contrastive Data and Learning for Natural Language Processing (2022.naacl-tutorials)
Copied to clipboard
| Challenge: | Current NLP models heavily rely on effective representation learning algorithms. |
| Approach: | This tutorial introduces contrastive learning and provides an introduction to the techniques. |
| Outcome: | This tutorial provides an introduction to the fundamentals of contrastive learning approaches and the theory behind them. |
Distantly Supervised Contrastive Learning for Low-Resource Scripting Language Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for code summarization are limited in resources and require atomic commands and category constraints to enhance code representations. |
| Approach: | They propose a framework that leverages limited atomic commands and category constraints to enhance code representations. |
| Outcome: | The proposed framework outperforms baseline methods in a number of domains and demonstrates superiority over competing frameworks. |
TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | The world’s more than 7000 languages are written in at least 293 scripts, which poses a difficulty for multilingual pretrained language models in learning crosslingual knowledge through lexical overlap. |
| Approach: | They propose a framework that optimizes the Transliteration Contrastive Modeling objective to fine-tune an mPLM by contrasting sentences in its training data and transliterations in a unified script. |
| Outcome: | The proposed model outperforms Glot500-m on zero-shot crosslingual transfer tasks while retaining uniformity across scripts. |
Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval (2023.acl-long)
Copied to clipboard
| Challenge: | Contrastive learning is the dominant paradigm for learning text representations from parallel text, but finding negative examples can be expensive in terms of compute or manual effort. |
| Approach: | They propose a generative model for learning multilingual text embeddings which encourages source separation in multilingual contexts by an approximation. |
| Outcome: | The proposed model outperforms both a strong contrastive and generative baseline on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval. |
HMCL: Task-Optimal Text Representation Adaptation through Hierarchical Contrastive Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Hierarchical Multilevel Contrastive Learning (HMCL) improves text representation for general large language models. |
| Approach: | a new contrastive learning framework is developed to improve general large language models . HMCL integrates 3-level semantic differentiation and unifies contrastive and pair classification into a strategy . |
| Outcome: | HMCL outperforms unsupervised methods and supervised fine-tuning approaches in multi-domain and multilingual benchmarks. |
Beyond Paragraphs: NLP for Long Sequences (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will introduce document-level representation learning techniques . document-based learning is challenging due to the limited sequence length of many models . |
| Approach: | They will provide an overview of established long sequence NLP techniques and discuss memory-saving methods that are key to processing long sequences. |
| Outcome: | The tutorial will introduce the latest and ongoing techniques for document-level representation learning. |
An End-to-End Contrastive Self-Supervised Learning Framework for Language Understanding (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to learning data representations using contrastive learning perform data augmentation and contrastive training separately. |
| Approach: | They propose a framework that performs data augmentation and contrastive learning end-to-end . they propose to combine data augmented with text encoders to optimize for contrastive training . |
| Outcome: | Experiments on GLUE and Gururangan datasets show the proposed framework is effective in NLP. |
What is the best recipe for character-level encoder-only modelling? (2023.acl-long)
Copied to clipboard
| Challenge: | aims to benchmark recent progress in language understanding models that output contextualised representations at the character level. |
| Approach: | They aim to find the best way to build and train character-level BERT-like models by comparing architectural innovations with pretraining objectives. |
| Outcome: | The proposed model outperforms a token-based model on a set of evaluation tasks with a fixed training procedure. |
CHIRON: Rich Character Representations in Long-Form Narratives (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing systems for character representation have simplified the problem of representing complex characters via graphs and brief character descriptions. |
| Approach: | They propose a ‘character sheet’ based representation that organizes and filters textual information about characters. |
| Outcome: | The proposed representation organizes and filters textual information about characters and is better and more flexible than previous models. |
HiCL: Hierarchical Contrastive Learning of Unsupervised Sentence Embeddings (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods that encode a sequence in its entirety for contrast with others often neglect local representation learning. |
| Approach: | They propose a hierarchical contrastive learning framework, HiCL, which considers local segment-level and global sequence-level relationships to improve training efficiency and effectiveness. |
| Outcome: | The proposed framework improves training efficiency and effectiveness by dividing a sequence into several segments and using local and global contrastive learning to model relationships. |