Label-aware Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to translate spoken language understanding into low-resource languages are limited to implicit alignment and disregard the inherent semantic structure in SLU. |
| Approach: | They propose to model utterance-slot-word structure by a multi-level contrastive learning framework . they also propose a label-aware joint model leveraging label semantics to enhance alignment . |
| Outcome: | The proposed model improves performance on two zero-shot cross-lingual datasets. |
Similar Papers
Code-Switching Can be Better Aligners: Advancing Cross-Lingual SLU through Representation-Level and Prediction-Level Alignment (2024.acl-short)
Copied to clipboard
| Challenge: | Existing code-switching-based cross-lingual spoken language understanding frameworks are limited to low-resource languages. |
| Approach: | They propose a cross-lingual spoken language understanding framework that leverages both code-switched and original sentences to achieve multi-level alignment. |
| Outcome: | The proposed framework can achieve multi-level alignment on two benchmarks across ten languages. |
GL-CLeF: A Global–Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot cross-lingual spoken language understanding rely on shared parameters, which can only perform implicit alignment across languages. |
| Approach: | They propose a global-local contrastive learning framework to achieve a fine-grained cross-lingual transfer . they employ bilingual dictionaries to construct multilingual views of the same utterance . |
| Outcome: | Experiments on MultiATIS++ show that GL-CLeF achieves the best performance . GL is based on dictionaries and encourages representations to be more similar than negative example pairs . |
Cross-lingual Spoken Language Understanding with Regularized Representation Alignment (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite promising results, current cross-lingual models suffer from imperfect cross-linguistic representation alignments between the source and target languages, which makes the performance sub-optimal. |
| Approach: | They propose a regularization approach to align word-level and sentence-level representations across languages without external resources. |
| Outcome: | The proposed model outperforms state-of-the-art models in few-shot and zero-shot scenarios and achieves comparable performance to supervised training with all training data. |
Cyclical Contrastive Learning Based on Geodesic for Zero-shot Cross-lingual Spoken Language Understanding (2024.findings-acl)
Copied to clipboard
| Challenge: | zero-shot cross-lingual SLU is a challenging task in low-resource languages . a lack of labeled training data makes it difficult to align representations of similar sentences . |
| Approach: | They propose a framework that uses cyclical contrastive learning to achieve consistency between languages . they propose to use geodesic to measure the similarity to construct positive and negative pairs . |
| Outcome: | The proposed framework achieves state-of-the-art performance on multiATIS++ and MTOP datasets. |
Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies show that multilingual generative models exhibit a strong language bias toward high-resource languages. |
| Approach: | They propose a cross-lingual alignment framework exploiting pairs of translation sentences to improve cross-linguistic abilities. |
| Outcome: | The proposed framework improves cross-lingual abilities and mitigates performance gap. |
Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognition (D19-1)
Copied to clipboard
| Challenge: | Existing work on name-switching focuses on word-level aspects but neglects subword-level characteristics shared across languages. |
| Approach: | They propose hierarchical meta-Embeddings that combine word-level and subword-level embeddings to create language-agnostic lexical representations. |
| Outcome: | The proposed model achieves state-of-the-art in English-Spanish code-switching scenarios. |
Improving Multi-lingual Alignment Through Soft Contrastive Learning (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to train multi-lingual sentence embeddings ruins the mono-lingual space. |
| Approach: | They propose a method to align multi-lingual embeddings based on similarity of sentences measured by a pre-trained mono-lingual teacher model. |
| Outcome: | The proposed method outperforms existing multi-lingual embeddings including LaBSE on five languages and on a translation pair for Tatoeba dataset. |
Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer (2024.lrec-main)
Copied to clipboard
| Challenge: | Unsupervised cross-lingual transfer is a process of transferring knowledge between languages without explicit supervision. |
| Approach: | They propose a framework that combines lexical and syntactic knowledge to enhance learning . they use a code-switching technique to implicitly teach lexica and a syntaktic-based graph attention network to help encode syntakic structure. |
| Outcome: | The proposed framework outperforms baselines of zero-shot cross-lingual transfer with 1.0 3.7 points on text classification, named entity recognition, and semantic parsing tasks. |
Exploring Alignment in Shared Cross-lingual Spaces (2024.acl-long)
Copied to clipboard
| Challenge: | a new study examines the degree of alignment between languages in multilingual embeddings . cross-lingual embeds are designed to encode linguistic concepts that bridge equivalent semantic meaning . a comprehensive approach is needed to address these questions. |
| Approach: | They employ clustering to uncover latent concepts within multilingual models . they introduce two metrics to quantify alignment and overlap of these concepts . |
| Outcome: | The proposed model can capture linguistic nuances across languages, but is not language-agnostic? the proposed model is able to capture nuances in multiple languages, the authors say. |
A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU (2024.lrec-main)
Copied to clipboard
| Challenge: | Multi-intent natural language understanding (NLU) models lack the rich information between the shared intents, especially in low-data scenarios. |
| Approach: | They propose a two-stage framework for multi-intent natural language understanding to harness shared intent information by word-level pre-training and prediction-aware contrastive fine-tuning. |
| Outcome: | The proposed framework surpasses baselines on low-data and full-data scenarios. |