Automatic Gloss-level Data Augmentation for Sign Language Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for enhancing sign language text data are insufficient . fewer studies have been performed on text data augmentation compared to video data . |
| Approach: | They propose three methods to augment sign language text data using Korean sign language gloss dictionary. |
| Outcome: | The proposed method improves translation performance by 0.204 and 0.170 compared to the original data. |
Similar Papers
Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies focus on the recognition step, while paying less attention to sign language translation. |
| Approach: | They propose a task-aware instruction network, namely TIN-SLT, for sign language translation, by introducing the isntruction module and the learning-based feature fuse strategy into a Transformer network. |
| Outcome: | The proposed system outperforms existing solutions on two benchmark datasets, PHOENIX-2014-T and ASLG-PC12, and outperformed previous best solutions by 1.65 and 1.42 in terms of BLEU-4. |
Cross-modality Data Augmentation for End-to-End Sign Language Translation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | End-to-end sign language translation (SLT) aims to convert sign language videos into spoken language texts without intermediate representations. |
| Approach: | They propose a cross-modality data-augmented framework to transfer gloss-to-text translation capabilities to end-to end sign language translation. |
| Outcome: | The proposed framework outperforms baseline models on two widely used SLT datasets. |
Gloss2Text: Sign Language Gloss translation using LLMs and Semantically Aware Label Smoothing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to sign language translation use gloss annotations as an intermediary . a new approach to use large language models and word embeddings to improve Gloss2Text translation is needed. |
| Approach: | They propose to leverage large language models pre-trained on expansive and diverse corpora to improve Gloss2Text translation stage by using data augmentation and label-smoothing loss function. |
| Outcome: | The proposed approach surpasses state-of-the-art methods on the PHOENIX Weather 2014T dataset . it shows that gloss annotations can be used to guide the translation process . |
Neural Machine Translation Methods for Translating Text to Sign Language Glosses (2023.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art techniques common to low resource Machine Translation (MT) are applied to improve MT of spoken language text to Sign Language glosses. |
| Approach: | They propose to use data augmentation, semi-supervised Neural Machine Translation, transfer learning and multilingual NMT to improve MT of spoken language to Sign Language glosses. |
| Outcome: | The proposed models outperform previous work on two German SL corpora and are confirmed by human evaluation. |
Korean Disaster Safety Information Sign Language Translation Benchmark Dataset (2024.lrec-main)
Copied to clipboard
Wooyoung Kim, TaeYong Kim, Byeongjin Kim, Myeong Jin MJ Lee, Gitaek Lee, Kirok Kim, Jisoo Cha, Wooju Kim
| Challenge: | Sign language is a crucial means of communication for deaf communities. |
| Approach: | They propose to refine Korean sign language translation datasets and release them . they show baseline performance varies depending on tokenization method applied to gloss sequences . |
| Outcome: | The proposed dataset outperforms baseline and spoken language tokenization methods. |
Sign Language Translation with Sentence Embedding Supervision (2024.acl-short)
Copied to clipboard
| Challenge: | State-of-the-art sign language translation systems facilitate learning through gloss annotations when available at scale. |
| Approach: | They propose to use sentence embeddings of the target sentences at training time that take the role of glosses to supervise the learning process. |
| Outcome: | The proposed method significantly outperforms gloss-free approaches on German and American sign languages and with mono- and multilingual sentence embeddings and translation systems. |
Getting More Data for Low-resource Morphological Inflection: Language Models and Data Augmentation (2020.lrec-1)
Copied to clipboard
| Challenge: | Morphological inflection is the process that generates the word form given its lexeme and morphological properties. |
| Approach: | They propose to use language models and data augmentation to improve morphological inflection without annotating more data. |
| Outcome: | The proposed model improves by 1.5% with the langauge model and by 9% with the data augmentation. |
Considerations for meaningful sign language machine translation based on glosses (2023.acl-short)
Copied to clipboard
| Challenge: | In machine translation, sign language translation based on glosses is becoming more popular . limitations of glossed approaches are not discussed in a transparent manner, and there is no common standard for evaluation. |
| Approach: | They propose to use a gloss-based approach to evaluate machine translation results . they propose to include realistic datasets, stronger baselines and convincing evaluation . |
| Outcome: | The proposed approach is based on a neural gloss translation model. |
How to Align Multiple Signed Language Corpora for Better Sign-to-Sign Translations? (2025.naacl-long)
Copied to clipboard
| Challenge: | despite the growing need for advanced signing technologies, signed language resources remain scarce. |
| Approach: | They propose a linguistically informed alignment algorithm that matches instances between signed languages . they compare similarities and differences across three signed languages to develop a model . |
| Outcome: | The proposed algorithm performs well on automatic metrics for sign-to-sign translation and generation. |
Signer Diversity-driven Data Augmentation for Signer-Independent Sign Language Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for sign language translation (SLT) rely on signer identity labels, which is often impractical and costly in real-world applications. |
| Approach: | They propose a signer diversity-driven data augmentation method that can generalize to signers not encountered during training. |
| Outcome: | The proposed method achieves state-of-the-art results without relying on signer identity labels. |