Using Neural Machine Translation Methods for Sign Language Translation (2022.acl-srw)
Copied to clipboard
| Challenge: | Sign languages are the main medium of exchanging information for the deaf and hard of hearing. |
| Approach: | They propose to use two NMT architectures to train models on parallel German Sign Language corpora . they achieve substantial improvement in BLEU scores for the models trained on the two corporales . |
| Outcome: | The proposed models achieve significant improvements on the two corpora trained on the german sign language . the proposed models outperform the models trained on both corporales . |
Similar Papers
Neural Machine Translation Methods for Translating Text to Sign Language Glosses (2023.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art techniques common to low resource Machine Translation (MT) are applied to improve MT of spoken language text to Sign Language glosses. |
| Approach: | They propose to use data augmentation, semi-supervised Neural Machine Translation, transfer learning and multilingual NMT to improve MT of spoken language to Sign Language glosses. |
| Outcome: | The proposed models outperform previous work on two German SL corpora and are confirmed by human evaluation. |
Considerations for meaningful sign language machine translation based on glosses (2023.acl-short)
Copied to clipboard
| Challenge: | In machine translation, sign language translation based on glosses is becoming more popular . limitations of glossed approaches are not discussed in a transparent manner, and there is no common standard for evaluation. |
| Approach: | They propose to use a gloss-based approach to evaluate machine translation results . they propose to include realistic datasets, stronger baselines and convincing evaluation . |
| Outcome: | The proposed approach is based on a neural gloss translation model. |
Better Sign Language Translation with STMC-Transformer (2020.coling-main)
Copied to clipboard
| Challenge: | Current SLT approaches use a sign language recognition system to extract sign language glosses from videos. |
| Approach: | They propose to use a Sign Language Recognition system to extract sign language glosses from videos and a translation system to generate spoken language translations from the glossed sign language. |
| Outcome: | The proposed system outperforms existing methods on gloss-to-text and video-to text translations on the ASLG-PC12 corpus. |
Gloss2Text: Sign Language Gloss translation using LLMs and Semantically Aware Label Smoothing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to sign language translation use gloss annotations as an intermediary . a new approach to use large language models and word embeddings to improve Gloss2Text translation is needed. |
| Approach: | They propose to leverage large language models pre-trained on expansive and diverse corpora to improve Gloss2Text translation stage by using data augmentation and label-smoothing loss function. |
| Outcome: | The proposed approach surpasses state-of-the-art methods on the PHOENIX Weather 2014T dataset . it shows that gloss annotations can be used to guide the translation process . |
How to Align Multiple Signed Language Corpora for Better Sign-to-Sign Translations? (2025.naacl-long)
Copied to clipboard
| Challenge: | despite the growing need for advanced signing technologies, signed language resources remain scarce. |
| Approach: | They propose a linguistically informed alignment algorithm that matches instances between signed languages . they compare similarities and differences across three signed languages to develop a model . |
| Outcome: | The proposed algorithm performs well on automatic metrics for sign-to-sign translation and generation. |
Phrase-Based & Neural Unsupervised Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent advances in machine translation have reported near human-level performance on several languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences. |
| Approach: | They propose two models that leverage a careful initialization of the parameters and denoising effect of language models. |
| Outcome: | The proposed models outperform the current methods on English-French and German-English benchmarks while being simpler and having fewer hyper-parameters. |
Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting (2023.findings-eacl)
Copied to clipboard
| Challenge: | Yin et al. ( 2021) calls for including sign language processing (SLP) in natural language processing research. |
| Approach: | They propose to use a sign language writing system to parse, factorize, decode and evaluate signed languages. |
| Outcome: | The proposed method achieves over 30 BLEU in a bilingual setup and over 20 BLUE in two multilingual setups. |
Sign Language Translation with Sentence Embedding Supervision (2024.acl-short)
Copied to clipboard
| Challenge: | State-of-the-art sign language translation systems facilitate learning through gloss annotations when available at scale. |
| Approach: | They propose to use sentence embeddings of the target sentences at training time that take the role of glosses to supervise the learning process. |
| Outcome: | The proposed method significantly outperforms gloss-free approaches on German and American sign languages and with mono- and multilingual sentence embeddings and translation systems. |
An Effective Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | a recent research line has managed to train both unsupervised and unsupervised machine translation systems using monolingual corpora only. |
| Approach: | They propose to use monolingual corpora to train both unsupervised and unsupervised machine translation systems. |
| Outcome: | The proposed system achieves 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more in the (supervised) shared task winner back in 2014. |
Challenges with Sign Language Datasets for Sign Language Recognition and Translation (2022.lrec-1)
Copied to clipboard
Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov, Horacio Saggion
| Challenge: | Sign Languages are the primary means of communication for at least half a million people in Europe . however, the development of SL recognition and translation tools is slowed down by resource scarcity and data formats are not suitable for machine learning. |
| Approach: | They propose a framework to unify available resources and facilitate SL research for different languages. |
| Outcome: | The proposed framework is based on a set of ELAN files and returns textual and visual data ready to train SL recognition and translation models. |