Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sign Language Translation evaluations remain largely signer-dependent, with overlapping signers across train/dev/test. |
| Approach: | We conduct signer-fold cross-validation on three leading SLT models . they find that under signer independent evaluation performance drops sharply . |
| Outcome: | a signer-dependent evaluation can substantially overestimate SLT capability, the authors say . they recommend adopting signer independent protocols to ensure generalisation to unseen signers . |
Similar Papers
Can Small Vision–Language Models Perform Sign Language Translation? (2026.findings-acl)
Copied to clipboard
| Challenge: | Vision-Language Models (VLMs) have shown strong generalization across multimodal tasks, but their capacity to handle sign language translation (SLT) remains unclear. |
| Approach: | They propose entity- and semantics-aware metrics tailored for SLT to evaluate their performance. |
| Outcome: | The proposed metrics highlight the limitations of general-purpose VLMs to SLT, unlike their applicability in other tasks. |
Signer Diversity-driven Data Augmentation for Signer-Independent Sign Language Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for sign language translation (SLT) rely on signer identity labels, which is often impractical and costly in real-world applications. |
| Approach: | They propose a signer diversity-driven data augmentation method that can generalize to signers not encountered during training. |
| Outcome: | The proposed method achieves state-of-the-art results without relying on signer identity labels. |
Multilingual Gloss-free Sign Language Translation: Towards Building a Sign Language Foundation Model (2025.acl-short)
Copied to clipboard
| Challenge: | Existing studies focus on translating a single SL into a spoken language (one-to-one SLT) however, multilingual SLT remains unexplored due to language conflicts and alignment difficulties across SLs and spoken languages. |
| Approach: | They propose a multilingual gloss-free model that can be used to translate a single SL into a spoken language and generate a token-level SL identification and spoken text. |
| Outcome: | The proposed model supports 10 SLs and handles one-to-one, many-to-1, and many- to-many SLT tasks. |
How to Align Multiple Signed Language Corpora for Better Sign-to-Sign Translations? (2025.naacl-long)
Copied to clipboard
| Challenge: | despite the growing need for advanced signing technologies, signed language resources remain scarce. |
| Approach: | They propose a linguistically informed alignment algorithm that matches instances between signed languages . they compare similarities and differences across three signed languages to develop a model . |
| Outcome: | The proposed algorithm performs well on automatic metrics for sign-to-sign translation and generation. |
Stable Signer: Hierarchical Sign Language Generative Model (2026.acl-long)
Copied to clipboard
| Challenge: | Sign Language Production (SLP) is the process of converting complex input text into a real video. |
| Approach: | They propose a new sign language generative model that streamlines redundant structure and optimizes the task objective. |
| Outcome: | The proposed model streamlines redundant structure and optimizes objective . it generates high-quality and multi-style sign language videos with hand gestures . |
Think in Latent Thoughts: A New Paradigm for Gloss-Free Sign Language Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to sign language translation (SLT) assume video segments are directly mappable to spoken-language words. |
| Approach: | They propose a reasoning-driven SLT framework that uses an ordered sequence of latent thoughts as an explicit middle layer between video and generated text. |
| Outcome: | The proposed model improves coherence and faithfulness over existing gloss-free methods. |
SAME: Signer-Aware Mixture-of-Experts for Test-Time Adaptation in Sign Language Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for supervised fine-tuning are limited due to labeled data . existing methods require long adaptation times and batch statistics are unavailable in streaming settings . |
| Approach: | They propose a plug-and-play, signer-aware Mixture-of-Experts (MoE) TTA architecture for SLT . they use a combination of lightweight MoE modules and unsupervised regularizers to decouple domain shift . |
| Outcome: | The proposed test-time adaptation outperforms existing TTA methods in sign language translation . the proposed architecture can be used in real-world deployments without labeling . |
Does BERT really agree ? Fine-grained Analysis of Lexical Dependence on a Syntactic Task (2022.findings-acl)
Copied to clipboard
| Challenge: | lexically-independent subject-verb number agreement (NA) is performed by transformer-based neural language models (NLMs) . but when as little as one attractor is present, the model fails to perform lexical generalization . |
| Approach: | They propose to disrupt lexical patterns found in naturally occurring stimuli for each targeted structure in a novel fine-grained analysis of BERT's behavior. |
| Outcome: | The proposed model generalizes well for simple templates, but fails to perform lexically-independent generalization when as little as one attractor is present. |
Challenges with Sign Language Datasets for Sign Language Recognition and Translation (2022.lrec-1)
Copied to clipboard
Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov, Horacio Saggion
| Challenge: | Sign Languages are the primary means of communication for at least half a million people in Europe . however, the development of SL recognition and translation tools is slowed down by resource scarcity and data formats are not suitable for machine learning. |
| Approach: | They propose a framework to unify available resources and facilitate SL research for different languages. |
| Outcome: | The proposed framework is based on a set of ELAN files and returns textual and visual data ready to train SL recognition and translation models. |
Multi-lingual Functional Evaluation for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Multilingual competence in large language models is often evaluated via static data benchmarks such as Belebele, M-MMLU and M-GSM. |
| Approach: | They extend existing functional benchmark templates from English to five additional languages that span the range of resources available for NLP: French, Spanish, Hindi, Arabic and Yoruba. |
| Outcome: | The proposed models are translated from English to French, Spanish, Hindi, Arabic and Yoruba. |