The Key Points: Using Feature Importance to Identify Shortcomings in Sign Language Recognition Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Pose estimation keypoints are widely used in sign language recognition (SLR) but they are difficult to achieve due to the large degree of variability between occurrences of the same sign, the lack of large datasets and the imbalanced nature of the data. |
| Approach: | They propose to use pose estimation keypoints to generalise to unseen signers by identifying potentially redundant features and identifying key points that are most informative to SLR . they propose to train models with large datasets and labelled data to find key points which are redundant to differentiating between signs . |
| Outcome: | The proposed model can be trained on large datasets and has more generalised features than would be possible with a small dataset. |
Similar Papers
Studying and Mitigating Biases in Sign Language Understanding Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using crowd-sourced sign language datasets to reduce performance disparities is critical to addressing potential biases and inequities. |
| Approach: | They use demographic information to study biases that may result from models trained on crowd-sourced sign datasets. |
| Outcome: | The proposed approach reduces performance disparities without decreasing accuracy. |
Challenges with Sign Language Datasets for Sign Language Recognition and Translation (2022.lrec-1)
Copied to clipboard
Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov, Horacio Saggion
| Challenge: | Sign Languages are the primary means of communication for at least half a million people in Europe . however, the development of SL recognition and translation tools is slowed down by resource scarcity and data formats are not suitable for machine learning. |
| Approach: | They propose a framework to unify available resources and facilitate SL research for different languages. |
| Outcome: | The proposed framework is based on a set of ELAN files and returns textual and visual data ready to train SL recognition and translation models. |
OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages (2022.acl-long)
Copied to clipboard
| Challenge: | a new study examines the performance of pretraining for sign language recognition in low-resource settings. |
| Approach: | They propose using pose extracted through pretrained models as the standard modality of data to reduce training time and enable efficient inference. |
| Outcome: | The proposed model reduces training time and allows efficient inference in sign languages. |
Evaluation of Manual and Non-manual Components for Sign Language Recognition (2020.lrec-1)
Copied to clipboard
Medet Mukushev, Arman Sabyrov, Alfarabi Imashev, Kenessary Koishybay, Vadim Kimmelman, Anara Sandygulova
| Challenge: | Deaf communities communicate via sign languages to express meaning and intent. |
| Approach: | They used sign samples from 20 commonly used signs in Kazakh-Russian Sign Language (K-RSL) to investigate whether non-manual components would improve sign’s recognition accuracy. |
| Outcome: | The results showed that using non-manual components would improve sign recognition accuracy. |
Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks fail to reflect real-world communication needs and are limited in their coverage. |
| Approach: | They present a comprehensive index of sign-language datasets, covering 120 resources across 35 sign languages. |
| Outcome: | The proposed index covers 120 resources across 35 sign languages. |
“Will You Find These Shortcuts?” A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on faithfulness evaluation is not conclusive and does not provide a clear answer as to how different methods are to be compared. |
| Approach: | They propose a protocol for faithfulness evaluation that makes use of partially synthetic data to obtain ground truth for feature importance ranking. |
| Outcome: | The proposed method is based on partially synthetic data and is compared with lexical shortcuts on a range of datasets and LSTM models. |
Including Signed Languages in Natural Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing research in Sign Language Processing (SLP) rarely explores signed languages . authors urge adoption of an efficient tokenization method and the collection of real-world signed language data . |
| Approach: | They propose to include signed languages as a research area with high social and scientific impact . they review the limitations of current SLP models and identify the open challenges . |
| Outcome: | The proposed model should include signed languages as a research area with high social and scientific impact. |
WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language (2022.acl-short)
Copied to clipboard
| Challenge: | Signed Language Processing (SLP) is a major form of NLP, but has been overlooked by the NLP community. |
| Approach: | They leverage existing resources to construct a large-scale dataset of American Sign Language signs annotated with six different phonological properties. |
| Outcome: | The proposed model outperforms existing approaches on signs unobserved during training. |
Fingerspelling within Sign Language Translation (2025.naacl-long)
Copied to clipboard
| Challenge: | Prior work has studied fingerspelling recognition, but little attention has been paid to how well models understand it in context of entire sentences. |
| Approach: | They annotate instances of fingerspelling within FLEURS-ASL and use it to evaluate how well translation models understand it. |
| Outcome: | The proposed model family significantly improves understanding of fingerspelling, but the effect of the mixed model is mixed. |
Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sign Language Translation evaluations remain largely signer-dependent, with overlapping signers across train/dev/test. |
| Approach: | We conduct signer-fold cross-validation on three leading SLT models . they find that under signer independent evaluation performance drops sharply . |
| Outcome: | a signer-dependent evaluation can substantially overestimate SLT capability, the authors say . they recommend adopting signer independent protocols to ensure generalisation to unseen signers . |