Handshape-Aware Sign Language Recognition: Extended Datasets and Exploration of Handshape-Inclusive Methods (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on sign language recognition encodes videos without acknowledging phonological attributes of signs. |
| Approach: | They propose a single-encoder network and a dual-encoding network for handshape-inclusive sign language recognition. |
| Outcome: | The proposed methods outperform baseline methods in the PHOENIX14T-HS dataset . the proposed methods consistently outperformed baseline methods . |
Similar Papers
Improving Handshape Representations for Sign Language Processing: A Graph Neural Network Approach (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for sign language recognition process a signing sequence holistically, leaving handshape information implicit, which limits both recognition accuracy and linguistic analysis. |
| Approach: | They propose a graph neural network that separates temporal dynamics from static handshape configurations in continuous signing sequences. |
| Outcome: | The proposed approach achieves 46% accuracy across 37 handshape classes, compared to 25% for baseline methods. |
How to Align Multiple Signed Language Corpora for Better Sign-to-Sign Translations? (2025.naacl-long)
Copied to clipboard
| Challenge: | despite the growing need for advanced signing technologies, signed language resources remain scarce. |
| Approach: | They propose a linguistically informed alignment algorithm that matches instances between signed languages . they compare similarities and differences across three signed languages to develop a model . |
| Outcome: | The proposed algorithm performs well on automatic metrics for sign-to-sign translation and generation. |
Improving Sign Recognition with Phonology (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work does not consider sign language phonology, but none leverages it . a recent study has shown that sign language recognition models lack structure . |
| Approach: | They explicitly recognize the role of phonology in sign production to train models for isolated sign language recognition . they train models that take in pose estimations of a signer producing a single sign to predict its phonological characteristics . |
| Outcome: | The proposed model improves sign recognition accuracy by 9% on the WLASL benchmark . the study could accelerate linguistic research in the domain of signed languages . |
WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language (2022.acl-short)
Copied to clipboard
| Challenge: | Signed Language Processing (SLP) is a major form of NLP, but has been overlooked by the NLP community. |
| Approach: | They leverage existing resources to construct a large-scale dataset of American Sign Language signs annotated with six different phonological properties. |
| Outcome: | The proposed model outperforms existing approaches on signs unobserved during training. |
SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work on sign language video processing focuses on the face, hands and body posture of the signer. |
| Approach: | They propose to learn the handshapes and rich facial expressions of sign languages in a self-supervised fashion by learning from individual frames rather than video sequences. |
| Outcome: | The proposed model is more efficient than previous work on sign language pre-training. |
Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies focus on the recognition step, while paying less attention to sign language translation. |
| Approach: | They propose a task-aware instruction network, namely TIN-SLT, for sign language translation, by introducing the isntruction module and the learning-based feature fuse strategy into a Transformer network. |
| Outcome: | The proposed system outperforms existing solutions on two benchmark datasets, PHOENIX-2014-T and ASLG-PC12, and outperformed previous best solutions by 1.65 and 1.42 in terms of BLEU-4. |
Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks fail to reflect real-world communication needs and are limited in their coverage. |
| Approach: | They present a comprehensive index of sign-language datasets, covering 120 resources across 35 sign languages. |
| Outcome: | The proposed index covers 120 resources across 35 sign languages. |
The American Sign Language Knowledge Graph: Infusing ASL Models with Linguistic Knowledge (2025.findings-naacl)
Copied to clipboard
| Challenge: | Sign language models could make language technologies more accessible to deaf and hard-of-hearing signers, but the supply of accurately labeled data struggles to meet the demand associated with training large, end-to-end architectures. |
| Approach: | They construct an American Sign Language Knowledge Graph from 11 sources of linguistic knowledge and use it to train neuro-symbolic models on ASL video input tasks. |
| Outcome: | The proposed model achieves 91% accuracies for isolated sign recognition, 14% for predicting the semantic features of unseen signs, and 36% for classifying the topic of Youtube-ASL videos. |
Listen, Decipher and Sign: Toward Unsupervised Speech-to-Sign Language Recognition (2023.findings-acl)
Copied to clipboard
Liming Wang, Junrui Ni, Heting Gao, Jialu Li, Kai Chieh Chang, Xulin Fan, Junkai Wu, Mark Hasegawa-Johnson, Chang Yoo
| Challenge: | Existing supervised sign language recognition systems rely on well-annotated data . instead, an unsupervised speech-to-sign language recognition system learns to translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Approach: | They propose an unsupervised speech-to-sign language recognition system that can translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Outcome: | The proposed approach outperforms baseline models on sign language corpora by 50% . the proposed approach is available at https://github.com/cactuswiththoughts/UnsupSpeech2Sign.git . |
Linguistically-driven Framework for Computationally Efficient and Scalable Sign Recognition (L18-1)
Copied to clipboard
| Challenge: | a new general framework for sign recognition from monocular video is presented . the framework exploits state-of-the-art learning methods while incorporating features based on what we know about the linguistic composition of lexical signs. |
| Approach: | They propose a general framework for sign recognition from monocular video . they exploit state-of-the-art learning methods while incorporating features from linguistic information . |
| Outcome: | The proposed framework exploits state-of-the-art learning methods while incorporating features based on what we know about linguistic composition of lexical signs. |