Automated Extraction of Prosodic Structure from Unannotated Sign Language Video (2024.lrec-main)
Copied to clipboard
| Challenge: | a new method for analyzing prosody in sign languages uses the velocity profile of the hands . the velocity profiles of hand movements can be used to analyse prosodic structure . |
| Approach: | They propose a method for extracting velocity information from unlabeled video of sign language using CoTracker. |
| Outcome: | The proposed method can extract prosodic information from unlabeled video clips. |
Similar Papers
Improving Handshape Representations for Sign Language Processing: A Graph Neural Network Approach (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for sign language recognition process a signing sequence holistically, leaving handshape information implicit, which limits both recognition accuracy and linguistic analysis. |
| Approach: | They propose a graph neural network that separates temporal dynamics from static handshape configurations in continuous signing sequences. |
| Outcome: | The proposed approach achieves 46% accuracy across 37 handshape classes, compared to 25% for baseline methods. |
SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work on sign language video processing focuses on the face, hands and body posture of the signer. |
| Approach: | They propose to learn the handshapes and rich facial expressions of sign languages in a self-supervised fashion by learning from individual frames rather than video sequences. |
| Outcome: | The proposed model is more efficient than previous work on sign language pre-training. |
Linguistically-driven Framework for Computationally Efficient and Scalable Sign Recognition (L18-1)
Copied to clipboard
| Challenge: | a new general framework for sign recognition from monocular video is presented . the framework exploits state-of-the-art learning methods while incorporating features based on what we know about the linguistic composition of lexical signs. |
| Approach: | They propose a general framework for sign recognition from monocular video . they exploit state-of-the-art learning methods while incorporating features from linguistic information . |
| Outcome: | The proposed framework exploits state-of-the-art learning methods while incorporating features based on what we know about linguistic composition of lexical signs. |
Sign Language Video Segmentation Using Temporal Boundary Identification (2025.acl-srw)
Copied to clipboard
| Challenge: | Sign language segmentation focuses on identifying temporal boundaries within video . previous methods have relied on frame-level and phrase-level segmentation. |
| Approach: | They propose to use synchronized subtitle data to facilitate temporal boundary recognition by a sequence-to-sequence model with and without attention for subtitle boundary identification. |
| Outcome: | The proposed model outperforms baseline models on optical flow data and aligned subtitles from BOBSL and YouTube-ASL. |
Handshape-Aware Sign Language Recognition: Extended Datasets and Exploration of Handshape-Inclusive Methods (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on sign language recognition encodes videos without acknowledging phonological attributes of signs. |
| Approach: | They propose a single-encoder network and a dual-encoding network for handshape-inclusive sign language recognition. |
| Outcome: | The proposed methods outperform baseline methods in the PHOENIX14T-HS dataset . the proposed methods consistently outperformed baseline methods . |
Linguistically Motivated Sign Language Segmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Sign language segmentation is a crucial task in sign language processing systems. |
| Approach: | They propose to combine two kinds of segmentation: segmentation into individual signs and segmentation to segment into phrases, larger units comprising several signs. |
| Outcome: | The proposed model is based on linguistic cues observed in sign language corpora and replaces the predominant IO tagging scheme with BIO taging to account for continuous signing. |
Modeling Intensification for Sign Language Generation: A Computational Approach (2022.findings-acl)
Copied to clipboard
| Challenge: | End-to-end sign language generation models do not accurately represent prosody in sign language. |
| Approach: | They propose to model intensification in a data-driven manner to improve prosody in generated sign languages by modeling temporal and spatial variations. |
| Outcome: | The proposed models improve the prosody of generated sign languages by using data-driven models. |
A Low-Cost Motion Capture Corpus in French Sign Language for Interpreting Iconicity and Spatial Referencing Mechanisms (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing tools for automatic translation of sign language videos into transcribed texts are limited. |
| Approach: | They propose to use deep learning methods to circumvent the use of models in spatial referencing recognition by a 3D skeleton and a software program to capture and post-process the LSF-SHELVES corpus. |
| Outcome: | The proposed system targets iconicity and spatial referencing in french sign language . it is light-weight and low-cost to collect data from a large panel of signers . |
Unsupervised Discrete Representations of American Sign Language (2024.emnlp-main)
Copied to clipboard
| Challenge: | Modern NLP models use discrete tokens to represent continuous signals, such as videos, audio, or gestures . modalities that are continuous are difficult to use with discrete models, such a LLM . |
| Approach: | They propose a method that discretizes sequences of fingerspelling signs into tokens . they also propose 'loss function' to improve interpretability of the tokens. |
| Outcome: | The proposed method improves the performance of the tokenizer on downstream tasks. |
Sign Languages and the Online World Online Dictionaries & Lexicostatistics (L18-1)
Copied to clipboard
| Challenge: | Several online dictionaries documenting the lexicon of a variety of sign languages are available . methodological issues must be addressed regarding how these resources are used for research purposes. |
| Approach: | They propose a web-based tool for annotating the articulatory features of signs . they compare handshapes for four Asian SLs and handshape for the entire sample . |
| Outcome: | The proposed tool compares handshapes and handsights of Asian SLs with European, American, and Brazilian SL samples. |