Papers by Amit Moryossef
Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting (2023.findings-eacl)
Copied to clipboard
| Challenge: | Yin et al. ( 2021) calls for including sign language processing (SLP) in natural language processing research. |
| Approach: | They propose to use a sign language writing system to parse, factorize, decode and evaluate signed languages. |
| Outcome: | The proposed method achieves over 30 BLEU in a bilingual setup and over 20 BLUE in two multilingual setups. |
Considerations for meaningful sign language machine translation based on glosses (2023.acl-short)
Copied to clipboard
| Challenge: | In machine translation, sign language translation based on glosses is becoming more popular . limitations of glossed approaches are not discussed in a transparent manner, and there is no common standard for evaluation. |
| Approach: | They propose to use a gloss-based approach to evaluate machine translation results . they propose to include realistic datasets, stronger baselines and convincing evaluation . |
| Outcome: | The proposed approach is based on a neural gloss translation model. |
sign.mt: Real-Time Multilingual Sign Language Translation Application (2024.emnlp-demo)
Copied to clipboard
| Challenge: | open-source application for real-time multilingual bi-directional translation between spoken and signed languages. |
| Approach: | They present an open-source application for real-time multilingual bi-directional translation between spoken and signed languages. |
| Outcome: | The open-source sign.mt application aims to address the communication divide between the hearing and the deaf. |
Including Signed Languages in Natural Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing research in Sign Language Processing (SLP) rarely explores signed languages . authors urge adoption of an efficient tokenization method and the collection of real-world signed language data . |
| Approach: | They propose to include signed languages as a research area with high social and scientific impact . they review the limitations of current SLP models and identify the open challenges . |
| Outcome: | The proposed model should include signed languages as a research area with high social and scientific impact. |
SwissSLi: The Multi-parallel Sign Language Corpus for Switzerland (2024.lrec-main)
Copied to clipboard
| Challenge: | Using a CC BY-NC-SA 4.0 license, this corpus contains parallel sign language videos and spoken language subtitles. |
| Approach: | They introduce SwissSLi, the first sign language corpus that contains parallel data of all three Swiss sign languages. |
| Outcome: | The proposed corpus contains parallel sign language videos and spoken language subtitles. |
SignCLIP: Connecting Text and Sign Language by Contrastive Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | SignCLIP is an efficient method of learning useful visual representations for sign language processing from large-scale, multilingual video-text pairs without optimizing for a specific task or sign language of limited size. |
| Approach: | They propose a method for learning visual representations for sign language processing from large-scale video-text pairs without directly optimizing for a specific task or sign language. |
| Outcome: | The proposed model can learn from multilingual video-text pairs without optimizing for a specific task or sign language of limited size. |
Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation (N19-1)
Copied to clipboard
| Challenge: | Modern neural generation systems conflate these two steps into a single end-to-end differentiable system. |
| Approach: | They propose to split the generation process into a symbolic text-planning stage that is faithful to the input, followed by a neural generation stage that focuses only on realization. |
| Outcome: | The proposed method improves reliability and adequacy while maintaining fluent output. |
Linguistically Motivated Sign Language Segmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Sign language segmentation is a crucial task in sign language processing systems. |
| Approach: | They propose to combine two kinds of segmentation: segmentation into individual signs and segmentation to segment into phrases, larger units comprising several signs. |
| Outcome: | The proposed model is based on linguistic cues observed in sign language corpora and replaces the predominant IO tagging scheme with BIO taging to account for continuous signing. |