| Challenge: | Existing methods to translate natural language descriptions into visualization queries focus on spoken languages, not sign languages. |
| Approach: | They propose a sign language interface that enables the DHH community to engage more fully with data analysis. |
| Outcome: | The proposed interface can be used by the deaf and hard-of-hearing community. |
Similar Papers
Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks fail to reflect real-world communication needs and are limited in their coverage. |
| Approach: | They present a comprehensive index of sign-language datasets, covering 120 resources across 35 sign languages. |
| Outcome: | The proposed index covers 120 resources across 35 sign languages. |
Text2Vis: A Challenging and Diverse Benchmark for Generating Multimodal Visualizations from Text (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown promise in generating visualizations from natural language, but lack of comprehensive benchmarks limits their capabilities. |
| Approach: | They propose a framework that jointly refines the textual answer and visualization code to improve GPT-4o's pass rate from 26% to 42% over direct approach. |
| Outcome: | The proposed framework increases GPT-4o’s pass rate from 26% to 42% over the direct approach and improves chart quality. |
How to Align Multiple Signed Language Corpora for Better Sign-to-Sign Translations? (2025.naacl-long)
Copied to clipboard
| Challenge: | despite the growing need for advanced signing technologies, signed language resources remain scarce. |
| Approach: | They propose a linguistically informed alignment algorithm that matches instances between signed languages . they compare similarities and differences across three signed languages to develop a model . |
| Outcome: | The proposed algorithm performs well on automatic metrics for sign-to-sign translation and generation. |
NLP+Vis: NLP Meets Visualization (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | This tutorial will introduce NLP+Vis with a focus on two main threads of work: NLP for Vis and Vis for NLP. |
| Approach: | tutorial will introduce NLP+Vis with a focus on two main threads of work . overview of research topics on combining NLP and Vis techniques will be covered . |
| Outcome: | The tutorial will introduce NLP+Vis with a focus on two main threads of work . it will provide an overview of research topics on combining NLP and Vis techniques . |
lingvis.io - A Linguistic Visual Analytics Framework (P19-3)
Copied to clipboard
Mennatallah El-Assady, Wolfgang Jentner, Fabian Sperrle, Rita Sevastjanova, Annette Hautli-Janisz, Miriam Butt, Daniel Keim
| Challenge: | Using a modular framework, linguistic visual analytics applications can be rapidly prototypized using a web-based framework. |
| Approach: | They propose a modular framework for rapid prototyping of linguistic, web-based, visual analytics applications. |
| Outcome: | The proposed framework supports rapid prototyping of linguistic, web-based, visual analytics applications. |
WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language (2022.acl-short)
Copied to clipboard
| Challenge: | Signed Language Processing (SLP) is a major form of NLP, but has been overlooked by the NLP community. |
| Approach: | They leverage existing resources to construct a large-scale dataset of American Sign Language signs annotated with six different phonological properties. |
| Outcome: | The proposed model outperforms existing approaches on signs unobserved during training. |
Gloss2Text: Sign Language Gloss translation using LLMs and Semantically Aware Label Smoothing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to sign language translation use gloss annotations as an intermediary . a new approach to use large language models and word embeddings to improve Gloss2Text translation is needed. |
| Approach: | They propose to leverage large language models pre-trained on expansive and diverse corpora to improve Gloss2Text translation stage by using data augmentation and label-smoothing loss function. |
| Outcome: | The proposed approach surpasses state-of-the-art methods on the PHOENIX Weather 2014T dataset . it shows that gloss annotations can be used to guide the translation process . |
Open-Domain Sign Language Translation Learned from Online Video (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on sign language translation has focused mainly on data collected in controlled environments or domains, which limits its applicability to real-world settings. |
| Approach: | They propose to use sign search as a pretext task and fusion of mouthing and handshape features to improve sign language translation in real-world settings. |
| Outcome: | The proposed techniques produce consistent and large improvements over baseline models based on prior work. |
Listen, Decipher and Sign: Toward Unsupervised Speech-to-Sign Language Recognition (2023.findings-acl)
Copied to clipboard
Liming Wang, Junrui Ni, Heting Gao, Jialu Li, Kai Chieh Chang, Xulin Fan, Junkai Wu, Mark Hasegawa-Johnson, Chang Yoo
| Challenge: | Existing supervised sign language recognition systems rely on well-annotated data . instead, an unsupervised speech-to-sign language recognition system learns to translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Approach: | They propose an unsupervised speech-to-sign language recognition system that can translate between spoken and sign languages by observing only non-parallel speech and sign-language corpora. |
| Outcome: | The proposed approach outperforms baseline models on sign language corpora by 50% . the proposed approach is available at https://github.com/cactuswiththoughts/UnsupSpeech2Sign.git . |
ISLTranslate: Dataset for Translating Indian Sign Language (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing sign language translation datasets are limited for the Indian sign language. |
| Approach: | They propose to use ISLTranslate to create a sign language translation dataset for Indian Sign Language consisting of 31k ISL-English sentence/phrase pairs. |
| Outcome: | The proposed dataset is the largest for Indian Sign Language translation dataset . it compares with a transformer-based model to validate the performance of existing systems. |