Papers by Vanessa Milan
ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articles (2024.emnlp-main)
Copied to clipboard
Kayo Yin, Chinmay Singh, Fyodor Minakov, Vanessa Milan, Hal Daumé Iii, Cyril Zhang, Alex Lu, Danielle Bragg
| Challenge: | Deaf and hard-of-hearing students face significant barriers in accessing STEM education due to the scarcity of STEM resources in signed languages. |
| Approach: | They develop models to identify fingerspelled words in American Sign Language (ASL) given an English sentence and a video, the model detects which English phrase is fingerspelled in the clip. |
| Outcome: | ASL STEM Wiki is the first continuous signing dataset focused on STEM . it detects fingerspelled words and queries them for appropriate signs to suggest to interpreters. |