Papers by Jiatong He
Wav2Gloss: Generating Interlinear Glossed Text from Speech (2024.acl-long)
Copied to clipboard
Taiqi He, Kwanghee Choi, Lindia Tjuatja, Nathaniel Robinson, Jiatong Shi, Shinji Watanabe, Graham Neubig, David Mortensen, Lori Levin
| Challenge: | Interlinear Glossed Text (IGT) is a form of linguistic annotation that can support documentation and resource creation for endangered languages. |
| Approach: | They propose a task in which these four annotation components are extracted automatically from speech and introduce a dataset to lay the groundwork for future research on IGT generation from speech. |
| Outcome: | The proposed dataset provides the first dataset to lay the groundwork for future research on IGT generation from speech, including end-to-end versus cascaded, monolingual versus multilingual, and single-task versus multiple-task approaches. |
Multimodal Neural Machine Translation: A Survey of the State of the Art (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal neural machine translation (MNMT) is a task that aims to translate text into the target language using neural networks. |
| Approach: | They propose to integrate other modalities with textual data to enhance translation performance. |
| Outcome: | The proposed task aims to integrate visual modality with textual data to improve translation quality. |