Papers by HyoJung Han
Measuring User’s Mental Models of Speech Translation in Human-AI Collaboration (2026.acl-long)
Copied to clipboard
| Challenge: | Existing research on machine translation tools has not revealed how users perceive MT errors and how they evolve through interaction. |
| Approach: | They propose a framework where users accept MT output or request professional re-translation to answer questions based on information presented in a foreign language. |
| Outcome: | The proposed framework can predict where the system is likely to be wrong and how it evolves through interaction. |
SpeechQE: Estimating the Quality of Direct Speech Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in automatic quality estimation for machine translation focus on written language, leaving the speech modality underexplored. |
| Approach: | They propose a new quality estimation system based on cascaded and end-to-end architectures. |
| Outcome: | The proposed system is better suited to estimating the quality of direct speech translation than existing systems designed for text translation. |
Bridging Background Knowledge Gaps in Translation with Automatic Explicitation (2023.emnlp-main)
Copied to clipboard
| Challenge: | despite its potential to help users, NLP research on explicitation is limited because of the lack of adequate evaluation methods. |
| Approach: | They propose automatic methods to generate explicitations from a Wikipedia dataset . they use both intrinsic and extrinsic evaluation to evaluate the system's effectiveness . |
| Outcome: | The proposed system bridges the gap between the source speaker and the target audience . it is effective based on intrinsic and extrinsic evaluation, the authors show . |
XLAVS-R: Cross-Lingual Audio-Visual Speech Representation Learning for Noise-Robust Speech Perception (2024.acl-long)
Copied to clipboard
| Challenge: | Speech recognition and translation systems perform poorly on noisy inputs, which are frequent in realistic environments. |
| Approach: | They propose a cross-lingual audio-visual speech representation model for noise-robust speech recognition and translation in over 100 languages. |
| Outcome: | The proposed model outperforms the previous state-of-the-art by 18.5% WER and 4.7 BLEU on downstream audio-visual speech recognition and translation tasks. |
SimQA: Detecting Simultaneous MT Errors through Word-by-Word Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | a good SimulMT system will allow the downstream QA system to answer correctly as quickly as possible. |
| Approach: | They propose a word-by-word question answering evaluation task to evaluate if models translate salient elements of a question correctly. |
| Outcome: | a new evaluation task aims to show whether models translate salient elements of a question accurately and quickly . evaluators can reveal weaknesses in existing neural systems, hallucinating or omitting facts . human evaluation is too costly and slow to guide system development, authors say . |
Can you map it to English? The Role of Cross-Lingual Alignment in the Multilingual Performance of LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can answer prompts in many languages despite being pre-trained mostly on English text. |
| Approach: | They propose a Discriminative Alignment Index to quantify instance-level alignment across 24 languages other than English and three distinct NLU tasks. |
| Outcome: | The proposed model can perform natural language understanding tasks in 24 languages other than English and three distinct NLU tasks. |