Papers by Truong-Son Hy
MultiMed: Multilingual Medical Speech Recognition via Attention Encoder Decoder (2025.acl-industry)
Copied to clipboard
Khai Le-Duc, Phuc Phan, Tan-Hanh Pham, Bach Phan Tat, Minh-Huong Ngo, Thanh Nguyen-Tang, Truong-Son Hy
| Challenge: | Multilingual automatic speech recognition (ASR) in the medical domain is a critical foundational task, serving a wide range of downstream applications such as speech translation, spoken language understanding, and voice-activated assistants. |
| Approach: | They present the first multilingual medical ASR dataset and the first collection of small-to-large end-to end medical APR models spanning five languages: Vietnamese, English, German, French, and Mandarin Chinese. |
| Outcome: | The proposed model covers Vietnamese, English, German, French, and Mandarin Chinese, and is the first multilingual ASR dataset across five languages. |
OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching (2025.acl-long)
Copied to clipboard
| Challenge: | Text-to-speech systems have seen significant advances in recent years, driven by improvements in deep learning and neural network architectures. |
| Approach: | They propose a method to explore optimal transport conditional flow matching with one-step sampling and a learned prior as the condition, effectively disregarding preceding states and reducing the number of sampling steps. |
| Outcome: | The proposed method achieves promising performance over existing methods in content accuracy, naturalness, prosody generation, and speaker style preservation. |
SilVar: Speech-Driven Multimodal Model for Reasoning Visual Question Answering and Object Localization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Visual Language Models have demonstrated remarkable capabilities across various tasks, including visual question answering and image captioning. |
| Approach: | They propose an end-to-end multimodal model that leverages speech instructions for reasoning-based visual question answering. |
| Outcome: | The proposed model can process and explain visual scenes from spoken input, moving beyond simple object recognition to reasoning-based interactions. |
Sentiment Reasoning for Healthcare (2025.acl-industry)
Copied to clipboard
| Challenge: | Sentiment Reasoning is an auxiliary task in sentiment analysis where the model predicts both the sentiment label and generates the rationale behind it based on the input transcript. |
| Approach: | They propose a task - Sentiment Reasoning - for both speech and textmodalities and propose 'multimodal multitask framework' . they propose to use a model that generates the rationale behind each predicted label and provides a rationale for model prediction with quality semantically comparable to humans. |
| Outcome: | The proposed task improves model transparency by providing rationale for model prediction with quality semantically comparable to humans while improving model’s classification performance. |
Medical Spoken Named Entity Recognition (2025.naacl-industry)
Copied to clipboard
Khai Le-Duc, David Thulke, Hung-Phong Tran, Long Vo-Dang, Khai-Nguyen Nguyen, Truong-Son Hy, Ralf Schlüter
| Challenge: | Named Entity Recognition (NER) aims to extract named entities from speech and categorise them into types like person, location, organization, etc. |
| Approach: | They present a spoken NER dataset in the medical domain using pre-trained models that are encoder-only and sequence-to-sequence. |
| Outcome: | The dataset is the largest spoken NER dataset in the world regarding the number of entity types, featuring 18 distinct types. |