Papers by Guan-Ting Lin
Can LLMs Understand the Implication of Emphasized Sentences in Dialogue? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Emphasis is a crucial component in human communication, which indicates speaker’s intention and implication beyond pure text in dialogue. |
| Approach: | They propose a benchmark dataset with annotated dialogue samples capturing the implications of emphasis. |
| Outcome: | The proposed evaluation pipeline achieves high correlation with human scoring and commercial LLMs perform better than open-source LLM. |
Full-Duplex-Bench-v2: A Multi-Turn Evaluation Framework for Duplex Dialogue Systems with an Automated Examiner (2026.acl-short)
Copied to clipboard
Guan-Ting Lin, Shih-Yun Shan Kuan, Jiatong Shi, Kai-Wei Chang, Siddhant Arora, Shinji Watanabe, Hung-yi Lee
| Challenge: | Full-duplex speech agents are often half-duplice, alternating turns between user and system. |
| Approach: | They propose a streaming framework that integrates with an examiner that enforces staged goals under two pacing setups. |
| Outcome: | The framework reports fluency, multi-turn instruction following, and task-specific competence. |
Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback (2025.acl-long)
Copied to clipboard
Guan-Ting Lin, Prashanth Gurunath Shivakumar, Aditya Gourav, Yile Gu, Ankur Gandhe, Hung-yi Lee, Ivan Bulyko
| Challenge: | Textless Spoken Language Models lag behind text-based Large Language Model (LLM) in semantic coherence and relevance. |
| Approach: | They propose a framework that leverages preference optimization inspired by Reinforcement Learning with Human Feedback to enhance the semantic understanding of SLMs. |
| Outcome: | The proposed framework achieves state-of-the-art performance of SLMs for most benchmarks . it leverages preference optimization inspired by Reinforcement Learning with Human Feedback . |
Introducing Semantics into Speech Encoders (2023.acl-long)
Copied to clipboard
Derek Xu, Shuyan Dong, Changhan Wang, Suyoun Kim, Zhaojiang Lin, Bing Liu, Akshat Shrivastava, Shang-Wen Li, Liang-Hsuan Tseng, Guan-Ting Lin, Alexei Baevski, Hung-yi Lee, Yizhou Sun, Wei Wang
| Challenge: | Existing self-supervised speech encoders contain primarily acoustic rather than semantic information. |
| Approach: | They propose a task-agnostic unsupervised way to incorporate semantic information from large language model (LLM) systems into self-supervised speech encoders without labeled audio transcriptions. |
| Outcome: | The proposed approach improves spoken language understanding (SLU) performance by over 5% on intent classification (IC), with modest gains in named entity resolution (NER) and slot filling (SF), and spoken question answering (SQA) score by over 22%. |
Continual Test-time Adaptation for End-to-end Speech Recognition on Noisy Speech (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current ASR TTA methods focus on non-continual TTA, which limits cross-sample knowledge learning compared to continual TTA. |
| Approach: | They propose a Fast-slow TTA framework that leverages the advantage of continual and non-continual TTA and a Dynamic SUTA method that automatically detects domain shifts and resets the model. |
| Outcome: | The proposed method outperforms non-continual and continual TTA methods while maintaining robustness to domain shifts without requiring domain boundary information. |
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations (2024.acl-long)
Copied to clipboard
| Challenge: | In spoken dialogue, even if two current turns are the same sentence, their responses might differ when they are spoken in different styles. |
| Approach: | They propose a language-to-speech dataset that can model linguistic content and speaking styles. |
| Outcome: | The proposed framework outperforms text-only baselines and prior speech LLMs methods. |