Papers by Heyang Yu
SParC: Cross-Domain Semantic Parsing in Context (P19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | Xu et al., 2017): a dataset for cross-domain semantic parsing in context with 4,298 question sequences. |
| Approach: | They present a dataset for cross-domainSemanticParsing inContext that consists of 4,298 coherent question sequences. |
| Outcome: | The proposed dataset demonstrates that it has greater semantic diversity and can be generalized to unseen domains due to its cross-domain nature and the unseened databases at test time. |
Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions (D19-1)
Copied to clipboard
Rui Zhang, Tao Yu, Heyang Er, Sungrok Shim, Eric Xue, Xi Victoria Lin, Tianze Shi, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | Generating SQL queries from user utterances is an important task to help end users acquire information from databases. |
| Approach: | They propose a context-dependent text-to-SQL generation task that edits previous queries . they use an utterance-table encoder and a table-aware decoder to incorporate context . |
| Outcome: | The proposed model is flexible to change individual tokens and robust to error propagation. |
M3AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Dataset (2024.acl-long)
Copied to clipboard
Zhe Chen, Heyang Liu, Wenyi Yu, Guangzhi Sun, Hongcheng Liu, Ji Wu, Chao Zhang, Yu Wang, Yanfeng Wang
| Challenge: | Publishing open-source academic video recordings is an emerging approach to sharing knowledge online. |
| Approach: | They propose a multimodal, multigenre, and multipurpose audio-visual academic lecture dataset with human annotations for multimodal content recognition and understanding tasks. |
| Outcome: | The proposed dataset can be used for multiple audio-visual recognition and understanding tasks. |
Post-decoder Biasing for End-to-End Speech Recognition of Multi-turn Medical Interview (2024.lrec-main)
Copied to clipboard
| Challenge: | End-to-end (E2E) models are replacing hybrid models for automatic speech recognition tasks. |
| Approach: | They propose a method to optimize E2E models for automatic speech recognition . they propose MED-IT, a multi-turn consultation speech dataset . |
| Outcome: | The proposed method improves on subsets of rare words appearing in training speech. |
VocalNet: Speech LLMs with Multi-Token Prediction for Faster and High-Quality Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show VocalNet outperforms existing open-source speech LLMs despite limited training data. |
| Approach: | They propose a scalable and model-agnostic training framework and a novel multi-token prediction paradigm for speech generation. |
| Outcome: | The proposed model outperforms open-source speech LLMs while outperforming existing open-sourced models. |
MM-SAP: A Comprehensive Benchmark for Assessing Self-Awareness of Multimodal Large Language Models in Perception (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in multimodal large language models (MLLMs) have demonstrated exceptional capabilities in visual perception and understanding, but they also suffer from hallucinations, which limit their reliability as AI systems. |
| Approach: | They propose a benchmark to evaluate self-awareness in perception for multimodal large language models (MLLMs) by integrating image information with knowledge quadrants, and propose MM-SAP to evaluate this capability. |
| Outcome: | The proposed benchmark offers detailed analysis of MLLMs with self-awareness in perception. |
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases (D19-1)
Copied to clipboard
Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, Dragomir Radev
| Challenge: | CoSQL is a corpus for building cross-domain, general-purpose database querying dialogue systems. |
| Approach: | They present a corpus for building cross-domain, general-purpose database querying dialogue systems . they use a Wizard-of-Oz collection of 3k turns plus 10k+ annotated SQL queries . |
| Outcome: | The proposed corpus is based on a Wizard-of-Oz dataset of 3k dialogues querying 200 complex DBs spanning 138 domains. |
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)
Copied to clipboard
Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains. |
| Approach: | They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries. |
| Outcome: | The proposed system outperforms baselines in the open domain task-solving benchmark. |