Papers by Haoyu Gao
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) have shown promise in MER, but their internal decision-making mechanisms under modality conflict and missingness remain underexplored. |
| Approach: | They propose a multimodal large language model that can detect and control modality conflicts and missing subsets by a lightweight mechanism that detects and controls modality conflict. |
| Outcome: | The proposed framework improves performance across settings, showing it can handle conflict and missing behaviors. |
HermEs: Interactive Spreadsheet Formula Prediction via Hierarchical Formulet Expansion (2023.acl-long)
Copied to clipboard
Wanrong He, Haoyu Dong, Yihuai Gao, Zhichao Fan, Xingzhuo Guo, Zhitao Hou, Xiao Lv, Ran Jia, Shi Han, Dongmei Zhang
| Challenge: | HermEs is a spreadsheet formula prediction language that is difficult for Excel users without programming experience to master. |
| Approach: | They propose a hierarchical approach to formula prediction via HiEraRchical forMulet ExpanSion . they propose generating formulas in a fixed order using hierarchically generated formulas . |
| Outcome: | The proposed approach improves formula prediction accuracy by guaranteeing correct grammar and streamlining token-level decoding with high-level Formulet. |
CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art large language models (LLMs) have demonstrated impressive code generation capabilities but struggle with real-world software engineering tasks such as revising source code to address code reviews. |
| Approach: | They propose a benchmark to evaluate large language models' ability to bridge both technical and conversational contexts by decomposing the generation task of code refinement into three essential reasoning steps. |
| Outcome: | The proposed benchmark exposes specific model weaknesses in code review comprehension disentangled from their generative automated code refinement results. |
BlendFilter: Advancing Retrieval-Augmented Large Language Models via Query Generation Blending and Knowledge Filtering (2024.emnlp-main)
Copied to clipboard
Haoyu Wang, Ruirui Li, Haoming Jiang, Jinjin Tian, Zhengyang Wang, Chen Luo, Xianfeng Tang, Monica Cheng, Tuo Zhao, Jing Gao
| Challenge: | Retrieval-augmented Large Language Models struggle with complex inputs and noisy knowledge retrieval hindering model effectiveness. |
| Approach: | They propose a query generation method that integrates query generation blending with knowledge filtering to enhance retrieval-augmented LLMs. |
| Outcome: | The proposed approach surpasses state-of-the-art benchmarks on open-domain question answering benchmarks. |
Do Multi-hop Readers Dream of Reasoning Chains? (D19-58)
Copied to clipboard
| Challenge: | Existing models for multihop reasoning are limited in their performance . multi-hop reasoning requires the ability to gather information from multiple passages . |
| Approach: | They propose a method that provides the full reasoning chain of multiple passages instead of just one final passage where the answer appears. |
| Outcome: | The proposed model improves on existing models by providing the full reasoning chain of multiple passages instead of just one final passage where the answer appears. |
HadSkip: Homotopic and Adaptive Layer Skipping of Pre-trained Language Models for Efficient Inference (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to exit pre-trained language models suffer from the limitation that they have to sequentially traverse through all layers prior to the selected exit layer, which degrades their performance. |
| Approach: | They propose a homotopic and adaptive layer skipping fine-tuning method that adaptively selects the layers to skip based on a predefined budget. |
| Outcome: | The proposed method outperforms all state-of-the-art baselines on the GLUE benchmark and shows that it is highly efficient. |
Knowledge-Guided Paraphrase Identification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for paraphrase identification (PI) are limited due to lack of professional knowledge. |
| Approach: | They propose to leverage Wikipedia knowledge to accurately identify paraphrases by mining outline knowledge of given sentences from Wikipedia. |
| Outcome: | The proposed framework outperforms state-of-the-art models on two public datasets: PARADE and clinicalSTS2019. |
Speech-Text Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment (2023.acl-long)
Copied to clipboard
Tianshu Yu, Haoyu Gao, Ting-En Lin, Min Yang, Yuchuan Wu, Wentao Ma, Chao Wang, Fei Huang, Yongbin Li
| Challenge: | Existing speech-text pre-training methods are limited to one or two specific tasks, despite their success in speech-language processing tasks. |
| Approach: | They propose a temporal position prediction task to capture the speech-text alignment . they use a textual dialog pre-training task to generalize a response selection task . |
| Outcome: | The proposed model is superior in learning speech-text alignment and multi-turn dialog context. |
RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have strong generalizability, but fine-tuning involves updating all parameters, rendering full fine-uning resource-intensive. |
| Approach: | They propose a parameter-efficient fine-tuning method that updates all pre-trained parameters during inference. |
| Outcome: | The proposed method outperforms baseline methods on five benchmarks across 20 datasets. |
HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation (2022.acl-long)
Copied to clipboard
Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, Dongmei Zhang
| Challenge: | Existing studies on table reasoning focus on flat tables and hierarchical tables . a new dataset, HiTab, aims to examine numerical reasoning over hierarchic tables based on hierarchically structured tables - a strong challenge for existing baselines and a valuable benchmark for future research. |
| Approach: | They propose a hierarchical question answering and natural language generation dataset to study hierarchic tables. |
| Outcome: | The proposed model shows that it is effective in QA and natural language generation over hierarchical tables. |
Context-Aware Conversation Thread Detection in Multi-Party Chat (D19-1)
Copied to clipboard
| Challenge: | In multi-party chat, it is common for multiple conversations to occur concurrently . a new model that automatically disentangles conversation threads is proposed . |
| Approach: | They propose a Context-Aware Thread Detection model that automatically disentangles conversation threads in chat logs. |
| Outcome: | The proposed model outperforms state-of-the-art models on four real-world chat logs. |
Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflows (2026.findings-acl)
Copied to clipboard
Haoyu Dong, Pengkun Zhang, Yan Gao, Xuanyu Dong, Yilin Cheng, Mingzhe Lu, Adina Yakefu, Shuxin Zheng
| Challenge: | FinWorkBench evaluates real-world enterprise-grade finance and accounting workflows . a human evaluation of GPT 5.1 Pro passes only 38.4% of workflows, a study finds . |
| Approach: | They propose a workflow construction process that combines LLM-assisted mining and expert annotation to build 172 composite workflows. |
| Outcome: | The proposed process combines expert annotation with LLM-assisted mining of workflows from authentic enterprise environments. |
Macedon: Minimizing Representation Coding Rate Reduction for Cross-Lingual Natural Language Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to learn cross-lingual models require limited data to perform cross-linguistic tasks. |
| Approach: | They propose a method to remove language-associated information via minimizing representation coding rate reduction. |
| Outcome: | The proposed model outperforms state-of-the-art models on cross-lingual tasks. |
Self-Explanation Prompting Improves Dialogue Understanding in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have achieved great success in various NLP tasks, but the vast model parameters pose challenges in downstream fine-tuning. |
| Approach: | They propose a task-agnostic prompting strategy that analyzes each dialogue utterance before task execution to enhance LLMs' comprehension in multi-turn dialogues. |
| Outcome: | The proposed strategy outperforms other zero-shot prompts and matches or exceeds efficacy of few-shot ones. |
Few-shot Temporal Pruning Accelerates Diffusion Models for Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing acceleration methods for text generation ignore the importance of the distribution of sampling steps, resulting in slow sampling rates. |
| Approach: | They propose a technique to accelerate diffusion models for text generation without additional training by using a Bayesian optimization approach. |
| Outcome: | The proposed technique achieves 400x acceleration even with minimal sampling steps after down to less than 1 minute of optimization yielding a competitive performance even with minimum sampling steps. |
NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries (2024.findings-eacl)
Copied to clipboard
Wei Zhao, Zhitao Hou, Siyuan Wu, Yan Gao, Haoyu Dong, Yao Wan, Hongyu Zhang, Yulei Sui, Haidong Zhang
| Challenge: | Creating spreadsheet formulas remains a tedious and error-prone task for many end-users . a novel task is proposed to generate spreadsheet formulae from a user's NL query . |
| Approach: | They propose a task to generate formulas that are grounded on a spreadsheet table given a Natural Language query as input. |
| Outcome: | The proposed task generates formulas that are grounded on a spreadsheet table, given a natural language query as input. |