Papers by Haoyu Wu
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding (2022.acl-long)
Copied to clipboard
Mengze Li, Tianbao Wang, Haoyu Zhang, Shengyu Zhang, Zhou Zhao, Jiaxu Miao, Wenqiao Zhang, Wenming Tan, Jin Wang, Peng Wang, Shiliang Pu, Fei Wu
| Challenge: | Existing methods for grounding video frames with dense annotations require enormous amount of human effort. |
| Approach: | They propose to ground natural language in video frames with only one frame labeled . they propose an end-to-end model that eliminates interference of irrelevant frames . |
| Outcome: | The proposed model can ground natural language in all video frames with only one frame labeled . the proposed model eliminates interference of irrelevant frames based on branch search and cropping techniques . |
Defending against Indirect Prompt Injection by Instruction Detection (2025.findings-emnlp)
Copied to clipboard
Tongyu Wen, Chenglong Wang, Xiyuan Yang, Haoyu Tang, Yueqi Xie, Lingjuan Lyu, Zhicheng Dou, Fangzhao Wu
| Challenge: | Indirect Prompt Injection attacks can be exploited by LLMs that are embedded with external data. |
| Approach: | They propose a detection-based approach that leverages the behavioral states of LLMs to identify potential IPI attacks. |
| Outcome: | The proposed approach reduces the success rate of attacks to 0.03% on the BIPIA benchmark. |
Cognitive Scaffold: From Fluid Context to Crystallized Memory for Long-Horizon DeepResearch Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Scaling LLM-based agents to long-horizon deep research is constrained by context-noise trade-off . solving a single query may require hundreds of interactions with noisy environments . |
| Approach: | They propose a factorized memory architecture that decouples the cognitive state into a Fluid Working Context for immediate reasoning and a persistent Knowledge Graph for long-term retention. |
| Outcome: | The Cognitive Scaffold outperforms baselines on Xbench-DeepSearch, BrowseComp-ZH, and GAIA . it achieves 74.7% Avg@3 and 87.0% Pass@3 on xbench, browseComp, and 88.3% Pass@3. |
Speech-Text Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment (2023.acl-long)
Copied to clipboard
Tianshu Yu, Haoyu Gao, Ting-En Lin, Min Yang, Yuchuan Wu, Wentao Ma, Chao Wang, Fei Huang, Yongbin Li
| Challenge: | Existing speech-text pre-training methods are limited to one or two specific tasks, despite their success in speech-language processing tasks. |
| Approach: | They propose a temporal position prediction task to capture the speech-text alignment . they use a textual dialog pre-training task to generalize a response selection task . |
| Outcome: | The proposed model is superior in learning speech-text alignment and multi-turn dialog context. |
AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)
Copied to clipboard
Jiaxin Bai, Wei Fan, Qi Hu, Qing Zong, Chunyang Li, Hong Ting Tsang, Hongyu Luo, Yauwai Yim, Haoyu Huang, Xiao Zhou, Feng Qin, Tianshi Zheng, Xi Peng, Xin Yao, Huiwen Yang, Leijie Wu, JI Yi, Gong Zhang, Renhai Chen, Yangqiu Song
| Challenge: | Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage. |
| Approach: | They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas. |
| Outcome: | The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality. |
SoMeLVLM: A Large Vision Language Model for Social Media Processing (2024.findings-acl)
Copied to clipboard
Xinnong Zhang, Haoyu Kuang, Xinyi Mou, Hanjia Lyu, Kun Wu, Siming Chen, Jiebo Luo, Xuanjing Huang, Zhongyu Wei
| Challenge: | Genereal domain large models lack nuanced multimodal understanding of social media . general domain models focus more on text than other modalities, which is not consistent with real-world user habits. |
| Approach: | They propose a Large Vision Language Model for Social Media Processing that combines five key capabilities to understand and generate real social media behavior. |
| Outcome: | The proposed model achieves state-of-the-art performance in multiple social media tasks. |
Detecting Continuously Evolving Scam Calls under Limited Annotation: A LLM-Augmented Expert Rule Framework (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect scam calls rely on labeled data and assume static distribution of scam narratives. |
| Approach: | They propose a method leveraging large language models to detect continuously evolving scam calls . scammers continuously evolve their tactics, making these methods less effective . |
| Outcome: | The proposed approach is based on large language models to detect continuously evolving scam calls. |
Evaluating the Expressive Appropriateness of Speech in Rich Contexts (2026.acl-long)
Copied to clipboard
Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, Yuheng Lu, Guanrou Yang, Xuanchen Li, Hexin Liu, Chunyu Qiang, Cheng Gong, Yifan Yang, Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, Xie Chen, Longbiao Wang, Jianwu Dang
| Challenge: | Existing methods for evaluating expressive speech focus on word accuracy, naturalness, signal quality, or emotional intensity at the utterance level. |
| Approach: | They propose a framework for Evaluating Expressive Appropriateness in speech that assesses whether a speech sample aligns with the underlying communicative intent implied by its discourse-level narrative context. |
| Outcome: | The proposed framework outperforms existing speech evaluation and analysis systems on a human-annotated test set. |
Self-Explanation Prompting Improves Dialogue Understanding in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have achieved great success in various NLP tasks, but the vast model parameters pose challenges in downstream fine-tuning. |
| Approach: | They propose a task-agnostic prompting strategy that analyzes each dialogue utterance before task execution to enhance LLMs' comprehension in multi-turn dialogues. |
| Outcome: | The proposed strategy outperforms other zero-shot prompts and matches or exceeds efficacy of few-shot ones. |
Can Large Language Models Tackle Graph Partitioning? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have remarkable capabilities in understanding complex tasks, but they can only handle graph partitioning tasks that require global perception abilities. |
| Approach: | They propose a pipeline for coarsening, reasoning, and refining to enable LLMs to perform graph partitioning on small-scale graphs. |
| Outcome: | The proposed pipeline can handle graph partitioning tasks on small graphs with coarsening, reasoning, and refining. |
FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining (2022.acl-long)
Copied to clipboard
| Challenge: | Tables store rich numerical data, but numerical reasoning over tables is still a challenge. |
| Approach: | They propose a spreadsheet formula is a valuable supervision for numerical reasoning in tables. |
| Outcome: | The proposed method outperforms state-of-the-art methods on three representative datasets of formula prediction, question answering, and cell type classification. |
NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries (2024.findings-eacl)
Copied to clipboard
Wei Zhao, Zhitao Hou, Siyuan Wu, Yan Gao, Haoyu Dong, Yao Wan, Hongyu Zhang, Yulei Sui, Haidong Zhang
| Challenge: | Creating spreadsheet formulas remains a tedious and error-prone task for many end-users . a novel task is proposed to generate spreadsheet formulae from a user's NL query . |
| Approach: | They propose a task to generate formulas that are grounded on a spreadsheet table given a Natural Language query as input. |
| Outcome: | The proposed task generates formulas that are grounded on a spreadsheet table, given a natural language query as input. |