Papers by Hao Ye
Optimizing Code Retrieval: High-Quality and Scalable Dataset Annotation through Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for code retrieval struggle to balance scalability and annotation quality. |
| Approach: | They propose a method that integrates functions called within the repository and information on third-party APIs to enhance the annotation context. |
| Outcome: | The proposed method improves the annotation context by incorporating functions called within the repository and information on third-party API functionalities. |
LoopCoder: Scaling Code Intelligence via Looped Language Models (2026.findings-acl)
Copied to clipboard
Jian Yang, Wei Zhang, Shuyue Guo, Yizhi LI, Linzheng Chai, Zhengmao Ye, Shukai Liu, Yuyang Song, Jiajun Wu, Che Liu, Tianyu Zheng, Siwei Wu, Leo L, Xudong Ma, Chuan Hao, Ran Tao, Yan Xing, Jianzhou Wang, Mingjie Tang, Aishan Liu, Zhoujun Li, Xianglong Liu, Weifeng Lv, Bryan Dai
| Challenge: | Large language models have mastered syntax-level code generation, but complex algorithmic reasoning remains a challenge. |
| Approach: | They propose a recurrent inductive bias that aligns with the recursive nature of programming logic. |
| Outcome: | The proposed model achieves comparable performance to standard dense models with more parameters. |
MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering (2025.findings-acl)
Copied to clipboard
Jingqun Tang, Qi Liu, Yongjie Ye, Jinghui Lu, Shu Wei, An-Lan Wang, Chunhui Lin, Hao Feng, Zhen Zhao, Yanjie Wang, Yuliang Liu, Hao Liu, Xiang Bai, Can Huang
| Challenge: | Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool. |
| Approach: | They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images . |
| Outcome: | The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images. |
VistaWise: Building Cost-Effective Agent with Cross-Modal Knowledge Graph for Minecraft (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown significant promise in embodied decision-making tasks within virtual open-world environments, but lack domain-specific knowledge. |
| Approach: | They propose a cost-effective agent framework that integrates cross-modal domain knowledge and finetunes a dedicated object detection model for visual analysis. |
| Outcome: | The proposed framework reduces the requirement for domain-specific training data from millions of samples to a few hundred. |
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)
Copied to clipboard
Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Mengmeng Yang, Zheng Fu, Jinyu Miao, Yining Shi, He Zhe Lim, Li Liu, Tianbao Zhou, Hongyi Wang, Huang Yu, Yifei Hu, Guang Li, Guang Chen, Hao Ye, Lijun Sun, Diange Yang
| Challenge: | Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning. |
| Approach: | AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks. |
| Outcome: | Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% . |
From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Tasks (2025.naacl-long)
Copied to clipboard
Xiaofeng Zhang, Yihao Quan, Chen Shen, Xiaosong Yuan, Shaotian Yan, Liang Xie, Wenxiao Wang, Chaochen Gu, Hao Tang, Jieping Ye
| Challenge: | Large visionlanguage models (LVLMs) are a powerful visual-language reasoning tool. |
| Approach: | They propose to integrate attention analysis with LLaVA-CAM to determine interactions between visual representations. |
| Outcome: | The proposed approach can be used to determine interactions between visual representations. |
Learning While Staying Curious: Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models (2026.acl-long)
Copied to clipboard
Hao Wang, Hao Gu, Hongming Piao, Kaixiong Gong, Yuxiao Ye, Xiangyu Yue, Sirui Han, Yike Guo, Dapeng Wu
| Challenge: | Recent advances establish "SFT-then-RL" as the defacto paradigm for enhancing large reasoning mod- els on automatically verifiable tasks. |
| Approach: | They propose an entropy-preserving SFT method to enhance exploration capabilities through intrinsic curiosity. |
| Outcome: | The proposed method outperforms the vanilla method on reasoning tasks by 2.5 points . it also outperformed the vanilla SFT by 2.9 points on out-of-distribution tasks . |
HCSpec: Two-Tier Horizontal Cascade Speculative Decoding for High-Efficiency Large Language Model Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to decode large language models adopt a homogeneous architecture . autoregressive decoding is a bottleneck because tokens must be generated sequentially . |
| Approach: | They propose a framework that organizes heterogeneous position-specialized draft modules into a horizontal cascade. |
| Outcome: | The proposed framework outperforms the current state-of-the-art (EAGLE3) and achieves 3.72x acceleration over vanilla decoding. |
A Bounding Box is Worth One Token - Interleaving Layout and Text in a Large Language Model for Document Understanding (2025.findings-acl)
Copied to clipboard
Jinghui Lu, Haiyang Yu, Yanjie Wang, Yongjie Ye, Jingqun Tang, Ziwei Yang, Binghong Wu, Qi Liu, Hao Feng, Han Wang, Hao Liu, Can Huang
| Challenge: | Existing methods for integrating spatial layouts with text have limitations . existing methods produce overly long text sequences or lack autoregressive traits of LLMs . |
| Approach: | They introduce Interleaving Layout and Text in a Large Language Model (LayTextLLM) they use OCR-derived text and spatial layouts to integrate with LLMs for document understanding . |
| Outcome: | The proposed model shows an increase in performance in KIE and VQA tasks. |
MPL: Multiple Programming Languages with Large Language Models for Information Extraction (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing research focuses on Python for code-style simulation, overlooking the potential of other widely-used PLs during the supervised fine-tuning phase. |
| Approach: | They propose a framework that incorporates programming languages into IE tasks . they introduce function-prompt with virtual running to simulate code-style inputs . |
| Outcome: | The proposed framework exploits the potential of different programming languages during the supervised fine-tuning phase. |
LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay (2024.emnlp-main)
Copied to clipboard
Yihuai Lan, Zhiqiang Hu, Lei Wang, Yang Wang, Deheng Ye, Peilin Zhao, Ee-Peng Lim, Hui Xiong, Hao Wang
| Challenge: | Existing studies on LLM agents' social behaviors are lacking . previous studies focused on positive social behaviors, leaving research on negative social behaviors relatively scarce. |
| Approach: | They propose a framework that features a multi-agent system facilitating efficient communication and interaction with LLM agents. |
| Outcome: | The proposed framework is based on Avalon and evaluates on game success and analyzes agents’ social behaviors. |
SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)
Copied to clipboard
Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li
| Challenge: | Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy. |
| Approach: | They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration. |
| Outcome: | The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent . |
Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputs (2025.coling-main)
Copied to clipboard
| Challenge: | Existing 3D AIGC methods don’t fully unleash human creativity. |
| Approach: | They propose a framework that generates 3D content from multimodal inputs . they propose 198 multimodal text inputs for 3D generation tasks . |
| Outcome: | The proposed framework generates 3D content from multimodal inputs without human intervention. |
LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing (2026.findings-acl)
Copied to clipboard
Hao Li, Yiqun Zhang, Zhaoyan Guo, Chenxu Wang, Shengji Tang, Qiaosheng Zhang, Yang Chen, Biqing Qi, Peng Ye, Lei Bai, Zhen Wang, Shuyue Hu
| Challenge: | Large language model (LLM) routing assigns each query to the best suitable model from an ensemble. |
| Approach: | They introduce a large-scale benchmark and unified framework for LLM routing . they find that many routing methods exhibit similar performance under unified evaluation . |
| Outcome: | The proposed benchmark provides comprehensive metrics for both performance-oriented and performance-cost trade-off routing. |
CausalMACE: Causality Empowered Multi-Agents in Minecraft Cooperative Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on multi-agent collaborative tasks in Minecraft is limited due to inefficiency and limited fault tolerance. |
| Approach: | They propose a framework that incorporates causality to manage dependencies among subtasks. |
| Outcome: | The proposed framework achieves state-of-the-art performance in multi-agent cooperative tasks of Minecraft. |
MAVIS: Multi-Agent Video Retrieval via Structured Video Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for video retrieval rely on embedding-based full-corpus scanning, but there is a bottleneck in semantic asymmetry and computational redundancy. |
| Approach: | They propose a multi-agent framework that rethinks retrieval as cooperative reasoning . they parse raw videos into a structured semantic library, enabling explicit attribute-level indexing . |
| Outcome: | The proposed framework bridges the granularity mismatch gap by parsing raw videos into a structured semantic library . it employs a Logic-aware Debate mechanism with a strict veto protocol . the proposed framework achieves competitive performance without task-specific fine-tuning . |
On the Influence of Masking Policies in Intermediate Pre-training (2021.emnlp-main)
Copied to clipboard
Qinyuan Ye, Belinda Z. Li, Sinong Wang, Benjamin Bolte, Hao Ma, Wen-tau Yih, Xiang Ren, Madian Khabsa
| Challenge: | Existing studies show that inserting an intermediate pre-training stage improves performance of masked language models. |
| Approach: | They propose methods to automate the discovery of optimal masking policies via direct supervision or meta-learning. |
| Outcome: | The proposed method outperforms the heuristic of masking named entities on TriviaQA and can be generalizable beyond that task. |
Tiny Scales, Great Challenges: The Limits of Multimodal LLMs in Scale Recognition (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on a single type of quantity or a specific format, lacking a comprehensive evaluation of scale recognition capabilities. |
| Approach: | They propose a visual scale recognition benchmark built using images from COCO, Open Images, and Flickr to evaluate scale recognition capabilities of multimodal large language models. |
| Outcome: | The proposed model achieves 42.60% accuracy, lower than the 97.40% of humans. |
VEG: Verbal 𝜖-greedy for Semantic Exploration in Multi-Turn RL Agents (2026.acl-industry)
Copied to clipboard
Yongchang Hao, Jie Hao, Yongsheng Mei, Ze Ye, Junyi Chai, Bin Guo, Benjamin Z. Yao, Chenlei Guo, Lili Mou
| Challenge: | Standard RL approaches suffer from reward sparsity and mode-seeking behavior . lack of diversity hinders exploration necessary for optimal learning . |
| Approach: | They propose a framework that leverages external feedback as a dynamic control variable to explicitly balance exploration and exploitation within the semantic space. |
| Outcome: | Experiments on Tau Bench and SearchQA show that the proposed framework outperforms standard RL baselines. |
Benchmarking Large Language Models on Communicative Medical Coaching: A Dataset and a Novel System (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing applications of natural language processing (NLP) focus on patient-centered services, but the potential of NLP to benefit inexperienced doctors remains unexplored. |
| Approach: | They propose a human-AI cooperative framework to assist medical learners in practicing communication skills during patient consultations. |
| Outcome: | The proposed framework enables medical learners to practice communication skills during patient consultations while a coach agent provides immediate, structured feedback. |
Demystifying Uncertainty in LLMs: Active Calibration between Concepts and Human Evaluations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static strategies for mitigating hallucinations do not explicitly model the information gain from interacting with the external environment. |
| Approach: | They propose a calibration-driven interactive learning strategy that selects clarification queries by optimizing calibration error. |
| Outcome: | The proposed method provides theoretical guarantees and empirical gains for reliability. |
Shallow Focus, Deep Fixes: Enhancing Shallow Layers Vision Attention Sinks to Alleviate Hallucination in LVLMs (2025.emnlp-main)
Copied to clipboard
Xiaofeng Zhang, Yihao Quan, Chen Shen, Chaochen Gu, Xiaosong Yuan, Shaotian Yan, Jiawei Cao, Hao Cheng, Kaijie Wu, Jieping Ye
| Challenge: | Multimodal large language models (MLLMs) demonstrate excellent abilities for understanding visual information, but the hallucination remains a challenging problem. |
| Approach: | They propose a training-free approach to enhance vision attention sinks to facilitate convergence of the image token attention sink within shallow layers. |
| Outcome: | The proposed approach improves the convergence of the image token attention sink within shallow layers and strengthens the layer’s focus on the image itself. |
Beyond Surface Features: Advancing Medical Vision-Language Alignment via Dynamic Evidence-Guided Preference Optimization (2026.acl-long)
Copied to clipboard
Zixuan Huang, Zhihong Zhu, Xiaolong Liu, Yanchao Hao, Manman Zhang, Zheng Wei, Bowen Xing, Xian Wu, Ye Li, Fen Miao, Yefeng Zheng
| Challenge: | Existing preference-based methods for medical large vision-Language Models face limitations in medical settings . existing methods are limited by overfitting to superficial cues and pseudo convergence of the preference signal. |
| Approach: | They propose a framework that enables evidence-aware and adaptive preference learning for Med-LVLMs. |
| Outcome: | The proposed framework improves evidence-aware and adaptive preference learning for Med-LVLMs. |
Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED (2022.acl-long)
Copied to clipboard
| Challenge: | Document-level relation extraction is a challenging task as it requires reasoning across multiple sentences. |
| Approach: | They propose to use a recommend-revise scheme to reduce the workload of annotators by providing them with candidate relation instances from distant supervision to supplement and remove relational facts. |
| Outcome: | The proposed dataset is the first large-scale and human-annotated dataset for relation extraction. |
LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains (2025.findings-emnlp)
Copied to clipboard
Liyao Li, Jiaming Tian, Hao Chen, Wentao Ye, Chao Ye, Haobo Wang, Ningtao Wang, Xing Fu, Gang Chen, Junbo Zhao
| Challenge: | Evaluating 52 LLMs reveals that only the strongest models maintain robust performance under increasing context lengths and format diversity. |
| Approach: | They propose a benchmark for evaluating long-context reasoning over semi-structured tables across diverse formats, tasks, and domains. |
| Outcome: | The proposed model outperforms compression-based approaches on tasks requiring semantic integration. |
The Stackelberg Speaker: Optimizing Persuasive Communication in Social Deduction Games (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches focus on information processing and strategy selection, overlooking the significance of persuasive communication in social deduction games. |
| Approach: | They propose a reinforcement learning framework that trains agents to optimize influential utterances for persuasive impact by formalizing turn-based dialogue as a Stackelberg competition . |
| Outcome: | The proposed framework outperforms baselines across four social deduction benchmarks and shows that it is effective in persuasive communication. |
CYCLE-INSTRUCT: Fully Seed-Free Instruction Tuning via Dual Self-Training and Cycle Consistency (2025.emnlp-main)
Copied to clipboard
Zhanming Shen, Hao Chen, Yulei Tang, Shaolin Zhu, Wentao Ye, Xiaomeng Hu, Haobo Wang, Gang Chen, Junbo Zhao
| Challenge: | Existing methods for instruction tuning rely on expensive human-annotated seed data or powerful external teacher models. |
| Approach: | They propose a framework that achieves fully seed-free instruction tuning by employing a dual self-training loop where two models are bootstrapped solely from raw, unlabeled text. |
| Outcome: | The proposed framework outperforms seed-driven back-translation baselines and achieves comparable performance to strongly supervised methods. |