Papers by Hao Fu
Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review (2025.findings-acl)
Copied to clipboard
Pei Fu, Tongkun Guan, Zining Wang, Zhentao Guo, Chen Duan, Hao Sun, Boming Chen, Qianyi Jiang, Jiayao Ma, Kai Zhou, Junfeng Luo
| Challenge: | Recent advances in vision-language models have unified perception and understanding tasks within Visual Question Answering paradigms. |
| Approach: | They propose to outline timeline, architecture, and pipeline of nearly all TIU MLLMs and review their performance on mainstream benchmarks. |
| Outcome: | The proposed models perform well on mainstream benchmarks and are compared with other models. |
Enhancing Hyperbole and Metaphor Detection with Their Bidirectional Dynamic Interaction and Emotion Knowledge (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for hyperbole and metaphor detection focus on superficial text features, ignoring the associations of hyperbola and metaphor . Existing frameworks focus on identifying superficial text, focusing on superficial features . |
| Approach: | They propose an emotion-guided hyperbole and metaphor detection framework based on bidirectional dynamic interaction. |
| Outcome: | The proposed framework outperforms baseline methods on four datasets. |
E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning (2022.findings-acl)
Copied to clipboard
Jiangjie Chen, Rui Xu, Ziquan Fu, Wei Shi, Zhongqiao Li, Xinbo Zhang, Changzhi Sun, Lei Li, Yanghua Xiao, Hao Zhou
| Challenge: | Existing benchmarks to test word analogy do not reveal the underneath process of analogical reasoning of neural models. |
| Approach: | They propose an explanation benchmark for analogical reasoning using a Civil Service exam . they use a free-text explanation scheme to explain whether an analogy should be drawn . |
| Outcome: | The proposed benchmark is very challenging for state-of-the-art models, it is found. |
HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for enhancing performance through increased use of expert knowledge often result in diminishing sparsity during expert selection. |
| Approach: | They propose a framework that integrates the computational processes of MoE with the concept of knowledge transferring in multi-task learning. |
| Outcome: | The proposed framework outperforms existing methods under identical conditions concerning the number of experts. |
VistaWise: Building Cost-Effective Agent with Cross-Modal Knowledge Graph for Minecraft (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown significant promise in embodied decision-making tasks within virtual open-world environments, but lack domain-specific knowledge. |
| Approach: | They propose a cost-effective agent framework that integrates cross-modal domain knowledge and finetunes a dedicated object detection model for visual analysis. |
| Outcome: | The proposed framework reduces the requirement for domain-specific training data from millions of samples to a few hundred. |
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving (2025.findings-emnlp)
Copied to clipboard
Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Mengmeng Yang, Zheng Fu, Jinyu Miao, Yining Shi, He Zhe Lim, Li Liu, Tianbao Zhou, Hongyi Wang, Huang Yu, Yifei Hu, Guang Li, Guang Chen, Hao Ye, Lijun Sun, Diange Yang
| Challenge: | Vision-Language Models struggle with hallucinations, inefficient reasoning, and limited real-world validation hinders accurate perception and robust step-by-step reasoning. |
| Approach: | AgentThink integrates Chain-of-Thought reasoning with dynamic, agent-style tool invocation for autonomous driving tasks. |
| Outcome: | Experiments on the DriveLMM-o1 benchmark show AgentThink significantly boosts overall reasoning scores by 53.91% and enhances answer accuracy by 33.54% . |
Can Large Language Models Effectively Support Decision-Making in Sudden Emergencies? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research has focused on the earlier stages of emergency response . lack of suitable datasets for reliable and compliance-aware decision-oriented modeling and evaluation is limiting current research . |
| Approach: | They propose a first real-world emergency decision-making dataset EDM-Bench . they propose 'rule-enhanced reasoning framework' that integrates external regulatory knowledge with constrained inference mechanisms to improve both decision safety and interpretability. |
| Outcome: | The proposed framework improves decision safety and interpretability by integrating regulatory knowledge with constrained inference mechanisms. |
CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models (2024.findings-acl)
Copied to clipboard
Yizhi Li, Ge Zhang, Xingwei Qu, Jiali Li, Zhaoqun Li, Noah Wang, Hao Li, Ruibin Yuan, Yinghao Ma, Kai Zhang, Wangchunshu Zhou, Yiming Liang, Lei Zhang, Lei Ma, Jiajun Zhang, Zuowen Li, Wenhao Huang, Chenghua Lin, Jie Fu
| Challenge: | a recent study shows that large language models have limited generalization in low-resource languages like Chinese. |
| Approach: | They propose to evaluate the zero-shot generalizability of large language models to the Chinese language . they release only half of the dataset publicly, with the remainder kept private . |
| Outcome: | The Chinese Instruction-Following Benchmark evaluates the generalizability of LLMs to the Chinese language. |
Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents (2025.emnlp-main)
Copied to clipboard
Haochen Sun, Shuwen Zhang, Lujie Niu, Lei Ren, Hao Xu, Hao Fu, Fangkun Zhao, Caixia Yuan, Xiaojie Wang
| Challenge: | Large Language Models (LLMs) based agent systems have made great strides in real-world applications beyond traditional NLP tasks. |
| Approach: | They propose a new LLM-based Multi-Agent System benchmark, Collab-Overcooked, built on the popular Overcooked-AI game with more applicable and challenging tasks in interactive environments. |
| Outcome: | The proposed benchmark provides a multi-agent framework supporting diverse tasks and objectives and encourages collaboration through natural language communication. |
ALPS: Attention Localization and Pruning Strategy for Efficient Adaptation of Large Language Models (2025.findings-acl)
Copied to clipboard
Hao Chen, Haoze Li, Zhiqing Xiao, Lirong Gao, Qi Zhang, Xiaomeng Hu, Ningtao Wang, Xing Fu, Junbo Zhao
| Challenge: | Prior research has focused on optimizing general-purpose large language models to downstream tasks . however, these approaches inherently introduce data dependency, which hinders generalization and reusability. |
| Approach: | They propose an algorithm that localizes the most task-sensitive attention heads and prunes by restricting attention training updates to these heads, thereby reducing alignment costs. |
| Outcome: | The proposed algorithm achieves 2% performance improvement over baselines on three tasks while localizing the most task-sensitive attention heads. |
Calibrating Inference Time Alignment with Sequence-level Risk Accumulation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to decode large language models (LLMs) often over-reject benign information, limiting their generalizability in real-world scenarios where harmful and benign information coexist. |
| Approach: | They propose a framework to regulate decoding alignments for Large Language Models (LLMs) they employ a reward-guided branch decoding paradigm to incorporate safety awareness during generation. |
| Outcome: | The proposed framework achieves superior performance on four attack benchmarks and two neutral datasets. |
Graph Neural Networks with Generated Parameters for Relation Extraction (P19-1)
Copied to clipboard
| Challenge: | Existing graph neural networks can only process multi-hop relational reasoning on pre-defined graphs and cannot be directly applied in natural language relational reasoning. |
| Approach: | They propose a graph neural network with generated parameters using natural language sentences as inputs. |
| Outcome: | The proposed model can process relational reasoning on graphs and in natural language processing tasks. |
Contextual Representation Learning beyond Masked Language Modeling (2022.acl-long)
Copied to clipboard
| Challenge: | masked language models adopt sampled embeddings as anchors to estimate and inject contextual semantics to representations. |
| Approach: | They propose a representation learning approach that uses embeddings as anchors to model contextual representations. |
| Outcome: | The proposed model achieves 5x speedup and 1.2 points average improvement over MLM. |
Look Beyond Feeling: Unveiling Latent Needs from Implicit Expressions for Proactive Emotional Support (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are gaining popularity as scalable tools for mental health support . however, nearly half of individuals do not receive timely support due to limited selfawareness or reluctance to seek help. |
| Approach: | They propose a proactive emotional support framework that leverages principles of active listening to uncover implicit user needs. |
| Outcome: | The proposed model elicits implicit emotional needs and delivers empathetic support compared to baselines . |
Prototype-based HyperAdapter for Sample-Efficient Multi-task Tuning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained language models are limited to low-data regimes and require learning different modules to adapt to diverse tasks. |
| Approach: | They propose a framework for parameter-efficient fine-tuning that trains modules per task . they use an instance-dense retriever and a prototypical hypernetwork to generate conditional modules . |
| Outcome: | The proposed framework outperforms existing methods on multi-task learning and few-shot transfer learning. |
ERCThinker: Fast-Slow Thinking for Emotion Recognition in Conversation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for ERC lack interpretability and shallow semantics capture deep semantics. |
| Approach: | They propose a Fast-Slow thinking framework for Emotion Recognition in Conversation . they use fine-grained emotion reasoning chains to capture deep semantics . |
| Outcome: | The proposed framework achieves state-of-the-art in explanation and judgment on a benchmark dataset. |
Improving Abstractive Dialogue Summarization with Hierarchical Pretraining and Topic Segment (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for meeting summary have limited the ability to deal with long-term dependency. |
| Approach: | They propose a hierarchical transformer encoder-decoder network with multi-task pre-training to capture key sentences at word level and generate them at word-level. |
| Outcome: | The proposed model is superior to the previous methods in meeting summary datasets AMI and ICSI. |
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)
Copied to clipboard
Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, Zili Wang
| Challenge: | Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence. |
| Approach: | They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included . |
| Outcome: | The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model. |
CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks (2026.acl-long)
Copied to clipboard
Lingyue Fu, Hao Guan, Bolun Zhang, Haowei Yuan, Yaoming Zhu, Lin Qiu, ZongYu Wang, Xuezhi Cao, Xunliang Cai, Weiwen Liu, Weinan Zhang, Yong Yu
| Challenge: | Existing large language models for software engineering rely on coarse-grained pass rates obscuring specific cognitive bottlenecks. |
| Approach: | They propose a repository-level benchmark that dissects coding capabilities through atomized tasks. |
| Outcome: | The proposed framework achieves a 78.55% validity yield, surpassing the 31.7% retention rate of SWE-bench-Verified. |
HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategy (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to cross-document relation extraction (RE) focus on identifying relations between head and tail entities from single sentence or document. |
| Approach: | They propose a hierarchical relation tree-based LLM-based hierarchic classification model for cross-document relation extraction (HCRE) based on predefined relations, the model can perform hierarchically classification level by level. |
| Outcome: | The proposed model outperforms existing baselines and validates its effectiveness. |
Demystifying Uncertainty in LLMs: Active Calibration between Concepts and Human Evaluations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static strategies for mitigating hallucinations do not explicitly model the information gain from interacting with the external environment. |
| Approach: | They propose a calibration-driven interactive learning strategy that selects clarification queries by optimizing calibration error. |
| Outcome: | The proposed method provides theoretical guarantees and empirical gains for reliability. |
JoPR: Joint Emotion Perception and Reasoning for Conversational Emotion Recognition (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for ERC lack human-like emotion reasoning and discrimination between similar emotions. |
| Approach: | They propose a multi-dimension curriculum with long CoT fine-tuning to clone human-like emotion reasoning for conversational emotion recognition. |
| Outcome: | The proposed model outperforms existing methods on three widely used datasets and shows that it is more intuitive and more accurate. |
Improving Text Generation with Student-Forcing Optimal Transport (2020.emnlp-main)
Copied to clipboard
Jianqiao Li, Chunyuan Li, Guoyin Wang, Hao Fu, Yuhchen Lin, Liqun Chen, Yizhe Zhang, Chenyang Tao, Ruiyi Zhang, Wenlin Wang, Dinghan Shen, Qian Yang, Lawrence Carin
| Challenge: | Maximum likelihood estimation (MLE) is used to train models, but during testing, the model is conditioned on previously generated tokens, resulting in exposure bias. |
| Approach: | They propose to use optimal transport to match the sequences generated in MLE and test modes to reduce exposure bias. |
| Outcome: | The proposed method is validated on machine translation, text summarization, and text generation tasks. |
LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains (2025.findings-emnlp)
Copied to clipboard
Liyao Li, Jiaming Tian, Hao Chen, Wentao Ye, Chao Ye, Haobo Wang, Ningtao Wang, Xing Fu, Gang Chen, Junbo Zhao
| Challenge: | Evaluating 52 LLMs reveals that only the strongest models maintain robust performance under increasing context lengths and format diversity. |
| Approach: | They propose a benchmark for evaluating long-context reasoning over semi-structured tables across diverse formats, tasks, and domains. |
| Outcome: | The proposed model outperforms compression-based approaches on tasks requiring semantic integration. |
EgoNormia: Benchmarking Physical-Social Norm Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing VLMs lack robust grounded norm understanding, a new study finds . current VLM models lack robust grounding, despite a high score for safety and privacy . |
| Approach: | They propose a pipeline to generate grounded MCQs from ego-centric videos of human interactions. |
| Outcome: | The proposed pipeline can generate grounded MCQs from egocentric video . it shows that current VLMs lack robust grounded norm understanding . |
NavA3: Understanding Any Instruction, Navigating Anywhere, Finding Anything (2026.acl-long)
Copied to clipboard
Lingfeng Zhang, Xiaoshuai Hao, Yingbo Tang, Haoxiang Fu, Xinyu Zheng, Pengwei Wang, Zhongyuan Wang, Wenbo Ding, Shanghang Zhang
| Challenge: | Existing embodied navigation methods struggle with such tasks due to their limitations in comprehending high-level human instructions and localizing objects with an open vocabulary. |
| Approach: | They propose a hierarchical framework for long-horizon navigation that integrates human instructions with 3D scene views. |
| Outcome: | The proposed model achieves SOTA results and can complete long-horizon navigation tasks across different robot embodiments in real-world environments. |
Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing (N19-1)
Copied to clipboard
| Challenge: | Variational autoencoders (VAEs) with an auto-regressive decoder have been applied for many natural language processing tasks. |
| Approach: | They propose a cyclical annealing schedule which repeats the process of increasing multiple times to learn more meaningful latent codes progressively by leveraging previous learning cycles as warm re-restart. |
| Outcome: | The proposed method improves on a broad range of NLP tasks, including language modeling, dialog response generation and semi-supervised text classification. |