Papers by Zihao He
RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Models (2025.findings-emnlp)
Copied to clipboard
Jingjing Liu, Zeming Liu, Zihao Cheng, Mengliang He, Xiaoming Shi, Yuhang Guo, Xiangrong Zhu, Yuanfang Guo, Yunhong Wang, Haifeng Wang
| Challenge: | Large Language Models (LLMs) have exhibited significant proficiency in code debugging, especially in automatic program repair. |
| Approach: | They propose a repository-level code debugging dataset with 22 subtypes of errors that supports 8 commonly used programming languages and 3 debug tasks. |
| Outcome: | The proposed dataset supports 8 commonly used programming languages and 3 debugging tasks. |
Generate-on-Graph: Treat LLM as both Agent and KG for Incomplete Knowledge Graph Question Answering (2024.emnlp-main)
Copied to clipboard
Yao Xu, Shizhu He, Jiabei Chen, Zihao Wang, Yangqiu Song, Hanghang Tong, Guang Liu, Jun Zhao, Kang Liu
| Challenge: | Existing methods to integrate LLMs with Knowledge Graphs (KGs) however, these methods are often incomplete to cover all the knowledge required to answer questions. |
| Approach: | They propose to integrate LLMs with Knowledge Graphs (KGs) to address insufficient knowledge and hallucination issues in Large Language Models. |
| Outcome: | The proposed method outperforms existing methods on two datasets. |
OpenHuEval: Evaluating Large Language Model on Hungarian Specifics (2025.findings-acl)
Copied to clipboard
Haote Yang, Xingjian Wei, Jiang Wu, Noémi Ligeti-Nagy, Jiaxing Sun, Yinfan Wang, Győző Zijian Yang, Junyuan Gao, Jingchao Wang, Bowen Jiang, Shasha Wang, Nanjun Yu, Zihao Zhang, Shixin Hong, Hongwei Liu, Wei Li, Songyang Zhang, Dahua Lin, Lijun Wu, Gábor Prószéky, Conghui He
| Challenge: | Recent advances in Large Language Models (LLMs) represent significant strides toward artificial general intelligence (AGI). |
| Approach: | They introduce OpenHuEval, the first benchmark for LLMs focusing on the Hungarian language and specifics. |
| Outcome: | The framework reveals intrinsic patterns and mechanisms of LLMs in non-English languages, with Hungarian serving as an example. |
ALCAP: Alignment-Augmented Music Captioner (2023.emnlp-main)
Copied to clipboard
| Challenge: | Traditional approaches to music captioning ignore the intricate interplay between the two . however, a comprehensive understanding of music necessitates the integration of both these elements. |
| Approach: | They propose a method to learn multimodal alignment between audio and lyrics through contrastive learning. |
| Outcome: | The proposed method achieves new state-of-the-art on two music captioning datasets. |
PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning (2026.acl-long)
Copied to clipboard
Afra Feyza Akyürek, Advait Gosai, Chen Bo Calvin Zhang, Vipul Gupta, Jaehwan Jeong, Anisha Gunjal, Tahseen Rabbani, Maria Mazzone, David Randolph IV, Mohammad Mahmoudi Meymand, Gurshaan Chattha, Paula Rodriguez, Diego A. Mares Buendia, Pavit Singh, Michael Liu, Subodh Chawla, Peter Cline, Lucy Ogaz, Ernesto Gabriel Hernández Montoya, Zihao Wang, Pavi Bhatter, Marcos Ayestaran, Bing Liu, Yunzhong He
| Challenge: | Frontier models often lack a view of performance on open-ended, economically consequential tasks in high-stakes professional domains where practical returns matter most. |
| Approach: | They introduce a professional reasoning benchmark that recruits 182 qualified professionals to contribute questions inspired by their workflows. |
| Outcome: | The proposed model outperforms other models in 114 countries and 47 US jurisdictions on hard subsets. |
Speaker Turn Modeling for Dialogue Act Classification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to DA classification model utterances without incorporating the turn changes among speakers throughout the dialogue, thus treating it no different than non-interactive written text. |
| Approach: | They propose to integrate the turn changes in conversations among speakers when modeling DAs by learning conversation-invariant speaker turn embeddings to represent speaker turns in a conversation. |
| Outcome: | The proposed model captures semantics from the dialogue content while accounting for different speaker turns in a conversation. |
Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing MLLMs are optimized for single-task scenarios and struggle to generalize to diverse contexts. |
| Approach: | They propose a framework that integrates multitask reinforcement learning and generalization capabilities of MLLMs to optimize the judge model across multiple tasks. |
| Outcome: | The proposed framework outperforms baseline models in judgment consistency and correlation with human preferences. |
PlanGenLLMs: A Modern Survey of LLM Planning Capabilities (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on developing LLMs to automate complex planning tasks. |
| Approach: | They propose to provide a comprehensive overview of current LLM planners to fill this gap . they examine performance criteria including completeness, executability, optimality, representation, generalization, and efficiency . |
| Outcome: | The proposed survey examines performance criteria for LLM planners and highlights their strengths and weaknesses. |
STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models can adapt outputs to align with community-specific norms, perspectives and communication styles. |
| Approach: | They propose a benchmark to assess community-specific steering using contrasting reddit communities. |
| Outcome: | STEER-BENCH assesses how well large language models understand community-specific instructions, their resilience to adversarial steering attempts, and their ability to accurately represent cultural and ideological perspectives. |
BigTokDetect: A Clinically-Informed Vision–Language Modeling Framework for Detecting Pro-Bigorexia Videos on TikTok (2026.eacl-long)
Copied to clipboard
Minh Duc Chu, Kshitij Pawar, Zihao He, Roxanna Sharifi, Ross M. Sonnenblick, Magdalayna Curry, Laura DAdamo, Lindsay Young, Stuart Murray, Kristina Lerman
| Challenge: | Social media platforms face escalating challenges in detecting harmful content that promotes muscle dysmorphic behaviors and cognitions (bigorexia). |
| Approach: | They propose a framework for detecting pro-bigorexia content on TikTok using an expert-annotated multimodal benchmark dataset of over 2,200 Tiktok videos labeled by clinical psychiatrists. |
| Outcome: | The proposed framework improves on fine-grained subcategories while commercial models achieve the highest accuracy on primary categories. |
CAST: Achieving Stable LLM-based Text Analysis for Data Analytics (2026.findings-acl)
Copied to clipboard
| Challenge: | Text analysis of tabular data relies on two core operations: summarization for corpus-level theme extraction and tagging for row-level labeling. |
| Approach: | They propose a framework that enhances output stability by constraining the model’s latent reasoning trajectory. |
| Outcome: | The proposed framework improves stability by constraining the model's latent reasoning trajectory. |
How Susceptible are Large Language Models to Ideological Manipulation? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have the potential to exert substantial influence on public perceptions and interactions with information. |
| Approach: | They examine how LLMs can learn and generalize ideological biases from their instruction-tuning data. |
| Outcome: | The LLMs show a startling ability to absorb ideology from one topic and generalize it to even unrelated ones. |
Reading Between the Tweets: Deciphering Ideological Stances of Interconnected Mixed-Ideology Communities (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies treat ideology as a liberal/conservative binary and fail to capture the spectrum of ideologies that may organically emerge in interconnected online communities. |
| Approach: | They propose a method that uses finetuning language models to probe nuanced ideologies of online communities by analyzing discussions of the 2020 election on Twitter. |
| Outcome: | The proposed approach shows higher alignment than baselines for the proposed approach. |
Query-Driven Multimodal GraphRAG: Dynamic Local Knowledge Graph Construction for Online Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to build knowledge graphs with LLMs are constrained by static knowledge bases and ineffective multimodal data integration. |
| Approach: | They propose a Query-Driven Multimodal GraphRAG framework that dynamically constructs local knowledge graphs tailored to query semantics. |
| Outcome: | The proposed framework outperforms unsupervised competitors in cross-modal understanding of complex queries. |
GLIMPSE: Do Large Vision-Language Models Truly Think With Videos or Just Glimpse at Them? (2025.emnlp-main)
Copied to clipboard
Yiyang Zhou, Linjie Li, Shi Qiu, Zhengyuan Yang, Yuyang Zhao, Siwei Han, Yangfan He, Kangqi Li, Haonian Ji, Zihao Zhao, Haibo Tong, Lijuan Wang, Huaxiu Yao
| Challenge: | Existing video benchmarks often resemble image-based questions with scans of only a few key frames, without deep temporal reasoning. |
| Approach: | They propose a video benchmark to assess whether large vision-language models can genuinely think with videos rather than perform superficial frame-level analysis. |
| Outcome: | The proposed benchmark consists of 3,269 videos and over 4,342 highly visual-centric questions across 11 categories, including Trajectory Analysis, Temporal Reasoning, and Forensics Detection. |
Not All Voices Are Rewarded Equally: Probing and Repairing Reward Models across Human Diversity (2025.findings-emnlp)
Copied to clipboard
Zihao Li, Feihao Fang, Xitong Zhang, Jiaru Zou, Zhining Liu, Wei Xiong, Ziwei Wu, Baoyu Jing, Jingrui He
| Challenge: | Using real-world datasets, we conduct the most comprehensive study to date, auditing various state-of-the-art reward models across nine sensitive attributes, including age, gender, ethnicity, etc. |
| Approach: | They propose a method to mitigate group disparities in reward modeling by using real-world data. |
| Outcome: | The proposed method is based on a population-based dataset with nine demographic attributes, including gender, ethnicity, age, gender, and ethnicity. |
Breaking the Reasoning Barrier A Survey on LLM Complex Reasoning through the Lens of Self-Evolution (2025.findings-acl)
Copied to clipboard
Tao He, Hao Li, Jingchang Chen, Runxuan Liu, Yixin Cao, Lizi Liao, Zihao Zheng, Zheng Chu, Jiafeng Liang, Ming Liu, Bing Qin
| Challenge: | OpenAI's O1 and subsequent projects like DeepSeek R1 have significantly advanced research on complex reasoning in LLMs. |
| Approach: | They analyze existing reasoning studies from the perspective of self-evolution and summarize O1-like works from open-source projects like DeepSeek R1 and Kimi-k1.5. |
| Outcome: | The proposed models are based on open-source models and pioneer advanced methodologies like Scaling Reinforcement Learning (RL). |
Community-Cross-Instruct: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities (2024.emnlp-main)
Copied to clipboard
| Challenge: | Social scientists use surveys to learn opinions and beliefs of populations, but these methods are slow, costly, and prone to biases. |
| Approach: | They propose a framework for aligning large language models to online communities by finetuning instruction-output pairs by an advanced LLM to elicit their beliefs. |
| Outcome: | The proposed framework enables cost-effective and automated surveying of diverse online communities. |
InternalInspector I2: Robust Confidence Estimation in LLMs through Internal States (2024.findings-emnlp)
Copied to clipboard
Mohammad Beigi, Ying Shen, Runing Yang, Zihao Lin, Qifan Wang, Ankith Mohan, Jianfeng He, Ming Jin, Chang-Tien Lu, Lifu Huang
| Challenge: | Large Language Models (LLMs) often struggle with generating reliable outputs, often producing high-confidence inaccuracies known as hallucinations. |
| Approach: | They propose a framework that leverages contrastive learning on internal states including attention states, feed-forward states, and activation states of all layers to enhance confidence estimation in LLMs. |
| Outcome: | The framework outperforms existing methods in the hallucination detection benchmark HaluEval and the previous methods at the same time. |
RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking (2026.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) integrates knowledge from tables with an external knowledge base to improve the answer relevance and accuracy. |
| Approach: | They propose a table-corpora-aware RAG framework called T-RAG to integrate external knowledge into Large Language Models (LLMs) they then develop a multi-table question answering benchmark called MultiTableQA which spans 3 different task types, 57,193 tables, and 23,758 questions in total. |
| Outcome: | The proposed framework achieves state-of-the-art accuracy, recall, and runtime performance, with improvements of up to 9.4%. |
JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse (2025.findings-acl)
Copied to clipboard
| Challenge: | Visual Language Action models have shown promise in decision-making tasks, but have been neglected in previous work . |
| Approach: | They propose a new paradigm for visual language action models that enhances the foundation model prior to action-specific tuning by first post-training it on a curated set of visual and linguistic tasks using self-supervised learning. |
| Outcome: | The proposed model outperforms the best agent baseline on a diverse set of atomic tasks and surpasses imitation learning-based policies in Minecraft. |
Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision? (2025.acl-long)
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) with CLIP pipeline are difficult because of the scarcity of labeled data and text supervision, different levels of downstream tasks, and conceptual gaps between domains. |
| Approach: | They propose a multi-modal prompt learning paradigm to adapt pre-trained GNNs to downstream tasks with weak text supervision. |
| Outcome: | The proposed model can generalize graphs to unseen classes with weak text supervision. |
Whose Emotions and Moral Sentiments do Language Models Reflect? (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing research has focused on positional alignment, which measures how closely the models mimic the opinions and stances of different social groups. |
| Approach: | They define the problem of affective alignment, which measures how LMs’ emotional and moral tone represents those of different groups. |
| Outcome: | The results show that the models represent the perspectives of some social groups better than others, suggesting a systemic bias within LMs. |
Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown promise in representing individuals and communities, but evaluating their fidelity remains a challenge. |
| Approach: | They propose a framework for aligning large language models with online communities via instruction-tuning and comprehensively evaluating alignment across various aspects of language. |
| Outcome: | The proposed framework shows that it can be used to create high-fidelity representations of people and communities. |
Detecting Polarized Topics Using Partisanship-aware Contextualized Topic Embeddings (2021.findings-emnlp)
Copied to clipboard
| Challenge: | polarization of the news media has been blamed for fanning disagreement, controversy and even violence. |
| Approach: | They propose a method to automatically detect polarized topics from partisan news sources by corpus-contextualized topic embedding a news corpus on a topic and using cosine distance to capture topical polarization. |
| Outcome: | The proposed method captures topical polarization and shows it can retrieve the most polarized topics. |