Papers by Zihao Yang
Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can solve reasoning and mathematical problems using the Chain-of-Thought technique, but require costly and long CoT data and fine-tuning. |
| Approach: | They propose a method that uses Sparse Autoencoders to extract interpretable features from vanilla CoT and use them to steer the LLM's internal states. |
| Outcome: | The proposed method uses Sparse Autoencoders (SAEs) to extract interpretable features from vanilla CoT and steer the LLM's internal states during generation. |
RippleCOT: Amplifying Ripple Effect of Knowledge Editing in Language Models via Chain-of-Thought In-Context Learning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | et al., 2022: ripple effect challenges knowledge editing for large language models. |
| Approach: | They propose a method to improve the accuracy of large language models by integrating Chain-of-Thought reasoning into the ICL editing approach. |
| Outcome: | RIPPLE-COT outperforms the state-of-the-art on the ripple effect, with gains ranging from 7.8% to 87.1%. |
MedQA-CS: Objective Structured Clinical Examination (OSCE)-Style Benchmark for Evaluating LLM Clinical Skills (2026.eacl-long)
Copied to clipboard
Zonghai Yao, Zihao Zhang, Chaolong Tang, Xingyu Bian, Youxia Zhao, Zhichao Yang, Junda Wang, Huixue Zhou, Won Seok Jang, Feiyun Ouyang, Hong Yu
| Challenge: | Current clinical LLM benchmarks fail to evaluate advanced clinical skills in AI and large language models (LLMs). |
| Approach: | They propose a framework to evaluate large language models (LLMs) using two instruction-following tasks designed to reflect real clinical scenarios. |
| Outcome: | The proposed framework evaluates LLMs through two instruction-following tasks designed to reflect real clinical scenarios. |
Enabling Stroke-Level Structural Analysis of Hieroglyphic Scripts without Language-Specific Priors (2026.findings-acl)
Copied to clipboard
Fuwen Luo, Zihao Wan, Ziyue Wang, Yaluo Liu, Pau Tong Lin Xu, Xuanjia Qiao, Xiaolong Wang, Peng Li, Yang Liu
| Challenge: | Existing structural analysis methods for hieroglyphic scripts are script-specific and labor-intensive. |
| Approach: | They propose a hieroglyphic Stroke Analyzer framework that captures character-internal structures and semantics without handcrafted data. |
| Outcome: | The proposed framework captures character-internal structures and semantics without priors . it can be used to generalize hieroglyphic scripts across languages . |
SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising (2022.findings-naacl)
Copied to clipboard
| Challenge: | Using sketch-based slot filling, text-to-SQL models suffer from over-complexity . et al., e.al., and d.albert, dr., propose a novel method for text- to-Sql generation . |
| Approach: | They propose to train sequence-to-sequence model with Schema-aware Denoising . they propose a clause-sensitive execution guided (EG) decoding strategy . |
| Outcome: | The proposed method improves performance in schema linking and grammar correctness . it also establishes new state-of-the-art on the WikiSQL benchmark . |
CharacterCraft: Bridging the Literature-Reality Dialogue Gap for Practical Role-Playing Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing dialogue datasets have a bias between query distributions and real-world user language usage. |
| Approach: | They propose a framework for Chinese role-playing and a robust evaluation method . they propose specialized Chinese dialogue extraction model and specialized memory retrieval module . |
| Outcome: | The proposed framework extracts character dialogue from novels and ensures high data quality. |
OpenHuEval: Evaluating Large Language Model on Hungarian Specifics (2025.findings-acl)
Copied to clipboard
Haote Yang, Xingjian Wei, Jiang Wu, Noémi Ligeti-Nagy, Jiaxing Sun, Yinfan Wang, Győző Zijian Yang, Junyuan Gao, Jingchao Wang, Bowen Jiang, Shasha Wang, Nanjun Yu, Zihao Zhang, Shixin Hong, Hongwei Liu, Wei Li, Songyang Zhang, Dahua Lin, Lijun Wu, Gábor Prószéky, Conghui He
| Challenge: | Recent advances in Large Language Models (LLMs) represent significant strides toward artificial general intelligence (AGI). |
| Approach: | They introduce OpenHuEval, the first benchmark for LLMs focusing on the Hungarian language and specifics. |
| Outcome: | The framework reveals intrinsic patterns and mechanisms of LLMs in non-English languages, with Hungarian serving as an example. |
ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models (2024.emnlp-main)
Copied to clipboard
Yuxiang Zhang, Jing Chen, Junjie Wang, Yaxin Liu, Cheng Yang, Chufan Shi, Xinyu Zhu, Zihao Lin, Hanwen Wan, Yujiu Yang, Tetsuya Sakai, Tian Feng, Hayato Yamana
| Challenge: | Currently, tool-augmented large language models (LLMs) only achieve total scores of 45.3 and 37.0, respectively, on a scale of 100. |
| Approach: | They propose a multi-level diagnostic process to assess the LLM's hallucinations through two perspectives: depth and breadth. |
| Outcome: | The proposed diagnostic process assesses the hallucinations of large language models through two perspectives: depth and breadth. |
Eliciting Implicit Acoustic Styles from Open-domain Instructions to Facilitate Fine-grained Controllable Generation of Speech (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current work relies on pre-defined rules or templates to control the style of speech. |
| Approach: | They propose to use open-domain instructions to generate speech with the acoustic style that meets users’ needs based on their instructions. |
| Outcome: | The proposed model can be used to generate speech with the acoustic style that meets users’ needs based on open-domain instructions. |
Improving Alignment in LVLMs with Debiased Self-Judgment (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for aligning LVLMs rely on external datasets, human annotations or complex post-processing. |
| Approach: | They propose a method that generates a debiased self-judgment score for LVLMs . this self-evaluation metric is created internally by the model without external resources . |
| Outcome: | The proposed approach outperforms existing methods in reducing hallucinations and safety concerns. |
Improving Automatic Grammatical Error Annotation for Chinese Through Linguistically-Informed Error Typology (2025.coling-main)
Copied to clipboard
| Challenge: | In educational settings, GEC systems provide immediate and consistent feedback to both native (L1) and non-native (L2) language learners. |
| Approach: | They propose a framework that provides detailed feedback on 12-16% of all errors by identifying them under a new error typology, specific enough to uncover subtle differences in error patterns between L1 and L2 writings. |
| Outcome: | The proposed framework can provide detailed feedback on 12-16% of all errors, revealing subtle differences in error patterns between L1 and L2 writings. |
Multi-Agent Collaboration via Cross-Team Orchestration (2025.findings-acl)
Copied to clipboard
Zhuoyun Du, Chen Qian, Wei Liu, Zihao Xie, YiFei Wang, Rennai Qiu, Yufan Dang, Weize Chen, Cheng Yang, Ye Tian, Xuantang Xiong, Lei Han
| Challenge: | Large Language Models (LLMs) have significantly impacted various domains, especially through organized LLM-driven autonomous agents. |
| Approach: | They propose a framework that enables orchestrated teams to jointly propose various task-oriented solutions and interact with their insights in a self-independence while cross-team collaboration environment for superior solutions generation. |
| Outcome: | Experiments show that the framework can generate better software quality compared to state-of-the-art frameworks. |
Intriguing Effect of the Correlation Prior on ICD-9 Code Assignment (2023.acl-srw)
Copied to clipboard
| Challenge: | The Ninth Revision of the International Classification of Diseases (ICD-9) is a standardized coding system used worldwide to classify and code diseases, injuries, and other health conditions. |
| Approach: | They evaluate the usefulness of correlation bias and suggest it could improve ICD-9 code assignment in some cases. |
| Outcome: | The proposed model improves on classes that are more imbalanced and less correlated with other codes, but the effect on individual class can be negative or positive. |
Refined Evaluation for End-to-End Grammatical Error Correction Using an Alignment-Based Approach (2025.coling-main)
Copied to clipboard
| Challenge: | errant is a new evaluation tool that can be used to evaluate end-to-end grammatical error correction systems. |
| Approach: | They propose a method to assess end-to-end grammatical error correction systems using alignment-based alignment methods that reproduce and improve results from existing evaluation tools. |
| Outcome: | The proposed method reproduces and improves results from existing evaluation tools, such as errant, even when applied to raw text input. |
Too Good to be Bad: On the Failure of LLMs to Role-Play Villains (2026.findings-acl)
Copied to clipboard
Zihao Yi, Qingxuan Jiang, Ruotian Ma, Xingyu Chen, Qu Yang, Mengru Wang, Fanghua Ye, Ying Shen, Zhaopeng Tu, Xiaolong Li, Liefeng Bo
| Challenge: | Large Language Models (LLMs) are increasingly tasked with creative generation, but their ability to portray non-prosocial, antagonistic personas remains largely unexamined. |
| Approach: | They propose a moral alignment benchmark to test the safety of large language models . they find that models struggle with traits directly antithetical to safety principles . |
| Outcome: | The proposed model fails to accurately portray morally ambiguous or villainous characters . the model fails most with traits directly antithetical to safety principles . |
Optimizing Packing and Shuffling Strategies for Enhanced Performance in Generative Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | Documents are typically concatenated to chunks of maximum sequence length (MSL) and shuffled in chunks (atom-size chunks). |
| Approach: | They propose to pack and shuff documents in chunks of tokens to prevent overfitting . they also propose to use padding to only include one document per chunk . |
| Outcome: | The proposed method reduces the risk of overfitting and improves generalizability. |
GLIMPSE: Do Large Vision-Language Models Truly Think With Videos or Just Glimpse at Them? (2025.emnlp-main)
Copied to clipboard
Yiyang Zhou, Linjie Li, Shi Qiu, Zhengyuan Yang, Yuyang Zhao, Siwei Han, Yangfan He, Kangqi Li, Haonian Ji, Zihao Zhao, Haibo Tong, Lijuan Wang, Huaxiu Yao
| Challenge: | Existing video benchmarks often resemble image-based questions with scans of only a few key frames, without deep temporal reasoning. |
| Approach: | They propose a video benchmark to assess whether large vision-language models can genuinely think with videos rather than perform superficial frame-level analysis. |
| Outcome: | The proposed benchmark consists of 3,269 videos and over 4,342 highly visual-centric questions across 11 categories, including Trajectory Analysis, Temporal Reasoning, and Forensics Detection. |
CODIS: Benchmarking Context-dependent Visual Comprehension for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Fuwen Luo, Chi Chen, Zihao Wan, Zhaolu Kang, Qidong Yan, Yingjie Li, Xiaolong Wang, Siyu Wang, Ziyue Wang, Xiaoyue Mi, Peng Li, Ning Ma, Maosong Sun, Yang Liu
| Challenge: | Multimodal large language models have demonstrated promising results in a variety of tasks that combine vision and language. |
| Approach: | They propose a benchmark to assess the ability of models to use contextual information in free-form text to enhance visual comprehension. |
| Outcome: | The proposed model fails to extract and utilize contextual information to improve understanding of images. |
InternalInspector I2: Robust Confidence Estimation in LLMs through Internal States (2024.findings-emnlp)
Copied to clipboard
Mohammad Beigi, Ying Shen, Runing Yang, Zihao Lin, Qifan Wang, Ankith Mohan, Jianfeng He, Ming Jin, Chang-Tien Lu, Lifu Huang
| Challenge: | Large Language Models (LLMs) often struggle with generating reliable outputs, often producing high-confidence inaccuracies known as hallucinations. |
| Approach: | They propose a framework that leverages contrastive learning on internal states including attention states, feed-forward states, and activation states of all layers to enhance confidence estimation in LLMs. |
| Outcome: | The framework outperforms existing methods in the hallucination detection benchmark HaluEval and the previous methods at the same time. |
Extending Complex Logical Queries on Uncertain Knowledge Graphs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies on logical queries on knowledge graphs overlook the incompleteness of KGs. |
| Approach: | They propose an ML-based approach to answer soft queries on uncertain knowledge . they propose to use forward inference and backward calibration to avoid catastrophic errors . |
| Outcome: | The proposed method ensures there are no catastrophic cascading errors while maintaining the same complexity as state-of-the-art inference algorithms for first-order queries. |
Experiential Co-Learning of Software-Developing Agents (2024.acl-long)
Copied to clipboard
Chen Qian, Yufan Dang, Jiahao Li, Wei Liu, Zihao Xie, YiFei Wang, Weize Chen, Cheng Yang, Xin Cong, Xiaoyin Che, Zhiyuan Liu, Maosong Sun
| Challenge: | Recent advances in large language models (LLMs) have brought significant changes to various domains, especially through autonomous agents. |
| Approach: | They propose a framework that lets agents learn shortcuts from their past tasks and use them for future task execution. |
| Outcome: | The proposed framework enables agents to tackle unseen software-developing tasks more effectively. |
Detecting AI-Generated Content on Social Media with Multi-modal Language Models (2026.acl-industry)
Copied to clipboard
Chenyang Yang, Shen Yan, Yibo Yang, Litao Hu, Yuchen Liu, Yuan Zeng, Hanchao Yu, Yinan Zhu, Sumedha Singla, Brian Vanover, Huijun Qian, Zihao Wang, Fujun Liu, Aashu Singh, Jianyu Wang, Xuewen Zhang
| Challenge: | Existing methods for AI-generated content detection face poor generalization to newer models, reliance on single modalities, and lack of interpretable explanations. |
| Approach: | They propose a model that curates diverse social media data and trains a vision-language model for detection and explanation. |
| Outcome: | The proposed model achieves state-of-the-art detection performance on public benchmarks and observes positive downstream impacts on user engagement. |
Following the Autoregressive Nature of LLM Embeddings via Compression and Alignment (2025.emnlp-main)
Copied to clipboard
Jingcheng Deng, Zhongtao Jiang, Liang Pang, Zihao Wei, Liwei Chen, Kun Xu, Yang Song, Huawei Shen, Xueqi Cheng
| Challenge: | Experimental results demonstrate that our method significantly outperforms traditional contrastive learning approaches when using the same amount of data. |
| Approach: | They propose a new contrastive learning method built on embedding conditional probability distributions that integrates two tasks: information compression and conditional distribution alignment. |
| Outcome: | The proposed method outperforms traditional contrastive learning approaches and achieves comparable performance to state-of-the-art models when using the same amount of data. |
COSMIC: Generalized Refusal Direction Identification in LLM Activations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods depend on predefined refusal templates detectable in output tokens or manual review. |
| Approach: | They propose a framework that optimally identifies steering directions and target layers using cosine similarity, entirely independent of output text. |
| Outcome: | The proposed framework achieves comparable steering effectiveness without any prior knowledge or assumptions of a model’s refusal behavior such as the use of certain refusal tokens. |