Papers by Shuzhou Yuan
GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit strong In-Context Learning (ICL) capabilities when prompts with demonstrations are used. |
| Approach: | They propose a prompt-based parameter-efficient fine-tuning approach that leverages insights into ICL’s information flow dynamics and hardwires the desired information flow into the GNN. |
| Outcome: | The proposed approach surpasses prompt-based fine-tuning methods in few-shot settings by updating just 0.2% to 0.5% of parameters. |
The Hidden Bias: A Study on Explicit and Implicit Political Stereotypes in Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly integral to information dissemination and decision-making processes. |
| Approach: | They investigate political bias and stereotype propagation across eight prominent LLMs using the two-dimensional Political Compass Test. |
| Outcome: | The political bias and stereotype propagation of large language models is investigated using the two-dimensional Political Compass Test (PCT) key findings reveal a left-leaning political alignment across all investigated models. |
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)
Copied to clipboard
Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong
| Challenge: | Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored. |
| Approach: | They propose a survey structured around the pipeline to identify and improve MI models. |
| Outcome: | The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency. |
Graph-Guided Textual Explanation Generation Framework (2025.emnlp-main)
Copied to clipboard
Shuzhou Yuan, Jingyi Sun, Ran Zhang, Michael Färber, Steffen Eger, Pepa Atanasova, Isabelle Augenstein
| Challenge: | Existing work has questioned their faithfulness, as they may not accurately reflect the model’s internal reasoning process regarding its predicted answer. |
| Approach: | They propose a Graph-Guided Textual Explanation Generation framework that generates a graph neural network layer that guides the NLE generation and generates explanations with greater semantic and lexical similarity to human-written ones. |
| Outcome: | The proposed framework improves NLE faithfulness by up to 12.12% compared to baseline methods on encoder-decoder and decoder-only models. |
ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks (2024.eacl-long)
Copied to clipboard
| Challenge: | Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. |
| Approach: | They propose a prompt-based method for token-level sequence labeling tasks . they propose to decompose an input sentence into single tokens and apply one prompt template to each token. |
| Outcome: | The proposed method outperforms Vanilla fine-tuning and Prompt-Tuning in zero-shot cross-lingual transfer . the method also attains state-of-the-art performance when employed with the mT5 model . |
GraSAME: Injecting Token-Level Structural Information to Pretrained Language Models via Graph-guided Self-Attention Mechanism (2024.findings-naacl)
Copied to clipboard
| Challenge: | Pretrained Language Models (PLMs) benefit from external knowledge stored in graph structures for various downstream tasks. |
| Approach: | They propose a graph-guided self-attention mechanism that integrates token-level structural information into PLMs without additional alignment or concatenation efforts. |
| Outcome: | The proposed model outperforms baseline models and achieves comparable results to state-of-the-art models on WebNLG datasets. |