Papers by Yushi Yang
Unveiling Project-Specific Bias in Neural Code Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) based neural code models struggle to generalize effectively to real-world inter-project out-of-distribution data. |
| Approach: | They propose a Cond-Idf measurement to measure the relatedness of a token with a label and its project-specificness. |
| Outcome: | The proposed framework improves both inter-project OOD generalization and adversarial robustness while not sacrificing accuracy on intra-project IID data. |
How Does DPO Reduce Toxicity? A Mechanistic Neuron-Level Analysis (2025.emnlp-main)
Copied to clipboard
| Challenge: | Direct Preference Optimization (DPO) is a popular choice of safety fine-tuning algorithms, but prior explanations of its effects only account for dampened toxic neurons in the MLP layers. |
| Approach: | They analysed four language models and found that toxic neurons only account for 2.5% to 24% of DPO’s effects across models. |
| Outcome: | The proposed method outperforms DPO in reducing toxicity while preserving perplexity, without requiring any weight updates. |
KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional Knowledge Graph Question Answering (KGQA) methods rely on semantic parsing to retrieve knowledge strictly necessary for answer generation. |
| Approach: | They propose a retrieval-filtering-summarization pipeline that enhances QA coverage by retrieving a broader subgraph likely to contain relevant information. |
| Outcome: | The proposed pipeline surpasses state-of-the-art solutions by about 7% in quality and exceeds GPT-4o (Tool) by 10-21%. |
Large Language Models Can Be Contextual Privacy Protection Learners (2024.emnlp-main)
Copied to clipboard
Yijia Xiao, Yiqiao Jin, Yushi Bai, Yue Wu, Xianjun Yang, Xiao Luo, Wenchao Yu, Xujiang Zhao, Yanchi Liu, Quanquan Gu, Haifeng Chen, Wei Wang, Wei Cheng
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable linguistic comprehension and generation capability, but when applied to specialized industries, they face challenges such as hallucination, insufficient domain knowledge, and failing to incorporate the latest domain knowledge. |
| Approach: | They propose a paradigm for fine-tuning LLMs that effectively injects domain-specific knowledge while safeguarding inference-time data privacy. |
| Outcome: | The proposed model protects private data while enhancing the model's knowledge. |
LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit (2024.emnlp-industry)
Copied to clipboard
Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Dacheng Tao, Xianglong Liu
| Challenge: | Existing quantization techniques have been categorized as 'simple' and 'highly efficient' however, their configurations vary from each other and cannot be fairly compared . |
| Approach: | They propose a plug-and-play compression toolkit to explore the impact of quantization. |
| Outcome: | The proposed toolkit explores the impact of quantization on large language models. |
Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for estimating attention importance for tokens are ineffective . dLLMs require bidirectional attention, which limits inference efficiency . |
| Approach: | They propose a training-free attention sparsification framework for efficient long-context inference . they propose 'sink-aware pruning strategy' to accurately estimate and remove redundant computation . |
| Outcome: | The proposed approach offers 29 lossless speedup under 32K context length. |
LLMs Don’t Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations (2025.emnlp-main)
Copied to clipboard
Harry Mayne, Ryan Othniel Kearns, Yushi Yang, Andrew M. Bean, Eoin D. Delaney, Chris Russell, Adam Mahdi
| Challenge: | Existing studies on language models' ability to explain their decisions in natural language have focused on self-generated counterfactual explanations (SCEs). |
| Approach: | They evaluate whether LLMs can generate valid counterfactuals and minimal ones . authors suggest that SCEs are, at best, an ineffective explainability tool . |
| Outcome: | The proposed language models can explain their decisions in natural language, the study finds . the models can produce valid counterfactual explanations, but make small edits that fail to change predictions. |
A Corpus of Adpositional Supersenses for Mandarin Chinese (2020.lrec-1)
Copied to clipboard
| Challenge: | Adpositions are frequent markers of semantic relations, but they are highly ambiguous and vary significantly from language to language. |
| Approach: | They propose to annotate Chinese adpositions in a corpus with all aforementioned supersenses . they adapt a framework that defined a set of supersens according to ostensibly language-independent criteria . |
| Outcome: | The proposed corpus is the first to be broadly annotated with adposition semantics in Chinese . it shows that the supersense categories are well-suited to Chinese adepositions despite syntactic differences from English . |