Papers by Pin-Yu Chen
STAR: Spectral Truncation and Rescale for Model Merging (2025.naacl-short)
Copied to clipboard
| Challenge: | Model merging is an efficient way of obtaining a multi-task model from several pretrained models without further fine-tuning. |
| Approach: | They propose a model merging technique that aims at mitigating "merging conflicts" by truncating small components in the respective spectral spaces and then an automatic parameter rescaling scheme to retain the nuclear norm of the original matrix. |
| Outcome: | The proposed model outperforms baseline models on flan-T5 by 4.2% and is robust to hyperparamater choice. |
OjaKV: Context-Aware Online Low-Rank KV Cache Compression (2026.findings-acl)
Copied to clipboard
Yuxuan Zhu, David H. Yang, Mohammad Mohammadi Amiri, Keerthiram Murugesan, Tejaswini Pedapati, Pin-Yu Chen
| Challenge: | Existing methods for inference use static, offline-learned subspaces that perform poorly under distribution shifts. |
| Approach: | They propose a framework that integrates a storage policy with an online subspace adaptation to preserve key-value tokens in full rank as high-fidelity anchors. |
| Outcome: | Experiments show that OjaKV maintains or improves zero-shot accuracy at high compression ratios, achieving the strongest gains on long-context benchmarks requiring complex reasoning. |
GRE Score: Generative Risk Evaluation for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models have revolutionized generative tasks, but concerns about their trustworthiness and vulnerability to adversarial attacks persist. |
| Approach: | They propose an attack-independent evaluation of LLM robustness using conditional generation for synthetic text creation and a method to quantify the model's resilience. |
| Outcome: | The proposed method achieves a consistent ranking of LLM robustness when compared to the attack-based model ranking on TrustLLM (CITATION). |
ImReasoner: Improving Memory-based Language Models for Reasoning-in-a-Haystack Tasks (2026.acl-long)
Copied to clipboard
Ching-Yun Ko, Payel Das, Sihui Dai, Georgios Kollias, Subhajit Chaudhury, Aurelie C. Lozano, Pin-Yu Chen
| Challenge: | despite advances, large language models exhibit brittleness on tasks that require multi-step reasoning over long contexts. |
| Approach: | They propose to explicitly encode contexts as ordered memory and perform iterative retrieval to construct reasoning chains. |
| Outcome: | The proposed frameworks fail to show emergent reasoning generalization in a weakly supervised scenario . the proposed framework is based on a synthetic benchmark to stress-test the models . |
Attention Tracker: Detecting Prompt Injection Attacks in LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are vulnerable to prompt injection attacks, where malicious inputs manipulate the model into ignoring original instructions and executing designated actions. |
| Approach: | They propose a training-free method that tracks attention patterns on instruction to detect prompt injection attacks without additional inference. |
| Outcome: | The proposed method shows an improvement of up to 10.0% over existing methods and performs well even on small LLMs. |
ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval (2026.acl-long)
Copied to clipboard
David H. Yang, Yuxuan Zhu, Mohammad Mohammadi Amiri, Keerthiram Murugesan, Tejaswini Pedapati, Subhajit Chaudhury, Pin-Yu Chen
| Challenge: | Large language models generate long chain of thoughts but memory footprint grows with output length . prior work on KV cache optimization focused on compressing long input context . |
| Approach: | They propose a new approach that compresses verbose reasoning thoughts into summaries . they use a dynamic KV cache selection policy that leverages these summary keys . |
| Outcome: | The proposed approach reduces memory usage while avoiding full-cache attention at each step. |
Locally Differentially Private Document Generation Using Zero Shot Prompting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing privacy protections are limited by pretrained large language models . a new mechanism that leverages the power of large language model de-anonymization is proposed . |
| Approach: | They propose a locally differentially private mechanism that leverages pretrained large language models to counter author de-anonymization attacks by minimizing the impact on downstream utility. |
| Outcome: | The proposed mechanism reduces the success rate of de-anonymization attacks against IMDB datasets while minimizing the impact on downstream utility. |
Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs (2026.findings-acl)
Copied to clipboard
Yue Huang, Haomin Zhuang, Jiayi Ye, Han Bao, Yanbo Wang, Hang Hua, Siyuan Wu, Pin-Yu Chen, Xiangliang Zhang
| Challenge: | prevailing taxonomies neglect robustness and honesty, yielding safer-on-paper but less useful systems. |
| Approach: | They propose a soft-gating pipeline where a guardian predicts a binary risk label plus a concise explanation and prepends this advice to the original query for re-inference. |
| Outcome: | The proposed model maintains safety while reducing over-refusal. |
Word Mover’s Embedding: From Word2Vec to Document Embedding (D18-1)
Copied to clipboard
Lingfei Wu, Ian En-Hsu Yen, Kun Xu, Fangli Xu, Avinash Balakrishnan, Pin-Yu Chen, Pradeep Ravikumar, Michael J. Witbrock
| Challenge: | Recent work has demonstrated that Word Mover’s Distance (WMD) that aligns semantically similar words yields unprecedented KNN classification accuracy. |
| Approach: | They propose a Word Mover’s Distance (WMD) method that aligns semantically similar words to generate unsupervised sentences or documents embeddings. |
| Outcome: | The proposed method consistently outperforms state-of-the-art techniques on 9 benchmark text classification datasets and 22 textual similarity tasks. |
Hey, That’s My Data! Token-Only Dataset Inference in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing dataset inference methods require logit access, but many modern LLMs restrict such access. |
| Approach: | They propose a token-only dataset inference framework that allows models to overwrite prior knowledge when trained on new data. |
| Outcome: | The proposed framework overwrites prior knowledge when trained on new data. |
Duwak: Dual Watermarks in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing watermark techniques are effective in embedding single human-imperceptible and machine-detectable patterns without significantly affecting generated text quality and semantics. |
| Approach: | They propose to embed dual secret patterns in token probability distribution and sampling schemes to enhance the efficiency of watermarking. |
| Outcome: | The proposed method achieves highest watermark quality at the lowest required token count for detection, up to 70% less than existing techniques, especially under post paraphrasing attacks. |
Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets (2026.acl-long)
Copied to clipboard
| Challenge: | Existing mitigation strategies focus on reactively addressing jailbreak incidents after safety guardrails have been compromised. |
| Approach: | They investigate the degradation of safety guardrails through the lens of representation similarity between upstream alignment datasets and downstream fine-tuning tasks. |
| Outcome: | The proposed model reduces harmfulness score by 10.33% when compared to baseline models. |
A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models are vulnerable to adversarial attacks, but their vulnerability is underexplored. |
| Approach: | They propose to concatenate a perturbed but semantically similar tweet into a model that fools stock prediction models. |
| Outcome: | The proposed method achieves consistent success rates and causes significant monetary loss in trading simulation by simply concatenating a perturbed but semantically similar tweet. |
Defensive Prompt Patch: A Robust and Generalizable Defense of Large Language Models against Jailbreak Attacks (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have showcased their ability to understand and generate text akin to human interaction. |
| Approach: | They propose a prompt-based defense mechanism specifically designed to protect LLMs against jailbreak attacks by introducing jailbreak prompts into malicious queries. |
| Outcome: | Empirical results show that the proposed defense outperforms existing defense strategies in balancing safety and utility while maintaining high utility. |
RiskLab: A Controlled Toolkit for Probing Emergent Risks in LLM-Based Multi-Agent Systems (2026.acl-demo)
Copied to clipboard
Yu Jiang, Wenjie Wang, Yue Huang, Yanbo Wang, Zhenhong Zhou, Xiuying Chen, Yang Liu, Pin-Yu Chen, Wei Wang, Xiangliang Zhang
| Challenge: | Recent advances in large language model (LLM) agents have accelerated deployment of multi-agent systems for complex tasks. |
| Approach: | They propose an open-source toolkit for instantiating, probing, and measuring emergent risks in LLM-based multi-agent systems under controlled conditions. |
| Outcome: | The proposed toolkit is based on a structured topology–environment–protocol–agent–task quintuple enabling reproducible studies of how communication structure, coordination mechanisms, and incentives shape system-level risks. |
Language Agnostic Code Embeddings (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies show that code language models have strong cross-lingual traits, but their multilingual representations can be dissected into a language-specific syntax component and a semantic component. |
| Approach: | They propose to isolate and eliminate language-specific components from multilingual code embeddings to improve downstream code retrieval tasks. |
| Outcome: | The proposed model improves retrieval tasks by removing language-specific components . the proposed model can be used to perform a variety of code generation tasks . |
Combining Domain and Alignment Vectors Provides Better Knowledge-Safety Trade-offs in LLMs (2025.acl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel in specific technical fields, but are not explicitly trained to be safe. |
| Approach: | They propose a model merging-based alignment method that allows for safer domain-specific models that preserve their utility. |
| Outcome: | The proposed method improves safety alignment on LLMs with minimal degradation on domain-specific benchmarks. |
Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning (P18-1)
Copied to clipboard
| Challenge: | Visual language grounding is widely studied in modern neural image captioning systems . a novel algorithm for crafting adversarial examples in image captions is proposed . |
| Approach: | They propose an algorithm to craft adversarial examples in machine vision and perception . their approach provides two evaluation approaches to check if they can mislead systems . |
| Outcome: | The proposed algorithm can craft visually-similar adversarial examples with randomly targeted captions or keywords, and the results are transferable to other image captioning systems. |
A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models are pre-trained on trillions of tokens and instruction-tuned or aligned to specific preferences. |
| Approach: | They propose guidelines to help researchers perform more effective parameter-efficient LLM alignment. |
| Outcome: | The proposed methods outperform preference optimization and outperformed pre-trained models on three key axes. |