Papers by Xinyue Shen
The Death and Life of Great Prompts: Analyzing the Evolution of LLM Prompts from the Structural Perspective (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has shown that high-quality prompts are essential for LLMs to produce accurate and relevant responses. |
| Approach: | They analyze 10,538 in-the-wild prompts collected from various platforms and develop a framework that decomposes the prompts into eight key components. |
| Outcome: | The proposed framework decomposes 10,538 in-the-wild prompts into eight components. |
JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been used to mitigate misuse and to align with human values. |
| Approach: | They propose to use large-scale evaluations of various jailbreak attacks to identify key patterns and test them under eight advanced defenses. |
| Outcome: | The proposed attacks achieve high success rates but are easy to mitigate by defenses. |
Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media (2025.acl-long)
Copied to clipboard
| Challenge: | Social media platforms are experiencing a growing presence of AI-Generated Texts (AIGTs) however, the misuse of AIGTs could have profound implications for public opinion . |
| Approach: | They collect a dataset with 2.4M posts from 3 major social media platforms . they then construct a diverse dataset to train and evaluate AIGT detectors . |
| Outcome: | The proposed dataset analyzes 2.4M posts from 3 major social media platforms from 2022 to 2024 . it finds that Medium and Quora show marked increases in AAR . |
LLMSurgeon: Diagnosing Data Mixture of Large Language Models (2026.acl-long)
Copied to clipboard
Yaxin Luo, Jiacheng Cui, Xiaohan Zhao, Xinyi Shang, Jiacheng Liu, Xinyue Bi, Zhaoyi Li, Zhiqiang Shen
| Challenge: | a lack of transparency in large language models makes auditing their "digital DNA" difficult. |
| Approach: | They propose a framework that casts DMS as an inverse problem under label-shift assumption . they propose LLMScan, a recipe-verifiable evaluation suite built from open-source LLMs . |
| Outcome: | The proposed framework casts DMS as an inverse problem under label-shift assumption . compared with existing frameworks, it recovers domain mixtures with high fidelity . |
ModSCAN: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalities (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large vision-language models have been widely used but stereotypical biases are unexplored. |
| Approach: | They propose a framework to SCAN stereotypical bias within large vision-language models . they examine stereotype biases with respect to gender and race in three scenarios . |
| Outcome: | The proposed framework can reduce stereotypical biases in large vision-language models . the currently popular models show significant stereotype biase . |
The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training (2026.findings-acl)
Copied to clipboard
Rui Zhang, Hongwei Li, Yun Shen, Xinyue Shen, Wenbo Jiang, Guowen Xu, Yang Liu, Michael Backes, Yang Zhang
| Challenge: | Misaligned large language models can magnify harm by exploiting them to undermine safety . et al., 2022b; Bai e.t., 2023): misalignment, realignment and model-specific resistance are important . |
| Approach: | They evaluate four methods to identify a mechanism asymmetry between attack and defense . they find that ORPO is most effective for misalignment, but DPO excels in realignment . |
| Outcome: | The proposed methods show a mechanism asymmetry between attack and defense . the proposed methods excel in realignment, but at the expense of model utility . |
Open Schrödinger’s Closed Box: Identifying Retrieval Augmented Generation in API-Accessible Large Language Model Services (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are powerful at question-answering but prone to hallucinations due to limited domain-specific or up-to-date knowledge. |
| Approach: | They propose a framework for IDentifying RAG properties in LLM services that integrates LLMs with retrieval systems and adds an external retriever and knowledge database to mitigate hallucinations. |
| Outcome: | The proposed framework detects RAG-enhanced LLMs with 99.97% accuracy with partial or no optional knowledge and nearly 100% when the LLM and database are known. |
When GPT Spills the Tea: Comprehensive Assessment of Knowledge File Leakage in GPTs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that adversarial prompts can induce GPTs to leak knowledge file content. |
| Approach: | They propose a workflow inspired by Data Security Posture Management to identify five leakage vectors for knowledge file leakage using 651,022 GPT metadata and 11,820 flows. |
| Outcome: | The proposed workflow analyzes 651,022 GPT metadata, 11,820 flows, and 1,466 responses to identify five leakage vectors: metadata, GPT initialization, retrieval, sandboxed execution environments, and prompts. |