Papers by Yangyi Chen
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023.findings-acl)
Copied to clipboard
Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji
| Challenge: | Existing models of robustness evaluation are incomprehensive, impractical, and invalid . |
| Approach: | They propose a framework for automatic robustness evaluation that shifts towards model-centric evaluation to further exploit the advantages of adversarial attacks. |
| Outcome: | The proposed framework is based on a model-centric evaluation protocol and a robustness evaluation protocol. |
Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks are vulnerable to backdoors . researchers add extra training task to distinguish poisoned and clean data . |
| Approach: | They propose two tricks that make existing backdoor attacks much more harmful . first trick is to add an extra task to distinguish poisoned and clean data . second trick is using all the clean training data rather than the original clean data. |
| Outcome: | The proposed tricks can significantly improve attack performance in three tough situations including clean data fine-tuning, low-poisoning-rate, and label-consistent attacks. |
Making Pre-trained Language Models both Task-solvers and Self-calibrators (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work shows that pre-trained language models can be effective for high-stake applications, but they become overconfident in their wrong predictions. |
| Approach: | They propose to use extra data to train pre-trained language models to effectively utilize training samples to make them both task-solvers and self-calibrators. |
| Outcome: | The proposed method can be used in three downstream applications, including selective classification, adversarial defense, and model cascading. |
Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP (2022.emnlp-main)
Copied to clipboard
| Challenge: | Textual adversarial samples are often misrepresented in research on security, evaluation, explainability, and data augmentation. |
| Approach: | They propose to use adversarial samples to evaluate their methods on security tasks to demonstrate the real-world concerns rather than developing impractical methods. |
| Outcome: | The proposed method has higher practical value than the current benchmark. |
Exploring the Universal Vulnerability of Prompt-based Learning Paradigm (2022.findings-naacl)
Copied to clipboard
| Challenge: | Prompt-based learning inherits the vulnerability from pre-training, where model predictions can be misled by inserting triggers into the text. |
| Approach: | They propose a potential solution to mitigate this vulnerability by injecting triggers into pre-trained language models using only plain text. |
| Outcome: | The proposed learning paradigm inherits the vulnerability from the pre-training stage . it can totally control or severely decrease the performance of prompt-based models . |
Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for adversarial samples are poorly applied in computer vision . however, textual adversarials are still vulnerable to small perturbations . |
| Approach: | They propose a framework to extend existing adversarial attack methods to textual adversarials by adding optimized perturbations to embedding layer and amplifying them in forward propagation process. |
| Outcome: | The proposed framework achieves better performance even using proxy gradient information and produces more fluent and grammatical adversarial samples compared to baseline methods. |
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Backdoor attacks can manipulate the output of deep neural networks and possess high insidiousness. |
| Approach: | They propose a textual backdoor defense based on outlier word detection that can handle all the textual attacks. |
| Outcome: | The proposed method can handle all the textual backdoor attack situations. |
Automatic Construction of Sememe Knowledge Bases via Dictionaries (2021.findings-acl)
Copied to clipboard
| Challenge: | Sememe knowledge bases (SKBs) are used to analyze natural language processing. |
| Approach: | They propose a method to build sememe knowledge bases from an existing dictionary . they propose to use existing dictionaries to build an English and a French SKB . |
| Outcome: | The proposed method is superior to HowNet, the most widely used SKB that takes decades to build manually. |
R-Tuning: Instructing Large Language Models to Say ‘I Don’t Know’ (2024.naacl-long)
Copied to clipboard
Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, Tong Zhang
| Challenge: | Existing methods for instruction tuning force the model to complete a sentence no matter whether it knows the knowledge or not. |
| Approach: | They propose a new approach to tuning large language models to refrain from answering questions beyond its parametric knowledge by identifying the disparity in parametric and parametric information. |
| Outcome: | The proposed approach improves a model’s ability to answer known questions and refrain from answering unknown questions. |
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to elicit confidence from large language models are limited to binary or inaccurate group-level confidence estimates. |
| Approach: | They propose a training framework that teaches LLMs to express more fine-grained confidence estimates. |
| Outcome: | The proposed training framework reduces the confidence calibration error and maintains the performance of the model. |
A Close Look into the Calibration of Pre-trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) may fail in giving reliable estimates of their predictive uncertainty. |
| Approach: | They conduct fine-grained control experiments to study the dynamic change in PLMs’ calibration performance in training. |
| Outcome: | The proposed methods significantly reduce PLMs’ confidence in wrong predictions. |
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for textual backdoor attacks insert additional contents into normal samples as triggers, causing detection and blocking of backdoors. |
| Approach: | They propose to use syntactic structure as trigger in textual backdoor attacks . they propose to achieve similar attack performance but have higher invisibility . |
| Outcome: | The proposed method achieves almost 100% success rate but has higher invisibility and stronger resistance to defenses than the insertion-based methods. |
Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer (2021.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that popular NLP models are vulnerable to both adversarial and backdoor attacks based on text style transfer. |
| Approach: | They propose to conduct adversarial and backdoor attacks based on text style transfer . the authors propose to use text style to alter the style of a sentence . |
| Outcome: | The proposed methods show that popular models are vulnerable to both attacks based on text style transfer . the results show that the proposed methods perform better than baselines in many aspects . |
Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT (2023.emnlp-main)
Copied to clipboard
Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu
| Challenge: | Large language models (LLMs) are capable of performing tasks but are likely to be misused. |
| Approach: | They propose a zero-shot black-box method to detect LLM-generated texts . they revise the text to be detected using the ChatGPT model . |
| Outcome: | The proposed method can detect LLM-generated texts with a zero-shot black-box model . it is based on intuition that the model will make fewer revisions to LLMs than to human-written texts . |
Multi-granularity Textual Adversarial Attack with Behavior Cloning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial attack models are vulnerable to adversarials crafted by human-imperceptible perturbations. |
| Approach: | They propose a multi-granularity adversarial attack model that generates high-quality adversarials with fewer queries to victim models. |
| Outcome: | The proposed model generates high-quality adversarial samples with fewer queries to victim models compared to baseline models . the proposed model also reduces query times for black-box models that only output labels without confidence scores . |
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Vision-language models have demonstrated strong efficacy as visual assistants . however, evaluation of their reasoning capabilities requires a costly benchmark . |
| Approach: | They propose a pipeline to measure the reasoning consistency of vision-language models . they propose supervised fine-tuning of VLMs and feedback from LLMs . |
| Outcome: | The proposed framework reduces cost while ensuring the generation of a high-quality dataset. |
ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation (2023.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art vision-language models have limited performance in structural knowledge extraction, such as relations between objects. |
| Approach: | They propose to leverage the inherent structure of programming language to depict visual structural information in a well-organized structured format. |
| Outcome: | The proposed framework improves visual structural knowledge extraction on visual structure prediction tasks. |