ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing word-level approaches to attack text are limited to a single word . existing methods ignore interactions between consecutive words, resulting in one-to-one attacks . |
| Approach: | They propose a black-box attack framework that misleads the language model by applying variable-length contextualized transformations to the original text. |
| Outcome: | The proposed framework outperforms existing methods on classification and inference tasks. |
Similar Papers
Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing gradient-based attacks quantize all tokens in a text at once, which creates a significant gap between adversarial loss for continuous and discrete text representations. |
| Approach: | They propose a gradient-based attack that quantizes tokens one by one and reoptimizes adversarial example after each quantization. |
| Outcome: | The proposed method outperforms other approaches on various natural language processing tasks. |
SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling (2025.emnlp-main)
Copied to clipboard
Krishna C Puvvada, Faisal Ladhak, Santiago Akle Serano, Cheng-Ping Hsieh, Shantanu Acharya, Somshubra Majumdar, Fei Jia, Samuel Kriman, Simeng Sun, Dima Rekesh, Boris Ginsburg
| Challenge: | Existing decoder-only models struggle with context lengths beyond their training distribution. |
| Approach: | They propose a causal Transformer architecture that generalizes robustly to sequence lengths longer than seen during training. |
| Outcome: | The proposed decoder-only architecture can generalize robustly to longer contexts . it is more computationally efficient than the standard Transformer architecture, the authors say . |
Improving the Transformer Translation Model with Document-Level Context (D18-1)
Copied to clipboard
| Challenge: | Existing models for document-level context translation ignore documentlevel context. |
| Approach: | They propose a document-level context encoder to represent document- level context and integrate it into the Transformer model. |
| Outcome: | Experiments on NIST Chinese-English and IWSLT French-English datasets show that the proposed translation model outperforms the Transformer model significantly. |
Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training (N19-1)
Copied to clipboard
| Challenge: | Recent work has shown superior performance for non-adversarial methods in more challenging language pairs. |
| Approach: | They propose to use adversarial autoencoder to map monolingual embeddings to a shared space and to put the target encoders as an adversary against the corresponding discriminator. |
| Outcome: | The proposed method is more robust and achieves better performance than previously proposed adversarial and non-adversarial methods. |
Contextualized Perturbation for Textual Adversarial Attack (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing techniques for generating adversarial examples are driven by local heuristic rules that are agnostic to the context, resulting in unnatural and ungrammatical outputs. |
| Approach: | They propose a ContextuaLized AdversaRial Example generation model that generates fluent and grammatical outputs through a mask-then-infill procedure. |
| Outcome: | The proposed model outperforms baseline models in terms of attack success rate, textual similarity, fluency and grammaticality. |
On Linearizing Structured Data in Encoder-Decoder Language Models: Insights from Text-to-SQL (2024.naacl-long)
Copied to clipboard
| Challenge: | Structured data is prevalent in tables, databases, and knowledge graphs, but there is a gap in our understanding of how these linearization-based methods handle structured data, which is inherently non-linear. |
| Approach: | They investigate the linear handling of structured data in encoder-decoder language models, specifically T5. |
| Outcome: | The proposed model can mimic human-designed processes such as schema linking and syntax prediction, and it can be compressed due to modality fusion redundancy. |
Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitution (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to attack transformer models are not effective at character level, but they are a natural attack scenario. |
| Approach: | They propose a character-level adversarial attack method against transformer models . they use a gradient-based method to find the most vulnerable words in a sentence . |
| Outcome: | The proposed method outperforms previous methods on sentence-level and token-level tasks. |
VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely applied to standard benchmarks due to the limited resources available in a domain. |
| Approach: | They propose a Transformer-based language model called VarMAE for domain-adaptive language understanding that encodes the context of a token into a smooth latent distribution. |
| Outcome: | Experiments on science- and finance-domain NLU tasks show that the proposed model can be efficiently adapted to new domains with limited resources. |
Targeted Adversarial Training for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing adversarial training approaches focus on making adversarials less expensive or regularizing rather than replacing the standard training objective. |
| Approach: | They propose an algorithm to introspect current mistakes and prioritize adversarial training steps to where the model errs the most. |
| Outcome: | The proposed algorithm improves adversarial training for natural language understanding by introspecting mistakes and prioritizing training steps to where the model errs the most. |
It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on societal bias in NLP focuses on race and gender . linguistic background is a unique attribute that has been largely ignored in the field . |
| Approach: | They examine linguistic background to craft plausible adversarial examples that expose biases in popular NLP models. |
| Outcome: | The proposed model improves robustness without sacrificing performance on clean data. |