| Challenge: | Recent advances in NLP are driven by a variety of Large Language Models (LLMs), such as GPT-3 (175B) and PaLM (540B). |
| Approach: | They propose a taxonomy that categorizes the methods into four groups and summarizes the metrics for evaluating the generation quality. |
| Outcome: | The proposed taxonomy categorizes the generation methods into four groups and summarizes the metrics for evaluating the quality. |
Similar Papers
Prompting Large Language Models for Counterfactual Generation: An Empirical Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have made remarkable progress in a wide range of natural language understanding and generation tasks, but their ability to generate counterfactuals has not been examined systematically. |
| Approach: | They propose a framework to evaluate LLMs' ability to generate counterfactuals based on key factors including intrinsic properties and prompt design. |
| Outcome: | The proposed framework examines the strengths and weaknesses of large language models (LLMs) and identifies factors that influence their ability to generate counterfactuals. |
Generating Realistic Natural Language Counterfactuals (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to explain ML tasks for natural language text are either unrealistic or introduce imperceptible changes. |
| Approach: | They propose a method that combines a conditional GAN and embeddings of a pretrained BERT encoder to model-agnostically generate realistic natural language text counterfactuals for explaining regression and classification tasks. |
| Outcome: | The proposed method outperforms baseline methods on fidelity and human judgments of naturalness across multiple datasets and multiple predictive models. |
Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation (2026.acl-long)
Copied to clipboard
Qianli Wang, Van Bach Nguyen, Yihong Liu, Fedor Splitt, Nils Feldhus, Christin Seifert, Hinrich Schuetze, Sebastian Möller, Vera Schmitt
| Challenge: | Large language models excel at generating English counterfactuals but their effectiveness in generating multilingual counterfacts remains unclear. |
| Approach: | They conduct automatic evaluations on both directly generated and derived counterfactuals in six languages and find that cross-lingual perturbations follow common strategic principles. |
| Outcome: | The proposed models show that translation-based counterfactuals offer higher validity than their directly generated counterparts, but still fall short of matching the quality of the original English counterf actuals. |
Generating Text from Language Models (2023.acl-tutorials)
Copied to clipboard
| Challenge: | a growing percentage of natural language processing tasks focus on the generation of text from probabilistic language models. |
| Approach: | They will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
| Outcome: | This tutorial will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing counterfactual generation methods rely on manual labor to create very few counterf actuals or only instantiate limited types of perturbations such as paraphrases or word substitutions. |
| Approach: | They propose a general-purpose counterfactual generator that allows for control over perturbation types and locations. |
| Outcome: | The proposed generator produces diverse sets of realistic counterfactuals that are useful in various applications. |
Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve performance of deep neural models are limited by the nature of spurious patterns in the data. |
| Approach: | They propose to use augmented data to generate spurious patterns in NLP models . they propose to generate counterfactual data for data augmentation and explanation . |
| Outcome: | The proposed approach improves performance on augmented data and on human-generated data. |
Investigating the Robustness of Natural Language Generation from Logical Forms via Counterfactual Samples (2022.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art methods based on pre-trained models have achieved remarkable performance on the standard test dataset. |
| Approach: | They propose to incorporate hierarchical structure of logical forms into the model and exploit automatically generated counterfactual data for training. |
| Outcome: | The proposed method is effective to alleviate spurious correlations between the headers of the tables and operators of the logical form. |
Natural Language Generation: Recently Learned Lessons, Directions for Semantic Representation-based Approaches, and the Case of Brazilian Portuguese Language (P19-2)
Copied to clipboard
| Challenge: | Natural Language Generation (NLG) is a promising area in Natural Language Processing (NLP) . |
| Approach: | They present a review of the literature on Natural Language Generation in Brazilian Portuguese. |
| Outcome: | The proposed approaches are based on the Abstract Meaning Representation formalism and have potential future directions. |
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to produce counterfactuals rely on small perturbations via minimal edits, resulting in simplistic changes. |
| Approach: | They propose a novel approach to produce counterfactuals that allow for larger edits and linguistic diversity while still bearing similarity to the original document. |
| Outcome: | The proposed approach outperforms existing methods for generalizing natural language models under select settings. |
A Systematic Review of Reproducibility Research in Natural Language Processing (2021.eacl-main)
Copied to clipboard
| Challenge: | Despite the recent progress in reproducibility, the field is far from reaching a consensus on how reproducibility should be defined, measured and addressed. |
| Approach: | They propose to provide a wide-angle snapshot of current work on reproducibility in NLP. |
| Outcome: | The proposed work will provide a wide-angle snapshot of current work on reproducibility in NLP. |