Effective Unsupervised Constrained Text Generation based on Perturbed Masking (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for constrained text generation stochastically sample edit positions and actions, which cause unnecessary search steps. |
| Approach: | They propose to extend perturbed masking technique to search for most incongruent token to edit and introduce four multi-aspect scoring functions to select edit action to further reduce search difficulty. |
| Outcome: | The proposed method achieves state-of-the-art in two representative tasks . it does not require supervised data, so it could be applied to different generation tasks. |
Similar Papers
Gradient-guided Unsupervised Lexically Constrained Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for lexically constrained generation fail when the search space is too large . a novel method to solve the problem is based on gradient-guided optimization . |
| Approach: | They propose a method to solve lexically-constrained generation as an unsupervised gradient-guided optimization problem. |
| Outcome: | The proposed method achieves state-of-the-art compared to previous methods . it is free of parallel data training, flexible to be used in the inference stage of any pre-trained generation model. |
Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown a powerful ability for text generation, but undesired behaviors such as toxicity and hallucinations can manifest. |
| Approach: | They propose to formalize text generation as a future-constrained generation problem to minimize undesirable behaviors and enforce faithfulness to instructions. |
| Outcome: | The proposed approach is effective across three tasks, including keyword-constrained generation, toxicity reduction, and factual correctness in question-answering. |
SGG: Learning to Select, Guide, and Generate for Keyphrase Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing keyphrase generation approaches synchronously generate present and absent keyphrases without explicitly distinguishing these two categories. |
| Approach: | They propose to deal with present and absent keyphrases separately with different mechanisms by using a hierarchical neural network with a pointing-based selector and a selection-guided generator. |
| Outcome: | The proposed model outperforms baselines on four keyphrase generation tasks and shows extensibility in natural language generation tasks. |
POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models cannot be directly employed to generate text under specified lexical constraints. |
| Approach: | They propose a method for insertion-based text generation that inserts tokens between existing tokens in a parallel manner. |
| Outcome: | The proposed method is intuitive and interpretable on Wikipedia and Yelp datasets. |
Conditional Augmentation for Aspect Term Extraction via Masked Sequence-to-Sequence Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Aspect term extraction is a task to extract aspect terms from review texts as opinion targets for sentiment analysis. |
| Approach: | They propose a conditional generation task for augmentation of aspect term extraction . they use a sequence-to-sequence method that generates a new sentence . results confirm that their method alleviates the data scarcity problem significantly . |
| Outcome: | The proposed method reduces the data scarcity problem significantly and boosts current models. |
Syntax-Guided Controlled Generation of Paraphrases (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work has explored the incorporation of complex syntactic-guidance as constraints in the task of controlled text generation. |
| Approach: | They propose an end-to-end framework for controlled paraphrase generation that incorporates complex syntactic-guidance constraints into the task. |
| Outcome: | The proposed framework generates syntax-conforming sentences while not compromising on relevance. |
FELIX: Flexible Text Editing Through Tagging and Insertion (2020.findings-emnlp)
Copied to clipboard
| Challenge: | FELIX is efficient in low-resource settings and fast at inference time, while being capable of modeling flexible input-output transformations. |
| Approach: | They propose a flexible text-editing approach that decomposes a text-generating task into two sub-tasks: tagging and insertion. |
| Outcome: | The proposed model is efficient in low-resource settings and fast at inference time while being capable of modeling flexible input-output transformations. |
Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders (2020.acl-main)
Copied to clipboard
| Challenge: | Existing conditional generation models cannot handle emerging conditions due to their joint end-to-end learning fashion. |
| Approach: | They propose a framework for conditional text generation that decouples the text generation module from the condition representation module to allow "one-to-many" conditional generation. |
| Outcome: | The proposed framework decouples the text generation module from the condition representation module to allow “one-to-many” conditional generation. |
Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to decode text to the most probable sequence have been proposed to address these challenges by improving coherence, diversity, and resemblance to human-generated text. |
| Approach: | They propose a novel decoding strategy that extends contrastive search by incorporating an adaptive degeneration penalty informed by the model’s estimated uncertainty at each generation step. |
| Outcome: | The proposed approach improves creativity and coherence while maintaining coherency across model architectures, languages, and datasets. |
ParaMac: A General Unsupervised Paraphrase Generation Framework Leveraging Semantic Constraints and Diversifying Mechanisms (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing unsupervised methods for paraphrase generation are weak in semantic equivalence or expression diversity. |
| Approach: | They propose a framework for unsupervised paraphrase generation that employs multi-aspect equivalence constraints and multi-granularity diversifying mechanisms to achieve good semantic equvalence and expressive diversity. |
| Outcome: | The proposed framework achieves 9.1% and 3.3% absolute gains over previous SOTA on Quora and MSCOCO and can improve to 18.0% and 4.6% on GLUE. |