| Challenge: | Existing sequence generation models produce outputs in one pass, usually left-to-right . current models model only a single edit step, and do not fully model editing . |
| Approach: | They propose to model editing processes, modeling the whole process of iteratively generating sequences. |
| Outcome: | The proposed model improves performance on a variety of axes compared to previous models . iterative refinement and editing are central parts of human creative workflow . |
Similar Papers
Text Generation with Text-Editing Models (2022.naacl-tutorials)
Copied to clipboard
Eric Malmi, Yue Dong, Jonathan Mallinson, Aleksandr Chuklin, Jakub Adamek, Daniil Mirylenka, Felix Stahlberg, Sebastian Krause, Shankar Kumar, Aliaksei Severyn
| Challenge: | Text-editing models are a popular alternative to seq2seq for monolingual text generation tasks such as text summarization and style transfer. |
| Approach: | They propose to use text-editing models to predict edit operations applied to the source sequence and to generate outputs word-by-word from scratch. |
| Outcome: | This paper provides an overview of the text-edit based models and their current state-of-the-art approaches. |
An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models (2022.acl-long)
Copied to clipboard
| Challenge: | et al. (2017) show that imitation learning algorithms for machine translation introduce mismatches between training and inference that lead to undertraining and poor generalization in editing scenarios. |
| Approach: | They propose a framework for training non-autoregressive sequence-to-sequence models for editing tasks where the original input sequence is iteratively edited to produce the output. |
| Outcome: | The proposed framework significantly improves output quality and controls complexity better on the simplification task. |
EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing (P19-1)
Copied to clipboard
| Challenge: | Current sentence simplification systems are variants of sequence-to-sequence models adopted from machine translation. |
| Approach: | They propose a sentence simplification model that learns explicit edit operations via a neural programmer-interpreter approach. |
| Outcome: | The proposed model outperforms state-of-the-art models on three benchmark text simplification corpora in terms of SARI (+0.95 WikiLarge, +1.89 WikiSmall, -1.41 Newsela) |
Editing Large Language Models: Problems, Methods, and Opportunities (2023.emnlp-main)
Copied to clipboard
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, Ningyu Zhang
| Challenge: | Recent advances in model editing for LLMs have created challenges and opportunities for the community. |
| Approach: | They propose to alter the behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
| Outcome: | The proposed method alters behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
Training Dynamics for Text Summarization Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have shown impressive results when fine-tuned on large summarization datasets. |
| Approach: | They analyze the training dynamics for generation models, focusing on summarization . they find that a propensity to copy the input is learned early in the training process . |
| Outcome: | The proposed model learns at different stages of fine-tuning, the authors show . they show that factual errors are learnt in later stages, but not at high-loss tokens . |
Recurrent Inference in Text Editing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing inference methods map the unedited text to the edited text or to the editing operations, but performance is degraded by the limited source text encoding and long, varying decoding steps. |
| Approach: | They propose a new inference method that iteratively performs editing actions . they introduce three types of editing tasks: AOR, AES, AEC . |
| Outcome: | The proposed method significantly narrows the problem space by iterating editing actions. |
Sentence-Level Content Planning and Style Specification for Neural Text Generation (D19-1)
Copied to clipboard
| Challenge: | Recent advances in text generation systems often produce incoherent and unfaithful outputs . a novel automated text generation system takes into account content selection, text planning, and surface realization. |
| Approach: | They propose an end-to-end trained two-step text generation model that considers sentence-level content planners and language styles. |
| Outcome: | The proposed model outperforms competing models in three domains with diverse topics and varying language styles. |
Paraphrase Generation by Learning How to Edit from Samples (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show the superiority of our retrieval-based paraphrase generation model in terms of both automatic metrics and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. |
| Approach: | They propose a retrieval-based method for paraphrase generation which uses a novel editor module to extract edits from paraphrase pairs. |
| Outcome: | The proposed model outperforms existing models in automatic metrics and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. |
QuickEdit: Editing Text & Translations by Crossing Words Out (N18-1)
Copied to clipboard
| Challenge: | Using statistical learning, a computer can rephrase a sentence by only pointing at words that should be avoided. |
| Approach: | They propose a framework for computer-assisted text editing that relies on simple interactions between human editors and tokens. |
| Outcome: | The proposed framework allows to get substantial modifications to a sentence without human intervention. |
Text Editing by Command (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has focused on making such models more controllable and factually grounded. |
| Approach: | They propose a novel interactive text generation setting in which the user interacts with the system by issuing commands to edit existing text. |
| Outcome: | The proposed model outperforms baseline models and obtains positive results in automatic and human evaluations. |