Eric Malmi, Yue Dong, Jonathan Mallinson, Aleksandr Chuklin, Jakub Adamek, Daniil Mirylenka, Felix Stahlberg, Sebastian Krause, Shankar Kumar, Aliaksei Severyn
| Challenge: | Text-editing models are a popular alternative to seq2seq for monolingual text generation tasks such as text summarization and style transfer. |
| Approach: | They propose to use text-editing models to predict edit operations applied to the source sequence and to generate outputs word-by-word from scratch. |
| Outcome: | This paper provides an overview of the text-edit based models and their current state-of-the-art approaches. |
Similar Papers
Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study (P19-1)
Copied to clipboard
| Challenge: | Sequence-to-sequence (seq2sequ) models have a weakness: they cannot always generate sentences without grammatical errors. |
| Approach: | They propose to use automatic grammatical error correction to improve seq2seq models . they conduct experiments on machine translation, formality style transfer, sentence compression and simplification . |
| Outcome: | The proposed system can improve grammaticality of generated text and improve formal style tasks. |
Automatic and Human-AI Interactive Text Generation (with a focus on Text Simplification and Revision) (2024.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we focus on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and then generates a revision that is improved according to some specific criteria. |
| Approach: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks that takes a piece of text as input and generates a revision that is improved according to some specific criteria. |
| Outcome: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and generates a revision that is improved according to some specificcriteria. |
How Do Seq2Seq Models Perform on End-to-End Data-to-Text Generation? (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models for data-to-text generation are based on pipelines and end-to end architectures. |
| Approach: | They use multidimensional quality metrics to evaluate models on end-to-end data-totext generation and compare their performance against pipeline models. |
| Outcome: | The proposed model improves in Omission and Inaccuracy Extrinsic errors but increases errors such as Addition. |
Simple and Effective Retrieve-Edit-Rerank Text Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Using retrieve-and-edit methods, text generation methods can be improved by reranking outputs from training sets and learning models to produce the final output. |
| Approach: | They propose to extend retrieve-and-edit seq2seq methods with a simple post-generation ranking approach that retrieves multiple outputs and edits each independently to produce the final output. |
| Outcome: | The proposed approach outperforms existing methods on two machine translation datasets and shows room for improvement with better candidate output selection in future work. |
Editing Large Language Models: Problems, Methods, and Opportunities (2023.emnlp-main)
Copied to clipboard
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, Ningyu Zhang
| Challenge: | Recent advances in model editing for LLMs have created challenges and opportunities for the community. |
| Approach: | They propose to alter the behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
| Outcome: | The proposed method alters behavior of LLMs efficiently within a specific domain without negatively impacting performance across other inputs. |
Learning to Model Editing Processes (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sequence generation models produce outputs in one pass, usually left-to-right . current models model only a single edit step, and do not fully model editing . |
| Approach: | They propose to model editing processes, modeling the whole process of iteratively generating sequences. |
| Outcome: | The proposed model improves performance on a variety of axes compared to previous models . iterative refinement and editing are central parts of human creative workflow . |
Seq2Edits: Sequence Transduction Using Span-level Edit Operations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Seq2Edits is an open-vocabulary approach to sequence editing for natural language processing tasks with a high degree of overlap between input and output texts. |
| Approach: | They propose an open-vocabulary approach to sequence editing for NLP tasks with a high degree of overlap between input and output texts. |
| Outcome: | The proposed approach speeds up inference by up to 5.2x compared to full sequence models . it improves explainability by associating each edit operation with a human-readable tag. |
Deep Learning Approaches to Text Production (N18-6)
Copied to clipboard
| Challenge: | Text production is a key component of many NLP applications . Claire Gardent is based in France and is pursuing research in text production . |
| Approach: | This tutorial will cover the fundamentals and state-of-the-art research on neural models for text production. |
| Outcome: | This tutorial will cover the fundamentals and the state-of-the-art research on neural models for text production. |
Text-to-Table: A New Way of Information Extraction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for information extraction are not well understood . text-to-table is a problem that aims to extract information from text data . |
| Approach: | They propose a new problem setting of information extraction, called text-to-table . they formalize text- to-table as a sequence-tosequence problem . |
| Outcome: | The proposed method outperforms existing methods on text-to-table tasks. |
Stylized Text Generation: Approaches and Applications (2020.acl-tutorials)
Copied to clipboard
| Challenge: | Text generation has played an important role in various applications of natural language processing. |
| Approach: | They present different settings of stylized text generation and introduce machine learning methods to represent style. |
| Outcome: | This paper presents a comprehensive literature review on stylized text generation . it focuses on the challenges and future directions of stylized generation based on machine learning . |