| Challenge: | Using GPT-2, long documents can ramble and do not follow human-like writing structure. |
| Approach: | They propose a controlled text generation task that generates documents with structure . they use a news article as a dataset to test different degrees of structural awareness . |
| Outcome: | The proposed task generates documents with a structure that is human-like, but long documents lack structure. |
Similar Papers
Automatic and Human-AI Interactive Text Generation (with a focus on Text Simplification and Revision) (2024.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we focus on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and then generates a revision that is improved according to some specific criteria. |
| Approach: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks that takes a piece of text as input and generates a revision that is improved according to some specific criteria. |
| Outcome: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and generates a revision that is improved according to some specificcriteria. |
Facts2Story: Controlling Text Generation by Key Facts (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for story generation struggle with staying coherent for long periods of time. |
| Approach: | They propose a controlled generation task which expands a sequence of facts into a longer narrative. |
| Outcome: | The proposed model produces competitive fluency while adhering to the requested facts. |
Data-to-text Generation with Variational Sequential Planning (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in data-to-text generation have greatly facilitated the task of generating textual output from non-linguistic input. |
| Approach: | They propose a neural model enhanced with a planning component responsible for organizing high-level information in a coherent and meaningful way. |
| Outcome: | The proposed model outperforms baseline models and is sample-efficient in the face of limited training data. |
Discourse-Aware Soft Prompting for Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in pre-trained langauge models (PLMs) have made great impact on text generation research. |
| Approach: | They propose to use hierarchical blocking to simulate a higher-level discourse structure of human written text and attention sparsity to learn sparse transformations on the softmax-function. |
| Outcome: | The proposed methods perform better on some generation tasks but don't generalize across all generation tasks. |
Plan-then-Generate: Controlled Data-to-Text Generation via Planning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on producing results that are close to the references, i.e. what to generate and in what order (the output structure) cannot be explicitly controlled by the users. |
| Approach: | They propose a Plan-then-Generate framework to improve the controllability of neural data-to-text models. |
| Outcome: | The proposed model can control both the intra-sentence and inter-sentent structure of the generated output. |
Learning to Select, Track, and Generate for Data-to-Text (P19-1)
Copied to clipboard
Hayate Iso, Yui Uehara, Tatsuya Ishigaki, Hiroshi Noji, Eiji Aramaki, Ichiro Kobayashi, Yusuke Miyao, Naoaki Okazaki, Hiroya Takamura
| Challenge: | Existing models often refer to the same data record multiple times. |
| Approach: | They propose a data-to-text generation model with two modules, one for tracking and the other for text generation. |
| Outcome: | The proposed model outperforms existing models even without writer information in all evaluation metrics and contributes to content planning and surface realization. |
Text-to-Text Automatic Story Generation: A Survey (2026.eacl-srw)
Copied to clipboard
| Challenge: | Automated story generation aims to produce coherent, engaging, and contextually consistent narratives with minimal or no human involvement . despite advances in large language models, maintaining narrative coherence, character consistency, storyline diversity, and plot controllability in generating stories is still challenging. |
| Approach: | They propose to develop new evaluation metrics and better data sets to support automatic story generation. |
| Outcome: | The proposed evaluation metrics and better datasets will improve narrative coherence and consistency and explore practical applications of story generation. |
Sentence-Level Content Planning and Style Specification for Neural Text Generation (D19-1)
Copied to clipboard
| Challenge: | Recent advances in text generation systems often produce incoherent and unfaithful outputs . a novel automated text generation system takes into account content selection, text planning, and surface realization. |
| Approach: | They propose an end-to-end trained two-step text generation model that considers sentence-level content planners and language styles. |
| Outcome: | The proposed model outperforms competing models in three domains with diverse topics and varying language styles. |
Semantics of the Unwritten: The Effect of End of Paragraph and Sequence Tokens on Text Generation with GPT2 (2021.acl-srw)
Copied to clipboard
| Challenge: | Experimental results show that pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage. |
| Approach: | They conduct experiments on an English essay dataset using Chinese-GPT2 . they find that the model can generate better continuations by learning to generate the in the fine-tuning stage. |
| Outcome: | The pre-trained language model GPT2 can generate better continuations by learning to generate the in the fine-tuning stage. |
Sentence Smith: Controllable Edits for Evaluating Text Embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Controllable and transparent text generation has been a long-standing goal in NLP . but previous approaches were hindered by parsing and generation insufficiencies . |
| Approach: | They propose a framework for English that has three steps: 1. Parsing a sentence into a semantic graph. 2. Applying human-designed semantic manipulation rules. 3. Generating text from the manipulated graph. |
| Outcome: | The proposed framework for English is based on a neural network and parsers. |