MOCHA: A Multi-Task Training Approach for Coherent Text Generation from Cognitive Perspective (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent pre-trained language models have produced impressive results, but there is still a gap between human written texts and machine-generated outputs. |
| Approach: | They propose a multi-task training strategy for long text generation grounded on the cognitive theory of writing. |
| Outcome: | The proposed model achieves better results on three open-ended generation tasks than baselines. |
Similar Papers
Text-to-Text Automatic Story Generation: A Survey (2026.eacl-srw)
Copied to clipboard
| Challenge: | Automated story generation aims to produce coherent, engaging, and contextually consistent narratives with minimal or no human involvement . despite advances in large language models, maintaining narrative coherence, character consistency, storyline diversity, and plot controllability in generating stories is still challenging. |
| Approach: | They propose to develop new evaluation metrics and better data sets to support automatic story generation. |
| Outcome: | The proposed evaluation metrics and better datasets will improve narrative coherence and consistency and explore practical applications of story generation. |
Hierarchical Neural Story Generation (P18-1)
Copied to clipboard
| Challenge: | a hierarchical model that generates a premise and then conditions on it creates fluent text . a novel form of model fusion improves the relevance of the story to the prompt . |
| Approach: | They use a hierarchical model that first generates a premise, then transforms it into a text . they use fusion to improve relevance of the story to the prompt and add a gated mechanism to model context . |
| Outcome: | The proposed model improves on strong baselines on automated and human evaluations. |
Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence (2021.acl-long)
Copied to clipboard
| Challenge: | Existing generation models struggle to maintain a coherent event sequence throughout the generated text. |
| Approach: | They propose a long text generation model which can represent prefix sentences at sentence level and discourse level in the decoding process. |
| Outcome: | The proposed model can generate more coherent texts than state-of-the-art models. |
Sentence-Level Content Planning and Style Specification for Neural Text Generation (D19-1)
Copied to clipboard
| Challenge: | Recent advances in text generation systems often produce incoherent and unfaithful outputs . a novel automated text generation system takes into account content selection, text planning, and surface realization. |
| Approach: | They propose an end-to-end trained two-step text generation model that considers sentence-level content planners and language styles. |
| Outcome: | The proposed model outperforms competing models in three domains with diverse topics and varying language styles. |
Discourse-Aware Neural Rewards for Coherent Text Generation (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to train text generation models using cross-entropy loss do not always correlate well with achieving high scores on commonly used evaluation measures. |
| Approach: | They propose to use discourse-aware rewards to model cross-sentence ordering to approximate desired discourse structure to train a model of long, coherent text. |
| Outcome: | The proposed model produces more coherent and less repetitive text than models trained with cross-entropy or with commonly used scores as rewards. |
The Amazing World of Neural Language Generation (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | Recent years have seen a paradigm shift in neural text generation due to advances in deep contextual language modeling and transfer learning. |
| Approach: | They will discuss how and why NLG models succeed/fail at generating coherent text. |
| Outcome: | This paper will discuss how and why these models succeed/fail at generating coherent text, and provide insights on several applications. |
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning (2023.acl-long)
Copied to clipboard
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Yang Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
| Challenge: | Existing methods for data-to-text generation focus on specific types of structured data. |
| Approach: | They propose a method that provides a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. |
| Outcome: | The proposed method improves zero-shot and few-shot scenarios and can adapt to new structured data. |
Facts2Story: Controlling Text Generation by Key Facts (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for story generation struggle with staying coherent for long periods of time. |
| Approach: | They propose a controlled generation task which expands a sequence of facts into a longer narrative. |
| Outcome: | The proposed model produces competitive fluency while adhering to the requested facts. |
Improving Neural Story Generation by Targeted Common Sense Grounding (D19-1)
Copied to clipboard
| Challenge: | Recent advances in language modeling have yielded thematic and stylistic coherence in story generation through large scale pretraining of Transformer models. |
| Approach: | They propose a multi-task learning scheme to achieve better common sense reasoning in language models by leveraging auxiliary training signals from datasets designed to provide common sense grounding. |
| Outcome: | The proposed model achieves improved common sense reasoning and state-of-the-art perplexity on the WritingPrompts dataset. |
Towards Content Transfer through Grounded Text Generation (N19-1)
Copied to clipboard
| Challenge: | Recent work in neural natural language generation has attracted significant interest in controlling the form of text, such as style, persona, and wordiness. |
| Approach: | They propose a task where the task is to generate a next sentence in a document that fits its context and is grounded in . external textual source such as a news story. |
| Outcome: | The proposed task is based on 640k Wikipedia referenced sentences paired with the source articles to show significant improvements against baselines. |