| Challenge: | Despite their local fluency, long-form text generated from RNNs is often generic, repetitive, and even self-contradictory. |
| Approach: | They propose a unified learning framework that can guide a base RNN generator towards more globally coherent generations by combining discriminators with a composite decoding objective. |
| Outcome: | The proposed framework can guide a base RNN generator towards more globally coherent generations by combining discriminators with the base RRN generator through a composite decoding objective. |
Similar Papers
FUDGE: Controlled Text Generation With Future Discriminators (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models allow us to generate increasingly realistic text by modeling a distribution P (X) over natural language sequences X. |
| Approach: | They propose a flexible and modular method for controlled text generation that uses a Bayesian decomposition of the conditional distribution of G given an attribute predictor and can easily compose predictors for multiple desired attributes. |
| Outcome: | The proposed method can be easily composed and performs three tasks. |
A Unified Neural Coherence Model (D19-1)
Copied to clipboard
| Challenge: | Existing models for coherence modeling fail on harder tasks with more realistic application scenarios. |
| Approach: | They propose a unified coherence model that incorporates sentence grammar, inter-sentence coherent relations, and global coherency patterns into a common neural framework. |
| Outcome: | The proposed model outperforms existing models on local and global discrimination tasks and outperformed existing models by a good margin. |
It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on societal bias in NLP focuses on race and gender . linguistic background is a unique attribute that has been largely ignored in the field . |
| Approach: | They examine linguistic background to craft plausible adversarial examples that expose biases in popular NLP models. |
| Outcome: | The proposed model improves robustness without sacrificing performance on clean data. |
LLM Agents Implement an NLG System from Scratch: Building Interpretable Rule-Based RDF-to-Text Generators (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing neural approaches to generate RDF-to-text are limited in their implementation. |
| Approach: | They propose a framework where the model is “trained” through collaborative interactions among multiple LLM agents rather than traditional backpropagation. |
| Outcome: | The proposed framework reduces hallucinations and fluency penalties on the WebNLG and OpenDialKG datasets. |
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models perform well on commonsense-reasoning benchmark datasets, but building machines with commonsence to compose plausible sentences remains challenging. |
| Approach: | They propose a constrained text generation task for generative commonsense reasoning that generates a coherent sentence using common concepts. |
| Outcome: | The proposed task generates a coherent sentence describing an everyday scenario using common concepts over 35k concept-sets. |
Code-switched Language Models Using Dual RNNs and Same-Source Pretraining (D18-1)
Copied to clipboard
| Challenge: | Using recurrent neural networks to build language models for code-switched text is an important problem with implications to downstream applications such as speech recognition and machine translation. |
| Approach: | They propose a novel recurrent neural network unit with dual components that focus on each language in the code-switched text separately and a generative model estimated using the training data. |
| Outcome: | The proposed techniques yield significant reductions in perplexity on Mandarin-English task and improve on baseline models. |
A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to train neural network-based models for code-mixing are limited due to language specificity of code-mixed text. |
| Approach: | They propose a deep learning approach to generate code-mixed text from English to multiple languages without any parallel data. |
| Outcome: | The proposed approach generates a code-mixed text from English to multiple languages without any parallel data. |
Discourse Understanding and Factual Consistency in Abstractive Summarization (2021.eacl-main)
Copied to clipboard
Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, Yejin Choi
| Challenge: | Existing abstractive summarization models often hallucinate information or generate factually incorrect summaries. |
| Approach: | They propose a general framework for abstractive summarization with factual consistency and distinct modeling of the narrative flow in an output summary. |
| Outcome: | The proposed framework generates abstracts with factual consistency and coherence significantly better than baselines. |
UniTRec: A Unified Text-to-Text Transformer and Joint Contrastive Learning Framework for Text-based Recommendation (2023.acl-short)
Copied to clipboard
| Challenge: | Existing text-based recommendation frameworks that use pretrained language models (PLMs) can improve performance on text-related tasks. |
| Approach: | They propose a unified local- and global-attention Transformer encoder to better model two-level contexts of user history. |
| Outcome: | The proposed framework improves on three text-based recommendation tasks. |
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification (2021.eacl-main)
Copied to clipboard
| Challenge: | Semi-supervised learning and multilingual pretraining have been shown to be effective for task-specific labelled data shortages. |
| Approach: | They propose to combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. |
| Outcome: | The proposed method outperforms state-of-the-art models in low-resource settings across several languages and outperformed existing models in English. |