| Challenge: | a perfectly informative agent would eschew utterances that are redundant or contradict that which they have already said. |
| Approach: | They propose to use a pre-trained NLI model to assess whether a sentence entails, contradicts, or is neutral to prompt and preceding text. |
| Outcome: | The proposed model is predictive of generation errors made by GPT-3. |
Similar Papers
A Survey on Natural Language Counterfactual Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in NLP are driven by a variety of Large Language Models (LLMs), such as GPT-3 (175B) and PaLM (540B). |
| Approach: | They propose a taxonomy that categorizes the methods into four groups and summarizes the metrics for evaluating the generation quality. |
| Outcome: | The proposed taxonomy categorizes the generation methods into four groups and summarizes the metrics for evaluating the quality. |
The Amazing World of Neural Language Generation (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | Recent years have seen a paradigm shift in neural text generation due to advances in deep contextual language modeling and transfer learning. |
| Approach: | They will discuss how and why NLG models succeed/fail at generating coherent text. |
| Outcome: | This paper will discuss how and why these models succeed/fail at generating coherent text, and provide insights on several applications. |
A synthetic data approach for domain generalization of NLI models (2024.acl-long)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) datasets are important benchmark tasks for LLMs . however, their realistic performance on out-of-distribution/domain data is less well-understood . a T5-small model trained with our data improves around 7% on average compared to the best alternative dataset . |
| Approach: | They propose a new approach for generating NLI data in diverse domains and lengths . they show that models trained on this data have the best generalization to completely new downstream test settings . |
| Outcome: | The proposed model can be trained on datasets with high-quality examples with meaningful premises and high accuracy. |
Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Neural abstractive summarization models generate factually inconsistent summaries . previous work has introduced the task of recognizing factual inconsistency as a downstream application of natural language inference (NLI). |
| Approach: | They propose a data generation pipeline that enables a task-oriented approach to detect factual inconsistencies in abstractive summarization models. |
| Outcome: | The proposed model improves the state-of-the-art performance across four benchmarks for recognizing factual inconsistency in generated summaries. |
Towards Controllable Biases in Language Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to induce societal biases in natural language generation is being developed . a method to equalize the amount of biased text across demographics is effective . |
| Approach: | They propose a method to induce societal biases in natural language generation by using demographic inequalities. |
| Outcome: | The proposed method is effective at equalizing biases across demographics while generating less negatively biased text overall. |
Generating Text from Language Models (2023.acl-tutorials)
Copied to clipboard
| Challenge: | a growing percentage of natural language processing tasks focus on the generation of text from probabilistic language models. |
| Approach: | They will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
| Outcome: | This tutorial will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
MorphNLI: A Stepwise Approach to Natural Language Inference Using Text Morphing (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing models fail to capture important semantic features of logic such as monotonicity and negation. |
| Approach: | They propose a modular step-by-step approach to natural language inference . they use a language model to generate edits to incrementally transform the premise into the hypothesis . |
| Outcome: | The proposed method outperforms baseline models in realistic cross-domain settings with improvements up to 12.6% (relative). |
An Empirical Study of Generating Texts for Search Engine Advertising (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing studies on neural language generation have not evaluated the effect of generated ads with actual serving included because it requires a large amount of training data and a particular environment. |
| Approach: | They propose to integrate a reinforcement learning framework into an end-to-end sequence-tosequence (Seq2S) model and demonstrate how to improve the ads’ impact, deploy models to a product, and evaluate the generated ads. |
| Outcome: | The proposed method improves the ads’ impact, deploys the models to a product, and evaluates the generated ads. |
Automatic and Human-AI Interactive Text Generation (with a focus on Text Simplification and Revision) (2024.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we focus on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and then generates a revision that is improved according to some specific criteria. |
| Approach: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks that takes a piece of text as input and generates a revision that is improved according to some specific criteria. |
| Outcome: | This tutorial focuses on text-to-text generation, a class of natural language generation tasks, that takes a piece of text as input and generates a revision that is improved according to some specificcriteria. |
Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference (P19-1)
Copied to clipboard
| Challenge: | Recent advances on abstractive summarization have led to fluent summaries, but factual errors in generated summary still severely limit their use in practice. |
| Approach: | They evaluate summaries produced by state-of-the-art models via crowdsourcing and show that factual errors occur frequently. |
| Outcome: | The proposed models can detect errors and reduce them by reranking alternative summaries. |