Invisible to People but not to Machines: Evaluation of Style-aware HeadlineGeneration in Absence of Reliable Human Judgment (2020.lrec-1)
Copied to clipboard
| Challenge: | Using a data alignment strategy and different training/testing settings, we aim at decoupling content from style and preserving the latter in generation. |
| Approach: | They propose a fine-grained evaluation strategy based on automatic classification to evaluate generated headlines' quality in terms of their newspaper-compliance. |
| Outcome: | The proposed model learns newspaper-specific style, but humans aren't reliable judges for this task, and deserves particular care in its design. |
Similar Papers
A Case Study on Neural Headline Generation for Editing Support (N19-2)
Copied to clipboard
Kazuma Murao, Ken Kobayashi, Hayato Kobayashi, Taichi Yatsuka, Takeshi Masuyama, Tatsuru Higurashi, Yoshimune Tabuchi
| Challenge: | a news-aggregator is a website or mobile application that aggregates web content . dozens of professional editors manually create their headlines, which are much shorter than the original headlines. |
| Approach: | They propose a neural headline generation model that automatically generates short headlines from news articles. |
| Outcome: | The proposed model is deployed to an editing support tool and compares editors' behavior before and after the release. |
StyleBART: Decorate Pretrained Model with Style Adapters for Unsupervised Stylistic Headline Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on unsupervised headline generation focus on a standard dataset and mono-style corpora. |
| Approach: | They propose an unsupervised approach for stylistic headline generation using a pretrained BART model decorated with adapters responsible for different styles. |
| Outcome: | The proposed method separates the task of style learning and headline generation, allowing for the generation of diverse headlines with diverse styles. |
Improving Truthfulness of Headline Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on abstractive summarization report ROUGE scores, but are concerned about the truthfulness of generated summaries. |
| Approach: | They propose to remove untruthful instances from supervision data to improve headline generation . they build a binary classifier that predicts an entailment relation between an article and its headline . |
| Outcome: | The proposed model improves on two popular datasets. |
Generating User-Engaging News Headlines (2023.acl-long)
Copied to clipboard
| Challenge: | Personalized news recommendation systems present the same headline to all users, making it difficult for them to understand the connection between their interests and the recommended article. |
| Approach: | They propose a framework that incorporates user profiling to generate personalized headlines and a combination of automated and human evaluation methods to determine user preference for personalized headline generation. |
| Outcome: | The proposed framework can generate personalized headlines that meet the needs of a diverse audience. |
Updated Headline Generation: Creating Updated Summaries for Evolving News Stories (2022.acl-long)
Copied to clipboard
| Challenge: | Existing systems that generate headlines for updated articles are not as efficient as static ones. |
| Approach: | They propose a task where a system generates a headline for an updated article, considering both the previous article and headline. |
| Outcome: | The proposed model produces headlines judged by humans to be as factual as gold headlines while making fewer unnecessary edits compared to a standard headline generation model. |
NumHG: A Dataset for Number-Focused Headline Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | a lack of fine-grained annotations for accurate numeral generation in headlines is a major roadblock . a new dataset, the NumHG, provides over 27,000 annotated numeral-rich news articles for detailed investigation . |
| Approach: | They propose a dataset that provides annotated numerals for headline generation . they evaluate five well-performing headline-generation models using human evaluation . |
| Outcome: | The proposed dataset provides annotated numeral-rich news articles for detailed investigation. |
Effectiveness of Data Augmentation and Pretraining for Improving Neural Headline Generation in Low-Resource Settings (2022.lrec-1)
Copied to clipboard
| Challenge: | Neural approaches for natural language generation (NLG) have mushroomed due to large textual resources. |
| Approach: | They propose to use a pretrained multilingual encoder-decoder model and a combination of two pretrained language models to train a model in a low-resource setting. |
| Outcome: | The proposed model outperforms the previous model on English and on a small subset of the same data. |
Hooks in the Headline: Learning to Generate Headlines with Controlled Styles (2020.acl-main)
Copied to clipboard
| Challenge: | Current summarization systems only produce plain, factual headlines, far from the practical needs for exposure and memorableness of the articles. |
| Approach: | They propose a task to generate relevant headlines with three style options . they propose combining summarization and reconstruction tasks into a multitasking framework . |
| Outcome: | The proposed method outperforms the state-of-the-art summarization model by 9.68% . it can generate relevant, fluent headlines with humor, romance and clickbait . |
Harnessing the power of LLMs: Evaluating human-AI text co-creation through the lens of news headline generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have shattered the ceiling of human-like text generation. |
| Approach: | They compared human-AI interaction types in LLM-assisted news headline generation to determine whether humans can best leverage them for writing. |
| Outcome: | The guiding and selecting model outputs added the most benefit with the lowest cost (in time and effort) Furthermore, AI assistance did not harm participants’ perception of control compared to freeform editing. |
Exploring the Limitations of Detecting Machine-Generated Text (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in the quality of the generation of text by large language models have spurred research into identifying machine-generated text. |
| Approach: | They audit classification performance for detecting machine-generated text by evaluating on texts with varying writing styles. |
| Outcome: | The proposed methods are highly sensitive to stylistic changes and complexity, and in some cases degrade entirely to random classifiers. |