Papers with Summarization
Multi Document Summarization Evaluation in the Presence of Damaging Content (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing metrics evaluate a summary based on relevance and consistency with the source documents. |
| Approach: | They propose to measure the ability of MDS systems to handle damaging documents in their input set by lexical similarity and language model likelihood. |
| Outcome: | The proposed metrics show that they can summarize a set of documents without damaging content. |
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating factual consistency in abstractive summarization systems have significant limitations, especially on refinement and interpretability. |
| Approach: | They propose a method for detecting summary factual inconsistency based on fine-grained atomic facts decomposition and adaptive granularity expansion. |
| Outcome: | The proposed method outperforms existing systems on the AGGREFACT benchmark dataset and achieves state-of-the-art performance. |
Not All Reviews Are Equal: Towards Addressing Reviewer Biases for Opinion Summarization (P19-2)
Copied to clipboard
| Challenge: | Existing research focuses on mining for opinions from review texts and ignores reviewers. |
| Approach: | They propose to model reviewer biases from review texts and learn a bias-aware opinion representation. |
| Outcome: | The proposed method includes balanced opinions from reviewers with different biases and preferences. |
BillSum: A Corpus for Automatic Summarization of US Legislation (D19-54)
Copied to clipboard
| Challenge: | In the US Congress, over 10,000 bills are introduced each year, with state legislatures introducing tens of thousands of bills. |
| Approach: | They introduce the first dataset for summarizing US Congressional and California state bills . they demonstrate that models built on Congressional bills can be used to summarize California billa . |
| Outcome: | The proposed summarization methods can be applied to states without human-written summaries. |
Abstractive Text Summarization Using the BRIO Training Paradigm (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing abstractive summarization models rely heavily on reference summaries and lack control over their performance. |
| Approach: | They propose a BRIO paradigm to reduce the dependence on reference summaries by fine-tuning pre-trained language models and training them with the paradigm. |
| Outcome: | The proposed paradigm outperforms existing models on Vietnamese and CNNDM datasets while maintaining the main content of the original text. |
SETSum: Summarization and Visualization of Student Evaluations of Teaching (2022.naacl-demo)
Copied to clipboard
| Challenge: | Student Evaluations of Teaching (SETs) are used in colleges and universities to assess student perceptions about their courses. |
| Approach: | They propose a system that leverages sentiment analysis, aspect extraction, summarization and visualization techniques to provide organized illustrations of SET findings to instructors and other reviewers. |
| Outcome: | The proposed system can be used by 10 professors from diverse departments to analyze SET results. |
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent studies have shown that even state-of-the-art pre-trained language models can generate inconsistent summaries in more than 70% of all cases. |
| Approach: | They propose a method that enables NLI models to be used for inconsistency detection by segmenting documents into sentence units and aggregating scores between pairs of sentences. |
| Outcome: | The proposed method achieves state-of-the-art accuracy of 74.4% on six large inconsistency detection datasets. |
Multi-Document Summarization with Determinantal Point Processes and Contextualized Representations (D19-54)
Copied to clipboard
| Challenge: | Determinantal point processes (DPP) is one of the best performing techniques for extractive summarization. |
| Approach: | They propose to combine determinantal point processes with surface indicators for effective identification of summary-worthy sentences. |
| Outcome: | The determinantal point processes (DPP) framework is one of the best performing in summarization competitions. |
WikiAsp: A Dataset for Multi-domain Aspect-based Summarization (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing aspects-based summarization models are domain-specific due to large differences in the type of aspects for different domains. |
| Approach: | They propose a large-scale dataset for multi-domain aspect-based summarization using Wikipedia articles from 20 different domains. |
| Outcome: | The proposed model is based on Wikipedia articles from 20 different domains and uses the section titles and boundaries of each article as a proxy for aspect annotation. |
Extractive Opinion Summarization in Quantized Transformer Spaces (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing work on opinion summarization focuses on aggregating opinions among reviews . et al., 2018; see etal., 2019; liu eto, 2019) demonstrate the potential of opinion summaries. |
| Approach: | They propose an unsupervised system for extractive opinion summarization based on vector-quantized variables and an extraction algorithm. |
| Outcome: | The proposed method is validated by human studies showing that judges prefer it over baselines. |
TL;DR Progress: Multi-faceted Literature Exploration in Text Summarization (2024.eacl-demo)
Copied to clipboard
| Challenge: | TL;DR Progress is a literature explorer designed specifically for the text summarization literature. |
| Approach: | They propose to organize 514 papers based on a comprehensive annotation scheme for text summarization approaches and a fine-grained, faceted search. |
| Outcome: | The proposed tool organizes 514papers based on a comprehensive annotation scheme for text summarization approaches and enables fine-grained, faceted search. |
Disordered-DABS: A Benchmark for Dynamic Aspect-Based Summarization in Disordered Texts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current research focuses on predefined aspects within structured texts, neglecting complexities of dynamic and disordered environments. |
| Approach: | They propose a benchmark for dynamic aspect-based summarization tailored to unstructured text. |
| Outcome: | The proposed benchmark addresses the complexities of dynamic and disordered environments in unstructured text. |
On the Role of Summary Content Units in Text Summarization Evaluation (2024.naacl-short)
Copied to clipboard
Marcel Nawrath, Agnieszka Nowak, Tristan Ratz, Danilo Walenta, Juri Opitz, Leonardo Ribeiro, João Sedoc, Daniel Deutsch, Simon Mille, Yixin Liu, Sebastian Gehrmann, Lining Zhang, Saad Mahamood, Miruna Clinciu, Khyathi Chandu, Yufang Hou
| Challenge: | a human written summary content unit (SCU) is used to judge the quality of a summary . a pyramid evaluation method is based on SCUs that decompose a reference summary into concise sentences . |
| Approach: | They propose to use automated SCUs to evaluate the quality of a candidate summary . they propose to generate SCU approximations from AMR meaning representations and large language models . |
| Outcome: | The proposed method can be fully automated, but lacks the human effort to validate it. |
Does the structure of textual content have an impact on language models for automatic summarization? (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing models for automatic summarization of long sequences suffer from context limitation. |
| Approach: | They propose to take into account textual information coming from distinct passages from the long texts to be summarized. |
| Outcome: | The proposed model improves on the performance of LongFormer on English. |
Exploring Iterative Controllable Summarization with Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel at abstractive summarization tasks, but their ability to precisely control summary attributes remains underexplored. |
| Approach: | They propose a guide-to-explain framework for controllable summarization that enables the model to identify misaligned attributes in the initial draft and guides it to self-explan errors in the previous output. |
| Outcome: | The proposed framework generates well-adjusted summaries that satisfy the desired attributes with robust effectiveness while requiring surprisingly fewer iterations than other iterative approaches. |
PASS: Perturb-and-Select Summarizer for Product Reviews (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on product reviews summarization focuses on generating concise, coherent and informative summaries, but this task is challenging. |
| Approach: | They propose a product reviews summarization task that employs a large pre-trained Transformer-based model and a method for ranking these summaries according to desired criteria. |
| Outcome: | The proposed system avoids the problem of self-contradiction by ranking the summaries according to desired criteria. |
Entity-Aware Abstractive Multi-Document Summarization (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for multidocument summarization do not focus on explicitly modeling the underlying semantic information across documents. |
| Approach: | They propose an entityaware model for abstractive multi-document summarization that augments the classical Transformer-based encoder-decoder framework with a heterogeneous graph consisting of text units and entities as nodes. |
| Outcome: | The proposed model can deal with saliency and redundancy issues explicitly and can be used with pre-trained language models, arriving at improved performance. |
HydraSum: Disentangling Style Features in Text Summarization with Multi-Decoder Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization systems implicitly encode “decisions” about summary properties, but these are not enforced. |
| Approach: | They propose a new summarization architecture that extends existing models to a mixture-of-experts version with multiple decoders. |
| Outcome: | The proposed architecture outperforms baseline models in obtaining stylistically-diverse summaries by sampling from individual decoders or their mixtures. |
End-to-End Aspect-Guided Review Summarization at Scale (2025.emnlp-industry)
Copied to clipboard
Ilya Boytsov, Vinny DeGenova, Mikhail Balyasin, Joseph Walt, Caitlin Eusden, Marie-Claire Rochat, Margaret Pierson
| Challenge: | Existing methods to generate concise product review summaries are prone to hallucination, omission of important facts, and factual errors. |
| Approach: | They propose a large language model-based system that combines aspect-based sentiment analysis with guided summarization to generate concise product review summaries. |
| Outcome: | The proposed system generates concise and interpretable product review summaries using a large language model (LLM) dataset. |
Embrace Divergence for Richer Insights: A Multi-document Summarization Benchmark and a Case Study on Summarizing Diverse Information from News Articles (2024.naacl-long)
Copied to clipboard
Kung-Hsiang Huang, Philippe Laban, Alexander Fabbri, Prafulla Kumar Choubey, Shafiq Joty, Caiming Xiong, Chien-Sheng Wu
| Challenge: | Existing studies on multi-document summarization focus on collating information that all sources agree upon, but the task of summarizing diverse information remains underexplored. |
| Approach: | They propose a task of summarizing diverse information encountered in multiple news articles encompassing the same event using a dataset curated by a large language model. |
| Outcome: | The proposed task aims to summarize diverse information in multiple news articles encompassing the same event . the proposed task is difficult due to its limited coverage and verbosity biases . |
Learning to Prioritize: Precision-Driven Sentence Filtering for Long Text Summarization (2022.lrec-1)
Copied to clipboard
| Challenge: | Neural text summarization models are limited by their maximum input length, posing a challenge to summarizing longer texts comprehensively. |
| Approach: | They propose a pre-processing layer that removes low-quality sentences in articles to improve existing summarization models. |
| Outcome: | The proposed approach improves state-of-the-art summarization models on WikiHow and Reddit TIFU datasets by 3.84 and 8.57 points on the full test set and the long article subset. |
AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for evaluating factual consistency of abstractive summarization lack coherence or error-type coverage. |
| Approach: | They propose a framework that generates perturbed summaries using Abstract Meaning Representations (AMRs) they use a selection module NegFilter to ensure the quality of the generated negative examples . |
| Outcome: | The proposed framework outperforms existing systems on the AggreFact-SOTA benchmark and provides high error-type coverage. |
Discourse Understanding and Factual Consistency in Abstractive Summarization (2021.eacl-main)
Copied to clipboard
Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, Yejin Choi
| Challenge: | Existing abstractive summarization models often hallucinate information or generate factually incorrect summaries. |
| Approach: | They propose a general framework for abstractive summarization with factual consistency and distinct modeling of the narrative flow in an output summary. |
| Outcome: | The proposed framework generates abstracts with factual consistency and coherence significantly better than baselines. |
Document Summarization with Latent Queries (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing benchmarks for query-focused summarization are small for training large neural models. |
| Approach: | They propose a unified modeling framework for query-focused summarization . they model queries as discrete latent variables over document tokens . |
| Outcome: | The proposed framework outperforms strong comparison systems across benchmarks, query types, document settings, and target domains. |
Multi-doc Hybrid Summarization via Salient Representation Learning (2023.acl-industry)
Copied to clipboard
| Challenge: | Multi-document summarization is gaining more and more attention . extractive multi-doc approaches intend to directly extract key facts from multiple sources . |
| Approach: | They propose a multi-document hybrid summarization approach that generates a human-readable summary and extracts corresponding key evidences based on multi-doc inputs. |
| Outcome: | The proposed method generates a human-readable summary and extracts key evidences based on multi-doc inputs. |
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization systems have been shown to be more prone to unfaithful facts . 30% of summaries generated by pre-trained language models suffer from hallucination . |
| Approach: | They propose a method to remedy entity-level extrinsic hallucinations with Entity Coverage Control . they first compute entity coverage precision and prepend the corresponding control code . a further fine-tuning is performed to unlock zero-shot summarization . |
| Outcome: | The proposed method leads to more faithful and salient abstractive summarization in fine-tuning and zero-shot settings. |
Toward Expanding the Scope of Radiology Report Summarization to Multiple Anatomies and Modalities (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies are limited to a single modality and a chest X-ray, making it difficult to replicate results or compare approaches. |
| Approach: | They propose a dataset to generate an impression section of a radiology report . they propose to use three new modalities and seven new anatomies to evaluate their models . |
| Outcome: | The proposed model is based on the MIMIC-III and MIMIC CXR datasets and evaluates their clinical efficacy via RadGraph, a factual correctness metric. |
Incorporating Question Answering-Based Signals into Abstractive Summarization via Salient Span Selection (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for generating summarizations using QA-based supervision produce higher quality summaries than baseline methods. |
| Approach: | They propose a method for incorporating question-answering signals into a summarization model by automatically marking document NPs as salient based on whether they are answered in the gold summaries. |
| Outcome: | The proposed method generates higher-quality summaries than baseline methods on benchmark summarization datasets. |
GO FIGURE: A Meta Evaluation of Factuality in Summarization (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in text generation systems produce fluent, coherent, relevant, and factually correct text. |
| Approach: | They propose a metaevaluation framework for evaluating factuality evaluation metrics . they propose five necessary conditions to evaluate factual metrics on diagnostic factuity data . |
| Outcome: | The proposed framework provides robust evaluation that is extensible to multiple types of factual consistency and standard generation metrics, including QA metrics. |
TracSum: A New Benchmark for Aspect-Based Summarization with Sentence-Level Traceability in Medical Domain (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evidence-based summarization tasks require tracing source evidence to assess their accuracy. |
| Approach: | They propose a benchmark for traceable, aspect-based summarization that pairs summaries with sentence-level citations to enable users to trace back to the original context. |
| Outcome: | The proposed benchmark can be used to evaluate document summarization with LLMs and human evaluations. |
Scaling Multi-Document Event Summarization: Evaluating Compression vs. Full-Text Approaches (2025.naacl-short)
Copied to clipboard
| Challenge: | Summarizing large text collections is a valuable tool for document research . a multi-stage pipeline and lack of global context are challenges for large-scale summarization systems. |
| Approach: | They compare compression and full-text systems for large-scale multi-document summarization . they find that compression-based methods outperform full-context methods . |
| Outcome: | The proposed methods outperform compression-based methods on three datasets . however, they suffer information loss due to their multi-stage pipeline and lack of global context. |
Using Question Answering Rewards to Improve Abstractive Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Neural abstractive summarization models have seen improvements in recent years, but they still suffer from multiple drawbacks. |
| Approach: | They propose a general framework to train abstractive summarization models to alleviate these issues by question-answering based rewards. |
| Outcome: | The proposed framework is preferred over general abstractive summarization models. |
Interactive Text Ranking with Bayesian Optimization: A Case Study on Community QA and Summarization (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing methods that focus on learning a ranking across the whole candidate space are lacking user or task-specific training data. |
| Approach: | They propose an interactive ranking approach that actively selects pairs of candidates, from which the user selects the best. |
| Outcome: | The proposed approach outperforms existing methods in community question answering and extractive multidocument summarization and is an effective reward function for reinforcement learning. |
SummHelper: Collaborative Human-Computer Summarization (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing approaches for text summarization are mostly automated, with limited space for human intervention and control. |
| Approach: | They propose a 2-phase summarization assistant that facilitates human-machine collaboration . it suggests possible content and generates a coherent summary from these selections . authors hope to improve the efficiency of the computer and human-involved approach . |
| Outcome: | The proposed summarization assistant is a 2-phase summarizing assistant . it suggests potential content and consolidates the output with visual mappings . the proposed system is available for free on youtube . |
FineSurE: Fine-grained Summarization Evaluation using LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text summarization evaluation do not correlate well with human judgments . evaluators that use Likert scale scores are limited in their ability to perform deeper analysis. |
| Approach: | They propose a fine-grained evaluator specifically tailored for the summarization task using large language models. |
| Outcome: | The proposed method improves on open-source and proprietary LLMs and shows better completeness and conciseness than existing methods. |
From Key Points to Key Point Hierarchy: Structured and Expressive Opinion Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Key Point Analysis (KPA) is a new method for analyzing textual comments . it uses a list of concise sentences or phrases to extract key points from data . |
| Approach: | They propose to organize key points into a hierarchy according to their specificity . they compare methods for predicting pairwise relations between key points . |
| Outcome: | The proposed method improves on predicting pairwise key point relations and weak supervision. |
Noisy Self-Knowledge Distillation for Text Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to summarize text using a single reference and noisy datasets are ill-suited to summarising on single reference datasets. |
| Approach: | They propose to use self-knowledge distillation to improve text summarization by generating smoothed labels for students and teachers to reduce model uncertainty. |
| Outcome: | The proposed framework improves on pretrained and non-pretrained models on three benchmarks. |
HIBRIDS: Attention with Hierarchical Biases for Structure-aware Long Document Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Document structure is critical for efficient information consumption, but it is difficult to encode it efficiently into the modern Transformer architecture. |
| Approach: | They propose a task which injects Hierarchical Biases foR Incorporating Document Structure into attention score calculation. |
| Outcome: | The proposed model produces better question-summary hierarchies than comparisons on hierarchy quality and content coverage, the authors show . |
InstructPTS: Instruction-Tuning LLMs for Product Title Summarization (2023.emnlp-industry)
Copied to clipboard
| Challenge: | E-commerce product catalogs contain billions of items with lengthy titles . this leads to a gap between how customers refer to these unnatural titles - and how they are used . |
| Approach: | They propose a novel approach to product title summarization that uses a fine-tuned instruction strategy to train a highly accurate model. |
| Outcome: | The proposed approach can generate more accurate product title summaries with an improvement of over 14 and 8 BLEU and ROUGE points. |
Summarizing Medical Conversations via Identifying Important Utterances (2020.coling-main)
Copied to clipboard
| Challenge: | Applying natural language processing (NLP) techniques to the medical field is a prevailing trend nowadays and has great potential in many applications, such as key information extraction in medical literature. |
| Approach: | They propose to use a hierarchical encoder-tagger model to generate medical conversation summarization by identifying important utterances. |
| Outcome: | The proposed model outperforms baseline models and models and adds conversation-related features to improve performance. |
Bias in Opinion Summarisation from Pre-training to Adaptation: A Case Study in Political Bias (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on extractive summarisation but limited attention has been paid to abstractive summaries. |
| Approach: | They propose to trace bias in abstractive summarisation models to social media opinions using different models and adaptation methods. |
| Outcome: | The proposed model is compared with other models and adaptation methods to summarise social media opinions using different models and adaption methods. |
Factual Relation Discrimination for Factuality-oriented Abstractive Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing factuality-oriented abstractive summarization models only consider the integration of factual information and ignore the causes of factuual errors. |
| Approach: | They propose a factuality-oriented abstractive summarization model that can identify the causes of factual errors. |
| Outcome: | The proposed model outperforms state-of-the-art models in factual metrics. |
Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for meeting summarization rely on transcripts and generate generic summaries, failing to contextualize long discussions and to tailor information to individual preferences and productivity requirements. |
| Approach: | They propose a multi-source approach that considers supplementary materials and generates a summary from this enriched transcript. |
| Outcome: | The proposed model increases summary relevance by 9% and produces more content-rich outputs. |
Detecting Incongruent News Articles Using Multi-head Attention Dual Summarization (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent studies on incongruity detection focus on estimating the similarity between the headline and the encoding of the body or its summary but most of these methods fail to handle inconvenient news articles created with embedded noise. |
| Approach: | They propose a method which generates two types of summaries that capture the congruent and incongruent parts in the body separately. |
| Outcome: | The proposed method outperforms the state-of-the-art methods over three publicly available datasets. |
Template-based Abstractive Microblog Opinion Summarization (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on Twitter uses extractive summarization to filter through information, but this approach often includes incomplete or redundant information. |
| Approach: | They propose to use Twitter data to generate 3100 gold-standard opinion summaries. |
| Outcome: | The proposed method outperforms previous work on extractive summarization models and fine-tunes to improve performance. |
Evaluating the Efficacy of Summarization Evaluation across Languages (2021.findings-acl)
Copied to clipboard
| Challenge: | Using multilingual summarization evaluation methods is more reliable and interpretable than manual methods. |
| Approach: | They propose to use multilingual BERT within BERTScore to evaluate summarization evaluation metrics . they use English datasets that are not representative of modern summarizing systems . |
| Outcome: | The proposed methods perform well across all languages, at a level above that for English. |
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models suffer from the problem of hallucinations, where a summary contains facts or entities not present in the original document. |
| Approach: | They propose an abstractive summarization model that addresses the problem of factuality during pre-training and fine-tuning. |
| Outcome: | Experiments on three downstream tasks show that FactPEGASUS significantly improves factuality compared to the original pre-training objective in zero-shot and few-shot settings. |
Faithful to the Document or to the World? Mitigating Hallucinations via Entity-Linked Knowledge in Abstractive Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing abstractive summarization systems are hampered by content hallucinations in which models generate text that is not directly inferable from the source alone. |
| Approach: | They propose to use external knowledge to latently connect entities and concepts to latences to lend provenance to many of these unfaithful yet factual entities. |
| Outcome: | The proposed model can be used to improve the factuality of summarizations without simply making them more extractive. |
Legal Case Document Summarization: Extractive and Abstractive Methods and their Evaluation (2022.aacl-main)
Copied to clipboard
Abhay Shukla, Paheli Bhattacharya, Soham Poddar, Rajdeep Mukherjee, Kripabandhu Ghosh, Pawan Goyal, Saptarshi Ghosh
| Challenge: | Summarization of legal case judgement documents is a challenging problem in Legal NLP. |
| Approach: | They propose to use extractive and abstractive summarization methods to evaluate legal document summarizing systems. |
| Outcome: | The proposed methods have been evaluated on three legal summarization datasets. |
Summarization Evaluation in the Absence of Human Model Summaries Using the Compositionality of Word Embeddings (C18-1)
Copied to clipboard
| Challenge: | Existing summary evaluation methods rely on multiple model summaries to evaluate quality of summary outputs. |
| Approach: | They propose a new summary evaluation approach that does not require human model summaries . they exploit compositional capabilities of word embeddings to develop features . |
| Outcome: | The proposed metric replicates human-generated summarization scores on data from TAC 2008 and 2009 . the features are then used to train a learning model for predicting the summary content quality in the absence of gold models. |
Analyzing Interpretability of Summarization Model with Eye-gaze Information (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have provided saliency scores for neural summarization models . eye-gaze information is often used as a proxy for human attention in reading tasks . |
| Approach: | They propose to compare model saliency to human eye-gaze data to determine whether it conforms to human gaze during summarization. |
| Outcome: | The proposed framework compares the model behavior to human summarization performance. |
Long Document Summarization with Top-down and Bottom-up Inference (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent models infer latent representations of words or tokens with a transformer encoder, which is bottom-up and thus does not capture long-distance context well. |
| Approach: | They propose a method to infer latent representations of words or tokens in documents . they assume a hierarchical structure of a document where top-level captures long range dependency . |
| Outcome: | The proposed model can summarize an entire book and achieve competitive performance on a wide range of document summarization benchmarks. |
Aspect and Sentiment Aware Abstractive Review Summarization (C18-1)
Copied to clipboard
| Challenge: | Abstractive summarization is a task that generates short and concise summaries of user generated reviews. |
| Approach: | They propose an interactive attention mechanism to learn the representations of context and aspect words within reviews, acted as an encoder. |
| Outcome: | The proposed model achieves impressive results compared to other strong competitors on a real-life dataset. |
From Information to Insight: Leveraging LLMs for Open Aspect-Based Educational Summarization (2025.acl-long)
Copied to clipboard
| Challenge: | a novel dataset summarizes student reflections on STEM lectures . ReflectASP eases the exploration of open-aspect-based summarization (OABS) despite the limitations of current datasets, it is still under-explored. |
| Approach: | They propose a dataset that summarizes student reflections on STEM lectures . they propose two refinement methods to improve summaries . |
| Outcome: | The proposed dataset summarizes student reflections on STEM lectures using automatic and human evaluations. |
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents (N18-2)
Copied to clipboard
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, Nazli Goharian
| Challenge: | Existing abstractive summarization models focus on summarizing sentences and short documents. |
| Approach: | They propose a hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary. |
| Outcome: | The proposed model significantly outperforms state-of-the-art models on two large-scale datasets of scientific papers. |
Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Tasks (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to evaluate summary faithfulness are sub-optimal due to the granularity level considered for premises and hypotheses. |
| Approach: | They propose a novel approach that uses a variable premise size and simplifies summary sentences into shorter hypotheses. |
| Outcome: | The proposed model performs better on diverse summarisation tasks than existing models. |
CaseSumm: A Large-Scale Dataset for Long-Context Summarization from U.S. Supreme Court Opinions (2025.findings-naacl)
Copied to clipboard
| Challenge: | CaseSumm is a dataset for long-context summarization in the legal domain . human groundtruth summaries are often not available for legal summarizing . |
| Approach: | They propose a dataset for long-context summarization that includes SCOTUS opinions and their official summaries. |
| Outcome: | The proposed dataset is the largest open legal case summarization dataset . it outperforms larger models on automatic metrics and human evaluation . |
Discourse-Driven Evaluation: Unveiling Factual Inconsistency in Long Document Summarization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing summarization systems can generate fluent summaries, but their ability to produce factually consistent summary remains questionable. |
| Approach: | They propose a framework that decomposes long texts into discourse-inspired chunks and utilizes discourse information to better aggregate sentence-level scores predicted by NLI models. |
| Outcome: | The proposed framework shows better performance over multiple benchmarks, focusing on long document summarization. |
Pruning Basic Elements for Better Automatic Evaluation of Summaries (N18-2)
Copied to clipboard
| Challenge: | Summarization studies work on increasing the scores that are given by automatic evaluation measures. |
| Approach: | They propose a simple but highly effective automatic evaluation measure of summarization, pruned Basic Elements. |
| Outcome: | The proposed measure outperforms ROUGE and BE in most cases and achieves highest correlation coefficient in TAC 2011 AESOP task. |
DomainSum: A Hierarchical Benchmark for Fine-Grained Domain Shift in Abstractive Text Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization is a crucial task in natural language processing . current research focuses on summarizing specific types of documents . domain shifts between documents affect summarisation performance . |
| Approach: | They propose a hierarchical benchmark to capture fine-grained domain shifts in abstractive summarization. |
| Outcome: | The proposed benchmark measures the generalization capabilities of pre-trained language models and large language models in in-domain and cross-domain settings. |
RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to integrate rhetorical structure theory into long document summarization models are unexplored. |
| Approach: | They propose to integrate rhetorical structure theory into a long document summarization model by explicitly incorporating rhetorical uncertainty into the model. |
| Outcome: | The proposed models outperform the vanilla LoRA and full-parameter fine-tuning models and outperformed previous state-of-the-art methods. |
Ensure the Correctness of the Summary: Incorporate Entailment Knowledge into Abstractive Sentence Summarization (C18-1)
Copied to clipboard
| Challenge: | Existing approaches focus on improving the informativeness of the summary, but ignore the correctness. |
| Approach: | They propose an entailment-aware encoder and an aML-based decoder to improve the correctness of the sentence summarization task. |
| Outcome: | The proposed model outperforms baselines on informativeness and correctness. |
OpenAsp: A Benchmark for Multi-document Open Aspect-based Summarization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models focus on a limited set of predefined aspects, resulting in a lack of realistic open aspect setting. |
| Approach: | They propose a benchmark for multi-document open aspect-based summarization using an annotation protocol. |
| Outcome: | The proposed benchmark satisfies the needs of users in real-world scenarios. |
Examining the State-of-the-Art in News Timeline Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on news timeline summarization (TLS) has left an unclear picture of how well it is currently solved and how it can be approached. |
| Approach: | They propose a combination of different TLS strategies that improves over the stateof-the-art on all tested benchmarks. |
| Outcome: | The proposed method improves over the state-of-the-art on all tested benchmarks. |
MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Current news summarization systems often contain 'extrinsic hallucinations', i.e. facts that are not present in the source document, which are often derived via world knowledge. |
| Approach: | They propose to use multiple supplementary resource documents to assist the task by pairing a single document with a human authored summary as the summary. |
| Outcome: | The proposed model reduces 55% of hallucinations when compared to single-document summarization models trained on the main article only. |
DTELS: Towards Dynamic Granularity of Timeline Summarization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing timeline summarizations lack flexibility to meet diverse granularity needs . a fine-grained timeline showing the technical details is preferred for news topics . |
| Approach: | They propose a new paradigm to construct adaptive timelines based on user instructions or requirements. |
| Outcome: | The proposed timelines are informative and granularly consistent, but they struggle to generate consistent timelines. |
Predicting Intervention Approval in Clinical Trials through Multi-Document Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Clinical trials are a key tool for assessing the effectiveness of health interventions. |
| Approach: | They propose a method to generate informative summaries from PubMed articles . they use the extracted summary to train a BERT-based classifier to predict effectiveness . |
| Outcome: | The proposed method generates informative summaries from multiple documents and trains a classifier based on the summary extracted from the abstracts to predict the effectiveness of the intervention. |
Self-Supervised and Controlled Multi-Document Opinion Summarization (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing unsupervised methods for summarizing reviews are based on bootstrapping and require a combination of loss functions or hierarchical latent variables to ensure that the generated summaries remain on-topic. |
| Approach: | They propose a self-supervised setup that considers an individual document as a target summary for a set of similar documents. |
| Outcome: | The proposed setup makes training simpler than previous approaches by relying only on standard log-likelihood loss and mainstream models. |
Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization (2023.findings-eacl)
Copied to clipboard
| Challenge: | generating aspect-specific and general opinion summaries is challenging due to the lack of annotated data. |
| Approach: | They propose two unsupervised approaches to generate aspect-specific and general opinion summaries by training on synthetic datasets constructed with aspect-related review contents. |
| Outcome: | The proposed method outperforms existing methods on space and Oposum+ and on other metrics. |
What’s Wrong? Refining Meeting Summaries with LLM Feedback (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for meeting summarization are limited and lack the robustness and context-based accuracy needed to maintain relevance. |
| Approach: | They propose a multi-LLM correction approach for meeting summarization using a two-phase process that mimics the human review process: mistake identification and summary refinement. |
| Outcome: | The proposed approach improves the quality of a given meeting summarization measured by relevance, informativeness, conciseness, and coherence. |
ROUGE-SciQFS: A ROUGE-based Method to Automatically Create Datasets for Scientific Query-Focused Summarization (2025.coling-main)
Copied to clipboard
| Challenge: | Scientific Query-Focused Summarization (Sci-QFS) has lagged in development due to the lack of data. |
| Approach: | They propose a method to take advantage of existing academic papers to obtain large-scale datasets for this task automatically. |
| Outcome: | The proposed method outperforms existing models on the datasets and shows that it is relatively straightforward for humans. |
Multi-View Source Ablation for Faithful Summarization (2023.findings-eacl)
Copied to clipboard
| Challenge: | MuFaSSa is a metric for evaluating faithfulness of abstractive summaries . it uses different strategies to remove information from source document to form multiple ablated views . |
| Approach: | They propose a metric for evaluating faithfulness of abstractive summaries using multiple ablated views. |
| Outcome: | The proposed metric outperforms existing models on summarization tasks and human-annotated faithfulness labels. |
Reference-free Summarization Evaluation via Semantic Correlation and Compression Ratio (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for summarization use human annotations as reference. |
| Approach: | They propose a new automatic reference-free evaluation metric that compares semantic distribution between source document and summary by pretrained language models and considers summary compression ratio. |
| Outcome: | The proposed metric is more consistent with human evaluation in terms of coherence, consistency, relevance and fluency. |
TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced tasks like text summarization, but their size and computational demands limit their use in resource-constrained and privacy-centric settings. |
| Approach: | They propose a framework for distilling LLMs’ text summarization abilities into a compact, local model using a curriculum learning strategy that evolves from simple to complex tasks. |
| Outcome: | The proposed framework outperforms baseline models on CNN/DailyMail, XSum, and ClinicalTrial, and improves interpretability by providing insights into the summarization rationale. |
MODABS: Multi-Objective Learning for Dynamic Aspect-Based Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for generating content specific summarization assume a fixed set of known aspects. |
| Approach: | They propose a dynamic aspect-based summarization framework that optimizes aspect number prediction and minimizes disparity between generated and reference summaries. |
| Outcome: | The proposed method outperforms baselines on three diverse datasets on different aspects of the input text. |
Accuracy is not enough: Evaluating Personalization in Summarizers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing accuracy measures cannot evaluate the degree of personalization of summarization models. |
| Approach: | They propose to use a PENS dataset to analyze the degree of personalization of ten different summarization models. |
| Outcome: | The proposed measure can evaluate the degree of personalization of summarization models using the PENS dataset. |
Attend to Medical Ontologies: Content Selection for Clinical Abstractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies have reported that clinicians read the IMPRESSION as they have less time to review findings. |
| Approach: | They propose to augment salient ontological terms into the abstractive summarizer by augmenting salient ontologies into the semantic summariser. |
| Outcome: | The proposed model significantly improves state-of-the-art results in terms of ROUGE metrics on two publicly available clinical data sets. |
AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Community Question Answering (CQA) fora lack a dataset to produce answer summarizations . a novel dataset of 4,631 CQA threads is used to generate answer summaries . |
| Approach: | They propose a dataset of 4,631 CQA threads for answer summarization curated by professional linguists. |
| Outcome: | The proposed approach boosts summarization performance according to automatic evaluation. |
Aspect-based summarization of pros and cons in unstructured product reviews (C18-1)
Copied to clipboard
| Challenge: | SynPat, a system based on syntactic phrases selected on the basis of valence scores, and a neural-network-based system trained on clusters of word-embedding encodings of similar pros and cons are compared to SynPat. |
| Approach: | They propose to use syntactic phrases selected on the basis of valence scores to generate pros and cons summaries. |
| Outcome: | The proposed systems outperform the baseline systems on held-out reviews with gold-standard pros and cons and on human annotators on relevance and completeness. |
DPL: Diverse Preference Learning Without A Reference Model (2025.naacl-long)
Copied to clipboard
Abhijnan Nath, Andrey Volozin, Saumajit Saha, Albert Aristotle Nanda, Galina Grunin, Rahul Bhotika, Nikhil Krishnaswamy
| Challenge: | Existing methods to direct preference alignment do not utilize diversity in preference annotations which limits their applicability. |
| Approach: | They propose a reference-model-free method that learns a baseline desirability in LLM responses while being robust to the diversity of preference annotations. |
| Outcome: | The proposed method learns a baseline desirability in LLM responses while being robust to the diversity of preference annotations. |
Beyond Multiple Choice: Evaluating Steering Vectors for Summarization (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent methods for controlling language models can often be classified into three main strategies: prompt engineering, trainable decoding mechanisms, fine-tuning according to specific objectives. |
| Approach: | They evaluate steering vectors for controlling topical focus, sentiment, toxicity, and readability in abstractive summaries across the SAMSum, NEWTS, and arXiv datasets. |
| Outcome: | The proposed method is effective in free-form generation, but high steering strengths induce degenerate repetition and factual hallucinations. |
Content Selection in Deep Learning Models of Summarization (D18-1)
Copied to clipboard
| Challenge: | Using deep learning models, we find that word embedding does not improve performance over simpler models. |
| Approach: | They propose to use sentence embedding to perform content selection across multiple domains . they propose to propose two alternative models that use auto-regressive sentence extraction . |
| Outcome: | The proposed models improve performance across news, personal stories, meetings, and medical articles. |
On the Summarization of Consumer Health Questions (P19-1)
Copied to clipboard
| Challenge: | Question understanding is one of the main challenges in question answering. |
| Approach: | They propose to use semantic augmentation to augment question datasets to improve their performance. |
| Outcome: | The proposed model outperforms sequence-to-sequence attentional models on the medical question summarization task with a ROUGE-1 score of 44.16%. |
Select and Summarize: Scene Saliency for Movie Script Summarization (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing models for summarizing long-form narrative texts are computationally and memory limited. |
| Approach: | They propose a scene saliency dataset that consists of human-annotated salient scenes for 100 movies. |
| Outcome: | The proposed model outperforms state-of-the-art models and reflects the information content of a movie more accurately than a model that takes the whole movie script as input. |
UniSumEval: Towards Unified, Fine-grained, Multi-dimensional Summarization Evaluation for LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks for summarization quality evaluation lack diverse input scenarios, focus on narrowly defined dimensions, and struggle with subjective and coarse-grained annotation schemes. |
| Approach: | They propose to use AI to help human annotations and identifie potentially hallucinogenic input texts. |
| Outcome: | The proposed benchmarks improve on existing benchmarks in terms of input diversity, granularity of human annotations, and evaluation dimensions. |
NeuS: Neutral Multi-News Summarization for Mitigating Framing Bias (2022.naacl-main)
Copied to clipboard
| Challenge: | a new task is proposed to reduce media news framing bias by generating a neutral summary from multiple news articles of the varying political leanings. |
| Approach: | They propose a task that generates a neutral summary from multiple news articles . they find title provides a good signal for framing bias and propose metric and model . |
| Outcome: | The proposed task can neutralize news content in hierarchical order from title to article . scalability remains a bottleneck due to the time-consuming human labor needed for composing the roundup . |
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (2023.eacl-main)
Copied to clipboard
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, Tatsunori Hashimoto
| Challenge: | Existing studies have shown that large language models contain linguistic and societal biases, but it is unclear how these biase amplify to downstream tasks. |
| Approach: | They investigate how name-nationality bias propagates from pre-training to downstream tasks . they show that these biases manifest themselves as hallucinations in summarization . |
| Outcome: | The proposed model can reduce the rate of hallucinations, but does not change the types of biases that do appear. |
Investigating Efficiently Extending Transformers for Long Input Summarization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large pretrained Transformer models have proven capable at tackling natural language tasks, but handling long sequence inputs still poses a significant challenge. |
| Approach: | They propose an extension of the PEGASUS model with additional long input pretraining to handle inputs of up to 16K tokens. |
| Outcome: | The proposed model achieves strong performance on long input summarization tasks comparable with much larger models. |
Few-shot Query-Focused Summarization with Prefix-Merging (2022.emnlp-main)
Copied to clipboard
| Challenge: | Query-focused summarization has been considered as an important extension for text summarizing . lack of large-scale datasets hinders its development . |
| Approach: | They propose to integrate text summarization and question answering into a prefix-based pretraining strategy for few-shot learning in query-focused summarizing. |
| Outcome: | The proposed prefix-based pretraining outperforms fine-tuning on query-focused summarization. |
Unfolding the Headline: Iterative Self-Questioning for News Retrieval and Timeline Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | a new approach to timeline summarization is proposed for open-domain news content . large language models (LLMs) can be used to extract and organize news events from multiple documents . |
| Approach: | They propose a method to integrate Large Language Models into news timeline summarization by iterating on how events are linked and posing new questions. |
| Outcome: | The proposed system is able to generate and refresh chronological summaries based on documents retrieved in each round. |
Summarizing Community-based Question-Answer Pairs (2022.emnlp-main)
Copied to clipboard
| Challenge: | Community-based question answering (CQA) has become an essential component of online services. |
| Approach: | They propose a novel task to summarize CQA pairs into a concise summary . they use a benchmark dataset and a sentence-type transfer and deduplication removal approach . |
| Outcome: | The proposed task aims to create a concise summary from CQA pairs . the proposed method is stronger than existing methods and is publicly available . |
Comparative Opinion Summarization via Collaborative Decoding (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing opinion summarization methods are insufficient to help users compare multiple choices. |
| Approach: | They propose a comparative opinion summarization task that generates two contrastive summaries and one common summary from two different candidate sets of reviews. |
| Outcome: | The proposed framework produces higher-quality contrastive and common summaries than state-of-the-art models. |
OASum: Large-Scale Open Domain Aspect-based Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing generic summarization methods generate only one summary for all different requests which is not optimal for diverse demands. |
| Approach: | They use crowd-sourced knowledge on Wikipedia to create a large-scale open-domain aspect-based summarization dataset with 1 million different aspects on 2 million Wikipedia pages. |
| Outcome: | The proposed model can generate diverse aspect-based summarizations on Wikipedia with zero/few-shot and fine-tuning on seven downstream datasets. |
Does Pretraining for Summarization Require Knowledge Transfer? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing theories claim that pretraining models learn linguistic knowledge from the pretraining corpus, but scientific explanations for these benefits remain unknown. |
| Approach: | They propose to use random character n-grams to test models on real corpora to see if the small residual benefit of using real data could be accounted for by the structure of the pretraining task. |
| Outcome: | The proposed task performs on documents consisting of character n-grams, whereas pretrained models perform on real corpora with no residual benefit. |
Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance (2021.findings-acl)
Copied to clipboard
| Challenge: | Query focused summarization models aim to generate summaries from source documents that can answer the given query. |
| Approach: | They propose a QFS-BART model that incorporates the explicit answer relevance of the source documents given the query via a question answering model. |
| Outcome: | Empirical results show that the proposed model achieves the new state-of-the-art performance. |
Domain Aligned Prefix Averaging for Domain Generalization in Abstractive Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on domain generalization have sophisticated training algorithms. |
| Approach: | They propose a lightweight, weight averaging approach to domain generalization for abstractive summarization using prefix tuning and weight adjusting. |
| Outcome: | The proposed method performs better on four diverse summarization domains compared to baselines. |
Benchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing QA-based summarization metrics must automatically determine whether the QA model’s prediction is correct or not. |
| Approach: | They benchmark lexical answer verification methods used by current QA-based metrics and two more sophisticated text comparison methods, BERTScore and LERC. |
| Outcome: | The proposed methods outperform the other methods in some settings while remaining statistically indistinguishable from lexical overlap in others. |
Attribute-aware Sequence Network for Review Summarization (D19-1)
Copied to clipboard
| Challenge: | Existing review summarization systems generate summary only based on review content and neglect the authors’ attributes (e.g., gender, age, and occupation). |
| Approach: | They propose an Attribute-aware Sequence Network (ASN) to take the aforementioned users’ characteristics into account by encoding their attributes over the words. |
| Outcome: | The proposed model outperforms existing systems on tripAtt and human evaluation by taking the authors' attributes into account and incorporating attribute embedding and word-using habits into word prediction. |
Contrastive Attention Mechanism for Abstractive Sentence Summarization (D19-1)
Copied to clipboard
| Challenge: | Existing attention mechanisms for abstractive sentence summarization are based on rule-based methods and large-scale training corpora. |
| Approach: | They propose a contrastive attention mechanism that extends the sequence-to-sequence framework for abstractive sentence summarization task. |
| Outcome: | The proposed mechanism improves the state-of-the-art on the abstractive sentence summarization task. |
What Matters to an LLM? Behavioral and Computational Evidences from Summarization (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly entrusted with the management of information. |
| Approach: | They combine behavioral and computational analyses to find out what LLMs prioritize . they generate length-controlled summaries and derive empirical importance distributions . |
| Outcome: | The proposed model converges on consistent importance patterns and clusters more by family than by size. |
LipKey: A Large-Scale News Dataset for Absent Keyphrases Generation and Abstractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work has addressed each element individually, but this study focuses on LipKey, the largest news corpus with human-written abstractive summaries, absent keyphrases, and titles. |
| Approach: | They propose a novel news dataset that consists of highly absent keyphrases . they combine lips keyphrase and TF-IDF to obtain abstractive summaries . |
| Outcome: | The proposed dataset is the largest news corpus with human-written abstractive summaries, absent keyphrases, and titles. |
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Contemporary leading-edge systems for abstractive (long) text summarization employ Transformer encoderdecoder architectures that only consider the nuclearity annotation . |
| Approach: | They propose to incorporate Rhetorical Structure Theory into a novel summarization model that incorporates both the types and uncertainty of rhetorical relations. |
| Outcome: | The proposed model outperforms state-of-the-art models on automatic metrics and human evaluation. |
Which Information Matters? Dissecting Human-written Multi-document Summaries with Partial Information Decomposition (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to characterize human-written summaries do not account for the nature of high-quality summary. |
| Approach: | They propose to characterize human-written summaries using partial information decomposition . they propose to decompose mutual information provided by all source documents into union, redundancy, synergy, and unique information . |
| Outcome: | The proposed approach decomposes the mutual information provided by all source documents into union, redundancy, synergy, and unique information. |
Multi-Dimensional Optimization for Text Summarization via Reinforcement Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing summarization methods target a specific dimension, resulting in poor quality summaries. |
| Approach: | They propose multi-objective reinforcement learning tailored to generate balanced summaries across all dimensions. |
| Outcome: | The proposed model achieves significant performance gains compared to baseline models on representative summarization datasets on four dimensions. |
Focus on the Action: Learning to Highlight and Summarize Jointly for Email To-Do Items Summarization (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing to-do item generation models focus on generating action mentions to provide more structured summaries of email text. |
| Approach: | They propose a learning to highlight and summarize framework to learn to identify the most salient text and actions and incorporate these structured representations to generate more faithful to-do items. |
| Outcome: | The proposed model outperforms baseline models and achieves state-of-the-art performance in terms of evaluation and human judgement. |
Earlier Isn’t Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization (D19-1)
Copied to clipboard
| Challenge: | despite recent advances in neural summarization systems, the underlying logic behind the improvements remains unexplored. |
| Approach: | They define three sub-aspects of summarization: position, importance, diversity . position exhibits substantial bias in news articles, but not with academic papers . |
| Outcome: | evaluators found that position bias is not present in academic papers and meeting minutes . elucidation provides useful lessons on analyzing summarization datasets . |
Convex Aggregation for Opinion Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in text autoencoders have significantly improved the quality of the latent space, allowing models to generate consistent text from aggregated latent vectors. |
| Approach: | They develop a framework which searches input-output word overlap for latent vector aggregation. |
| Outcome: | The proposed framework improves the quality of the latent space and establishes state-of-the-art performance on two opinion summarization benchmarks. |
AWESOME: GPU Memory-constrained Long Document Summarization using Memory Mechanism and Global Salient Content (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing solutions focus on efficient attentions or divide-and-conquer strategies, but these methods sacrifice global context, leading to incoherent and uninformative summaries. |
| Approach: | They propose to leverage the memory-efficient nature of divide-and-conquer methods while preserving global context. |
| Outcome: | The proposed framework improves informativeness, faithfulness, and coherence over baselines on government reports, meeting transcripts, screenplays, scientific papers, and novels. |
EntSUMv2: Dataset, Models and Evaluation for More Abstractive Entity-Centric Summarization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Entity-centric summarization is a form of controllable summarizing that aims to generate a summary for a specific entity given a document. |
| Approach: | They propose to use a more abstract version of the original entity-centric ENTSUM summarization dataset to generate a shorter annotated summary for downstream users. |
| Outcome: | The proposed method is more abstract and uses supervised fine-tuning and large-scale instruction tuning to provide more specific and useful summaries for downstream users. |
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback (2023.acl-long)
Copied to clipboard
Paul Roit, Johan Ferret, Lior Shani, Roee Aharoni, Geoffrey Cideron, Robert Dadashi, Matthieu Geist, Sertan Girgin, Leonard Hussenot, Orgad Keller, Nikola Momchev, Sabela Ramos Garea, Piotr Stanczyk, Nino Vieillard, Olivier Bachem, Gal Elidan, Avinatan Hassidim, Olivier Pietquin, Idan Szpektor
| Challenge: | Recent advances in abstractive summarization systems produce factually inconsistent text . this is emphasized in tasks like summarizing, which often produce inconsistent text with no input article . |
| Approach: | They use reinforcement learning to optimize for factual consistency and explore trade-offs . they use textual-entailment rewards to optimize the accuracy of the generated summaries . |
| Outcome: | The proposed method improves faithfulness, salience and conciseness of the generated summaries. |
Leveraging Information Bottleneck for Scientific Document Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract salient sentences from document are unsupervised and rely on graph-based methods for sentence ranking. |
| Approach: | They propose an unsupervised extractive approach to document level summarization based on the Information Bottleneck principle. |
| Outcome: | The proposed framework can be extended to a multi-view framework by different signals. |
Bias in News Summarization: Measures, Pitfalls and Corpora (2024.findings-acl)
Copied to clipboard
| Challenge: | Pretrained large language models can reproduce harmful social biases in constrained settings, such as summarization. |
| Approach: | They propose a method to generate input documents with carefully controlled demographic attributes and then apply it to a controlled setting. |
| Outcome: | The proposed method allows to generate input documents with carefully controlled demographic attributes while working with real-world input documents. |
ACLSum: A New Dataset for Aspect-based Summarization of Scientific Publications (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing statistical phrasal or hierarchical machine translation systems relies on a large set of translation rules which results in engineering challenges. |
| Approach: | They propose to use factorized grammar from the field of linguistics as more general translation rules from XTAG English Grammar to generate a manually crafted summarization dataset. |
| Outcome: | The proposed method outperforms existing methods on low-resource language translation tasks with less training data. |
Questioning the Validity of Summarization Datasets and Improving Their Factual Consistency (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization systems have a lack of a defined definition for the task . factual consistency is a key factor in summarizing, but there are still deficiencies . a new study shows that summarized summarisation models achieve improved performance . |
| Approach: | They propose a filtered summarization dataset with improved factual consistency to address this problem . they argue that the dataset should become a valid benchmark for developing and evaluating summarizing systems . |
| Outcome: | The proposed model improves on a popular summarization dataset with improved factual consistency. |
DA-Pred: Performance Prediction for Text Summarization under Domain-Shift and Instruct-Tuning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often don’t perform as expected under Domain Shift or after Instruct-tuning. |
| Approach: | They propose a method that uses the known performance in high-resource domains and fine-tuning settings to predict performance in low-resourced domains or base models. |
| Outcome: | The proposed method can help researchers decide if resources should be allocated for data labeling and LLM Instruct-tuning. |
Question Answering as an Automatic Evaluation Metric for News Article Summarization (N19-1)
Copied to clipboard
| Challenge: | Recent work on summarization and headline generation focuses on maximizing ROUGE scores. |
| Approach: | They propose an extrinsic evaluation metric that maximizes ROUGE scores for automatic summarization and headline generation. |
| Outcome: | The proposed model maximizes ROUGE scores while increasing competitive results. |
CTRLsum: Towards Generic Controllable Text Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing summarization systems produce generic summaries that are disconnected from users’ preferences and expectations. |
| Approach: | They propose a generic framework to control generated summaries through a set of keywords. |
| Outcome: | The proposed framework is comparable or better than strong pretrained systems on three domains of summarization datasets and five control tasks. |
Leveraging Locality in Abstractive Text Summarization (2022.emnlp-main)
Copied to clipboard
Yixin Liu, Ansong Ni, Linyong Nan, Budhaditya Deb, Chenguang Zhu, Ahmed Hassan Awadallah, Dragomir Radev
| Challenge: | Neural attention models have improved on many natural language processing tasks, but their quadratic memory complexity hinders their applications in long text summarization. |
| Approach: | They propose to use a restricted context to study locality in text summarization . they propose to employ a quadratic memory growth with respect to the input length . |
| Outcome: | The proposed model has better performance than baseline models with efficient attention modules. |
CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning (2022.naacl-main)
Copied to clipboard
Xiangru Tang, Arjun Nair, Borui Wang, Bingyao Wang, Jai Desai, Aaron Wade, Haoran Li, Asli Celikyilmaz, Yashar Mehdad, Dragomir Radev
| Challenge: | Factual inconsistencies in generated summaries severely limit the practical applications of abstractive dialogue summarization. |
| Approach: | They propose a typology of factual errors to better understand hallucinations generated by current models and a contrastive fine-tuning strategy to improve the factual consistency and overall quality of summaries. |
| Outcome: | The proposed model significantly reduces all kinds of factual errors on both SAMSum dialogue summarization and AMI meeting summarizing datasets. |
Automatic, Meta and Human Evaluation for Multimodal Summarization with Multimodal Output (2024.naacl-long)
Copied to clipboard
| Challenge: | Multimodal summarization with multimodal output (MSMO) has attracted increasing research interest . evaluation is an emerging yet underexplored research topic . |
| Approach: | They propose a framework that studies three research questions of MSMO evaluation . they propose an automatic evaluation metric and a meta-evaluation benchmark dataset . |
| Outcome: | The proposed evaluation metric and human-annotated meta-evaluation benchmark are used to assess the quality of evaluation metrics and show the framework is effective. |
On Positional Bias of Faithfulness for Long-form Summarization (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models exhibit positional bias in long-context settings, under-attending to information in the middle. |
| Approach: | They compile eight human-annotated long-form summarization datasets to evaluate faithfulness . they find that LLMs faithfully summarize beginning and end of documents but neglect middle content . |
| Outcome: | The proposed methods show that LLMs under-attend to information in the middle of inputs. |
ASPECTNEWS: Aspect-Oriented Summarization of News Documents (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating generic summarizations can't be used to generalize to these domains without seeing in-domain training data. |
| Approach: | They use a dataset of real-world aspect-oriented summaries to annotate articles from two different news sub-domains. |
| Outcome: | The proposed approach produces better focused summaries than existing systems without seeing in-domain training data. |
Asking and Answering Questions to Evaluate the Factual Consistency of Summaries (2020.acl-main)
Copied to clipboard
| Challenge: | Existing automatic evaluation metrics for summarization are insensitive to factual inconsistencies. |
| Approach: | They propose an automatic evaluation protocol that detects factual inconsistencies in a model-generated summary. |
| Outcome: | QAGS has higher correlations with human judgments of factual consistency than other evaluation metrics. |
FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing automatic metrics do not capture errors in abstractive summarization models. |
| Approach: | They propose an automatic question answering metric for faithfulness that leverages recent advances in reading comprehension. |
| Outcome: | The proposed metric has significantly higher correlation with human faithfulness scores on highly abstracted summaries. |
Rationale-based Opinion Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to generate concise summaries of reviews are generic and lack supporting details. |
| Approach: | They propose a rationale-based opinion summarization paradigm that outputs representative opinions and corresponding rationales. |
| Outcome: | The proposed method is more useful than conventional summarizations. |
FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence (2024.acl-long)
Copied to clipboard
Sebastian Joseph, Lily Chen, Jan Trienes, Hannah Göke, Monika Coers, Wei Xu, Byron Wallace, Junyi Jessy Li
| Challenge: | FactPICO is a factuality benchmark for plain language summarization of medical texts describing randomized controlled trials . existing metrics for factual summarizing medical evidence are poorly correlated with expert judgments on the instance level. |
| Approach: | They propose a factuality benchmark for plain language summarization of medical texts . they assess factuality of critical elements of RCTs in those summaries . |
| Outcome: | The proposed benchmark assesses the factuality of medical summaries using LLMs . the summary summators are based on 345 plain language summaires with fine-grained evaluation . |
Enhancing Abstractiveness of Summarization Models through Calibrated Distillation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to generate abstractive summarizations are slow and abstractive, but we propose a novel approach to enhance the level of abstractiveness without sacrificing the informativeness of generated summaries. |
| Approach: | They propose a novel approach to enhance the level of abstractiveness without sacrificing the informativeness of generated summaries by exposing diverse pseudo summary with two supervision to the student model. |
| Outcome: | The proposed method outperforms previous methods in abstractive summarization distillation, producing highly abstractive and informative summaries. |
Corpus for Automatic Structuring of Legal Documents (2022.lrec-1)
Copied to clipboard
Prathamesh Kalamkar, Aman Tiwari, Astha Agarwal, Saurabh Karn, Smita Gupta, Vivek Raghavan, Ashutosh Modi
| Challenge: | In populous countries, pending legal cases are growing exponentially. |
| Approach: | They propose a corpus of legal judgment documents in English that is annotated with a label coming from a list of pre-defined rhetorical roles. |
| Outcome: | The proposed corpus of legal judgment documents is annotated with a label coming from a list of pre-defined rhetorical roles. |
MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checking (2024.acl-long)
Copied to clipboard
| Challenge: | Fact-checking claims on social media platforms poses a significant challenge due to the large volume of new claims constantly being posted without sufficient methods for verification. |
| Approach: | They propose a model that generates claim-specific summaries from multimodal multi-document datasets using a perceiver-based model that is able to handle inputs from multiple modalities of arbitrary lengths. |
| Outcome: | The proposed model outperforms the SOTA approach by 4.6% in the claim verification task on the MOCHEG dataset and shows strong performance on the new multi-document claims dataset. |
Focus Attention: Promoting Faithfulness and Diversity in Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Currently, document summarization is challenging even for humans. |
| Approach: | They propose a focus attention mechanism which encourages decoders to generate tokens that are topically similar to the input document. |
| Outcome: | The proposed method outperforms top-k and nucleus sampling methods on the BBC extreme summarization task and is more accurate than focus attention-based models. |
Length Control in Abstractive Summarization by Pretraining Information Selection (2022.acl-long)
Copied to clipboard
| Challenge: | Existing length-controllable summarization models generate summaries as long as training data . current methods only control lengths at decoding stage, but adapt to desired lengths . |
| Approach: | They propose a length-aware attention mechanism to adapt the encoding of the source based on the desired length. |
| Outcome: | The proposed method produces high-quality summaries with desired lengths and even those short lengths never seen in the training data. |
Lexical Repetitions Lead to Rote Learning: Unveiling the Impact of Lexical Overlap in Train and Test Reference Summaries (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Ideal summarization models should generalize to novel summary-worthy content without remembering reference training summaries by rote. |
| Approach: | They propose to partition test set based on lexical similarity of reference test summaries with training summary to determine model competencies. |
| Outcome: | The proposed evaluation protocol improves generalization and generalization on novel test cases while maintaining average performance. |
BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text summarization datasets include short-form source documents that lack long-range causal and temporal dependencies and contain strong layout and stylistic biases. |
| Approach: | They propose a dataset for long-form narrative summarization that uses human written summaries on three levels of difficulty. |
| Outcome: | The proposed dataset covers documents from the literature domain, such as novels, plays and stories, and includes highly abstractive, human written summaries on three levels of difficulty. |
Flight of the PEGASUS? Comparing Transformers on Few-shot and Zero-shot Multi-document Abstractive Summarization (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work has shown that pre-trained transformers obtain remarkable performance on many natural language processing tasks including automatic summarization. |
| Approach: | They propose to use transformers to generate multi-document summarization where the summary is explicitly conditioned on a user-given topic statement or question. |
| Outcome: | The proposed models perform well on four challenging summarization datasets from the general domain and one from consumer health. |
HOLMS: Alternative Summary Evaluation with Large Language Models (2020.coling-main)
Copied to clipboard
| Challenge: | Efficient document summarization requires evaluation measures that can rank a set of systems based on an average score and highlight which individual summary is better than another. |
| Approach: | They propose a hybrid evaluation measure for document summarization called HOLMS that combines both language models pre-trained on large corpora and lexical similarity measures. |
| Outcome: | The proposed measure outperforms ROUGE and BLEU on several extractive summarization datasets for both linguistic quality and pyramid scores. |
Abstractive Text Summarization Based on Deep Learning and Semantic Content Generalization (P19-1)
Copied to clipboard
| Challenge: | Abstractive text summarization is a demanding, time expensive and generally laborious task. |
| Approach: | They propose a framework for enhancing abstractive text summarization using deep learning techniques and semantic data transformations. |
| Outcome: | The proposed method is evaluated on two popular datasets with encouraging results. |
Background Summarization of Event Timelines (2023.emnlp-main)
Copied to clipboard
| Challenge: | Generating concise summaries of news events is a challenging task for newcomers to a news story. |
| Approach: | They propose a task of background news summarization that complements each timeline update with a background summary of relevant preceding events. |
| Outcome: | The proposed system performs well on a question-answering-based evaluation metric, Background Utility Score (BUS). |
Factual Error Correction for Abstractive Summarization Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are unable to ensure factual consistency of generated summaries. |
| Approach: | They propose a post-editing corrector module to identify and correct factual errors in generated summaries. |
| Outcome: | The proposed model outperforms existing models on CNN/DailyMail dataset on factual consistency evaluation. |
Can Large Language Model Summarizers Adapt to Diverse Scientific Communication Goals? (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work on the evaluation of large language models (LLMs) has shown unprecedented performance on diverse language generation tasks. |
| Approach: | They investigate the controllability of large language models on scientific summarization tasks by controlling stylistic and content coverage factors. |
| Outcome: | The proposed model outperforms humans on the MuP review generation task in terms of similarity to reference summaries and human preferences. |
Better Highlighting: Creating Sub-Sentence Summary Highlights (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarizations are considered to be less reliable because they distort the original meaning and can be confusing for readers. |
| Approach: | They propose a method to generate summary highlights that are understandable on their own to avoid confusion. |
| Outcome: | The proposed method allows summaries to be understood in context and avoids misdirecting readers to false conclusions. |
PyrEval: An Automated Method for Summary Content Analysis (L18-1)
Copied to clipboard
| Challenge: | PyrEval automates manual summarization evaluation of abstractive summarizing systems . extractive summaries that select complete sentences have shifted in recent years . |
| Approach: | They propose a method for automatic summarization evaluation that automates the manual pyramid method by using pre-trained vectors and a greedy algorithm to evaluate the pyramid content. |
| Outcome: | The proposed method can be applied to human and machine summaries with no retraining and in excellent time. |
StreamHover: Livestream Transcript Summarization and Annotation (2021.emnlp-main)
Copied to clipboard
Sangwoo Cho, Franck Dernoncourt, Tim Ganter, Trung Bui, Nedim Lipka, Walter Chang, Hailin Jin, Jonathan Brandt, Hassan Foroosh, Fei Liu
| Challenge: | StreamHover is a framework for annotating and summarizing livestream transcripts . the problem is that there is n't enough annotated datasets to summarize livestreams based on the informal nature of spoken language . |
| Approach: | They propose a framework for annotating and summarizing livestream transcripts using a text preview. |
| Outcome: | The proposed model generalizes better and improves over strong baselines. |
QuestEval: Summarization Asks for Fact-based Evaluation (2021.emnlp-main)
Copied to clipboard
Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang, Patrick Gallinari
| Challenge: | Existing evaluation metrics for summarization evaluation are limited and do not correlate well with human judgments. |
| Approach: | They propose to extend existing evaluation metrics to include question answering models to assess whether a summary contains all relevant information in its source document. |
| Outcome: | The proposed framework significantly improves the correlation with human judgments over four evaluation dimensions. |
Improving Factual Consistency of Abstractive Summarization via Question Answering (2021.acl-long)
Copied to clipboard
Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, Bing Xiang
| Challenge: | Recent studies show that about 30% of summaries generated by neural text summarization suffer from fact fabrication. |
| Approach: | They propose an automatic evaluation metric to measure factual consistency and a learning algorithm that maximizes the metric during model training. |
| Outcome: | The proposed method improves factual consistency and overall quality of summarization models. |
Data Selection Curriculum for Abstractive Text Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive Text Summarization (ATS) models are commonly trained using large-scale data that is randomly shuffled. |
| Approach: | They propose a data selection curriculum scoring system that measures the learning difficulty of an ATS model and expected performance on an instance. |
| Outcome: | The proposed system surpasses baselines on CNN/DailyMail dataset, utilizing 20% of available instances. |
Summarization of Opinionated Political Documents with Varied Perspectives (2025.coling-main)
Copied to clipboard
| Challenge: | Political ideologies can lead people to develop misperceptions of groups with opposing opinions, such as the 2024 US presidential election, French legislative election, or the Brexit referendum. |
| Approach: | They propose a dataset and task for independently summarizing political perspectives in a set of opinionated news articles. |
| Outcome: | The proposed dataset and task evaluates models of varying sizes and architectures on a set of opinionated news articles. |
HeterGraphLongSum: Heterogeneous Graph Neural Network with Passage Aggregation for Extractive Long Document Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models for extractive document summarization are based on sequence-to-sequence (Seq2Sequency) but long-form document summaries using graph-based methods are still an open research issue. |
| Approach: | They propose a heterogeneous graph neural network model to improve the performance of extractive document summarization using graph-based methods. |
| Outcome: | The proposed model can achieve state-of-the-art performance without pre-trained language models. |
Multi-hop Inference for Question-driven Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for summarizing source document for non-factoid questions are lacking in factoidic QA. |
| Approach: | They propose a question-driven abstractive summarization method that incorporates multi-hop reasoning into question-based summarizing. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two non-factoid QA datasets. |
Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluations (2023.acl-long)
Copied to clipboard
Lucy Lu Wang, Yulia Otmakhova, Jay DeYoung, Thinh Hung Truong, Bailey Kuehl, Erin Bransom, Byron Wallace
| Challenge: | Prior work has shown that models may exploit shortcuts that are difficult to detect using standard n-gram similarity metrics such as ROUGE. |
| Approach: | They propose to use human-assessed summary quality facets and pairwise preferences to improve MDS evaluation methods. |
| Outcome: | The proposed methods improve the quality of literature review summarization models . they use human-assessed summary quality facets and pairwise preferences . |
SumeCzech: Large Czech News-Based Summarization Dataset (L18-1)
Copied to clipboard
| Challenge: | Summarization of documents is a well-studied NLP task, but only a few datasets are available for Czech. |
| Approach: | They propose to use a Czech news-based summarization dataset to evaluate document summarizing . they propose a language-agnostic variant of the ROUGE metric to enable automatic evaluation . |
| Outcome: | The proposed dataset contains more than a million Czech news articles . the proposed approach is strong abstractive and language-agnostic . |
Composing Elementary Discourse Units in Abstractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Abstractive summarization uses a single document sentence to generate a summary, but this can cause performance degradation. |
| Approach: | They propose to use elementary discourse unit (EDU) as the summarization unit to extract and group informative EDUs and then an EDU fusion model to fuse the EDU in each group into one sentence. |
| Outcome: | The proposed model can be used to combine informative EDUs into one sentence and reward selection actions. |
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in efficient attention mechanisms have led to the expansion of the context length of large language models. |
| Approach: | They propose a procedure to synthesize Haystacks of documents and generate a summary that identifies relevant insights and precisely cites the source documents. |
| Outcome: | The proposed evaluation can score summaries on Coverage and Citation . the proposed evaluation lags human performance estimates by 10+ points on SummHay . |
LCFO: Long Context and Long Form Output Dataset and Benchmarking (2025.findings-acl)
Copied to clipboard
Marta R. Costa-jussà, Pierre Andrews, Mariano Coria Meglioli, Joy Chen, Joe Chuang, David Dale, Christophe Ropers, Alexandre Mourachko, Eduardo Sánchez, Holger Schwenk, Tuan A. Tran, Arina Turkatenko, Carleigh Wood
| Challenge: | Using long text outputs to evaluate progress in summarization and summary expansion tasks is challenging. |
| Approach: | They propose a framework for assessing gradual summarization and summary expansion capabilities across diverse domains. |
| Outcome: | The proposed framework provides alignments between specific QA pairs and corresponding summaries in 7 domains. |
STORYSUMM: Evaluating Faithfulness in Story Summarization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating abstractive summarization are lacking in faithfulness evaluation. |
| Approach: | They propose a dataset that measures faithfulness of LLM summaries with localized errors and faithfulness labels for evaluation methods. |
| Outcome: | The proposed method does not achieve more than 70% accuracy on this task. |
Citance-Contextualized Summarization of Scientific Papers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Current automatic summarization approaches generate abstracts, but abstracts do not show relationship between paper and references. |
| Approach: | They propose a contextualized summarization approach that generates an informative summary . they extract and model the citances of a paper, retrieve relevant passages from cited papers, and generate abstractive summaries tailored to each citance. |
| Outcome: | The proposed method extracts and models the citances of a paper, retrieves relevant passages from cited papers, and generates abstractive summaries tailored to each citance. |
Prompted Aspect Key Point Analysis for Quantitative Review Summarization (2024.acl-long)
Copied to clipboard
| Challenge: | Recent abstractive approaches generate KPs based on sentences, resulting in overlapping and hallucinated opinions. |
| Approach: | They propose to use supervised learning to extract short sentences as key points before matching them to review comments for quantification of KP prevalence. |
| Outcome: | The proposed framework achieves state-of-the-art performance on Yelp and SPACE. |
ACUEval: Fine-grained Hallucination Evaluation and Correction for Abstractive Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent-proposed evaluation metrics for large language models have a preference-bias . however, such metrics often lack interpretability and only offer a single score . |
| Approach: | They propose a metric that leverages the power of large language models to perform two sub-tasks: decomposing summaries into atomic content units and validating them against the source document. |
| Outcome: | The proposed metric improves faithfulness scores on three summarization evaluation benchmarks by 3% compared to the next-best metric. |
Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses (D19-1)
Copied to clipboard
| Challenge: | Sentence position is a strong feature for news summarization, since the lead often summarizes the key points of the article. |
| Approach: | They propose two techniques to make neural systems sensitive to the importance of content in different parts of the article by using random shuffled sentences to pretrain the model. |
| Outcome: | The proposed techniques improve the performance of a competitive reinforcement learning based extractive system, with the auxiliary loss being more powerful than pretraining. |
Inducing Document Structure for Aspect-based Summarization (P19-1)
Copied to clipboard
| Challenge: | Abstractive summarization systems treat documents as unstructured and generate a single generic summary per document. |
| Approach: | They propose to incorporate document structure into automatic summarization systems . they induce latent document structure and abstractive summarizing objective . |
| Outcome: | The proposed model improves on topic-agnostic baselines and can produce abstractive and extractive aspect-based summaries. |
A Bag of Tricks for Dialogue Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using a pretrained sequence-to-sequence language model, we explore speaker name substitution, negation scope highlighting, multi-task learning with relevant tasks, and pretraining on in-domain data. |
| Approach: | They propose a pretrained sequence-to-sequence language model that can handle different parts of dialogue belonging to multiple speakers and combine them to produce a coherent monologue summary. |
| Outcome: | The proposed techniques outperform baseline models on a dialogue summarization dataset. |
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often generate inconsistent summaries containing factual errors or fabricated content. |
| Approach: | They propose to generate representative examples of non-factual summaries through infilling language models and train a robust fact-correction model to post-edit them to improve factual consistency. |
| Outcome: | The proposed model outperforms previous methods in correcting factual errors on two popular summarization datasets. |
FaMeSumm: Investigating and Improving Faithfulness of Medical Summarization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing summarization models produce unfaithful outputs for medical text summarizing . a framework to improve faithfulness is proposed to improve medical text summary accuracy . |
| Approach: | They propose a framework to improve faithfulness by fine-tuning pre-trained language models based on medical knowledge. |
| Outcome: | The proposed framework improves faithfulness on medical summarization tasks. |
Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trends (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have improved summarization, but they still face a challenge of hallucination. |
| Approach: | They propose a taxonomy of errors to address the problem of hallucination in LLMs . they propose two prompt-based approaches for fine-grained error detection . |
| Outcome: | The proposed model outperforms existing metrics in identifying the novel "Contextual Inference" error type. |
Pre-trained Personalized Review Summarization with Effective Salience Estimation (2023.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) are a new paradigm in text generation for the strong ability of natural language comprehension. |
| Approach: | They propose a pre-trained personalized review summarization method that incorporates personalized information into the salience estimation of input reviews. |
| Outcome: | The proposed method performs better than the state-of-the-art methods on real-world datasets. |
OpineSum: Entailment-based self-training for abstractive opinion summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization is promising for fluently comparing opinions from a set of reviews about a place or product. |
| Approach: | They propose a novel method that automatically leverages common opinions across reviews to create powerful abstractive models. |
| Outcome: | The proposed method outperforms strong peer systems in both settings. |
COSMIC: Mutual Information for Task-Agnostic Summarization Evaluation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for summarizing text are not well aligned with human judgments. |
| Approach: | They propose a task-oriented evaluation approach that assesses the quality of summarizers based on their capacity to produce summaries while preserving task outcomes. |
| Outcome: | The proposed method is able to predict task performance in a variety of contexts and tasks. |
GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews (2024.acl-long)
Copied to clipboard
| Challenge: | Scientific peer review is essential for the quality of academic publications. |
| Approach: | They propose a method that summarises scholarly reviews using a Rational Speech Act framework and novel uniqueness scores. |
| Outcome: | The proposed method generates more discriminative summaries than baseline methods in terms of human evaluation while achieving comparable performance with these methods in term of automatic metrics. |
Entity-Based Evaluation of Political Bias in Automatic Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that NLP systems may encode social biases, but the *political* bias of summarization models remains relatively unknown. |
| Approach: | They use an entity replacement method to examine the portrayal of politicians in automatically generated summaries. |
| Outcome: | The proposed model can control for the content of the source document and can be used to predict the ideal quality of summarization models. |
Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages (2025.acl-long)
Copied to clipboard
Hyangsuk Min, Yuho Lee, Minjeong Ban, Jiaqi Deng, Nicole Hee-Yeon Kim, Taewon Yun, Hang Su, Jason Cai, Hwanjun Song
| Challenge: | Existing evaluation frameworks for text summarization lack domain-specific assessment criteria and are predominantly English-centric. |
| Approach: | They propose a multi-dimensional, multi-domain evaluation of summarization in English and Chinese that incorporates specialized assessment criteria for each domain and leverages a debate system to enhance annotation quality. |
| Outcome: | The proposed evaluation framework provides a multi-dimensional, multi-domain evaluation of summarization in English and Chinese. |
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics (2023.acl-long)
Copied to clipboard
Liang Ma, Shuyang Cao, Robert L Logan IV, Di Lu, Shihao Ran, Ke Zhang, Joel Tetreault, Alejandro Jaimes
| Challenge: | Existing benchmarks measure the correlation with human judgements of faithfulness on model-generated summaries, but they are insufficient for diagnosing whether metrics are consistent, effective on human-written texts, and sensitive to different error types. |
| Approach: | They propose to use unfaithful minimal pairs to measure the consistency of automatic faithfulness metrics by comparing human-written summary pairs with a dataset of 889 human-writing, minimally different summary pairs. |
| Outcome: | The proposed benchmarks show that the most discriminative metrics tend not to be the most consistent, and that the best performing metrics are sensitive to errors. |
Narrative License and Model Sycophancy in LLM Summaries of Scientific Work (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used to summarize academic work . however, they can exaggerate or mischaracterize findings . |
| Approach: | They examine how Narrative License (NL) emerges in large language models summaries . authors find that stated stances and user personas produce predictable shifts . |
| Outcome: | The proposed models can exaggerate or mischaracterize findings in scholarly articles . the authors show that the models' "sycophancy" can reduce NL . |
Improving Radiology Summarization with Radiograph and Anatomy Prompts (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on automatic impression generation, but this task is time-consuming and in high demand. |
| Approach: | They propose to use an anatomy-enhanced multimodal model to generate automatic impressions by combining radiology images with textual features. |
| Outcome: | The proposed model achieves state-of-the-art on two benchmark datasets and compares with existing models. |
Rehearse With User: Personalized Opinion Summarization via Role-Playing based on Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that large language models can achieve stateof-the-art performance on standard summarization benchmarks without the need for large-scale training data. |
| Approach: | They propose a personalized opinion summarization framework via LLM-based role-playing to better understand the user's personalized needs. |
| Outcome: | The proposed framework can improve the level of personalization in large model-generated summaries by taking into account user characteristics and interests while summarizing multiple product reviews. |
Improving Consistency for Text Summarization with Energy Functions (2023.findings-emnlp)
Copied to clipboard
Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang
| Challenge: | Current abstractive summarization models generate inconsistent content due to the inherently noisy dataset and the discrepancy between maximum likelihood estimation based training objectives and consistency measurements. |
| Approach: | They propose a new consistency taxonomy that categorizes inconsistent content into faithfulness, factuality, and self-supportiveness. |
| Outcome: | Experiments on XSUM and CNN/DM datasets show that EnergySum mitigates the trade-off between accuracy and consistency. |
Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions for word probability distributions are limited and the output softmax layer is inherently limited. |
| Approach: | They propose to use the output softmax layer to compute the word probability distribution instead of using pointer networks to break the bottleneck. |
| Outcome: | The proposed method improves factCC score by 2 points in CNN/DM and XSUM dataset, and MAUVE scores by 30% in bookSum paragraph-level dataset. |
A Summarization Dataset of Slovak News Articles (2020.lrec-1)
Copied to clipboard
| Challenge: | a number of studies on document summarization have focused on the English language . however, most of the work on this task is done on English datasets . |
| Approach: | They propose to use a news site's ROUGE metric to adapt it to Slovak texts . they propose to introduce a large-scale news-based summarization dataset . |
| Outcome: | The proposed approach is better suited for Slovak texts than the dominant ROUGE metric. |
Towards Unifying Multi-Lingual and Cross-Lingual Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing work on multilingual summarization and cross-lingual summmarization has been limited due to their different definitions. |
| Approach: | They propose to unify MLS and CLS into a more general setting, i.e. many-to-many summarization. |
| Outcome: | The proposed model outperforms the state-of-the-art models in the zero-shot directions. |
Not Lost After All: How Cross-Encoder Attribution Challenges Position Bias Assumptions in LLM Summarization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Position bias is a key limitation in automatic summarization. |
| Approach: | They propose a cross-encoder-based alignment method that processes summary-source sentence pairs . |
| Outcome: | The proposed method allows better identification of semantic correspondences even when summaries substantially rewrite the source. |
On The Persona-based Summarization of Domain-Specific Documents (2024.findings-acl)
Copied to clipboard
Ankan Mullick, Sombit Bose, Rounak Saha, Ayan Bhowmick, Pawan Goyal, Niloy Ganguly, Prasenjit Dey, Ravi Kokku
| Challenge: | In an ever-expanding world of domain-specific knowledge, summarization of information is a complex task . persona-based summarizing of domain specific information by humans is deemed not preferred . |
| Approach: | They propose a framework for efficient training of a small foundation LLM on a healthcare corpus. |
| Outcome: | The proposed framework fine-tunes a domain-specific small foundation LLM using a healthcare corpus and evaluates its quality using AI-based critiquing. |
Leveraging Entailment Judgements in Cross-Lingual Summarisation (2024.findings-acl)
Copied to clipboard
| Challenge: | Synthetically created cross-lingual summarisation datasets are prone to include document-summary pairs where the reference summary is unfaithful to the corresponding document. |
| Approach: | They propose to use off-the-shelf cross-lingual Natural Language Inference to evaluate faithfulness of reference and model generated summaries and use unlikelihood loss to teach a model about unfaithful summary sequences. |
| Outcome: | The proposed approach evaluates faithfulness of reference and model generated summaries and uses unlikelihood loss to teach a model about unfaithful summary sequences. |
Synthesize, if you do not have: Effective Synthetic Dataset Creation Strategies for Self-Supervised Opinion Summarization in E-commerce (2023.findings-emnlp)
Copied to clipboard
Tejpalsingh Siledar, Suman Banerjee, Amey Patil, Sudhanshu Singh, Muthusamy Chelliah, Nikesh Garera, Pushpak Bhattacharyya
| Challenge: | Existing approaches to generate general and aspect-specific opinion summarization are limited due to their reliance on human-specified aspects and seed words. |
| Approach: | They propose synthetic dataset creation approaches for general and aspect-specific opinion summarization . general opinion summaries struggle to generate faithful to the input reviews, they say . aspect- specific opinion summarisation models are limited due to reliance on human-specified aspects . |
| Outcome: | The proposed approach outperforms existing models on three e-commerce test sets on general and aspect-specific opinion summarization. |
MeetingBank: A Benchmark Dataset for Meeting Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | a lack of annotated meeting corpora hinders the development of meeting summarization technology. |
| Approach: | They present a new benchmark dataset of city council meetings over the past decade . they use a divide-and-conquer approach to divide professionally written minutes into shorter passages . |
| Outcome: | The proposed dataset provides a testbed for various meeting summarization systems and allows the public to gain insight into how council decisions are made. |
LexAbSumm: Aspect-based Summarization of Legal Decisions (2024.lrec-main)
Copied to clipboard
| Challenge: | LexAbSumm is a dataset designed for aspect-based summarization of legal documents . it is based on a set of ECtHR fact sheets, and is available for download. |
| Approach: | They propose a dataset designed for aspect-based summarization of legal case decisions . they evaluate abstractive summarizing models tailored for longer documents . |
| Outcome: | The proposed dataset is designed for aspect-based summarization of legal cases . it reveals a challenge in conditioning models to produce aspect-specific summaries . |
Re-Examining Summarization Evaluation across Multiple Quality Criteria (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a number of automated evaluation metrics are evaluated by multiple quality criteria, such as relevance, consistency, fluency and coherence. |
| Approach: | They propose a method that removes the confounding variable and detects unreliable correlations. |
| Outcome: | The proposed method detects unreliable correlations between QCs and human scores . it is based on a multi-QC setup, but it fails to detect summary corruptions . |
Longform Multimodal Lay Summarization of Scientific Papers: Towards Automatically Generating Science Blogs from Research Articles (2024.lrec-main)
Copied to clipboard
| Challenge: | Science blogs and lay-speak are critical to communicating scientific information to the general public and policymakers. |
| Approach: | They propose to use presentation transcripts and slides to generate a scientific blog from a research article in layperson's terms. |
| Outcome: | The proposed approach can generate a blog text and select the most relevant figures to explain a research article in layperson’s terms, essentially a science blog. |
No perspective, no perception!! Perspective-aware Healthcare Answer Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Healthcare Community Question Answering forums are prone to off-topic discussions and diverse answers can be challenging for readers to sift through. |
| Approach: | They propose a task of perspective-specific answer summarization to identify different perspectives within healthcare-related responses and frame a perspective-driven abstractive summary covering all responses. |
| Outcome: | The proposed model outperforms existing models against five baselines and shows that it is more accurate than existing models. |
From Outcomes to Processes: Guiding PRM Learning from ORM for Inference-Time Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to align large language models with human preferences suffer from inconsistent scoring and suboptimal alignment. |
| Approach: | They propose a dual-consistency framework that aligns partial sequences with human preferences. |
| Outcome: | The proposed framework significantly reduces granularity discrepancies and improves GPT-4 evaluation scores. |
Beyond the Context Window: Scaling Agentic RL via End-to-end Optimized Context Compression (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reinforcement learning pipelines suffer from degraded instruction following, excessive rollout costs, and strict context limits. |
| Approach: | They propose a reinforcement learning (RL) fine-tuning of large language model (LLM) agents for long-horizon multi-turn tool use where context length quickly becomes a bottleneck. |
| Outcome: | The proposed framework improves the success rate while maintaining the same or even lower working context length compared to baselines. |
Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Query-focused Summarization (QfS) is a system that generates summaries from document(s) based on a query. |
| Approach: | They propose a Query-focused Summarization approach that uses a generalization of Reinforcement Learning (RL) for Natural Language Generation and a better semantic similarity reward. |
| Outcome: | The proposed approach improves on the ROUGE-L metric and in a benchmark dataset. |
LMGQS: A Large-scale Dataset for Query-focused Summarization (2023.findings-emnlp)
Copied to clipboard
Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Pengcheng He, Chenguang Zhu, Michael Zeng
| Challenge: | Lack of large-scale datasets for query-focused summarization hinders model development . lack of data limits the ability of QFS models to train robust neural models . |
| Approach: | They propose to generate a query for each summary sentence in a generic summarization annotation using a pretrained language model. |
| Outcome: | The proposed model achieves state-of-the-art zero-shot and supervised performance on multiple existing QFS benchmarks. |
PerDucer: Keyphrase-Driven Personalization Inducer for Summarization from User Histories (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work reported that prepending long interaction histories to LLMs leads to unstable personalization, especially for multi-aspect documents. |
| Approach: | They propose a personalization inducer for frozen language models that maps latent preference signals to a small set of personalized keyphrases for the query document. |
| Outcome: | The proposed model outperforms the strongest history-prompting LLMs and SLMs in the PENS and OpenAI-Reddit benchmarks. |
Re-FRAME the Meeting Summarization SCOPE: Fact-Based Summarization and Personalization via Questions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | FRAME reframes summarization as a semantic enrichment task . SCOPE is a reason-out-loud protocol that has the model build a reasoning trace . |
| Approach: | They propose a modular pipeline that reframes summarization as a semantic enrichment task. |
| Outcome: | The proposed pipeline reduces hallucinations and omissions by 2 out of 5 points . SCOPE improves knowledge fit and goal alignment over prompt-only baselines . |
PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing summarization strategies are abstractive and extractive, but are hard to control. |
| Approach: | They propose a PhRase-level cOpying Mechanism that enhances attention on n-grams and calculates an auxiliary loss for the copying prediction. |
| Outcome: | Empirical studies show that PROM improves copying accuracy and faithfulness on benchmarks. |
Prophecy Distillation for Boosting Abstractive Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive summarization models with maximum likelihood estimation generate unfaithful facts alongside ambiguous focus. |
| Approach: | They propose a framework which learns a regular summarization model to mimic the behavior of being guided by prophecy for boosting abstractive summaries. |
| Outcome: | The proposed model achieves new or matched state-of-the-art on four well-known datasets. |
Evaluation of LLMs in Medical Text Summarization: The Role of Vocabulary Adaptation in High OOV Settings (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been successful in medical text summarization . however, they do not perform fine-grained evaluations under difficult settings . |
| Approach: | They show that large language models show a significant performance drop for data points with high concentration of out-of-vocabulary words or with high novelty. |
| Outcome: | The proposed model shows a significant performance drop for data points with high concentration of out-of-vocabulary words or with high novelty. |
Behavioral Analysis of Information Salience in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models excel at text summarization, but the exact notion of salience remains unclear. |
| Approach: | They propose a framework to derive and investigate information salience in Large Language Models (LLMs) using length-controlled summarization as a behavioral probe into the content selection process. |
| Outcome: | The proposed framework derives a proxy for how models prioritize information in large language models. |
Agent-as-Judge for Factual Summarization of Long Narratives (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated near-human performance in summarization tasks based on traditional metrics such as ROUGE and BERTScore . however, these metrics do not adequately capture critical aspects of summarizing quality, such as factual accuracy, especially for long narratives. |
| Approach: | They propose a framework that evaluates and refines factuality in narrative summarization by leveraging a Character Knowledge Graph extracted from input narrative. |
| Outcome: | The proposed framework evaluates factuality and provides actionable guidance for refinement. |
ReflectSumm: A Benchmark for Course Reflection Summarization (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing research has focused on standard summarization benchmarks within domains like news, scientific articles, and opinions. |
| Approach: | They propose a summarization dataset specifically designed for summarizing students’ reflective writing. |
| Outcome: | The proposed summarization dataset can be used in opinion summarizing scenarios and in educational domains. |
Decomposed Opinion Summarization with Verified Aspect-Aware Modules (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for summarizing opinions from large-scale online reviews are not available for crowdsourcing and are difficult to crowdsource. |
| Approach: | They propose a domain-agnostic modular approach guided by review aspects to separate tasks of aspect identification, opinion consolidation, and meta-review synthesis to enable greater transparency and ease of inspection. |
| Outcome: | The proposed approach generates more grounded summaries than baseline models, as verified through automated and human evaluations. |
Real-World Summarization: When Evaluation Reaches Its Limits (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study examines the evaluation of hotel highlights in the context of hotel data. |
| Approach: | They examine evaluation of faithfulness to input data in the context of hotel highlights . they compare traditional metrics, trainable methods, and LLM-as-a-judge approaches . |
| Outcome: | The results show that simple metrics outperform human judgments on LLM-generated summaries . the results also highlight challenges in crowdsourced evaluations. |
Aspect-Oriented Summarization for Psychiatric Short-Term Readmission Prediction (2025.emnlp-main)
Copied to clipboard
WonJin Yoon, Boyu Ren, Spencer Thomas, Chanhwi Kim, Guergana K Savova, Mei-Hua Hall, Timothy A. Miller
| Challenge: | Recent advances in large language models have enabled the automated processing of lengthy documents even without supervised training on a task-specific dataset. |
| Approach: | They propose a method for processing the summaries of long documents using different aspect-oriented prompts and integrate the information signals from these different prompts for supervised training of transformer models. |
| Outcome: | The proposed method improves on a high-impact task predicting readmissions from a psychiatric discharge using real-world data from four hospitals. |
ThreadSumm: Summarization of Nested Discourse Threads Using Tree of Thoughts (2026.acl-long)
Copied to clipboard
| Challenge: | Discussion forums have nested structures that are difficult to navigate and can attract off-topic replies that get interleaved with the author's own continuation posts. |
| Approach: | They propose a multi-stage LLM framework that treats thread summarization as a hierarchical reasoning problem over explicit aspect and content unit representations. |
| Outcome: | The proposed framework improves the quality of nested discussion thread summarizations while retaining aspect retention and opinion coverage. |
Learning to Control Summaries with Score Ranking (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in summarization focus on improving summary quality across multiple dimensions, but they overlook the challenge of controlling summary generation with respect to individual dimensions. |
| Approach: | They propose a loss function that aligns model outputs with fine-grained, model-based evaluation scores to enable both improvement in summary quality and dimension-specific control. |
| Outcome: | The proposed method improves the overall quality of summaries while maintaining strong control over individual quality dimensions. |
TRAC: Token-level Reward Assignment for Coherent Abstractive Summarization (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have achieved remarkable success in text summarization, but maintaining logical coherence and contextual consistency remains a pervasive challenge in long-form generation. |
| Approach: | They propose a framework that introduces a token-level reward function by integrating relative sentence gain, inter-sentence attention, and a Gaussian length penalty. |
| Outcome: | The proposed model outperforms the sequence-level baseline by 11.05% in fluency and 10.61% in Relevance. |