| Challenge: | Existing research has focused on standard summarization benchmarks within domains like news, scientific articles, and opinions. |
| Approach: | They propose a summarization dataset specifically designed for summarizing students’ reflective writing. |
| Outcome: | The proposed summarization dataset can be used in opinion summarizing scenarios and in educational domains. |
Similar Papers
From Information to Insight: Leveraging LLMs for Open Aspect-Based Educational Summarization (2025.acl-long)
Copied to clipboard
| Challenge: | a novel dataset summarizes student reflections on STEM lectures . ReflectASP eases the exploration of open-aspect-based summarization (OABS) despite the limitations of current datasets, it is still under-explored. |
| Approach: | They propose a dataset that summarizes student reflections on STEM lectures . they propose two refinement methods to improve summaries . |
| Outcome: | The proposed dataset summarizes student reflections on STEM lectures using automatic and human evaluations. |
Bringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents (2021.acl-short)
Copied to clipboard
| Challenge: | Faceted summarization provides briefings of a document from different perspectives. |
| Approach: | They propose a faceted summarization benchmark built on Emerald journal articles . they propose faceted models that bring structure into faceted documents . |
| Outcome: | The proposed benchmark is based on Emerald journal articles and covers a diverse range of domains. |
WikiSum: Coherent Summarization Dataset for Efficient Human-Evaluation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing summarization datasets are limited in their ability to evaluate output . a human evaluation is necessary to understand and improve summarizing systems . |
| Approach: | They propose a dataset based on how-to articles and coherent paragraph summaries written in plain language. |
| Outcome: | The proposed dataset makes human evaluation easier and more effective . the authors compare the proposed dataset to existing ones on PubMed and the literature. |
SummEval: Re-evaluating Summarization Evaluation (2021.tacl-1)
Copied to clipboard
Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | a lack of comprehensive studies on evaluation metrics for text summarization hinders progress . a new study aims to improve evaluation metrics that correlate with human judgments . |
| Approach: | They propose to re-evaluate automatic evaluation metrics and share a toolkit for evaluation . they hope to promote a more complete evaluation protocol for text summarization . |
| Outcome: | The proposed evaluation metrics are inconsistent with existing evaluation protocols. |
SETSum: Summarization and Visualization of Student Evaluations of Teaching (2022.naacl-demo)
Copied to clipboard
| Challenge: | Student Evaluations of Teaching (SETs) are used in colleges and universities to assess student perceptions about their courses. |
| Approach: | They propose a system that leverages sentiment analysis, aspect extraction, summarization and visualization techniques to provide organized illustrations of SET findings to instructors and other reviewers. |
| Outcome: | The proposed system can be used by 10 professors from diverse departments to analyze SET results. |
A Critical Look at Meta-evaluating Summarisation Evaluation Metrics (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Effective summarisation evaluation metrics enable researchers and practitioners to compare different summarization systems efficiently. |
| Approach: | They argue that evaluation metrics are primarily meta-evaluated on news summarisation datasets and that there has been a noticeable shift in research focus towards evaluating the faithfulness of generated summaries. |
| Outcome: | The evaluation metrics are primarily meta-evaluated on news summarisation datasets and there has been a noticeable shift in research focus towards evaluating the faithfulness of generated summaries. |
Fair Abstractive Summarization of Diverse Perspectives (2024.naacl-long)
Copied to clipboard
Yusen Zhang, Nan Zhang, Yixin Liu, Alexander Fabbri, Junru Liu, Ryo Kamoi, Xiaoxin Lu, Caiming Xiong, Jieyu Zhao, Dragomir Radev, Kathleen McKeown, Rui Zhang
| Challenge: | Existing work on summarization metrics and large language models has not explored fair abstractive summarizing. |
| Approach: | They propose four reference-free automatic metrics to measure the differences between target and source perspectives. |
| Outcome: | The proposed methods alleviate fair abstractive summarization on user-generated data. |
How to Find Strong Summary Coherence Measures? A Toolbox and a Comparative Study for Summary Coherence Measure Evaluation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to evaluate summary coherence are often evaluated using disparate datasets and metrics. |
| Approach: | They propose to use automatic evaluation to evaluate coherence of summaries by selecting high-scoring candidates. |
| Outcome: | The proposed methods show that they can perform better on an even playing field. |
AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Community Question Answering (CQA) fora lack a dataset to produce answer summarizations . a novel dataset of 4,631 CQA threads is used to generate answer summaries . |
| Approach: | They propose a dataset of 4,631 CQA threads for answer summarization curated by professional linguists. |
| Outcome: | The proposed approach boosts summarization performance according to automatic evaluation. |
The State and Fate of Summarization Datasets: A Survey (2025.naacl-long)
Copied to clipboard
| Challenge: | Summarization is the task of shortening a text while preserving the most important information it contains. |
| Approach: | They propose a novel ontology covering sample properties, collection methods and distribution covering sample characteristics, collection method and distribution. |
| Outcome: | The proposed ontology covers sample properties, collection methods and distribution, and can be used to streamline future research into a more coherent body of work. |