ALIGNMEET: A Comprehensive Tool for Meeting Annotation, Alignment, and Evaluation (2022.lrec-1)
Copied to clipboard
| Challenge: | Summarization is a challenging problem, and it is difficult to create, correct, and evaluate the summaries manually. |
| Approach: | They propose an open-source tool for meeting annotation, alignment, and evaluation . the tool aims to provide an efficient and clear interface for fast annotation . |
| Outcome: | The proposed tool is open-source and installable from PyPI. |
Similar Papers
Align then Summarize: Automatic Alignment Methods for Summarization Corpus Creation (2020.lrec-1)
Copied to clipboard
| Challenge: | Summarizing text is not a straightforward task. |
| Approach: | They propose to use automated transcriptions to generate reports from automatic transcriptions as a dataset for neural summarization. |
| Outcome: | The proposed model improves on publicmeetings corpus on a dataset of aligned public meetings. |
SUMMARY WORKBENCH: Unifying Application and Evaluation of Text Summarization Models (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Summary Workbench is a tool for developing and evaluating text summarization models. |
| Approach: | They propose a tool for developing and evaluating text summarization models that integrates with Docker plugins and provides visual analysis of models’ strengths and weaknesses. |
| Outcome: | The proposed model and evaluation measures can be easily integrated as Docker-based plugins and provide insights into the models’ strengths and weaknesses. |
MeetingBank: A Benchmark Dataset for Meeting Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | a lack of annotated meeting corpora hinders the development of meeting summarization technology. |
| Approach: | They present a new benchmark dataset of city council meetings over the past decade . they use a divide-and-conquer approach to divide professionally written minutes into shorter passages . |
| Outcome: | The proposed dataset provides a testbed for various meeting summarization systems and allows the public to gain insight into how council decisions are made. |
ExplainMeetSum: A Dataset for Explainable Meeting Summarization Aligned with Human Intent (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for meeting summarization use extract-thengenerate method to select "salient" contents . extract-thangenerates method typically selects "selected" content in a distantly supervised manner . |
| Approach: | They propose a novel extractor-guided method to generate a summary from evidence sentences that "explain" a meeting summary. |
| Outcome: | The proposed method outperforms existing methods with gains of up to 3.13 in the ROUGE-1 score. |
A Modular Tool for Automatic Summarization (P19-3)
Copied to clipboard
| Challenge: | Abstractive automatic summarization methods are supervized, but they require large corpora to perform tasks. |
| Approach: | They propose to use a modular tool for automatic summarization that is as simple as possible for end-users. |
| Outcome: | The proposed tool is open source and written in Java . it could be used as a baseline for future work and evaluate methods on different corpora. |
SummEval: Re-evaluating Summarization Evaluation (2021.tacl-1)
Copied to clipboard
Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, Dragomir Radev
| Challenge: | a lack of comprehensive studies on evaluation metrics for text summarization hinders progress . a new study aims to improve evaluation metrics that correlate with human judgments . |
| Approach: | They propose to re-evaluate automatic evaluation metrics and share a toolkit for evaluation . they hope to promote a more complete evaluation protocol for text summarization . |
| Outcome: | The proposed evaluation metrics are inconsistent with existing evaluation protocols. |
jp-evalb: Robust Alignment-based PARSEVAL Measures (2024.naacl-demo)
Copied to clipboard
| Challenge: | evalb is used for constituency parsing evaluation, but imposes constraints and requires consistent tokenization and sentence boundary outcomes. |
| Approach: | They propose an evaluation system designed to compute PARSEVAL measures, offering a viable alternative to evalb commonly used for constituency parsing evaluation. |
| Outcome: | The proposed evaluation system is based on an alignment method that aligns sentences and words when discrepancies arise. |
Abstractive Meeting Summarization: A Survey (2023.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have improved language generation systems, opening the door to improved forms of abstractive summarization. |
| Approach: | They propose to use neural encoder-decoder architectures to generate abstractive meeting summarizations that are particularly well-suited for multi-party conversation. |
| Outcome: | The proposed system could be used in a wide variety of real-world contexts, from business meetings to medical consultations to customer service calls. |
The Power of Summary-Source Alignments (2024.findings-acl)
Copied to clipboard
Ori Ernst, Ori Shapira, Aviv Slobodkin, Sharon Adar, Mohit Bansal, Jacob Goldberger, Ran Levy, Ido Dagan
| Challenge: | Multi-document summarization (MDS) is a challenging task, often decomposed to subtasks of salience and redundancy detection, followed by text generation. |
| Approach: | They propose to extend the summary-source alignment framework by applying it at the more fine-grained proposition span level and annotating alignment manually in a multi-document setup. |
| Outcome: | The proposed framework can yield several datasets for at least six different tasks. |
CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning (2022.naacl-main)
Copied to clipboard
Xiangru Tang, Arjun Nair, Borui Wang, Bingyao Wang, Jai Desai, Aaron Wade, Haoran Li, Asli Celikyilmaz, Yashar Mehdad, Dragomir Radev
| Challenge: | Factual inconsistencies in generated summaries severely limit the practical applications of abstractive dialogue summarization. |
| Approach: | They propose a typology of factual errors to better understand hallucinations generated by current models and a contrastive fine-tuning strategy to improve the factual consistency and overall quality of summaries. |
| Outcome: | The proposed model significantly reduces all kinds of factual errors on both SAMSum dialogue summarization and AMI meeting summarizing datasets. |