Papers by Yael Amsterdamer
Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation (N19-1)
Copied to clipboard
Ori Shapira, David Gabay, Yang Gao, Hadar Ronen, Ramakanth Pasunuru, Mohit Bansal, Yael Amsterdamer, Ido Dagan
| Challenge: | Manual evaluation methods are perceived as insufficient due to the high cost of the Pyramid method and the required expertise. |
| Approach: | They propose a crowdsourced method that compares system summaries to references and uses crowdsourced scripts to analyze the results. |
| Outcome: | The proposed method shows higher correlation relative to the original Pyramid method. |
Extending Multi-Document Summarization Evaluation to the Interactive Setting (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to interactive summarization are incomparable and divergent . a key gap in the development and adoption of interactive summaries is the lack of evaluation methodologies and benchmarks for meaningful comparison of systems. |
| Approach: | They propose an end-to-end evaluation framework for interactive summarization based on expansion-based interaction . framework includes procedure of collecting real user sessions, evaluation measures relying on summarizing standards, but adapted to reflect interaction. |
| Outcome: | The proposed evaluation framework is based on evaluations of baseline implementations and is available publicly as a benchmark. |
Evaluating Multiple System Summary Lengths: A Case Study (D18-1)
Copied to clipboard
| Challenge: | Practical summarization systems are expected to produce summaries of varying lengths, per user needs. |
| Approach: | They propose to use ROUGE metric to evaluate system summaries of multiple lengths. |
| Outcome: | The evaluation protocol in question is competitive, the authors show . they found that the evaluation protocol is competitive with existing benchmarks. |
Interactive Query-Assisted Summarization via Deep Reinforcement Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing systems that can perform interactive summarization cannot ingest the full document set or operate at sufficient speed for interactivity. |
| Approach: | They propose two deep reinforcement learning models for interactive summarization task . they use interactive session state and history to refrain from redundancy . |
| Outcome: | The proposed model improves informativeness while preserving positive user experience. |