Papers by Mousumi Akter
Revisiting Automatic Evaluation of Extractive Summarization Task: Can We Do Better than ROUGE? (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to evaluate text summarization tasks using ROUGE have been criticized for lack of semantic understanding. |
| Approach: | They propose a semantic-aware metric for extractive summarization task that is semantic-based . they use CNN/DailyMail dataset to study the new metric . |
| Outcome: | The proposed metric is semantic-aware and shows higher correlation with human judgement and yields a large number of disagreements with the original ROUGE metric. |
SEM-F1: an Automatic Way for Semantic Evaluation of Multi-Narrative Overlap Summaries at Scale (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has introduced an important yet relatively under-explored NLP task called Semantic Overlap Summarization (SOS) that entails generating a summary from multiple alternative narratives which conveys the common information provided by those narratives. |
| Approach: | They propose to use a sentence-level precision-recall style automated evaluation metric to evaluate a new NLP task called Semantic Overlap Summarization (SOS) they propose to employ the popular ROUGE metric and use it to compare the two tasks. |
| Outcome: | The proposed metric yields higher correlation with human judgment and higher inter-rater agreement compared to the existing metric. |
Learning to Generate Overlap Summaries through Noisy Synthetic Data (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing training data for seq-to-seq models is limited due to the lack of available training data. |
| Approach: | They propose a data augmentation technique which allows to create large amount of synthetic data for training a seq-to-seq model. |
| Outcome: | The proposed technique performs better than pre-trained models on news domains and is close to the existing methods on golden training data. |
Benchmarking LLMs on Semantic Overlap Summarization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are the most capable text generation models in a variety of tasks and fields. |
| Approach: | They benchmark Large Language Models (LLMs) on SOS and introduce PrivacyPolicyPairs (3P) a dataset of 135 high-quality privacy policy documents is used to evaluate the model. |
| Outcome: | The proposed dataset complements existing resources and broadens domain coverage. |
Semantic Overlap Summarization among Multiple Alternative Narratives: An Exploratory Study (2022.coling-1)
Copied to clipboard
| Challenge: | Existing tasks for summarizing multiple alternate narratives with different perspectives are under-explored. |
| Approach: | They propose a task which entails generating a single summary from multiple alternative narratives . they use a web-based dataset and human annotations to evaluate the task . |
| Outcome: | The proposed task is based on a novel dataset and human annotations. |
LLMs as Meta-Reviewers’ Assistants: A Case Study (2025.naacl-long)
Copied to clipboard
Eftekhar Hossain, Sanjeev Kumar Sinha, Naman Bansal, R. Alexander Knipper, Souvika Sarkar, John Salvador, Yash Mahajan, Sri Ram Pavan Kumar Guttikonda, Mousumi Akter, Md. Mahadi Hassan, Matthew Freestone, Matthew C. Williams Jr., Dongji Feng, Santu Karmaker
| Challenge: | Meta-reviews are a critical step in the overall scientific peer-reviewed process, which focuses on understanding the consensus of expert opinions on a scholarly work and making informed judgments on its scientific merit. |
| Approach: | They propose to use large language models to generate a controlled multi-perspective-summary (MPS) of their opinions to help meta-reviewers better comprehend multiple experts' perspectives. |
| Outcome: | The proposed model can help meta-reviewers better comprehend multiple experts’ perspectives by generating a controlled multi-perspective-summary (MPS) of their opinions. |
On Evaluation of Bangla Word Analogies (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing word embeddings in Bangla struggle to perform well on low-resource data sets. |
| Approach: | They propose to use a benchmark dataset of Bangla word analogies to evaluate the quality of existing Bangla embeddings. |
| Outcome: | The proposed evaluation set includes 16,678 unique word analogies in Bangla and a translated and curated version of the original Mikolov dataset (10,594 samples) . |