Papers by Mobashir Sadat
DelucionQA: Detecting Hallucinations in Domain-specific Question Answering (2023.findings-emnlp)
Copied to clipboard
Mobashir Sadat, Zhengyu Zhou, Lukas Lange, Jun Araki, Arsalan Gundroo, Bingqing Wang, Rakesh Menon, Md Parvez, Zhe Feng
| Challenge: | Hallucination is a well-known phenomenon in text generated by large language models . state-of-the-art LLMs still have a number of weaknesses, including the tendency to generate hallucinatory statements without considering the factuality . |
| Approach: | They propose a dataset that captures hallucinations made by retrieval-augmented LLMs . they propose to use these methods to help detect hallucinosity in QA tasks . |
| Outcome: | The proposed method captures hallucinations made by retrieval-augmented LLMs for QA tasks. |
Hierarchical Multi-Label Classification of Scientific Documents (2022.emnlp-main)
Copied to clipboard
| Challenge: | Automated topic classification is a useful tool for managing scientific documents in a digital collection. |
| Approach: | They propose a hierarchical multi-label text classification dataset with keyword labeling as an auxiliary task. |
| Outcome: | The proposed model achieves a Macro-F1 score of 34.57% and is publicly available. |
SciNLI: A Corpus for Natural Language Inference on Scientific Text (2022.acl-long)
Copied to clipboard
| Challenge: | Existing Natural Language Inference (NLI) datasets are not related to scientific text. |
| Approach: | They propose a large dataset for NLI that captures the formality in scientific text and contains 107,412 sentence pairs extracted from scholarly papers on NLP and computational linguistics. |
| Outcome: | The proposed model achieves a Macro F1 score of only 78.18% and an accuracy of 78.23%. |
A MISMATCHED Benchmark for Scientific Natural Language Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for scientific NLI are derived from various computer science domains, whereas non-CS domains are completely ignored. |
| Approach: | They propose a scientific natural language inference benchmark called MisMatched that incorporates sentence pairs having an implicit scientific NLI relation into model training. |
| Outcome: | The proposed benchmark covers three non-CS domains and contains 2,700 human annotated sentence pairs. |
Co-training for Low Resource Scientific Natural Language Inference (2024.acl-long)
Copied to clipboard
| Challenge: | Scientific Natural Language Inference (NLI) is a task to predict the semantic relation between sentences extracted from research articles. |
| Approach: | They propose a co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. |
| Outcome: | The proposed method achieves an improvement of 1.5% in Macro F1 over the distant supervision baseline and substantial improvements over several other strong SSL baselines. |
MSciNLI: A Diverse Benchmark for Scientific Natural Language Inference (2024.naacl-long)
Copied to clipboard
| Challenge: | a dataset containing 132,320 sentence pairs from five new scientific domains is used for scientific Natural Language Inference (NLI) the availability of multiple domains makes it possible to study domain shift for scientific NLI. |
| Approach: | They propose a dataset with 132,320 sentence pairs from five new scientific domains to introduce diversity in scientific NLI. |
| Outcome: | The proposed dataset contains 132,320 sentence pairs extracted from five new scientific domains. |
Learning to Infer from Unlabeled Data: A Semi-supervised Learning Approach for Robust Natural Language Inference (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a popular technique for reducing the reliance on human annotations for NLI tasks. |
| Approach: | They propose a way to incorporate unlabeled data into semi-supervised learning (SSL) using a conditional language model, they propose to generate hypotheses for unlabed sentences . |
| Outcome: | The proposed framework significantly improves the performance of four NLI datasets in low-resource settings. |