Papers by Pouya Pezeshkpour
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study investigated hallucinations in multi-document summarization tasks . but, it is unclear how challenges arising from handling multiple documents affect outputs . |
| Approach: | They investigate how hallucinations manifest in large language models when summarizing topic-specific information from a set of documents. |
| Outcome: | The proposed benchmarks show that the models generate more hallucinations than baselines . the results highlight the need for more effective approaches to mitigate hallucinosity in MDS . |
LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive capabilities in mathematical reasoning, but their effectiveness is limited to specific mathematical topics. |
| Approach: | They propose to use the MaTT benchmark to assess large language models' accuracy in multiple-choice scenarios. |
| Outcome: | The proposed model achieved 54% accuracy in a multiple-choice scenario, while the Chain-of-Thought prompting did not improve. |
An Empirical Comparison of Instance Attribution Methods for NLP (2021.naacl-main)
Copied to clipboard
| Challenge: | Influence functions provide machinery for identifying training instances that may have led to a specific prediction, but are computationally expensive and prohibitive in many cases. |
| Approach: | They evaluate the degree to which different potential instance attribution agrees with respect to the importance of training samples. |
| Outcome: | The proposed methods exhibit desirable characteristics similar to more complex methods, but are computationally expensive. |
Mixed Signals: Decoding VLMs’ Reasoning and Underlying Bias in Vision-Language Conflict (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Vision-language models have demonstrated impressive performance by effectively integrating visual and textual information to solve complex tasks. |
| Approach: | They build upon existing benchmarks to create five datasets containing mismatched image-text pairs and examine how they reason over visual and textual data . |
| Outcome: | The proposed model reasoned over visual and textual data in real-world applications but not in the visual and visual descriptions. |
Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications (N19-1)
Copied to clipboard
| Challenge: | Existing approaches focus on improving accuracy and overlook other aspects such as robustness and interpretability. |
| Approach: | They propose adversarial modifications for link prediction models that identify influential facts and evaluate their sensitivity to addition of fake facts. |
| Outcome: | The proposed model evaluates the robustness of the model to the addition of fake facts and the interpretability of the models. |
ParsiNLU: A Suite of Language Understanding Challenges for Persian (2021.tacl-1)
Copied to clipboard
Daniel Khashabi, Arman Cohan, Siamak Shakeri, Pedram Hosseini, Pouya Pezeshkpour, Malihe Alikhani, Moin Aminnaseri, Marzieh Bitaab, Faeze Brahman, Sarik Ghazarian, Mozhdeh Gheini, Arman Kabiri, Rabeeh Karimi Mahabagdi, Omid Memarrast, Ahmadreza Mosallanezhad, Erfan Noury, Shahab Raji, Mohammad Sadegh Rasooli, Sepideh Sadeghi, Erfan Sadeqi Azer, Niloofar Safi Samghabadi, Mahsa Shafaei, Saber Sheybani, Ali Tazarv, Yadollah Yaghoobzadeh
| Challenge: | Despite progress in natural language understanding, most progress is concentrated on resource-rich languages like English . despite high-quality benchmarks, there are few available NLU datasets for Persian language . |
| Approach: | They propose a benchmark for Persian language that includes a range of language understanding tasks . they present their results on monolingual and multilingual pre-trained language models . |
| Outcome: | The proposed benchmarks compare human performance with monolingual and multilingual models on Persian language with high quality evaluation datasets. |
Combining Feature and Instance Attribution to Detect Artifacts (2022.findings-acl)
Copied to clipboard
| Challenge: | In this paper, we evaluate use of different attribution methods for aiding identification of training data artifacts. |
| Approach: | They propose hybrid methods that combine saliency maps and instance attribution methods to aid in identifying training data artifacts. |
| Outcome: | The proposed methods can be used to efficiently uncover artifacts in training data when a challenging validation set is available. |
From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Tool-augmented Language Models can invoke external tools to solve problems beyond their parametric capacity. |
| Approach: | They propose a preference-optimization-based framework that realigns TaLMs to use tool outputs as assistive evidence. |
| Outcome: | The proposed framework improves accuracy and reasoning depth under tool use. |
Multi-Conditional Ranking with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to rank documents using large language models are limited by the complexity of the items and conditions. |
| Approach: | They propose a novel decomposed reasoning method to evaluate multi-conditional ranking across various item types and conditions to overcome this limitation. |
| Outcome: | The proposed method improves LLMs performance 14.4% over existing methods. |
Less is More for Long Document Summary Evaluation by LLMs (2024.eacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational cost and the Lost-in-the-middle problem where important information in the middle of long documents is often overlooked. |
| Approach: | They propose a novel method which extracts key sentences from a long source document and then evaluates the summary by prompting LLMs. |
| Outcome: | The proposed method significantly reduces evaluation costs and exhibits a higher correlation with human evaluations. |
Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education (2025.naacl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have potential to automate hiring but inherent biases may lead to unfair hiring practices. |
| Approach: | They evaluate how factors such as gender, race, and educational background influence model decisions. |
| Outcome: | The proposed model reduces biases related to gender and race, but implicit biase concerning educational background remains significant. |
Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models have demonstrated impressive capabilities in various NLP tasks, but previous studies have shown they are sensitive to prompt wording and few-shot demonstrations and their order. |
| Approach: | They focus on LLMs robustness on multiple-choice questions . they find a performance gap of 13% to 85% when options are reordered . |
| Outcome: | The proposed model outperforms supervised models on multiple choice questions even outperforming humans. |
Embedding Multimodal Relational Data for Knowledge Base Completion (D18-1)
Copied to clipboard
| Challenge: | Existing approaches focus on a finite set of entities, ignoring the variety of data types used in knowledge bases. |
| Approach: | They propose multimodal knowledge base embeddings that use different neural encoders for observed data and different neural decoders to learn embedded entities and multimodal data. |
| Outcome: | The proposed models outperform existing methods with 5-7% accuracy over existing methods. |