Papers by Estevam Hruschka
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study investigated hallucinations in multi-document summarization tasks . but, it is unclear how challenges arising from handling multiple documents affect outputs . |
| Approach: | They investigate how hallucinations manifest in large language models when summarizing topic-specific information from a set of documents. |
| Outcome: | The proposed benchmarks show that the models generate more hallucinations than baselines . the results highlight the need for more effective approaches to mitigate hallucinosity in MDS . |
Mixed Signals: Decoding VLMs’ Reasoning and Underlying Bias in Vision-Language Conflict (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Vision-language models have demonstrated impressive performance by effectively integrating visual and textual information to solve complex tasks. |
| Approach: | They build upon existing benchmarks to create five datasets containing mismatched image-text pairs and examine how they reason over visual and textual data . |
| Outcome: | The proposed model reasoned over visual and textual data in real-world applications but not in the visual and visual descriptions. |
Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models generate fluent text with minimal task-specific supervision, but their ability to generate rationales for knowledge-intensive tasks (KITs) remains under-explored. |
| Approach: | They propose to generate retrieval-augmented rationalization of KIT model predictions via external knowledge guidance within a few-shot setting. |
| Outcome: | The proposed rationales were compared with crowd-sourced rationale models on factuality, sufficiency, and convincingness. |
A Dynamic Self-Evolving Extraction System (2026.acl-demo)
Copied to clipboard
| Challenge: | High-quality information extractions often require domain-specific accuracy, up-to-date understanding of specialized taxonomies, and the ability to incorporate emerging jargon and rare outliers. |
| Approach: | They propose a Dynamic Self-Evolving Extraction and Curation Toolkit which continuously improves as it is used to extract structured information from raw text. |
| Outcome: | The proposed toolkit continuously improves as it is used in medical, legal, and HR domains. |
RECAP: REwriting Conversations for Intent Understanding in Agentic Planning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Understanding user intent is essential for effective conversational assistants . however, real-world dialogues are often ambiguous, underspecified, or dynamic . |
| Approach: | They propose a benchmark to evaluate intent rewriting in user-agent dialogues . they propose rewriters that reframe user-goal dialogues into concise representations of user goals . |
| Outcome: | The proposed benchmark outperforms baselines in terms of plan preference and fine-tuning two DPO-based rewriters yields additional utility gains. |
From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Tool-augmented Language Models can invoke external tools to solve problems beyond their parametric capacity. |
| Approach: | They propose a preference-optimization-based framework that realigns TaLMs to use tool outputs as assistive evidence. |
| Outcome: | The proposed framework improves accuracy and reasoning depth under tool use. |
Multi-Conditional Ranking with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to rank documents using large language models are limited by the complexity of the items and conditions. |
| Approach: | They propose a novel decomposed reasoning method to evaluate multi-conditional ranking across various item types and conditions to overcome this limitation. |
| Outcome: | The proposed method improves LLMs performance 14.4% over existing methods. |
Low-resource Entity Set Expansion: A Comprehensive Study on User-generated Text (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing benchmarks for entity set expansion (ESE) are limited to well-formed text and well-defined concepts. |
| Approach: | They propose to use user-generated text to assess the generalizability of ESE methods by identifying phenomena such as non-named entities, multifaceted entities and vague concepts. |
| Outcome: | The proposed methods are based on user-generated text to assess their generalizability and performance. |
Less is More for Long Document Summary Evaluation by LLMs (2024.eacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational cost and the Lost-in-the-middle problem where important information in the middle of long documents is often overlooked. |
| Approach: | They propose a novel method which extracts key sentences from a long source document and then evaluates the summary by prompting LLMs. |
| Outcome: | The proposed method significantly reduces evaluation costs and exhibits a higher correlation with human evaluations. |
Verification-Aware Planning for Multi-Agent Systems (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language model (LLM) agents are increasingly deployed to tackle complex tasks . multi-agent collaboration introduces new challenges in planning, coordination, and verification . |
| Approach: | They propose a framework for multi-agent collaboration with verification-aware planning . the framework decomposes tasks, models subtask dependencies, and encodes planner-defined passing criteria as subtask verification functions (VFs) |
| Outcome: | The proposed framework outperforms baselines on diverse datasets while improving system robustness and interpretability. |
AIPOM: Agent-aware Interactive Planning for Multi-Agent Systems (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Large language models (LLMs) are being used for planning in orchestrated multi-agent systems . existing LLMs fall short of human expectations and lack effective mechanisms for users to inspect, understand, and control their behaviors. |
| Approach: | They propose a system supporting human-in-the-loop planning through conversational and graph-based interfaces. |
| Outcome: | AIPOM enables users to transparently inspect, refine, and collaboratively guide LLM-generated plans, significantly enhancing user control and trust in multi-agent workflows. |
Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education (2025.naacl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have potential to automate hiring but inherent biases may lead to unfair hiring practices. |
| Approach: | They evaluate how factors such as gender, race, and educational background influence model decisions. |
| Outcome: | The proposed model reduces biases related to gender and race, but implicit biase concerning educational background remains significant. |
Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models have demonstrated impressive capabilities in various NLP tasks, but previous studies have shown they are sensitive to prompt wording and few-shot demonstrations and their order. |
| Approach: | They focus on LLMs robustness on multiple-choice questions . they find a performance gap of 13% to 85% when options are reordered . |
| Outcome: | The proposed model outperforms supervised models on multiple choice questions even outperforming humans. |
Natural Language Processing for Human Resources: A Survey (2025.naacl-industry)
Copied to clipboard
| Challenge: | Recent advances in NLP have the potential to transform HR processes, from recruitment to employee management. |
| Approach: | They analyze key tasks such as information extraction and text classification and their roles in downstream applications like recommendation and language generation while discussing ethical concerns. |
| Outcome: | The proposed frameworks can be applied to HR tasks and to recommendation, language generation, and interaction. |
FactLens: Benchmarking Fine-Grained Fact Verification (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive capability in language generation and understanding, but their tendency to hallucinate and produce factually incorrect information remains a key limitation. |
| Approach: | They propose a benchmark to evaluate fine-grained fact verification where claims are broken down into smaller sub-claims for individual verification. |
| Outcome: | The proposed model enables more precise identification of inaccuracies, improved transparency, and reduced ambiguity in evidence retrieval. |
Low-resource Interactive Active Labeling for Fine-tuning Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing active learning methods for fine-tuning language models are underperforming in low-resource, interactive labeling setting. |
| Approach: | They propose a novel active learning method that employs a hybrid sampling strategy to minimize labeling cost and acquisition latency while providing a framework for adapting to dataset diversity. |
| Outcome: | The proposed method reduces labeling cost and acquisition latency while providing a framework for adapting to dataset diversity via user guidance. |