INFACT: A Diagnostic Benchmark for Induced Faithfulness and Factuality Hallucinations in Video-LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks only evaluate models in clean settings due to hallucinations . |
| Approach: | They propose a diagnostic benchmark that evaluates models in four modes for faithfulness and factuality. |
| Outcome: | The proposed benchmark evaluates models in four modes: Base (clean), Visual Degradation, Evidence Corruption, and Temporal Intervention for order-sensitive items. |
Similar Papers
The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | a growing number of researchers are studying the hallucination issue in large language models. |
| Approach: | They propose a hallucination detection benchmark and a method to detect hallucines in LLMs. |
| Outcome: | The proposed method detects hallucinations and mitigates them using different training stages. |
Harmful Factuality: LLMs Correcting What They Shouldn’t (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are trained for factual accuracy, but can conflict with the critical demand for source fidelity. |
| Approach: | They propose a reproducible framework to elicit and measure HFH using controlled entity-level perturbations and strategic entity selection. |
| Outcome: | The proposed framework reduces HFH rates by 50% across summarization, rephrasing, and QA tasks. |
Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have extensive world knowledge, but often generate inaccurate geospatial knowledge. |
| Approach: | They propose a framework for evaluation of large language models to mitigate hallucinations . they use Kahneman-Tversky Optimization to align LLMs with their reality . |
| Outcome: | The proposed evaluation framework uncovers hallucinations in 20 advanced LLMs. |
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization (2024.naacl-long)
Copied to clipboard
Liyan Tang, Igor Shalyminov, Amy Wong, Jon Burnsky, Jake Vincent, Yu’an Yang, Siffi Singh, Song Feng, Hwanjun Song, Hang Su, Lijia Sun, Yi Zhang, Saab Mansour, Kathleen McKeown
| Challenge: | Existing LLMs hallucinate significant amounts of factual errors in the dialogue domain, regardless of the model’s size. |
| Approach: | They propose to evaluate topic-focused dialogue summarization by using large language models (LLMs) they use human annotations to evaluate factual consistency and explain factually inconsistent sentences. |
| Outcome: | The proposed evaluation benchmark on topic-focused dialogue summarization shows that existing LLMs hallucinate significant amounts of factual errors regardless of the model’s size. |
FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Language models (LMs) generate false or unverifiable content, often known as hallucination, despite ongoing efforts to enhance their factuality. |
| Approach: | They propose a tool that measures LMs’ factuality in real-world user interactions by evaluating their factual accuracy and categorizing content units as Supported, Unsupported, or Undecidable based on Web-retrieved evidence. |
| Outcome: | The proposed evaluation pipeline measures language models’ factuality in real-world user interactions. |
FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) lack the capacity to handle multimodal inputs effectively. |
| Approach: | They introduce a reference-free and fine-grained evaluation metric that measures the faithfulness of the generated free-form answers from large vision-language models. |
| Outcome: | The proposed metric measures the faithfulness of free-form answers from large vision-language models. |
Distorted or Fabricated? A Survey on Hallucination in Video LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Despite significant advances in video-language modeling, hallucinations remain a persistent challenge in video large language models. |
| Approach: | They present a systematic taxonomy that categorizes hallucinations into two core types: dynamic distortion and content fabrication. |
| Outcome: | The proposed taxonomy categorizes hallucinations into two core types: dynamic distortion and content fabrication. |
LAFaCT: Attribution-based Localization and Focused Sequential Analysis of Fact-Critical Tokens for Hallucination Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models suffer from hallucinations, severely undermining their reliability. |
| Approach: | They propose a framework that localizes fact-critical tokens and performs sequential analysis on their hidden states. |
| Outcome: | The proposed framework localizes fact-critical tokens using Factual Criticality . it then performs a focused sequential analysis on their hidden states . |
When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation (2026.eacl-short)
Copied to clipboard
| Challenge: | Existing studies on LLM factuality evaluation have not investigated the reliability of static evaluation benchmarks. |
| Approach: | They examine five popular factuality benchmarks and eight LLMs released over different years to assess their reliability. |
| Outcome: | The proposed method compared five popular factuality benchmarks and eight LLMs released over different years. |
LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Current evaluation frameworks are static and vulnerable to benchmark data contamination . current models are ineffective at assessing reasoning under temporal uncertainty . |
| Approach: | They propose a live-based benchmark that simulates the real-world "fog of war" they propose evaluating models on their ability to reason with evolving, incomplete information . |
| Outcome: | The proposed model outperforms proprietary state-of-the-art models in classification and evidence mode . it also provides a component to monitor BDC explicitly . |