From Factuality to Meta-Factivity: A Cognitive Blueprint for Trustworthy LLMs

Challenge:	Current evaluation paradigms on Event Factuality Prediction (EFP) focus on static classification tasks and shortcut learning and unreliable reasoning.
Approach:	They propose a meta-factivity framework that moves evaluation beyond surface recognition to belief trajectory reasoning and epistemic regulation.
Outcome:	The proposed framework shifts from event factuality to meta-factivity . the proposed framework lays the groundwork for a more rigorous benchmark for explainable self-governance .

Similar Papers

Overview

Challenge

Approach

Outcome

Harmful Factuality: LLMs Correcting What They Shouldn’t (2026.findings-eacl)

Copied to clipboard

Mingchen Li, Hanzhi Zhang, Heng Fan, Junhua Ding, Yunhe Feng

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) are trained for factual accuracy, but can conflict with the critical demand for source fidelity.
Approach:	They propose a reproducible framework to elicit and measure HFH using controlled entity-level perturbations and strategic entity selection.
Outcome:	The proposed framework reduces HFH rates by 50% across summarization, rephrasing, and QA tasks.

Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation (2024.acl-long)

Copied to clipboard

Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Lifeng Jin, Linfeng Song, Haitao Mi, Helen Meng

Reasoning

Challenge:	Existing approaches to addressing factual inaccuracies require high-quality human factuality annotations to mitigate these hallucinations.
Approach:	They propose to leverage the self-evaluation capability of an LLM to provide training signals that steer the model towards factuality.
Outcome:	The proposed approach significantly improves factual accuracy over LLMs across three key knowledge-intensive tasks on TruthfulQA and BioGEN.

The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models (2024.acl-long)

Copied to clipboard

Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Language Models

Challenge:	a growing number of researchers are studying the hallucination issue in large language models.
Approach:	They propose a hallucination detection benchmark and a method to detect hallucines in LLMs.
Outcome:	The proposed method detects hallucinations and mitigates them using different training stages.

INFACT: A Diagnostic Benchmark for Induced Faithfulness and Factuality Hallucinations in Video-LLMs (2026.acl-long)

Copied to clipboard

null Yangjunqi, Yuecong Min, Jie Zhang, Shiguang Shan, Xilin Chen

Multimodality

Challenge:	Existing benchmarks only evaluate models in clean settings due to hallucinations .
Approach:	They propose a diagnostic benchmark that evaluates models in four modes for faithfulness and factuality.
Outcome:	The proposed benchmark evaluates models in four modes: Base (clean), Visual Degradation, Evidence Corruption, and Temporal Intervention for order-sensitive items.

Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization (2022.acl-long)

Copied to clipboard

Meng Cao, Yue Dong, Jackie Cheung

Text Generation

Challenge:	State-of-the-art abstractive summarization systems often generate hallucinations, i.e., content that is not directly inferable from the source document.
Approach:	They propose a detection approach that separates factual from non-factual hallucinations of entities by masked language models.
Outcome:	The proposed method outperforms baselines in accuracy and F1 scores and has a strong correlation with human judgments on factuality classification tasks.

GO FIGURE: A Meta Evaluation of Factuality in Summarization (2021.findings-acl)

Copied to clipboard

Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, Yejin Choi, Jianfeng Gao

Summarization

Challenge:	Recent advances in text generation systems produce fluent, coherent, relevant, and factually correct text.
Approach:	They propose a metaevaluation framework for evaluating factuality evaluation metrics . they propose five necessary conditions to evaluate factual metrics on diagnostic factuity data .
Outcome:	The proposed framework provides robust evaluation that is extensible to multiple types of factual consistency and standard generation metrics, including QA metrics.

From Remembering to Metacognition: Do Existing Benchmarks Accurately Evaluate LLMs? (2025.findings-emnlp)

Copied to clipboard

Geng Zhang, Yizhou Ying, Sihang Jiang, Jiaqing Liang, Guanglei Yue, Yifei Fu, Hailin Hu, Yanghua Xiao

Reasoning

Challenge:	Existing benchmark datasets focus on low-level cognitive tasks while providing limited coverage of higher-level reasoning skills.
Approach:	They analyze the cognitive depth of popular LLM benchmarks using Bloom’s Taxonomy to evaluate both the cognitive and knowledge dimensions.
Outcome:	The results show that incorporating higher-level cognitive instructions into the current instruction fine-tuning process improves model performance.

TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization (2024.naacl-long)

Copied to clipboard

Liyan Tang, Igor Shalyminov, Amy Wong, Jon Burnsky, Jake Vincent, Yu’an Yang, Siffi Singh, Song Feng, Hwanjun Song, Hang Su, Lijia Sun, Yi Zhang, Saab Mansour, Kathleen McKeown

Natural Language Interfaces

Challenge:	Existing LLMs hallucinate significant amounts of factual errors in the dialogue domain, regardless of the model’s size.
Approach:	They propose to evaluate topic-focused dialogue summarization by using large language models (LLMs) they use human annotations to evaluate factual consistency and explain factually inconsistent sentences.
Outcome:	The proposed evaluation benchmark on topic-focused dialogue summarization shows that existing LLMs hallucinate significant amounts of factual errors regardless of the model’s size.

KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge (2026.findings-eacl)

Copied to clipboard

Alex Robertson, Huizhi Liang, Mahbub Gani, Rohit Kumar, Srijith Rajamohan

Structured Data in NLP

Challenge:	Existing benchmarks for large language models are limited by static and narrow questions, leading to limited coverage and misleading evaluations.
Approach:	They propose a Knowledge Graph-based hallucination benchmark that assesses Large Language Models across the breadth and depth of their knowledge and provides a fairer and more comprehensive insight into LLM truthfulness.
Outcome:	The proposed framework assesses LLMs across breadth and depth of their knowledge, and provides a fairer and more comprehensive insight into LLM truthfulness.

Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMs (2025.acl-long)

Copied to clipboard

Giovanni Servedio, Alessandro De Bellis, Dario Di Palma, Vito Walter Anelli, Tommaso Di Noia

Challenge:	Recent studies suggest that LLMs encode internal representations of factuality when generating inaccurate or fabricated content.
Approach:	They propose a strategy for sampling plausible true-false factoid sentences from tabular data and a procedure for generating realistic, LLM-dependent true-False datasets from Question Answering collections.
Outcome:	The proposed approach lays the groundwork for future research on factuality in LLMs and offers practical guidelines for more effective evaluation.