FASTTRACK: Reliable Fact Tracing via Clustering and LLM-Powered Evidence Validation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to fact tracing rely on assessing the similarity between training samples and the query along a certain dimension, such as lexical similarity, gradient, or embedding space. |
| Approach: | They propose a new approach that harnesses the capabilities of Large Language Models to validate supportive evidence for queries and clusters the training database towards a reduced extent for LLMs to trace facts. |
| Outcome: | The proposed approach outperforms existing methods in accuracy and efficiency while being x33 faster than TracIn. |
Similar Papers
FaStFact: Faster, Stronger Long-Form Factuality Evaluations in LLMs (2025.findings-emnlp)
Copied to clipboard
Yingjia Wan, Haochen Tan, Xiao Zhu, Xinyu Zhou, Zhiwei Li, Qingsong Lv, Changxuan Sun, Jiaqi Zeng, Yi Xu, Jianqiao Lu, Yinhong Liu, Zhijiang Guo
| Challenge: | Prior evaluation pipelines fail to evaluate factuality of long-form LLMs due to inefficiency and costly human assessment. |
| Approach: | They propose a fast and strong evaluation pipeline that can evaluate factuality of long-form LLMs . they propose 'faStFact' to reduce cost of web searching and inference calling . |
| Outcome: | The proposed evaluation pipeline achieves highest alignment with human evaluation and efficiency among existing baselines. |
Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have demonstrated effectiveness in zero-shot retrieval tasks using large language models. |
| Approach: | They challenge this assumption by analyzing whether knowledge leakage in benchmarks contributes to performance gains. |
| Outcome: | The proposed methods have demonstrated significant performance gains across multiple benchmarks. |
GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checking (2025.acl-long)
Copied to clipboard
Yingjian Chen, Haoran Liu, Yinhong Liu, Jinxiang Xie, Rui Yang, Han Yuan, Yanran Fu, Peng Yuan Zhou, Qingyu Chen, James Caverlee, Irene Li
| Challenge: | Existing fact-checking methods that use large language models often generate subtle factual errors. |
| Approach: | They propose a fact-checking framework that uses extracted knowledge graphs to enhance text representation. |
| Outcome: | GraphCheck outperforms existing specialized fact-checkers on seven benchmarks spanning general and medical domains . Graph Neural Networks process extracted knowledge graphs as a soft prompt, enabling efficient fact- checking in a single inference call. |
MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current methods for fact-checking are based on verifying each piece of a model against potential evidence using an LLM. |
| Approach: | They propose a method that builds small fact-checking models that have GPT-4-level performance but 400x lower cost. |
| Outcome: | The proposed model outperforms other models and reaches GPT-4 accuracy. |
Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection (2026.acl-long)
Copied to clipboard
Yuwei Zhang, Wenhao Yu, Shangbin Feng, Yifan Zhu, Letian Peng, Jayanth Srinivasa, Gaowen Liu, Jingbo Shang
| Challenge: | Existing knowledge injection benchmarks for large language models lack standardized testing grounds. |
| Approach: | They propose a knowledge injection benchmark that leverages recently-added and expert-curated facts from Wikipedia’s “Did You Know...” entries. |
| Outcome: | The proposed framework improves reliability accuracy by 29.1%. |
Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers (2024.findings-emnlp)
Copied to clipboard
Yuxia Wang, Revanth Gangi Reddy, Zain Mujahid, Arnav Arora, Aleksandr Rubashevskii, Jiahui Geng, Osama Mohammed Afzal, Liangming Pan, Nadav Borenstein, Aditya Pillai, Isabelle Augenstein, Iryna Gurevych, Preslav Nakov
| Challenge: | Large language models generate naturally sounding answers over a broad range of human inquiries, but they often generate answers that contradict real-world facts. |
| Approach: | They propose a framework for annotating and evaluating the factuality of large language models . they propose 'factcheck-bench' which provides a multi-stage annotation scheme . |
| Outcome: | The proposed framework outperforms several popular LLM fact-checkers in claim, sentence, and document levels. |
TruthTrap: A Bilingual Benchmark for Evaluating Factually Correct Yet Misleading Information in Question Answering (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used to answer factual, information-seeking questions (ISQs). |
| Approach: | They propose to use a dataset to evaluate large language models to generate human-like text on ISQs in two languages, English and Farsi, and then use it to evaluate nine LLMs. |
| Outcome: | The proposed dataset shows that accuracy drops by 25% when models encounter misleading yet factual hints. |
FactLens: Benchmarking Fine-Grained Fact Verification (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive capability in language generation and understanding, but their tendency to hallucinate and produce factually incorrect information remains a key limitation. |
| Approach: | They propose a benchmark to evaluate fine-grained fact verification where claims are broken down into smaller sub-claims for individual verification. |
| Outcome: | The proposed model enables more precise identification of inaccuracies, improved transparency, and reduced ambiguity in evidence retrieval. |
Towards Faithful and Robust LLM Specialists for Evidence-Based Question-Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Evidence-Based QA has proved insufficiently faithful with Large Language Models . a typical application of LLMs is in Evidence-based Question Answering (QA). |
| Approach: | They propose a data generation pipeline with automated data quality filters to fine-tune LLMs for better source quality and answer attributability. |
| Outcome: | The proposed model can synthesize high-quality training and testing data at scale. |
EviReport: From Reasoned Outlines to Evidence Tracked Long-Form Reports (2026.findings-acl)
Copied to clipboard
Zihan Liu, Jianhui li, Zexin Wang, Fei Sun, Jingjing LI, Zheyuan Li, Ke Xiang, Hang Cui, Houhua Gong, Changhua Pei, Gaogang Xie
| Challenge: | Evidence-intensive reports often produce fluent but under-supported drafts . eviReport is an evidence-grounded workflow for automated long-form report generation . |
| Approach: | They propose an evidence-tracked workflow that organizes corpus evidence into compact, traceable units and retrieves query-relevant subgraphs into retrieval-ready packages. |
| Outcome: | The proposed workflow outperforms baselines in factual coverage, factual accuracy and visual evidence integration. |