FacLens: Transferable Probe for Foreseeing Non-Factuality in Fact-Seeking Question Answering of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-factuality detection methods require response generation, which incurs significant computational overhead. |
| Approach: | They propose a lightweight model called Factuality Lens which effectively probes hidden representations of fact-seeking questions for the NFP task. |
| Outcome: | The proposed model is able to probe hidden representations of fact-seeking questions and reduce development costs. |
Similar Papers
Factuality of Large Language Models: A Survey (2024.emnlp-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Georgiev, Rocktim Das, Preslav Nakov
| Challenge: | Large language models (LLMs) are factually incorrect, which limits their applicability in real-world scenarios. |
| Approach: | They analyze existing work to identify major challenges and their associated causes . they propose to evaluate LLMs using a variety of measures to mitigate factual errors . |
| Outcome: | The proposed methods are based on a variety of datasets and proposed strategies to mitigate factual errors. |
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models are trained on vast unlabeled data, rich in world knowledge. |
| Approach: | They propose a categorization scheme for factual probing methods based on how inputs, outputs and probed PLMs are adapted . they synthesize insights about knowledge retention and prompt optimization in PLM models and analyze obstacles to adopting them as knowledge bases . |
| Outcome: | The proposed method synthesizes insights about knowledge retention and prompt optimization in PLMs, analyzes obstacles to adopting them as knowledge bases and outline directions for future work. |
Enabling LLM Knowledge Analysis via Extensive Materialization (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have majorly advanced NLP and AI, and a major success factor is their internalized factual knowledge. |
| Approach: | They propose a method to comprehensively materialize an LLM’s factual knowledge through recursive querying and result consolidation. |
| Outcome: | The proposed method provides constructive insights into the scope and structure of LLM knowledge (or beliefs) it provides scale, accuracy, bias, cutoff and consistency at the same time. |
Tracing and Dissecting How LLMs Recall Factual Knowledge for Real World Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have shown promising ability to perform commonsense reasoning. |
| Approach: | They propose a two-dimensional analysis framework that incorporates token back-tracing and token decoding to uncover how LLMs conduct factual knowledge recall. |
| Outcome: | The proposed framework shows that LLMs lack relevant knowledge but struggle to select the most accurate information based on context during the retrieval and rerank phase. |
Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models outperform information retrieval techniques for downstream knowledge-intensive tasks when being prompted to generate world knowledge. |
| Approach: | They propose a COmpreheNsive kNowledge Evaluation framework to evaluate generated knowledge from six important perspectives . they conduct extensive empirical analysis of generated knowledge on two widely studied knowledge-intensive tasks . |
| Outcome: | The proposed framework evaluates generated knowledge from six important perspectives on two knowledge-intensive tasks. |
TruthTrap: A Bilingual Benchmark for Evaluating Factually Correct Yet Misleading Information in Question Answering (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used to answer factual, information-seeking questions (ISQs). |
| Approach: | They propose to use a dataset to evaluate large language models to generate human-like text on ISQs in two languages, English and Farsi, and then use it to evaluate nine LLMs. |
| Outcome: | The proposed dataset shows that accuracy drops by 25% when models encounter misleading yet factual hints. |
FactAlign: Long-form Factuality Alignment of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have demonstrated significant potential as the next-generation information access engines, but reliability is hindered by issues of hallucination and generating non-factual content. |
| Approach: | They propose a novel alignment framework that enhances the factuality of LLMs’ long-form responses while maintaining their helpfulness. |
| Outcome: | The proposed framework improves factuality of LLMs while maintaining helpfulness. |
FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Models (2025.findings-emnlp)
Copied to clipboard
Radu Marinescu, Debarun Bhattacharjya, Junkyu Lee, Tigran T. Tchrakian, Javier Carnerero-Cano, Yufang Hou, Elizabeth M. Daly, Alessandra Pascale
| Challenge: | Large language models often fail to ensure factual accuracy of outputs thus limiting reliability in real-world applications. |
| Approach: | They propose a neuro-symbolic based factuality assessment framework that employs probabilistic reasoning to evaluate the truthfulness of long-form generated responses. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt-based methods in factual accuracy and recall. |
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language models (LMs) capture factual knowledge by filling in the blanks of cloze-style prompts. |
| Approach: | They propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
| Outcome: | The proposed method improves the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
Comprehensiveness Metrics for Automatic Evaluation of Factual Recall in Text Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) produce incomplete or selectively omit key information . omissions of key information or misrepresentation of conflicting evidence can cause harm . |
| Approach: | They propose a method that decomposes texts into atomic statements and uses natural language inference to identify missing facts and a Q A-based metric that extracts question-answer pairs and compares responses across sources. |
| Outcome: | The proposed evaluation metrics show they perform better than more complex metrics, but at a cost. |