Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are trained on vast corpora that contain substantial knowledge but their outputs often contain confidently stated inaccuracies. |
| Approach: | They propose to encode truthfulness as a distinct linear feature, termed the "truth direction", which can classify truthfulness reliably. |
| Outcome: | The proposed model can generalize to logical transformations, question-answering tasks, in-context learning, and external knowledge sources. |
Similar Papers
On the Universal Truthfulness Hyperplane Inside LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored hallucinations through the lens of internal representations, proposing mechanisms to decipher LLMs’ adherence to facts. |
| Approach: | They propose to train a universal truthfulness hyperplane that distinguishes the model’s factually correct and incorrect outputs on a diverse collection of over 40 datasets and examine its cross-task, cross-domain, and in-domain generalization. |
| Outcome: | The proposed model is able to distinguish factual outputs from incorrect outputs on a diverse collection of over 40 datasets. |
LLM Beliefs Are in Their Heads (2026.acl-long)
Copied to clipboard
| Challenge: | Using linear controlled probes, we investigate belief-like representations in decoder-only autoregressive LLMs using residual stream activations and single attention heads. |
| Approach: | They develop four different experiments on decoder-only autoregressive LLMs and examine how they fare against these standards. |
| Outcome: | The proposed representations exhibit strong truth sensitivity and consistent accuracy across models and data sets. |
Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Quantization enables efficient deployment of large language models in resource-constrained environments . but impact on truthfulness remains largely unexplored . |
| Approach: | They propose a framework to assess the truthfulness of quantized large language models . they find quantized models retain internally truthful representations but produce false outputs . |
| Outcome: | The framework assesses the truthfulness of quantized models across three dimensions . it finds that quantized model models retain internally truthful representations but are more susceptible to false outputs . |
AI-LieDar : Examine the Trade-off Between Utility and Truthfulness in LLM Agents (2025.naacl-long)
Copied to clipboard
| Challenge: | LieDar is a framework to study how LLM-based agents navigate these scenarios in a multi-turn interactive setting. |
| Approach: | They propose a framework to study how LLM-based agents navigate these scenarios in an interactive multi-turn setting. |
| Outcome: | The proposed framework shows that all models are truthful less than 50% of the time, although truthfulness and goal achievement rates vary across models. |
Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that LLMs encode internal representations of factuality when generating inaccurate or fabricated content. |
| Approach: | They propose a strategy for sampling plausible true-false factoid sentences from tabular data and a procedure for generating realistic, LLM-dependent true-False datasets from Question Answering collections. |
| Outcome: | The proposed approach lays the groundwork for future research on factuality in LLMs and offers practical guidelines for more effective evaluation. |
How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work shows that large language models encode whether a statement is true as a vector in residual stream activations. |
| Approach: | They study how truth vectors change when context is introduced in Large Language Models . they measure directional change between truth vector with and without context and relative magnitude of truth vector upon adding context. |
| Outcome: | The results show that large models distinguish relevant from irrelevant context mainly through directional change () |
Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Past work has found that query–probe disagreements occur in situations where either probes or queries are uncertain. |
| Approach: | They propose to identify three different classes of disagreement, which they term confabulation, deception, and heterogeneity. |
| Outcome: | The proposed methods are more accurate than LM outputs and better on uncertain answers than queries and probes. |
Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinations (2026.acl-long)
Copied to clipboard
Wen Luo, Guangyue Peng, Wei Li, Shaohang Wei, Feifan Song, Liang Wang, Nan Yang, Xingxing Zhang, Jing Jin, Furu Wei, Houfeng Wang
| Challenge: | Previous work shows that large language models generate hallucinations, yet the origins and mechanisms of these signals remain unclear. |
| Approach: | They propose to validate and disentangle two different pathways for truthfulness cues . they also propose to use the same mechanism to derive self-contained evidence from the generated answer . |
| Outcome: | The proposed applications improve hallucination detection performance by integrating two different inputs. |
Truth Knows No Language: Evaluating Truthfulness Beyond English (2025.acl-long)
Copied to clipboard
Blanca Calvo Figueras, Eneko Sagarzazu, Julen Etxaniz, Jeremy Barnes, Pablo Gamallo, Iria de-Dios-Flores, Rodrigo Agerri
| Challenge: | a new benchmark evaluates the truthfulness of large language models (LLMs) based on imitative falsehoods. |
| Approach: | They propose a professionally translated extension of the TruthfulQA benchmark . it evaluates truthfulness in Basque, Catalan, Galician, and Spanish . |
| Outcome: | The proposed extension of the TruthfulQA benchmark evaluates truthfulness in Basque, Catalan, Galician, and Spanish. |
The Internal State of an LLM Knows When It’s Lying (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown exceptional performance in various tasks, but one of their main drawbacks is generating inaccurate or false information with a confident tone. |
| Approach: | They propose to train a classifier that outputs the probability that a statement is truthful based on the hidden layer activations of the LLM as it reads or generates the statement. |
| Outcome: | The proposed classifier achieves an average of 71% to 83% accuracy labeling which sentences are true versus false, depending on the LLM base model. |