Do We Know What LLMs Don’t Know? A Study of Consistency in Knowledge Probing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for probing knowledge gaps in large language models are inconsistent and inconsistent. |
| Approach: | They propose a process based on input variations and quantitative metrics to evaluate probing methods that are inconsistent on knowledge gaps. |
| Outcome: | The proposed process exposes two dimensions of inconsistency in knowledge gap probing. |
Similar Papers
What Did I Do Wrong? Quantifying LLMs’ Sensitivity and Consistency to Prompt Engineering (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have significantly improved productivity in a number of routine tasks. |
| Approach: | They propose two metrics for classification tasks, namely *sensitivity* and *consistency*, which are complementary to task performance. |
| Outcome: | The proposed metrics are complementary to task performance and can be used to guide prompt engineering and obtain LLMs that balance robustness and performance. |
To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have remarkable performance in a variety of tasks due to factual knowledge accumulated during pre-training. |
| Approach: | They propose an evaluation protocol that disentangles knowing from applying knowledge and test state-of-the-art LLMs on 49 ambiguous entities. |
| Outcome: | The proposed evaluation protocol disentangles knowing from applying knowledge and tests state-of-the-art LLMs on 49 ambiguous entities. |
Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025.emnlp-main)
Copied to clipboard
Hexiang Tan, Fei Sun, Sha Liu, Du Su, Qi Cao, Xin Chen, Jingang Wang, Xunliang Cai, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng
| Challenge: | Existing detection methods fail to account for **self-consistent error** . study identifies self-consistency errors and evaluates them . |
| Approach: | They propose a method that fuses hidden state evidence from an external verifier LLM to detect self-consistent errors. |
| Outcome: | The proposed method significantly enhances performance on self-consistent errors across three LLM families. |
Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method (2024.naacl-long)
Copied to clipboard
Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Chong Meng, Shuaiqiang Wang, Zhicong Cheng, Zhaochun Ren, Dawei Yin
| Challenge: | Recent literature reveals that Large Language Models (LLMs) hallucinate intermittently, which impedes their reliability for further utilization. |
| Approach: | They propose a self-detection method to detect which questions an LLM does not know by combining the two components to identify whether the model generates a non-factual response to the question. |
| Outcome: | The proposed method can detect which questions an LLM does not know across factoid question-answering, arithmetic reasoning, and commonsense reasoning tasks. |
Detecting LLM Hallucination Through Layer-wise Information Deficiency: Analysis of Ambiguous Prompts and Unanswerable Questions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) often generate confident yet inaccurate responses, introducing significant risks for deployment in safety-critical domains. |
| Approach: | They propose a method to detect model hallucination by systematic analysis of information flow across model layers. |
| Outcome: | The proposed approach improves model reliability by immediately integrating with universal LLMs without additional training or architectural modifications. |
Do Large Language Models Know What They Don’t Know? (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have vast knowledge that allows them to excel in various NLP tasks. |
| Approach: | They propose an automated method to detect uncertainty in the responses of large language models and a dataset to measure their self-knowledge. |
| Outcome: | The proposed method detects uncertainty in the responses of large language models and provides a novel measure of their self-knowledge. |
You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments (2024.naacl-long)
Copied to clipboard
Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, David Jurgens
| Challenge: | Large Language Models (LLMs) are popular for research in social sciences . currently, prompting LLMs is insufficient to accurately and reliably capture model perceptions, and we discuss potential alternatives to improve this. |
| Approach: | They construct a dataset that contains 693 questions encompassing 39 different instruments of persona measurement on 115 persona axes and a set of questions containing minor variations. |
| Outcome: | The proposed model can generate answers and negate statements in a consistent and robust manner. |
Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) tend to be unreliable on fact-based answers. |
| Approach: | They propose a framework for comparing LLMs' confidence over fact-based answers with hidden-state probes that are more reliable than hidden-status probes. |
| Outcome: | The proposed methods show that hidden-state probes provide the most reliable confidence estimates despite requiring access to weights and supervision data. |
Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Known-unknown questions are characterized by high uncertainty due to the absence of definitive answers. |
| Approach: | They introduce a dataset with known-unknown questions and establish a categorization framework to clarify the origins of uncertainty in such queries. |
| Outcome: | The proposed model improved in distinguishing between known and unknown queries within open-ended question-answering scenarios. |
Demystifying Uncertainty in LLMs: Active Calibration between Concepts and Human Evaluations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static strategies for mitigating hallucinations do not explicitly model the information gain from interacting with the external environment. |
| Approach: | They propose a calibration-driven interactive learning strategy that selects clarification queries by optimizing calibration error. |
| Outcome: | The proposed method provides theoretical guarantees and empirical gains for reliability. |