Token Knowledge: A New Perspective For Knowledge in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Predicting the presence and absence of certain knowledge in large language models could aid hallucination avoidance. |
| Approach: | They propose a token knowledge dataset construction method and use the intermediate states during inference to train probes. |
| Outcome: | The proposed method increases the model's latent potential by 60% to 90% with strong out-of-distribution generalization by training on just a few dozen prompts. |
Similar Papers
Estimating Knowledge in Large Language Models Without Generating a Single Token (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate knowledge in large language models require querying and evaluating the model's generated responses. |
| Approach: | They ask whether it is possible to estimate how knowledgeable a model is about a subject entity only from its internal computation. |
| Outcome: | The proposed model performs well with QA accuracy and FActScore . it can be leveraged to guide decisions on how to apply further training or augment queries with retrieval. |
On Large Language Models’ Hallucination with Regard to Known Facts (2024.naacl-long)
Copied to clipboard
Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou
| Challenge: | Large language models are successful in answering factoid questions but are also prone to hallucination. |
| Approach: | They propose self-reporting to the model when faced with such limitations. |
| Outcome: | The proposed classifier can detect hallucinations with an 88% success rate and can be used to answer factoid questions with correct answer knowledge. |
Tutorial Proposal: Hallucination in Large Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Grasping the intricacies of hallucination in LLMs can be daunting, especially for those new to the field. |
| Approach: | This tutorial aims to bridge the gap between the field and the field of hallucination . it will explore the key aspects of hallucinonation, including benchmarking, detection, and mitigation techniques . |
| Outcome: | This tutorial will explore the key aspects of hallucination in LLMs . it will also explore the specific constraints and shortcomings of current approaches . |
Exploring the Hidden Capacity of LLMs for One-Step Text Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can reconstruct surprisingly long texts via autoregressive generation from just one trained input embedding. |
| Approach: | They show that large language models can reconstruct surprisingly long texts via autoregressive generation from just one trained input embedding. |
| Outcome: | The proposed model can generate hundreds of accurate tokens in one token-parallel forward pass, when provided with only two learned embeddings. |
Enabling LLM Knowledge Analysis via Extensive Materialization (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have majorly advanced NLP and AI, and a major success factor is their internalized factual knowledge. |
| Approach: | They propose a method to comprehensively materialize an LLM’s factual knowledge through recursive querying and result consolidation. |
| Outcome: | The proposed method provides constructive insights into the scope and structure of LLM knowledge (or beliefs) it provides scale, accuracy, bias, cutoff and consistency at the same time. |
Improving Consistency in LLM Inference using Probabilistic Tokenization (2025.findings-naacl)
Copied to clipboard
| Challenge: | Prior work has shown that probabilistic tokenizations can generate multiple tokenization of the same input string. |
| Approach: | They propose a method to leverage the multiple tokenization capabilities of modern LLM tokenizers. |
| Outcome: | The proposed method improves the self-consistency of large language models by generating multiple tokenizations. |
Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations (2026.acl-long)
Copied to clipboard
Shanghao Li, Jinda Han, Yibo Wang, Yuanjie Zhu, Zihe Song, Langzhou He, Kenan Kamel A Alghythee, Philip S. Yu
| Challenge: | Existing literature primarily addresses this problem through external interventions such as retrieval augmentation and prompt engineering at the input or output level. |
| Approach: | They find that LLMs can still produce hallucinated outputs when using structured external knowledge. |
| Outcome: | The proposed models fail to ground the provided knowledge, causing the model to revert to parametric memory. |
Token Prediction as Implicit Classification to Identify LLM-Generated Text (2023.emnlp-main)
Copied to clipboard
| Challenge: | a novel approach for identifying large language models (LLMs) involved in text generation is proposed . instead of adding an additional classification layer, we reframe the classification task as a next-token prediction task . |
| Approach: | They propose a novel approach for identifying large language models involved in text generation . instead of adding an additional classification layer, they reframe the task as a next-token prediction task . |
| Outcome: | The proposed method performs exceptionally well in the text classification task . it can distinguish distinctive writing styles among various LLMs even without an explicit classifier. |
The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models fail at simple character-level tasks due to low mutual information, study finds . authors propose a lightweight architectural modification that improves character- level reasoning . |
| Approach: | They propose a lightweight architectural modification that improves character-level reasoning while preserving the inductive advantages of subword models. |
| Outcome: | The proposed model improves character-level reasoning while preserving the advantages of subword models. |
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification (2024.findings-acl)
Copied to clipboard
Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov
| Challenge: | Large language models are notorious for producing erroneous claims in their output. |
| Approach: | They propose a fact-checking and hallucination detection pipeline based on token-level uncertainty quantification that removes the impact of uncertainty about what claim to generate on the current step and what surface form to use. |
| Outcome: | The proposed method can fact-check the atomic claims in the output of large language models. |