Papers by Jakub Binkowski
Hallucination Detection in LLMs Using Spectral Features of Attention Maps (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance across tasks but remain prone to hallucinations. |
| Approach: | They propose a method that uses attention maps to detect hallucinations . they propose to use top-k eigenvalues of the attention maps as input to probes . |
| Outcome: | The proposed method achieves state-of-the-art hallucination detection performance among attention-based methods. |
FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to detect hallucinated content are limited by their tendency to generate factual errors. |
| Approach: | They propose a black-box sampling-based method that enables fine-grained fact-level detection by representing text as interpretable knowledge graphs consisting of facts in the form of triples. |
| Outcome: | The proposed method improves hallucination correction by 35.5% compared to baseline methods while sentence-level SelfCheckGPT yields only 10.6% improvement. |
Empowering Small-Scale Knowledge Graphs: A Strategy of Leveraging General-Purpose Knowledge Graphs for Enriched Embeddings (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to augment LLMs with Knowledge Graphs (KGs) Knowledge-intensive tasks are prone to errors and require a large amount of knowledge to be understood. |
| Approach: | They propose a framework for augmenting LLMs through Knowledge Graphs (KGs) they propose KGs can be used to enhance performance in knowledge-intensive tasks . |
| Outcome: | Experimental results show that a small domain-specific KG can benefit from a performance boost in downstream tasks when linked to a substantial general-purpose KG. |
When Will the Tokens End? Graph-Based Forecasting for LLMs Output Length (2025.acl-srw)
Copied to clipboard
Grzegorz Piotrowski, Mateusz Bystroński, Mikołaj Hołysz, Jakub Binkowski, Grzegorz Chodak, Tomasz Jan Kajdanowicz
| Challenge: | Large Language Models (LLMs) are typically trained to predict the next token in a sequence. However, their internal representations encode signals that go beyond immediate next-token prediction. |
| Approach: | They propose an aggregation-based model that combines hidden states from multiple transformer layers l 8, dots, 15 using element-wise operations such as mean or sum. |
| Outcome: | The proposed model reduces NMAE by over 50% on the Alpaca dataset. |
The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs (2025.emnlp-main)
Copied to clipboard
Denis Janiak, Jakub Binkowski, Albert Sawczyn, Bogdan Gabrys, Ravid Shwartz-Ziv, Tomasz Jan Kajdanowicz
| Challenge: | Large language models (LLMs) have revolutionized natural language processing, but their tendency to hallucinate poses serious challenges for reliable deployment. |
| Approach: | They propose to use ROUGE to assess lexical overlap to determine accuracy of hallucination detection methods. |
| Outcome: | The proposed evaluation frameworks can rival complex methods, exposing a fundamental flaw in current evaluation practices. |