Papers by Andrei Andriiainen
Quantifying Logical Consistency in Transformers via Query-Key Alignment (2025.emnlp-main)
Copied to clipboard
Eduard Tulchinskii, Laida Kushnareva, Anastasia Voznyuk, Andrei Andriiainen, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov
| Challenge: | Existing solutions for multi-step logical reasoning are unreliable . Existing methods generate intermediate steps but provide no internal check of coherence . |
| Approach: | They propose a method that uses internal Query-Key interactions within transformer attention heads as a proxy for logical consistency. |
| Outcome: | The proposed method reveals latent reasoning structure in large language models and provides a mechanistic alternative to ablation-based analysis. |