Revealing Redundant Syntax in Large Language Models through Multi-Hop Dependency Paths (2026.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on attention–syntax alignment has focused on single-hop Universal Dependency edges (DPs). |
| Approach: | They extract 2–3 hop MDPs from UD-parsed English and quantify head–relation alignment with an Unlabeled Attachment Score (UAS)-style metric modified for causal masking in decoder-only models. |
| Outcome: | The authors show that head alignments are overlapped and specialized . the head alignment is measurable in large language models trained on raw text . |
Similar Papers
On the Continued Value of Universal Dependencies in the Era of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | a growing belief that explicit linguistic representations are no longer necessary is questioned in large language models . a recent study examines whether and in what ways this cross-lingual syntactic framework can still benefit LLMs . |
| Approach: | They use Universal Dependencies (UD) to examine whether and in what ways it can still benefit LLMs. |
| Outcome: | The proposed model outperforms its syntax-agnostic counterparts in a cross-lingual evaluation task. |
Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation (2025.acl-long)
Copied to clipboard
| Challenge: | Multi-hop Question Answering (MHQA) adds layers of complexity to question answering tasks. |
| Approach: | They explore how LMs respond to multi-hop questions by permuting search results under various configurations. |
| Outcome: | The proposed model outperforms decoder-only models in MHQA tasks despite being significantly smaller in size . |
Failure Modes in Multi-Hop QA: The Weakest Link Effect and the Recognition Bottleneck (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have identified a position bias in Large Language Models that causes them to overlook information at certain positions. |
| Approach: | They propose a semantic probe to disentangle position bias in Large Language Models . they propose MFAI to steer attention towards selected positions . |
| Outcome: | The proposed model can locate and integrate information at certain positions even in noisy, long-context settings. |
Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? (2024.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art Large Language Models (LLMs) are accredited with a number of different capabilities, including reading comprehension, mathematical and reasoning skills, and possessing scientific knowledge. |
| Approach: | They propose a benchmark to generate seemingly plausible multi-hop reasoning chains that ultimately lead to incorrect answers. |
| Outcome: | The proposed model circumvents the reasoning requirement but in subtle ways . it shows that it is more difficult to generate plausible alternatives . |
GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing LLM leaderboards often reference scores reported in other papers without consistent settings and prompts, which may encourage cherry-picking favored settings and for better results. |
| Approach: | They propose an open-source and reproducible LLM evaluation suite built on top of OpenAI Evals that systematically evaluates 10+ leading LLMs and OpenAI’s legacy models on 20+ curated benchmarks across 7 capability categories. |
| Outcome: | The evaluation suite is built on top of OpenAI Evals and evaluates 10+ leading LLMs and OpenAI’s legacy models on 20+ curated benchmarks across 7 capability categories. |
Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally. |
| Approach: | They propose a "back-patching" analysis method to solve multi-hop queries . they propose resolving the bridge entity into the bridge and the second hop into the target entity into latent steps. |
| Outcome: | The proposed method solves multi-hop queries that require two information extraction steps . it shows that the later layers lack the necessary knowledge to correctly generate the answer . |
Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization (2026.acl-long)
Copied to clipboard
Weixu Zhang, Ye Yuan, Changjiang Han, Yuxing Tian, Zipeng Sun, Linfeng Du, Jikun Kang, Hong Kang, Xue Liu, Haolun Wu
| Challenge: | Large Language Models exhibit strong implicit personalization ability, but most approaches treat this behavior as a black box. |
| Approach: | They propose a mechanistic interpretation perspective and propose 'sparse' set of Preference Heads . they compute a Preference Contribution Score for each attention head and compare their predictions . |
| Outcome: | The proposed framework computes a Preference Contribution Score (PCS) for each attention head and measures its causal impact on user aligned outputs. |
Multi-component Causal Tracing in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are prone to various forms of safety risks, such as learning and propagating societal biases and even creating harmful or deceptive content through jailbreak attacks. |
| Approach: | They propose a framework for causally tracing multiple components simultaneously that systematically identifies the subsets of components most critical to a desired performance metric. |
| Outcome: | The proposed method outperforms existing methods in identifying components critical to a desired performance metric. |
Attention Can Reflect Syntactic Structure (If You Let It) (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study has attempted to decode linguistic structure from the Transformer . but, much of the work focused on English, a language with rigid word order and a lack of inflectional morphology. |
| Approach: | They propose to fine-tune a feature encoder for BERT to learn linguistic structure from its multi-head attention mechanism. |
| Outcome: | The proposed model can decode full trees above baseline accuracy from single attention heads across languages. |
GALLa: Graph Aligned Large Language Models for Improved Source Code Understanding (2025.acl-long)
Copied to clipboard
| Challenge: | Programming languages have rich semantics that are represented by graphs and not available from the surface form of source code. |
| Approach: | They propose to use graph neural networks and cross-modal alignment technologies to inject structural information of code into LLMs as an auxiliary task during finetuning. |
| Outcome: | The proposed framework improves on five code tasks with six different baseline LLMs, while incurring no cost at inference time. |