Tug-of-war between idioms’ figurative and literal interpretations in LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | We find that idioms have non-compositional figurative interpretations that diverge from the idiomatic literal interpretation. |
| Approach: | They employ causal tracing to analyze how pretrained causal transformers deal with idiom ambiguity. |
| Outcome: | The proposed model leverages the idiom's context and refines it if it conflicts with the retrieved interpretation. |
Similar Papers
Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Unlike literal expressions, idioms’ meanings do not follow from their parts, posing a challenge for neural machine translation (NMT). |
| Approach: | They examine the mechanics of the dominant NMT model, Transformer, and their effect on their understanding of idioms. |
| Outcome: | The proposed model over-generates compositional, literal translations and is unable to translate idioms accurately. |
It’s not Rocket Science: Interpreting Figurative Language in Narratives (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing text representations by design rely on compositionality, while figurative language is often non-compositional. |
| Approach: | They propose to use a pre-trained language model to interpret figurative language types to adopt human strategies for interpreting figurativ language types: inferring meaning from context and relying on constituent words’ literal meanings. |
| Outcome: | The proposed models perform significantly worse than humans on discriminative and generative tasks, bridging the gap from human performance. |
When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality (2024.acl-long)
Copied to clipboard
| Challenge: | In incremental models, one interpretation is possible, but models that can revise can do so if the ambiguity is resolved. |
| Approach: | They propose an interpretable way to analyse incremental states in a bidirectional way . they propose to use a model that can update internal states to reflect the garden path effect . |
| Outcome: | The proposed model shows that it can perform revisions and recover if the label is incorrect. |
That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for translation of ambiguous text use context to disambiguate meaning . current models for MTs consistently translate English idioms literally, whereas LMs are context-aware . |
| Approach: | They use a dataset of 512 pairs of English sentences to study semantic ambiguities . they use literal and figurative idioms to disambiguate intended meaning . |
| Outcome: | The results show that current models translate English idioms literally, even when the context suggests a figurative interpretation. |
Causal interventions expose implicit situation models for commonsense language understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Classical psycholinguistic accounts have suggested that world knowledge enters into language understanding through structured schemas called situation models. |
| Approach: | They apply causal intervention techniques to transformer models to analyze performance on the Winograd Schema Challenge . |
| Outcome: | The proposed model performs well on the Winograd Schema Challenge . |
Mechanistic Insights into Deferred Semantic Drift in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) face a fundamental challenge with delayed disambiguation: how can a model update the meaning of an early, ambiguous token when clarifying context only appears later in the sequence? |
| Approach: | They propose a method to defer semantic re-evaluation to subsequent tokens in a process they call "Deferred Semantic Drift" they demonstrate this mechanism in metaphor comprehension and provide causal validation by steering model outputs towards literal or metaphorical meanings via targeted activation interventions. |
| Outcome: | The proposed model can update the meaning of an ambiguous word when clarifying context arrives only after it has been processed. |
Semantic Contrastive Adaptation for Multimodal Figurative Language Understanding (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing models for understanding figurative language in images perform well on literal recognition but fail on multimodal figurativ benchmarks. |
| Approach: | They propose a model that adapts to idiomatic and figurative language using literal alignment bias rather than limited model capacity. |
| Outcome: | The proposed model generalizes across five idiom-rich languages despite being trained on English supervision. |
No Context Needed: Contextual Quandary In Idiomatic Reasoning With Pre-Trained Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | idiomatic expressions (IEs) are a non-compositional aspect of a text that makes it difficult for a model to comprehend . general purpose PTLMs are negatively affected by the context, as performance increases with its removal. |
| Approach: | They propose to use idiomatic expressions to infer additional meaning from IEs . they argue that only IE-aware models are suitable for idiom- matic reasoning tasks . |
| Outcome: | The proposed models can reason in the presence of idiomatic expressions, the authors show . they show that general purpose PTLMs are negatively affected by the context . |
Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting (2023.emnlp-main)
Copied to clipboard
| Challenge: | idioms are common in everyday language, but often pose a challenge to translators because their meanings do not follow from the meanings of their parts. |
| Approach: | They propose to use retrieval-augmented models to increase the accuracy of a strong pretrained machine translation model on idiomatic sentences by up to 13%. |
| Outcome: | The proposed techniques improve the accuracy of a strong pretrained model on idiomatic sentences by up to 13% in absolute accuracy, and holds potential benefits for non-idiomatic phrases. |
Bridging Internal Consistency and External Alignment: A Causal and Dynamic Interpretability Framework for LLM Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing interpretability methods focus on internal and external aspects of the model . existing explanations often focus on surface correlations or static dependencies . |
| Approach: | They propose a causal and dynamic interpretability framework for Large Language Models . they characterize backdoor-adjusted causal effects of generated prefix and prompt . |
| Outcome: | The proposed framework provides a unified causal view of internal consistency and external alignment in LLM generation dynamics. |