No Context Needed: Contextual Quandary In Idiomatic Reasoning With Pre-Trained Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | idiomatic expressions (IEs) are a non-compositional aspect of a text that makes it difficult for a model to comprehend . general purpose PTLMs are negatively affected by the context, as performance increases with its removal. |
| Approach: | They propose to use idiomatic expressions to infer additional meaning from IEs . they argue that only IE-aware models are suitable for idiom- matic reasoning tasks . |
| Outcome: | The proposed models can reason in the presence of idiomatic expressions, the authors show . they show that general purpose PTLMs are negatively affected by the context . |
Similar Papers
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions (2022.tacl-1)
Copied to clipboard
| Challenge: | Prior work has identified deficiencies in their contextualized representation stemming from the underlying compositional paradigm of representation. |
| Approach: | They propose to use an adapter as a lightweight non-compositional language expert trained on idiomatic sentences to build idiomity into BART. |
| Outcome: | The proposed approach improves idiomaticity over baselines and up to 25% higher sequence accuracy on idiom processing tasks. |
Rolling the DICE on Idiomaticity: How LLMs Fail to Grasp Context (2025.acl-long)
Copied to clipboard
| Challenge: | Existing models fail to resolve idiomaticity when it depends on contextual understanding . idiom frequency influences performance but does not guarantee accurate interpretation. |
| Approach: | They propose a novel contrastive dataset to assess whether large language models can effectively leverage context to disambiguate idiomatic meanings. |
| Outcome: | The proposed model performs better on sentences deemed more likely by the model . collocational frequency and sentence probability influence performance but not accuracy . |
Can Large Language Models Understand Context? (2024.findings-eacl)
Copied to clipboard
Yilun Zhu, Joel Moniz, Shruti Bhargava, Jiarui Lu, Dhivya Piraviperumal, Site Li, Yuan Zhang, Hong Yu, Bo-Hsiang Tseng
| Challenge: | Existing evaluation methodologies for Large Language Models (LLMs) have been inadequate to evaluate their ability to understand contextual features. |
| Approach: | They propose a benchmark to assess large language models' ability to understand context by adapting existing datasets to suit their evaluation. |
| Outcome: | The proposed model performs better under the in-context learning pretraining scenario than state-of-the-art models. |
Idiomatic Expression Identification using Semantic Compatibility (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to localize idiomatic expressions have limited views of their generalizability to new idioms. |
| Approach: | They propose a multi-stage neural architecture to detect whether a sentence has an idiomatic expression and localize it when it occurs in a figurative sense. |
| Outcome: | The proposed model achieves state-of-the-art on three of the largest datasets with idiomatic expressions of varied syntactic patterns and degrees of non-compositionality. |
AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets are limited to providing the degree of idiomaticity of expressions along with the literal and, where applicable, (a single) non-literal interpretation of MWEs. |
| Approach: | They propose to use a dataset to test the effectiveness of a language model in generating representations of sentences containing idioms. |
| Outcome: | The proposed model performs reasonably well on the one-shot and few-shot scenarios, but there is scope for improvement in the zero-shot scenario. |
IEKG: A Commonsense Knowledge Graph for Idiomatic Expressions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on IE comprehension has focused on detecting idiomaticity, but this fails to account for IEs' non-compositionality. |
| Approach: | They construct a commonsense knowledge graph for figurative interpretations of IEs that can be used to convert PTLMs into knowledge models that encode and infer commonsensical knowledge related to IE use. |
| Outcome: | The proposed model can generalize to IEs unseen during training. |
ePiC: Employing Proverbs in Context as a Benchmark for Abstract Language Understanding (2022.acl-long)
Copied to clipboard
| Challenge: | Large language models have shown exciting progress on several NLP benchmarks . however, evaluating their ability for complex analogical reasoning remains under-explored . |
| Approach: | They propose a dataset of narratives for employing proverbs in context as a benchmark for abstract language understanding. |
| Outcome: | The proposed dataset provides fine-grained annotation of aligned spans between proverbs and narratives and contains minimal overlaps between narratives with proverb . the results show that large language models struggle on these tasks compared to humans, and these tasks pose multiple learning challenges. |
Negation, Coordination, and Quantifiers in Contextualized Language Models (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work has focused on specific tasks and on the learning outcome. |
| Approach: | They propose to decouple the weaknesses from specific tasks and focus on the embeddings per se and their mode of learning. |
| Outcome: | The proposed model can learn semantic constraints and how the context impacts their embeddings. |
Learning from Omission (P19-1)
Copied to clipboard
| Challenge: | a recent study shows that pragmatic reasoning improves language understanding systems . end-to-end training produces more accurate utterance interpretation models . |
| Approach: | They show that pragmatic reasoning can improve the quality of learned meanings . they draw pragmatic inferences from listening to what a speaker says and not to what they do . |
| Outcome: | The proposed model improves the quality of language understanding models when sparse data is used. |