Talking the Talk Does Not Entail Walking the Walk: On the Limits of Large Language Models in Lexical Entailment Recognition (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Verbs are crucial for expressing actions and relationships between entities, making it essential to properly capture their nuances. |
| Approach: | They propose to use prompting strategies and zero-shot prompting to recognize entailment relations among verbs from two lexical databases, WordNet and HyperLex. |
| Outcome: | The proposed models can tackle the lexical entailment recognition task with moderately good performance, although at varying degree of effectiveness and under different conditions. |
Similar Papers
Traffic Light or Light Traffic? Investigating Phrasal Semantics in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Phrases are fundamental linguistic units through which humans convey semantics. |
| Approach: | They assess the capacity of API-based large language models to comprehend phrase semantics . they use three human-annotated datasets to analyze their results . |
| Outcome: | The proposed model outperforms embedding-based methods in phrase semantic reasoning tasks . the proposed model does not show significant advantage over fine-tuned methods . |
Lexical Semantics with Large Language Models: A Case Study of English “break” (2023.findings-eacl)
Copied to clipboard
| Challenge: | Large neural language models (LLMs) can be powerful tools for research in lexical semantics. |
| Approach: | They argue that large neural language models can be powerful tools for research in lexical semantics by capturing known sense distinctions and identifying informative new sense combinations. |
| Outcome: | The proposed models capture many of the sense distinctions found in the English verb break and can be used to identify informative new sense combinations for further analysis. |
It is not a piece of cake for GPT: Explaining Textual Entailment Recognition in the presence of Figurative Language (2025.coling-main)
Copied to clipboard
| Challenge: | Figure-based language is used to convey opinions, ideas, or emotions in texts and dialogues. |
| Approach: | They evaluate the capabilities of Large Language Models to address TER and generate textual explanations of TER predictions. |
| Outcome: | The proposed model outperforms the open-source models in Zero- and Few-Shot Learning settings and shows significant performance improvements. |
Unveiling the Limits of Large Language Models in Inferring Pragmatic Meaning from Non-Verbal Responses (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused mainly on LLMs' comprehension of verbal behavior, with non-verbal behavior considered only in conjunction with verbal responses. |
| Approach: | They present the first systematic evaluation of LLMs’ ability to infer pragmatic meaning in dialogue consisting solely of non-verbal responses. |
| Outcome: | The proposed model fails to capture non-verbal intent and has accuracy dropping by 60% compared to verbal ones. |
When Large Language Models Meet Speech: A Survey on Integration Approaches (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models have spurred interest in expanding their application beyond text-based tasks. |
| Approach: | They propose to categorize the integration of speech with LLMs into three main approaches . they demonstrate how these methods are applied across various speech-related applications . |
| Outcome: | The proposed methods are applied across speech-related applications and highlight the challenges in this field to offer inspiration for future research. |
LVLMs are Bad at Overhearing Human Referential Communication (2025.emnlp-main)
Copied to clipboard
| Challenge: | a crucial skill for embodied AI agents working with humans is grounding in referential communication. |
| Approach: | They use large vision language models to overhear spontaneous conversations between humans . they find that current LVLMs fail to show consistent performance improvement . |
| Outcome: | The proposed models fail to show consistent performance improvement over previous models . the authors release the results to facilitate future research . |
Assessing the Capabilities of Large Language Models in Coreference: An Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are a new approach to coreference resolution, but their performance is not yet fully understood. |
| Approach: | They propose that future efforts should improve scope, data, and evaluation methods of traditional coreference research to adapt to the development of LLMs. |
| Outcome: | The proposed methods improve scope, data, and evaluation methods of traditional coreference research to adapt to the development of LLMs. |
Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogism (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) take advantage of step-by-step reasoning instructions . negation is a core linguistic phenomenon that is difficult to process . |
| Approach: | They examine the step-by-step reasoning ability of large language models with a focus on negation . negation is a core linguistic phenomenon that is difficult to process . |
| Outcome: | The proposed models perform better when using chain-of-thought prompting . the results highlight unique limitations in each LLM family . |
How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are impressive in solving tasks, but they can quickly be outdated after deployment. |
| Approach: | They provide a review of recent advances in aligning deployed large language models with the ever-changing world knowledge. |
| Outcome: | The proposed models can be used to perform various tasks directly through in-context learning or for further fine-tuning for domain-specific uses. |
Probing Pretrained Language Models for Lexical Semantics (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on morphosyntactic, semantic, and world knowledge, but it remains unclear to what extent LMs derive lexical type-level knowledge from words in context. |
| Approach: | They propose to use multilingual and monolingual LMs to extract lexical type-level knowledge from words in context. |
| Outcome: | The proposed models perform well across six typologically diverse languages and five lexical tasks. |