Can ChatGPT’s Performance be Improved on Verb Metaphor Detection Tasks? Bootstrapping and Combining Tacit Knowledge (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for supervised metaphor detection are limited by their performance. |
| Approach: | They propose to use ChatGPT to detect most prevalent verb metaphors among metaphors . they use literal collocations of target verbs and subject-object pairs of verbs to detect them . |
| Outcome: | The proposed method achieves the best performance on the unsupervised verb metaphors detection task compared to existing unsupervised methods or direct prediction using ChatGPT. |
Similar Papers
Merely Judging Metaphor is Not Enough: Research on Reasonable Metaphor Detection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current metaphor detection tasks only provide labels without interpreting how to understand them. |
| Approach: | They propose to improve the current metaphor detection task by using mainstream Large Language Models. |
| Outcome: | The proposed model is based on the original sentence, target word, and usage . the model is then evaluated using manual evaluation . |
Improving Neural Metaphor Detection with Visual Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | a new method for metaphor detection uses text from visual datasets to identify words . a metaphor is a complex interaction between two terms, creating an "implicationcomplex" |
| Approach: | They propose a technique for sampling text from visual datasets to create a visibility word embedding. |
| Outcome: | The proposed method improves on previous approaches that use more complex neural networks and richer linguistic features for verb classification. |
Verifying Claims About Metaphors with Large-Scale Automatic Metaphor Identification (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing studies on metaphors have focused on a small number of examples, whereas few studies verify claims with large corpus. |
| Approach: | They propose to use a large corpus to verify existing claims about verb metaphors . they apply metaphor detection to sentences extracted from Common Crawl . |
| Outcome: | The proposed method identifies verb metaphors with lower concreteness, imageability, familiarity and more emotional and subjective sentences. |
Neural Metaphor Detection in Context (D18-1)
Copied to clipboard
| Challenge: | Existing models focus on limited forms of linguistic context, such as unigrams. |
| Approach: | They propose end-to-end neural models for detecting metaphorical word use in context . they show that bi-directional biLSTM models which operate on complete sentences work well . |
| Outcome: | The proposed models show that they can learn rich contextual word representations . they are compared to previous models which focused on limited linguistic context . |
On the Impact of Temporal Representations on Metaphor Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | State-of-the-art approaches for metaphor detection compare their literal - or core - meaning and their contextual meaning using neural networks. |
| Approach: | They propose to use temporal and static word embeddings to account for different representations of literal meanings to examine metaphor detection tasks. |
| Outcome: | The proposed method outperforms static methods but may provide representations of the core meaning of the metaphor too close to their contextual meaning, causing confusion. |
MetaPro 2.0: Computational Metaphor Processing on the Effectiveness of Anomalous Language Modeling (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for metaphor interpretation are slow due to lack of annotated datasets and effective pre-trained language models. |
| Approach: | They propose a large annotated dataset and a PLM for the metaphor interpretation task. |
| Outcome: | The proposed method improves on metaphor identification and interpretation with comparable baselines on the new dataset. |
Does GPT-3 Grasp Metaphors? Identifying Metaphor Mappings with Generative Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to detect whether natural language sequences are metaphoric or literal focus on detecting the transfer of knowledge structures to pre-trained language models. |
| Approach: | They propose to probe the ability of GPT-3 to detect metaphoric language and predict the metaphor’s source domain without any pre-set domains. |
| Outcome: | The proposed model generates the correct source domain for a new sample with an accuracy of 65.15% in English and 34.65% in Spanish. |
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023.findings-acl)
Copied to clipboard
Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, Jimmy Huang
| Challenge: | Currently, the evaluation of large language models (LLMs) such as ChatGPT in academic datasets is difficult due to the difficulty of evaluating the generative outputs produced by this model against the ground truth. |
| Approach: | They evaluate ChatGPT across 140 tasks and analyze 255K responses it generates in academic datasets. |
| Outcome: | The proposed model performs well on 140 tasks and generates 255K responses in these datasets. |
Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on metaphor processing have focused on single datasets and specific task settings, often using artificially constructed data through lexical replacement. |
| Approach: | They propose to evaluate the capabilities of Large Language Models (LLMs) in metaphor interpretation across multiple datasets, tasks, and prompt configurations. |
| Outcome: | The proposed frameworks are more realistic and efficient than current models and are more efficient than existing models. |
On the Generalization of Training-based ChatGPT Detection Methods (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that training-based methods are ineffective to detect LLM generated texts from unseen tasks or topics which are not collected during training. |
| Approach: | They propose to train classification models to distinguish LLMs from human texts by a distribution shift caused by prompts, text lengths, topics, and language tasks. |
| Outcome: | The proposed methods can detect LLMs from black-box models, but they suffer from distribution shifts due to a wide range of factors, including prompts, text lengths, topics, and language tasks. |