Large Language Models are Few-Shot Training Example Generators: A Case Study in Fallacy Recognition (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on fallacy recognition is still in its early stages, with limited datasets available. |
| Approach: | They propose to use GPT3.5 to generate synthetic examples and explore prompt settings to improve the representation of the infrequent classes. |
| Outcome: | The proposed model improves on existing models and generates synthetic examples with GPT3.5. |
Similar Papers
Multitask Instruction-based Prompting for Fallacy Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Fallacies are used as seemingly valid arguments to support a position and persuade the audience about its validity. |
| Approach: | They propose to use instruction-based prompting to recognize 28 unique fallacies across datasets . they also analyze the effect of model size and prompt choice on model performance . |
| Outcome: | The proposed approach can recognize 28 unique fallacies across domains and genres. |
It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners (2021.naacl-main)
Copied to clipboard
| Challenge: | Pretraining ever-larger language models on massive corpora requires enormous amounts of compute. |
| Approach: | They propose to convert textual inputs into cloze questions that contain a task description . they also exploit unlabeled data to improve their performance . |
| Outcome: | The proposed model outperforms GPT-3 with PET/iPET with cloze questions and unlabeled data. |
Few-shot Learning with Multilingual Generative Language Models (2022.emnlp-main)
Copied to clipboard
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li
| Challenge: | Large-scale generative language models such as GPT-3 are competitive few-shot learners. |
| Approach: | They train multilingual generative language models on a corpus covering a diverse set of languages and study their few- and zero-shot learning capabilities. |
| Outcome: | The proposed model outperforms GPT-3 on 171 out of 182 directions with 32 training examples and surpasses the official supervised baseline in 45 directions. |
Large Language Models Are Better Logical Fallacy Reasoners with Counterargument, Explanation, and Goal-Aware Prompt Formulation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent large language models have demonstrated improved processing of complex language, but detecting logical fallacies remains a challenge. |
| Approach: | They propose a prompt formulation approach for logical fallacy detection that integrates contextual information into input text and queries for validity within the argument’s context. |
| Outcome: | The proposed approach improves over state-of-the-art models by 0.57 in F1-scores and 0.45 in fine-tuned models. |
Contrastive Learning for Prompt-based Few-shot Language Learners (2022.naacl-main)
Copied to clipboard
| Challenge: | a recent study has shown that GPT-3 fine-tuning models with limited examples is effective . a contrastive learning framework clusters inputs from the same class under different augmented “views” and repels those from different classes. |
| Approach: | They propose a supervised contrastive framework that clusters inputs from the same class under different augmented "views" they combine a contrastive loss with the standard masked language modeling loss in prompt-based few-shot learners . |
| Outcome: | The proposed framework improves on the state-of-the-art methods in a diverse set of 15 language tasks. |
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |
Are LLMs Good Zero-Shot Fallacy Classifiers? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing fallacy classifiers lack sufficient labeled data for training, limiting their out-of-distribution (OOD) generalization abilities. |
| Approach: | They propose to use Large Language Models (LLMs) for zero-shot fallacy classification. |
| Outcome: | The proposed schemes outperform existing classifiers in OOD inference scenarios and opendomain tasks. |
Reason from Fallacy: Enhancing Large Language Models’ Logical Reasoning through Logical Fallacy Understanding (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated good performance in many reasoning tasks, but struggle with some more complex reasoning tasks including logical reasoning. |
| Approach: | They propose five concrete tasks from three cognitive dimensions of WHAT, WHY, and HOW to evaluate LLMs’ capability of logical fallacy understanding. |
| Outcome: | The proposed dataset can be used to evaluate LLMs’ LFU capability and to fine-tune LLM models to obtain significantly enhanced performance on logical reasoning. |
Leveraging Synthetic Targets for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Using synthetic target data, training models on synthetic targets outperforms training on actual ground-truth data. |
| Approach: | They propose a recipe for training machine translation models on synthetic target data by leveraging a large pre-trained model. |
| Outcome: | The proposed model outperforms training on real-world translation datasets. |
Language Models for Text Classification: Is In-Context Learning Enough? (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing research on text classification models with prompts is limited in scale and lacks understanding of how these methods compare to more established methods. |
| Approach: | They compare the performance of large and smaller language models with prompts to achieve state-of-the-art performance in many NLP tasks. |
| Outcome: | The proposed models outperform the more standard approaches in binary, multiclass, and multilabel tasks in a large scale evaluation of 16 text classification datasets. |