High-quality argumentative information in low resources approaches improve counter-narrative generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that fine-tuning improves the performance of language models . large language models generate acceptable texts in a number of scenarios, a study shows . |
| Approach: | They show that fine-tuning improves the task of hate speech counter-narrative generation . they provide a subset of arguments and a good base model is required for the fine-uning to have a positive impact. |
| Outcome: | The proposed model produces counter-narratives that are as satisfactory as the whole set. |
Similar Papers
Generating Counter Narratives against Online Hate Speech: Data and Strategies (2020.acl-main)
Copied to clipboard
| Challenge: | Hate Speech (HS) is a pervasive issue that spreads quickly and widely . research has focused on avoiding undesired effects that come with content moderation . |
| Approach: | They propose to use large scale unsupervised language models to generate responses to hate effectively using large scale models. |
| Outcome: | The proposed methods lack quality data and produce generic/repetitive responses. |
Leveraging Pre-existing Resources for Data-Efficient Counter-Narrative Generation in Korean (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets and methods for detecting hate speech are limited by resource-intensive nature and only focus on the primary language. |
| Approach: | They propose a Korean Hate Speech Counter Punch (KHSCP) method that generates fact-based responses to hate speech in the Korean language and propose to use existing resources to overcome data scarcity. |
| Outcome: | The proposed method can overcome data scarcity in low-resource environments by leveraging existing resources. |
Data-Efficient Strategies for Expanding Hate Speech Detection into Under-Resourced Languages (2022.emnlp-main)
Copied to clipboard
| Challenge: | Hate speech datasets focus on English-language content, hindering effective models . annotating hateful content is expensive, time-consuming and potentially harmful to annotators. |
| Approach: | They propose to use ISO 639-1 codes to fine-tune models on one source language and apply them to another language. |
| Outcome: | The proposed approach performs well on some tasks, but fails on many others. |
Selecting Informative Contexts Improves Language Model Fine-tuning (2021.acl-long)
Copied to clipboard
| Challenge: | Language model fine-tuning is computationally expensive and time-consuming . however, the inclusion of training examples that negatively affect performance is limited . |
| Approach: | They propose a general fine-tuning method that incorporates information gain filtration . they propose to release pre-trained secondary learners on common corpora to promote efficient fine-uning. |
| Outcome: | The proposed method achieves a median perplexity of 54.0 on a books dataset compared to 57.3 for standard fine-tuning. |
IMHO Fine-Tuning Improves Claim Detection (N19-1)
Copied to clipboard
| Challenge: | Empirical results show that using this approach improves the state of art performance across four benchmark argumentation data sets by an average of 4 absolute F1 points in claim detection. |
| Approach: | They propose to fine-tune a language model using a Reddit corpus of opinionated claims and use the internet acronyms IMO/IMHO to identify claims. |
| Outcome: | The proposed approach improves state of art performance across four benchmark argumentation data sets by an average of 4 absolute F1 points. |
Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to combat online hatred using informed textual responses - called counter narratives - produce generic/repetitive responses and lack grounded and up-to-date evidence such as facts, statistics, or examples. |
| Approach: | They propose to automatically generate counter narratives using an external knowledge repository to provide more informative content to fight online hatred. |
| Outcome: | The proposed pipeline can generate suitable and informative counter narratives in in-domain and cross-domain settings. |
Fine-tuning with HED-IT: The impact of human post-editing for dialogical language models (2024.findings-acl)
Copied to clipboard
Daniela Occhipinti, Michele Marchi, Irene Mondella, Huiyuan Lai, Felice Dell’Orletta, Malvina Nissim, Marco Guerini
| Challenge: | a recent study has focused on the quality of data generated by automatic methods for fine-tuning Language Models in languages less resourced than English. |
| Approach: | They investigate whether human intervention improves the quality of machine-generated dialogues . they use a large-scale dataset to fine-tune three different sizes of an LM . |
| Outcome: | The results show that human intervention can improve the quality of training data . larger models are less sensitive to data quality, while smaller models are more sensitive . |
Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on generating hate speech/counter narratives have failed to reach high-quality datasets. |
| Approach: | They propose a human-in-the-loop data collection methodology that refines a generative language model iteratively by using its own data from previous loops to generate new training samples. |
| Outcome: | The proposed method is the only expert-based multi-target HS/CN dataset available to the community. |
XAutoLM: Efficient Fine-Tuning of Language Models via Meta-Learning and AutoML (2025.emnlp-main)
Copied to clipboard
Ernesto Luis Estevanell Valladares, Suilan Estevez-Velarde, Yoan Gutierrez, Andrés Montoyo, Ruslan Mitkov
| Challenge: | XAutoLM is a meta-learning-augmented framework that can be used to optimize discriminative and generative LM fine-tuning pipelines. |
| Approach: | They propose a meta-learning-augmented AutoML framework that reuses past experiences to optimize discriminative and generative LM fine-tuning pipelines efficiently. |
| Outcome: | XAutoLM surpasses zero-shot optimizer’s peak F1 on five of six tasks, reduces mean evaluation time of pipelines by up to 4.5x, and uncovers 50% more pipelines above zero- shot Pareto front. |
Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide (2026.tacl-1)
Copied to clipboard
| Challenge: | Pre-trained language models provide strong foundations, but effective adaptation under data scarcity requires efficient and efficient fine-tuning techniques. |
| Approach: | They propose to review parameter-efficient fine-tuning techniques that lower training and deployment costs and domain and cross-lingual adaptation methods for both encoder and decoder models. |
| Outcome: | The proposed techniques lower training and deployment costs, domain and cross-lingual adaptation methods, and model specialization strategies. |