A (More) Realistic Evaluation Setup for Generalisation of Community Models on Malicious Content Detection (2024.findings-naacl)
Copied to clipboard
| Challenge: | despite the performance of community models for malicious content detection, misinformation and hate speech continue to propagate on social media networks. |
| Approach: | They propose a new evaluation setup for community models for malicious content detection based on a few-shot subgraph sampling approach to test generalisation of models using local explorations of a larger graph. |
| Outcome: | The proposed evaluation setup outperforms existing models on real-world graphs on a training graph. |
Similar Papers
How to Solve Few-Shot Abusive Content Detection Using the Data We Actually Have (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for abusive language detection are expensive and lack of knowledge about the target is a challenge. |
| Approach: | They propose to build models cheaply for a new target label set and/or language, using only a few training examples of the target domain. |
| Outcome: | The proposed model improves monolingually and across languages using existing datasets and only a few-shots of the target domain. |
CACL: Community-Aware Heterogeneous Graph Contrastive Learning for Social Media Bot Detection (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for social media bot detection neglect community structure and poor model generalization due to the relatively small scale of the dataset. |
| Approach: | They propose a framework that constructs social networks as heterogeneous graphs and uses community-aware modules to mine hard positive and hard negative samples for supervised graph contrastive learning. |
| Outcome: | The proposed framework outperforms baselines on three social media bot benchmarks. |
MetaAdapt: Domain Adaptive Few-Shot Misinformation Detection via Meta Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for misinformation detection are limited by data scarcity . existing methods fail to detect early-stage misinformation on emerging topics . |
| Approach: | They propose a meta learning based approach for domain adaptive few-shot misinformation detection that leverages limited target examples to provide feedback and guide the knowledge transfer from the source to the target domain. |
| Outcome: | The proposed method improves performance on real-world datasets with reduced parameters. |
On the Effectiveness of Sentence Encoding for Intent Detection Meta-Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies on few-shot intent detection have attempted to formulate the task as a meta-learning problem. |
| Approach: | They propose to modify a few-shot intent detection task to produce a non-trivially strong performance without further domain-specific adaptation. |
| Outcome: | The proposed model improves on the prototypical network variants with task-specific fine-tuning. |
Towards Realistic Few-Shot Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that few-shot relation classification models can be used to extract any relation of interest from a collection of text with only a few example instances. |
| Approach: | They propose to modify the training routine to encourage models to better discriminate between relations involving similar entity types. |
| Outcome: | The proposed models outperform human models on relation extraction tasks while relying on entity type information. |
Few-shot Event Detection: An Empirical Study and a Unified View (2023.acl-long)
Copied to clipboard
| Challenge: | Extensive studies have been carried out on fewshot event detection (ED) however, there are noticeable discrepancies among existing methods from three aspects. |
| Approach: | They propose a unified view of ED models and a better unified baseline for fair evaluation. |
| Outcome: | The proposed framework outperforms existing methods by a large margin on three datasets. |
Probing LLMs for hate speech detection: strengths and vulnerabilities (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent efforts to detect hateful or toxic language using large language models have not used explanation, additional context and victim community information in the detection process. |
| Approach: | They use different prompt variations, input information and victim community information to evaluate large language models in zero shot setting without adding any in-context examples. |
| Outcome: | The proposed models perform significantly better when included in the pipeline than baseline models. |
Do Models of Mental Health Based on Social Media Data Generalize? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing literature on the validity of proxy-based methods for annotating mental health status in social media has raised new concerns regarding their use in clinical applications. |
| Approach: | They explore the generalization ability of machine learning classifiers trained to detect depression in individuals across multiple social media platforms. |
| Outcome: | The proposed methods show that they can be used to train and analyze large datasets and that they are robust to large dataset sizes. |
Towards Realistic Few-Shot Relation Extraction: A New Meta Dataset and Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for few-shot relation extraction are not realistic due to the large amount of training data required. |
| Approach: | They propose a meta dataset for few-shot relation extraction based on existing supervised relation extraction datasets and a few-shot form of the TACRED dataset. |
| Outcome: | The proposed methods perform poorly on the few-shot relation extraction task. |
Using RL to Identify Divisive Perspectives Improves LLMs Abilities to Identify Communities on Social Media (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show improvements on Reddit and Twitter data . |
| Approach: | They propose to take advantage of Large Language Models (LLMs) to better identify user communities. |
| Outcome: | The proposed model improves on Reddit and Twitter data and tasks of community detection, bot detection, and news media profiling. |