Papers by Yida Mu
Enhancing Data Quality through Simple De-duplication: Navigating Responsible Computational Social Science Research (2024.emnlp-main)
Copied to clipboard
| Challenge: | Social media data exhibits distinctive characteristics such as rapid and continual topic evolution. |
| Approach: | They propose new protocols and best practices for improving dataset development from social media data and its usage. |
| Outcome: | The proposed protocols and best practices improve the performance of social media datasets and their usage. |
It’s about Time: Rethinking Evaluation on Rumor Detection Benchmarks using Chronological Splits (2023.findings-eacl)
Copied to clipboard
| Challenge: | Current rumor detection benchmarks use random splits as training, development and test sets which results in topical overlaps. |
| Approach: | They propose to use chronological rather than random splits for rumor classification . they propose to always use chronological splits to minimize topical overlaps . |
| Outcome: | The proposed model overestimates performance on four popular rumor detection benchmarks considering chronological instead of random splits. |
Large Language Models Offer an Alternative to the Traditional Approach of Topic Modelling (2024.lrec-main)
Copied to clipboard
| Challenge: | Topic modelling has found extensive use in automatically detecting significant topics within a corpus of documents, but there are certain drawbacks. |
| Approach: | They propose a framework that prompts large language models to generate topics from a given set of documents and establish evaluation protocols to assess the clustering efficacy of LLMs. |
| Outcome: | The proposed model generates relevant topic titles and adheres to human guidelines to refine and merge topics. |
Don’t waste a single annotation: improving single-label classifiers through soft labels (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for annotating data are limited by ambiguity and lack of context in data samples. |
| Approach: | They challenge the traditional approach of annotating data by only providing a single label for each sample and annotator disagreement is discarded . instead, they use additional annotation information such as confidence, secondary label and disagreement to generate soft labels. |
| Outcome: | The proposed method improves model performance and calibration on the hard label test set. |
Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science (2024.lrec-main)
Copied to clipboard
Yida Mu, Ben P. Wu, William Thorne, Ambrose Robinson, Nikolaos Aletras, Carolina Scarton, Kalina Bontcheva, Xingyi Song
| Challenge: | Existing instruction-tuned Large Language Models (LLMs) have impressive language understanding and the capacity to generate responses that follow specific prompts. |
| Approach: | They evaluate the zero-shot performance of two publicly accessible LLMs, ChatGPT and OpenAssistant, in the context of six Computational Social Science classification tasks. |
| Outcome: | The proposed LLMs perform better than state-of-the-art models on social science tasks. |
Examining Temporalities on Stance Detection towards COVID-19 Vaccination (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have highlighted the importance of vaccination as an effective strategy to control the transmission of the COVID-19 virus. |
| Approach: | They evaluate a range of transformer-based models using chronological and random splits of social media data to examine the impact of temporal concept drift on stance detection towards COVID-19 vaccination. |
| Outcome: | The proposed models show that the models performed better with chronological and random splits than with random split models. |
Examining the Limitations of Computational Rumor Detection Models Trained on Static Datasets (2024.lrec-main)
Copied to clipboard
| Challenge: | Past research has indicated that content-based rumor detection models perform less effectively on unseen rumors. |
| Approach: | They propose to use data split strategies to minimize the effects of temporal concept drift in static datasets during the training of rumor detection methods. |
| Outcome: | The proposed model over-relys on the information derived from the rumors’ source post and overlooks the significant role that contextual information can play. |