Error Analysis of NLP Models and Non-Native Speakers of English Identifying Sarcasm in Reddit Comments (2024.lrec-main)
Copied to clipboard
| Challenge: | sarcasm detection remains an issue for both humans and natural language processing models . |
| Approach: | They analysed 300 comments from the FigLang 2020 Reddit Dataset and 39 non-native speakers of English to see if they were sarcastic. |
| Outcome: | The results show that the models and models have similar performance and weaknesses when the comments include political topics or are phrased as questions. |
Similar Papers
BESSTIE: A Benchmark for Sentiment and Sarcasm Classification for Varieties of English (2025.findings-acl)
Copied to clipboard
| Challenge: | despite large language models showing bias against non-mainstream varieties, there are no labeled datasets for sentiment analysis of English. |
| Approach: | They propose a benchmark for sentiment and sarcasm classification for three varieties of English . they manually annotate the datasets with sentiment and the sarcasmatic labels . |
| Outcome: | The proposed benchmark is based on a web-based content from Google Place reviews and Reddit comments. |
Sarcasm Detection is Way Too Easy! An Empirical Comparison of Human and Machine Sarcasm Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | sarcasm detection datasets focus on intended, rather than perceived sarcasm, but there is no comparison between human and machine performance. |
| Approach: | They collect author-annotated sarcasm datasets that focus on intended, rather than perceived sarcasticism . they compare human-level benchmarks to that of state-of-the-art sarkasmatic detection systems . |
| Outcome: | The proposed datasets compare human and machine performance on sarcastic tasks in English and Arabic. |
Representing Social Media Users for Sarcasm Detection (D18-1)
Copied to clipboard
| Challenge: | Existing annotated corpus of Reddit comments is limited by available annotation methods. |
| Approach: | They propose a Bayesian approach that directly represents authors’ propensities to be sarcastic and a dense embedding approach that can learn interactions between the author and the text. |
| Outcome: | The proposed approach performs better in homogeneous contexts, whereas the dense embeddings prove valuable in more diverse contexts. |
A Large Self-Annotated Corpus for Sarcasm (L18-1)
Copied to clipboard
| Challenge: | Existing datasets for sarcasm detection have unbalanced and self-annotated labels, allowing for learning in both balanced and unbalanciated label regimes. |
| Approach: | They introduce the Self-Annotated Reddit Corpus (SARC) which has 1.3 million sarcastic statements and many times more instances of non-sarcasm statements. |
| Outcome: | The proposed corpus has 1.3 million sarcastic statements and many more instances of non-sarcasm statements, allowing for learning in both balanced and unbalanced label regimes. |
A Survey in Automatic Irony Processing: Linguistic, Cognitive, and Multi-X Perspectives (2022.coling-1)
Copied to clipboard
| Challenge: | figurative language research has focused on sarcasm and irony, but there is still a gap in the field. |
| Approach: | They propose to review computational irony, cognitive science, and neural models of irony processing . they aim to encourage a balanced and equal research environment in figurative languages . |
| Outcome: | The proposed multi-X irony processing perspectives will provide an overview of computational irony, insights from linguisic theory and cognitive science, and interactions with downstream NLP tasks. |
‘Am I the Bad One’? Predicting the Moral Judgement of the Crowd Using Pre–trained Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on NLP touch upon moral contexts in text. |
| Approach: | They construct a dataset that can be used for moral judgement tasks on a popular reddit subreddit. |
| Outcome: | The proposed model passes moral judgements on posts from a popular reddit subreddit . it shows that the model can be fine tuned and improves across the datasets . |
Ruddit: Norms of Offensiveness for English Reddit Comments (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect offensive language have been limited by categorical labels . however, there are several challenges in the detection of such content . |
| Approach: | They analyze Reddit comments with fine-grained, real-valued offensiveness scores . they evaluate the ability of widely-used neural models to predict offensiveness . |
| Outcome: | The proposed method produces highly reliable offensiveness scores and can predict scores on reddit comments. |
SarcNet: A Multilingual Multimodal Sarcasm Detection Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Sarcasm is an implicit form of sarcasm, involving an intended meaning that contradicts the literal expression . human use conflict between factual information and a statement as cues to detect sarcasm . sarkasmatic analysis is challenging due to its implicit nature . |
| Approach: | They propose a multimodal sarcasm detection dataset that uses multiple modalities to detect sarcasm. |
| Outcome: | The proposed model improves on previous models based on a single label . human sarcasm cannot be detected using a unified label across multiple modalities . |
Perturbation Sensitivity Analysis to Detect Unintended Model Biases (D19-1)
Copied to clipboard
| Challenge: | Recent research shows that data-driven NLP models may inadvertently capture, reflect and sometimes amplify various social biases present in the language data they are trained on. |
| Approach: | They propose a generic evaluation framework that detects unintended model biases related to named entities and requires no new annotations or corpora. |
| Outcome: | The proposed framework detects unintended model biases related to named entities and requires no new annotations or corpora. |
An Ensemble of Humour, Sarcasm, and Hate Speechfor Sentiment Classification in Online Reviews (D19-55)
Copied to clipboard
| Challenge: | sarcasm, humor, hate speech, and sentiment are a complex language attribute . sentiment classification models are used for complex language understanding tasks . |
| Approach: | They propose a two-step model that extracts features pertaining to sarcasm, humour, hate speech, as well as sentiment from online reviews and feeds them to inform sentiment classification. |
| Outcome: | The proposed model improves on sarcasm, humor, hate speech and sentiment classification . it can be combined with other models to achieve similar results . |