| Challenge: | a new method to characterize, quantify and measure the impact of hard instances is proposed . a method to label hard instances can shed light on why and when classifiers fail, authors say . |
| Approach: | They propose a method to characterize, quantify and measure the impact of hard instances in polarity classification of movie reviews. |
| Outcome: | The proposed method can quantify the impact of hard instances in polarity classification . it can shed light on why and when classifiers fail, the authors say . |
Similar Papers
Benchmarks and models for entity-oriented polarity detection (N18-3)
Copied to clipboard
| Challenge: | a dataset of 17,000 manually labeled documents is large for determining entity-oriented polarity in business news. |
| Approach: | They propose a convolutional neural network-based approach to classify entity-oriented polarity in business news. |
| Outcome: | The proposed model is based on convolutional neural networks and is small on the scale of existing models. |
Transformers in the loop: Polarity in neural models of language (2022.acl-long)
Copied to clipboard
| Challenge: | Recent Transformer-based language representation models (LRMs) show impressive results on practical text analysis tasks, but do they have access to complex linguistic notions? |
| Approach: | They propose to use polarity as a case study to compare metrics derived from language models to human judgments obtained in psycholinguistic experiments. |
| Outcome: | The proposed model is more accurate than linguistic theory predictions for polarity, and allows us to use language models to discover new insights into natural language grammar beyond existing linguistic theories. |
Re-Examining FactBank: Predicting the Author’s Presentation of Factuality (2022.coling-1)
Copied to clipboard
| Challenge: | Previously published results on FactBank are no longer valid. |
| Approach: | They propose to correct a subset of FactBank data to improve performance . they use multiple training paradigms, data smoothing techniques, and polarity classifiers . |
| Outcome: | The proposed model improves performance on the FactBank dataset. |
Polarized Opinion Detection Improves the Detection of Toxic Language (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for estimating polarized annotations are un-normalized and difficult to exploit in machine learning. |
| Approach: | They propose a method for K-class text classification that exploits polarized texts in the dataset. |
| Outcome: | The proposed method exploits polarized texts in a dataset and can improve classification performance. |
Language Models Use Monotonicity to Assess NPI Licensing (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural language models (LMs) have become powerful approximators of human language . fewer studies have been done on what kind of formal semantic features are encoded by LMs . |
| Approach: | They propose a series of experiments that investigate the semantic knowledge of language models . they use diagnostic classifiers, linguistic acceptability tasks and a ranking method to investigate the models' inner workings. |
| Outcome: | The proposed method can be applied to LMs trained on filtered corpora and gain stronger insights into their generalizations. |
From Polarity to Intensity: Mining Morality from Semantic Space (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to compute moral intensity are limited to word-level measurement and heavily rely on human labelling. |
| Approach: | They propose a weakly-supervised framework that can automatically measure moral intensity from text. |
| Outcome: | The proposed framework can measure moral intensity from text with moral polarity labels, which are more robust and easier to acquire. |
Biasly: An Expert-Annotated Dataset for Subtle Misogyny Detection and Mitigation (2024.findings-acl)
Copied to clipboard
Brooklyn Sheppard, Anna Richter, Allison Cohen, Elizabeth Smith, Tamara Kneese, Carolyne Pelletier, Ioana Baldini, Yue Dong
| Challenge: | the Biasly dataset captures misogyny in movies in ways unique within the literature. |
| Approach: | The Biasly dataset captures misogyny in North American film by combining annotations of movie subtitles with common NLP algorithms. |
| Outcome: | The Biasly dataset captures misogyny expressions in North American film . it contains annotations of movie subtitles and text generation for rewrites . |
Polar Quantification of Actor Noun Phrases for German (2022.lrec-1)
Copied to clipboard
| Challenge: | Polanyi and Zaenen (2006) focused on the negative polar load of noun phrases, especially those denoting actors. |
| Approach: | They propose a method to measure the negative polar load of noun phrases by using a silver standard and a BERT-based intensity regressor. |
| Outcome: | The proposed model is based on a lexicon-based silver standard and tested empirically. |
On Evaluation of Document Classification with RVL-CDIP (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing document classification benchmarks have label noise, ambiguous documents, and sensitive information. |
| Approach: | They argue that RVL-CDIP is unsuitable for benchmarking document classifiers . they advocate for a new document classification benchmark with ambiguous labels . |
| Outcome: | The RVL-CDIP benchmark is widely used for document classification . the authors argue that its limited scope, presence of errors and lack of diversity make it less than ideal for benchmarking. |
POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization (2026.findings-acl)
Copied to clipboard
Usman Naseem, Robert Geislinger, Juan Ren, Sarah Kohail, Rudy Alexandro Garrido Veliz, P Sam Sahil, Yiran Zhang, Idris Abdulmumin, Marco Antonio Stranisci, Özge Alacam, Cengiz Acarturk, Aisha Jabr, Saba Anwar, Abinew Ali Ayele, Simona Frenda, Alessandra Teresa Cignarella, Elena Tutubalina, Oleg Rogov, Aung Kyaw Htet, Xintong Wang, Surendrabikram Thapa, Kritesh Rauniyar, Tanmoy Chakraborty, MD Arfeen Zeeshan, Dheeraj Kodati, Satya Keerthi, Sahar Moradizeyveh, Firoj Alam, Md Arid Hasan, Syed Ishtiaque Ahmed, Ye Kyaw Thu, Shantipriya Parida, Ihsan Ayyub Qazi, Lilian Diana Awuor Wanzare, Nelson Odhiambo Onyango, Clemencia Siro, Jane Wanjiru Kimani, Ibrahim Said Ahmad, Adem Chanie Ali, Martin Semmann, Chris Biemann, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam
| Challenge: | polarization is a pervasive threat to democratic institutions, civil discourse, and social cohesion worldwide . most existing datasets focus on English or high-resource languages, reflecting a widespread trend across NLP tasks . |
| Approach: | They propose a multilingual, multicultural, and multi-event dataset with over 110K instances in 22 languages drawn from diverse online platforms and real-world events. |
| Outcome: | The proposed dataset analyzes polarization detection, type, and manifestation using a variety of annotation platforms adapted to each cultural context. |