Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models (2023.findings-acl)
Copied to clipboard
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z. HaoChen, James Zou, Percy Liang, Serena Yeung
| Challenge: | Recent studies show that some tasks exhibit inverse scaling, or U-shaped scaling, where the performance degrades as models are scaled up. |
| Approach: | They propose a task that asks questions with negation to show positive scaling . they hypothesize that solving NeQA depends on question answering and negation understanding . |
| Outcome: | The proposed task can exhibit inverse scaling, U-shaped scaling, or positive scaling, and the scaling trends shift as the task is more powerful. |
Similar Papers
Towards the Roots of the Negation Problem: A Multilingual NLI Dataset and Model Scaling Analysis (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Negations are key to determining sentence meaning, making them essential for logical reasoning. |
| Approach: | They construct and publish two new textual entailment datasets in four languages with paired examples differing in negation. |
| Outcome: | The results show that increasing the model size may improve the models’ ability to handle negations. |
Inverse Scaling Can Become U-Shaped (2023.emnlp-main)
Copied to clipboard
| Challenge: | Scaling up language models has been shown to improve performance on a wide range of downstream tasks, but are there any tasks for which performance gets worse as models scale? |
| Approach: | They evaluate models trained on five times more compute and evaluated them on 280B parameters and 500 zettaFLOPs of training compute. |
| Outcome: | The proposed tasks show that performance decreases as models scale and increases again as models get larger. |
NegVQA: Can Vision Language Models Understand Negation? (2025.findings-acl)
Copied to clipboard
| Challenge: | NegVQA is a visual question answering (VQA) benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions. |
| Approach: | They propose a visual question answering benchmark consisting of 7,379 two-choice questions covering diverse negation scenarios and image-question distributions. |
| Outcome: | The proposed model fails to correctly interpret negation, leading to critical errors in interactive AI systems. |
This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have grammatical knowledge but fail to interpret negation . a recent study shows that LLMs struggle with negative sentences . |
| Approach: | They propose to use a dataset to grasp LLMs' generalization and inference capability . they also fine-tuned models to assess whether the understanding of negation can be trained . |
| Outcome: | The proposed model is able to generalize and infer negation in 400,000 sentences . but it is suboptimal when it comes to negation, a key step in natural language processing . |
Making Language Models Robust Against Negation (2025.naacl-long)
Copied to clipboard
| Challenge: | Negation is a semantic phenomenon that alters an expression to convey the opposite meaning. |
| Approach: | They propose a self-supervised method to make language models more robust against negation by pre-training models. |
| Outcome: | The proposed task outperforms the off-the-shelf versions on nine negation-related benchmarks. |
Pitfalls of Scale: Investigating the Inverse Task of Redefinition in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable results in several linguistic, reasoning and knowledge retrieval tasks. |
| Approach: | They propose to scale Large Language Models (LLMs) to scale up to reveal potential reasoning gaps as LLMs scale up. |
| Outcome: | The proposed redefinition task shows that model performance degrades with scale, and false confidence rises. |
Understanding by Understanding Not: Modeling Negation in Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Negation is a core construction in natural language, but state-of-the-art pre-trained language models often handle it incorrectly. |
| Approach: | They propose to augment language modeling objective with unlikelihood objective based on negated generic sentences from a raw text corpus. |
| Outcome: | The proposed approach reduces the top 1 error rate to 4% on negated LAMA dataset and improves on negating NLI benchmarks. |
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Negation is fundamental to human communication. |
| Approach: | They propose a dataset which requires reasoning about implications of negated statements in paragraphs . they collect paragraphs with diverse negation cues and crowdworkers ask questions about implications . |
| Outcome: | The first dataset in english requires reasoning about implications of negated statements in paragraphs . it features 14,182 question-answer pairs with over 200 unique negation cues based on crowd-workers . the best performing model achieves only 42% on consistency metric, well below human performance of 81%. |
Paraphrasing in Affirmative Terms Improves Negation Understanding (2024.acl-short)
Copied to clipboard
| Challenge: | Negation is a common linguistic phenomenon in human languages . however, language models face challenges with negation in many tasks . |
| Approach: | They propose to incorporate affirmative interpretations into models to make them more robust against negation. |
| Outcome: | The proposed models are more robust against negation when negation is present in input . the proposed models can be used to analyze large corpus and natural language understanding tasks . |
Developmental Negation Processing in Transformer Language Models (2022.acl-short)
Copied to clipboard
| Challenge: | Negation is an important construct in language for reasoning over the truth of propositions, garnering interest from philosophy (Horn, 1989) and psycholinguistics (Zwaan, 2012). |
| Approach: | They propose to frame a natural language inference task as a problem and examine how well transformers can process negation categories. |
| Outcome: | The proposed models perform better on certain categories, suggesting clear differences in how they are processed. |