How Furiously Can Colorless Green Ideas Sleep? Sentence Acceptability in Context (2020.tacl-1)
Copied to clipboard
| Challenge: | a recent study shows that context affects our perception of sentence acceptability, but few studies investigate how it affects language models. |
| Approach: | They compare acceptability ratings of sentences judged in isolation with a relevant context and with an irrelevant context. |
| Outcome: | The proposed model achieves state-of-the-art for unsupervised acceptability prediction. |
Similar Papers
The Influence of Context on Sentence Acceptability Judgements (P18-2)
Copied to clipboard
| Challenge: | a paper examining the influence of document context on acceptability judgements for English sentences is published in journal journal of linguistics. |
| Approach: | They propose to use document context to assess acceptability judgements for English sentences . they also test the accuracy of neural models that incorporate document context during training . |
| Outcome: | The proposed model improves acceptability ratings for ill-formed sentences, but reduces them for well-formed ones. |
Assessing the Effect of Context in Multi-domain Acceptability Judgment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies evaluate sentences in isolation and do not consider how context influences LLM acceptability judgments. |
| Approach: | They examine how contextual cues affect model-generated acceptability ratings across multiple domains and several LLMs, using different forms of domain-specific contextual cueeds to situate sentences in intended usage settings. |
| Outcome: | The findings support the development of more context-aware evaluation frameworks. |
Language model acceptability judgements are not always robust to context (2023.acl-long)
Copied to clipboard
Koustuv Sinha, Jon Gauthier, Aaron Mueller, Kanishka Misra, Keren Fuentes, Roger Levy, Adina Williams
| Challenge: | a recent study found that models prefer acceptable inputs over acceptable ones. |
| Approach: | They find that model judgements are generally robust when placed in randomly sampled linguistic contexts, but unstable when contexts match the test stimuli in syntactic structure. |
| Outcome: | The proposed model performance improves when contexts match syntactic structure, and declines when they are unacceptable. |
Predicting Reference: What do Language Models Learn about Discourse Models? (2020.emnlp-main)
Copied to clipboard
| Challenge: | a growing literature that probes neural language models to assess their latent acquisition of grammatical knowledge has not investigated their acquisition of discourse modeling ability. |
| Approach: | They draw on a psycholinguistic literature that has established how different contexts affect referential biases concerning who is likely to be referred to next. |
| Outcome: | The proposed models do not resemble human language users, the authors show . their models capture the linguistic knowledge required to perform discourse modeling . |
Role of Context in Unsupervised Sentence Representation Learning: the Case of Dialog Act Modeling (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Unsupervised learning of word representations involves capturing the contextual information surrounding word occurrences. |
| Approach: | They propose to use text-based dialog act tags to compare content- and context-oriented sentence representations inferred on telephone conversations to examine whether a contextual signal is of any significant benefit to general-purpose sentence representation. |
| Outcome: | The proposed model outperforms content-based and context-oriented representations on telephone conversations and shows that it increases the dimensionality of the vectors. |
Don’t Take This Out of Context!: On the Need for Contextual Models and Evaluations for Stylistic Rewriting (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing stylistic text rewriting methods ignore the context of the text, causing generic, incoherent, and generic outputs. |
| Approach: | They propose a contextual evaluation metric that integrates preceding context into stylistic text rewriting. |
| Outcome: | The proposed metric integrates the preceding textual context into rewriting and evaluation stages . human preferences are better reflected by the proposed criterio and other metrics . |
Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predicting Quantifiers (P18-2)
Copied to clipboard
| Challenge: | cloze deletion test is a test that requires the learner to understand the context and vocabulary in order to identify the correct word. |
| Approach: | They collect data from human participants and test various models in a local and a global context condition to examine the role of linguistic context in predicting quantifiers. |
| Outcome: | The proposed models outperform humans in a local and global context and are only slightly better in the latter. |
A large-scale study of the effects of word frequency and predictability in naturalistic reading (N19-1)
Copied to clipboard
| Challenge: | Recent studies have shown separable effects of word frequency and predictability on human sentence processing . other theories hold that apparent effects of frequency are underlyingly effects of predictability . |
| Approach: | They examine the generalizability of this finding to more realistic conditions of sentence processing by studying effects of frequency and predictability in three large-scale naturalistic reading corpora. |
| Outcome: | The results show that word frequency and predictability are significant in isolation but not over and above predictability, and raise doubts about the existence of such a distinction in everyday sentence comprehension. |
Rolling the DICE on Idiomaticity: How LLMs Fail to Grasp Context (2025.acl-long)
Copied to clipboard
| Challenge: | Existing models fail to resolve idiomaticity when it depends on contextual understanding . idiom frequency influences performance but does not guarantee accurate interpretation. |
| Approach: | They propose a novel contrastive dataset to assess whether large language models can effectively leverage context to disambiguate idiomatic meanings. |
| Outcome: | The proposed model performs better on sentences deemed more likely by the model . collocational frequency and sentence probability influence performance but not accuracy . |
Contextualized Word Representations for Reading Comprehension (N18-2)
Copied to clipboard
| Challenge: | Reading comprehension (RC) is a high-level task in natural language understanding that requires reading a document and answering questions about its content. |
| Approach: | They propose to provide a standard neural network for reading a document and answering a question about its content. |
| Outcome: | The proposed model improves on the competitive SQuAD dataset by providing rich contextualized word representations and allowing it to choose between context-dependent and context-independent representations. |