Papers by Kanishka Misra
Language model acceptability judgements are not always robust to context (2023.acl-long)
Copied to clipboard
Koustuv Sinha, Jon Gauthier, Aaron Mueller, Kanishka Misra, Keren Fuentes, Roger Levy, Adina Williams
| Challenge: | a recent study found that models prefer acceptable inputs over acceptable ones. |
| Approach: | They find that model judgements are generally robust when placed in randomly sampled linguistic contexts, but unstable when contexts match the test stimuli in syntactic structure. |
| Outcome: | The proposed model performance improves when contexts match syntactic structure, and declines when they are unacceptable. |
Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods that decompose multi-hop questions into single hop sub-questions are difficult to implement. |
| Approach: | They propose to use random-walks to guide pre-trained language models to map multi-hop questions to random-walked paths that lead to the answer. |
| Outcome: | The proposed methods improve on two T5 LMs. |
Hey, wait a minute: on at-issue sensitivity in Language Models (2026.eacl-short)
Copied to clipboard
| Challenge: | Existing methods to evaluate dialogue naturalness are limited. |
| Approach: | They propose a method to assess dialogue naturalness using linguistic notion of at-issueness. |
| Outcome: | The proposed method mitigates bias in linguistic analyses of LMs and tests discourse-sensitive behavior. |
Exploring BERT’s Sensitivity to Lexical Cues using Tests from Semantic Priming (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Using English lexical stimuli, we find that BERT models show "priming" predicting a word with greater probability when the context includes a related word versus an unrelated one. |
| Approach: | They analyze a pre-trained BERT model with tests informed by semantic priming . they find that BERT too shows "priming" predicting a word with greater probability when context includes a related word versus an unrelated one. |
| Outcome: | The proposed model shows a tendency to be distracted by related prime words as context becomes more informative, and lower probability of related words. |
COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models (PLMs) lack robustness in demonstrating simple reasoning, despite having the prerequisite knowledge. |
| Approach: | They propose to test pre-trained language models' ability to attribute properties to concepts and their ability to demonstrate property inheritance behavior. |
| Outcome: | The proposed model can easily distinguish between concepts on the basis of a property when they are trivially different, but find it relatively difficult when concepts are related on the base of nuanced knowledge representations. |
Bears, all bears, and some bears. Language Constraints on Language Models’ Inductive Inferences (2026.findings-acl)
Copied to clipboard
| Challenge: | Language places subtle constraints on how we make inductive inferences. |
| Approach: | They propose to use language to constrain inductive inferences by replicating an experiment . they find subtle differences arise in general purpose statistical learners like VLMs . |
| Outcome: | The proposed model can be used to extend inductive inferences to humans using language . the model can extend properties of a category to other members of the population, the authors show . |
Wugnectives: Novel Entity Inferences of Language Models from Discourse Connectives (2026.eacl-long)
Copied to clipboard
| Challenge: | Using context + knowledge of discourse connectives to make predictions about discourse connective . |
| Approach: | They present a dataset of 8,880 stimuli that evaluates LMs’ inferences about novel entities in contexts where connectives link the entities to particular attributes. |
| Outcome: | The proposed dataset evaluates LMs’ inferences about new entities in contexts where connectives link the entities to particular attributes. |
Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs (2025.findings-acl)
Copied to clipboard
William Berkeley Sheffield, Kanishka Misra, Valentina Pyatkin, Ashwini Deo, Kyle Mahowald, Junyi Jessy Li
| Challenge: | Discourse particles are crucial elements that subtly shape the meaning of text. |
| Approach: | They examine the capacity of linguists to distinguish fine-grained senses of English *just* . they find that they struggle to fully capture more subtle nuances of discourse particles . |
| Outcome: | The study shows that linguists struggle to capture subtle nuances of discourse particles. |
Characterizing the Role of Similarity in the Property Inferences of Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Property inheritance is a phenomenon where novel properties are projected from higher level categories to lower level ones. |
| Approach: | They investigate how LMs perform property inheritance with behavioral and causal analysis experiments. |
| Outcome: | The results provide insight into the conceptual structure of language models and may suggest new psycholinguistic experiments for human subjects. |
Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent zero-shot evaluations have highlighted important limitations in the abilities of language models (LMs) to perform meaning extraction. |
| Approach: | They propose to use in-context examples and instructions to improve LMs' robustness in performing property inheritance. |
| Outcome: | The proposed model can perform non-trivial property inheritance on in-context examples and instructions, but it is inconsistent with the task. |
Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language models learn rare syntactic phenomena by generalization vs. memorization, a study finds . aannalysis experiments show that humans learn rare grammatical structures by generalizing from less rare phenomena. |
| Approach: | They iteratively trained transformer language models on a systematically manipulated corpus and evaluated their learning of a rare grammatical phenomenon. |
| Outcome: | The results show that language models learn rare grammatical phenomena by generalization vs. memorization . human-scale corpora are used to train the models and compare their learning to counterfactual corpors . |
Cross-Modal Taxonomic Generalization in (Vision-) Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that language models learn from surface form to learn from more grounded evidence. |
| Approach: | They propose to use a vision-language model to learn hypernyms from images . they find that the model can recover this knowledge and generalize even when there is no hypernomia in the image. |
| Outcome: | The proposed model can recover this knowledge and generalize even when the model receives no evidence of hypernyms during training. |