Papers by Niranjan Balasubramanian
CaT-Bench: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on reasoning in plans focus on classical problems, simulated environments, or restricted language such as PDDL, but real-world plans cannot be tested to test for correctness and reliability. |
| Approach: | They propose a benchmark question that tests whether a step must necessarily occur before or after another in cooking recipe plans. |
| Outcome: | The proposed question-driven evaluation shows that SOTA LLMs are underwhelming and biased towards predicting dependence more often, but the best F1 result is 0.73. |
Toward Diverse Precondition Generation (2021.starsem-1)
Copied to clipboard
| Challenge: | a typical goal for language understanding is to logically connect the events of a discourse, but connective events are not described due to their commonsense nature. |
| Approach: | They propose a system that generates unique and diverse preconditions by using an event sampler, candidate generator, and post-processor. |
| Outcome: | The proposed system can generate unique and diverse preconditions without training on diverse examples. |
Evaluation of LLMs-based Hidden States as Author Representations for Psychological Human-Centered NLP Tasks (2025.findings-naacl)
Copied to clipboard
| Challenge: | Many human-centered NLP tasks focus on assessing human-attributes of a user based on their language. |
| Approach: | They evaluate different ways of representing documents and users using different LM and HuLM architectures to predict task outcomes as dynamically changing states and averaged trait-like user-level attributes. |
| Outcome: | The proposed representations predict valence, arousal, empathy, and distress as well as trait-like user-level attributes. |
Using Commonsense Knowledge to Answer Why-Questions (2022.emnlp-main)
Copied to clipboard
Yash Kumar Lal, Niket Tandon, Tanvi Aggarwal, Horace Liu, Nathanael Chambers, Raymond Mooney, Niranjan Balasubramanian
| Challenge: | Existing approaches to integrating commonsense knowledge into large language models are implicit and explicit. |
| Approach: | They analyze the effects of model size and methods of injecting knowledge into TellMeWhy datasets to determine what aspects of commonsense knowledge are available in large language models. |
| Outcome: | The largest models yield substantial improvements over base models, but the amount of improvement decreases with larger model size. |
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | DeFormer is a transformer-based QA model that uses input-wide self-attention at all layers, causing them to be slow and memory-intensive. |
| Approach: | They propose a transformer which substitutes the full self-attention with question-wide and passage-wide self- attentions in the lower layers. |
| Outcome: | The proposed model can be used to speed up QA by over 4.3x and with simple distillation-based losses they incur only a 1% drop in accuracy. |
Human Language Modeling (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing language modeling models treat text sequences as if they were created independently. |
| Approach: | They propose a hierarchical extension to the language modeling problem whereby a human-level exists to connect sequences of documents and capture the notion that human language is moderated by changing human states. |
| Outcome: | The proposed model outperforms the current state-of-the-art in terms of language modeling and fine-tuning for 4 downstream tasks spanning document- and user-levels. |
Hierarchical Quantized Representations for Script Generation (D18-1)
Copied to clipboard
| Challenge: | Scripts define knowledge about how everyday scenarios are expected to unfold . language models tend towards local coherency, which is a major issue . |
| Approach: | They propose an autoencoder model with a latent space defined by a hierarchy of categorical variables . they use a vector quantization based approach which allows continuous embeddings to be associated with each latent variable value . |
| Outcome: | The proposed model outperforms a language modeling-based method on several tasks and lowers perplexity scores. |
Modeling Label Semantics for Predicting Emotional Reactions (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for predicting how events induce emotions ignore the semantics of the labels themselves. |
| Approach: | They propose that the semantics of emotion labels can guide a model’s attention when representing the input story. |
| Outcome: | The proposed model can model the semantics of emotion labels and track correlations on unlabeled data. |
Modeling Preconditions in Text with a Crowd-sourced Dataset (2020.findings-emnlp)
Copied to clipboard
Heeyoung Kwon, Mahnaz Koupaee, Pratyush Singh, Gargi Sawhney, Anmol Shukla, Keerthi Kumar Kallur, Nathanael Chambers, Niranjan Balasubramanian
| Challenge: | Existing methods for modeling preconditions in text are limited due to the lack of large scale labeled data grounded in text. |
| Approach: | They propose a crowd-sourced annotation of preconditions between event pairs in newswire that is larger than prior annotations. |
| Outcome: | The proposed model outperforms existing models on two task sets, showing that precondition knowledge is not easily accessible in LM-derived representations alone. |
BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examples (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Biomedical research has progressed at a tremendous pace, with PubMed2 indexing well over 1M publications per year in the past eight years. |
| Approach: | They propose a semi-supervised procedure that bootstraps biomedical NLI datasets from positive entailment examples present in biomedically published texts. |
| Outcome: | The proposed procedure bootstraps biomedical NLI datasets from positive entailment examples from biomedically challenging texts. |
IrEne-viz: Visualizing Energy Consumption of Transformer Models (2021.emnlp-demo)
Copied to clipboard
Yash Kumar Lal, Reetu Singh, Harsh Trivedi, Qingqing Cao, Aruna Balasubramanian, Niranjan Balasubramanian
| Challenge: | IrEne is an energy prediction system that accurately predicts inference energy consumption of transformer-based NLP models. |
| Approach: | They present an online platform for visualizing and exploring energy consumption of transformer-based NLP models. |
| Outcome: | The proposed system predicts energy consumption of transformer-based models and their components. |
PASTA: A Dataset for Modeling PArticipant STAtes in Narratives (2023.tacl-1)
Copied to clipboard
Sayontan Ghosh, Mahnaz Koupaee, Isabella Chen, Francis Ferraro, Nathanael Chambers, Niranjan Balasubramanian
| Challenge: | Existing models that understand narratives should infer these implicit states and their causal relationships with the narrative's explicit events. |
| Approach: | They propose a dataset that contains inferable participant states, a counterfactual perturbation to each state and the changes to the story that would be necessary if the counterfact was true. |
| Outcome: | The proposed model can reason about the impact of changes to the story that would be necessary if the counterfactual were true. |
Residualized Similarity for Faithfully Explainable Authorship Verification (2025.findings-emnlp)
Copied to clipboard
Peter Zeng, Pegah Alipoormolabashi, Jihu Mun, Gourab Dey, Nikita Soni, Niranjan Balasubramanian, Owen Rambow, H. Schwartz
| Challenge: | Neural methods achieve high accuracy, but their representations lack direct interpretability. |
| Approach: | They propose a method that supplements systems using interpretable features with a neural network to improve their performance while maintaining interpretability. |
| Outcome: | The proposed method improves the performance of state-of-the-art models while maintaining interpretability. |
Teaching an Old LLM Secure Coding: Localized Preference Optimization on Distilled Preferences (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve security of LLM generated code are ineffective and lack localized regions of code. |
| Approach: | They propose a method for distilling a preference dataset of insecure and secure code pairs from frontier LLMs and a security reasoning that explains the issues and the fix. |
| Outcome: | The proposed method reduces code insecurity while improving overall code quality. |
Efficient Methods for Natural Language Processing: A Survey (2023.tacl-1)
Copied to clipboard
Marcos Treviso, Ji-Ung Lee, Tianchu Ji, Betty van Aken, Qingqing Cao, Manuel R. Ciosici, Michael Hassid, Kenneth Heafield, Sara Hooker, Colin Raffel, Pedro H. Martins, André F. T. Martins, Jessica Zosa Forde, Peter Milder, Edwin Simpson, Noam Slonim, Jesse Dodge, Emma Strubell, Niranjan Balasubramanian, Leon Derczynski, Iryna Gurevych, Roy Schwartz
| Challenge: | Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data, but using only scale to improve performance means resource consumption also grows. |
| Approach: | They propose to use data, time, storage, or energy to improve model performance. |
| Outcome: | The proposed methods and findings provide guidance for conducting NLP under limited resources and point towards promising research directions for developing more efficient methods. |
Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to multi-hop Reading Comprehension (RC) have greatly improved its explainability, models ability to explain their own answers. |
| Approach: | They propose to generate a question-focused abstractive summary of input paragraphs and feed it to an RC system. |
| Outcome: | The proposed explanation generates more compact explanations than an extractive explainer with limited supervision while maintaining sufficiency. |
TellMeWhy: A Dataset for Answering Why-Questions in Narratives (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models do not have the ability to answer "why" questions that require commonsense knowledge external to the narrative. |
| Approach: | They propose a crowd-sourced dataset that asks why characters perform actions . they show that state-of-the-art models are far below human performance on answering such questions . |
| Outcome: | The proposed dataset shows that state-of-the-art models are far below human performance on answering such questions. |
Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models exploit dataset artifacts to produce correct answers without connecting information across multiple facts. |
| Approach: | They formalize disconnected reasoning across subsets of supporting facts to reduce disconnected reasoning . they propose an automatic transformation of existing datasets that reduces disconnected reasoning. |
| Outcome: | The proposed model-agnostic probe reduces disconnected reasoning in a reading comprehension setting. |
MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detection (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Much of natural language processing is focused on leveraging large capacity language models, typically trained over single messages with a task of predicting one or more tokens. |
| Approach: | They propose a hierarchical message-encoder pre-trained over Twitter for stance prediction task. |
| Outcome: | The proposed model achieves 67% performance on stance prediction task using a pre-trained message-encoder over Twitter. |
Repurposing Entailment for Multi-Hop Question Answering Tasks (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to use entailment models for question answering are limited . large scale datasets are typically framed at a sentence level, whereas question answering requires verifying whether multiple sentences, taken together as a premise, entitle a hypothesis. |
| Approach: | They propose a general architecture that can use entailment models for multi-hop QA tasks. |
| Outcome: | The proposed model outperforms QA models trained on target datasets and the OpenAI transformer models. |
Large Human Language Models: A Need and the Challenges (2024.naacl-long)
Copied to clipboard
| Challenge: | a growing recognition of the importance of modeling human and social factors into human-centered NLP models . authors advocate for three positions toward creating large human language models based on psychological and behavioral sciences . |
| Approach: | et al. advocate for three positions toward creating large human language models . they argue that LM training should include the human context and recognize that people are more than their group . |
| Outcome: | a new study shows that learning language from linguistic signals alone is not adequate, according to a recent paper . authors advocate for three positions toward creating large human language models . a human-centered model should include the human context, and account for the dynamic nature of the human environment, they say . |
Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on action anticipation models treat it as a vision modality problem, but knowledge about action sequences can be obtained from textual data. |
| Approach: | They show how knowledge in pretrained language models can be adapted and distilled into vision based action anticipation models. |
| Outcome: | The proposed model achieves a 3.5% relative gain on EGTEA-GAZE+ and 7.2% relative gain for two action anticipation datasets. |
♫ MuSiQue: Multihop Questions via Single-hop Question Composition (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing multihop reasoning benchmarks are largely solvable via shortcuts . a bottom–up approach allows us to create a multihop QA dataset that requires proper multihop thinking. |
| Approach: | They propose a bottom–up approach that selects composable pairs of single-hop questions that are connected and adds stringent filters to the construction process. |
| Outcome: | The proposed approach creates a multihop question answering dataset with 25K 2–4 hop questions. |
Quantifying Misattribution Unfairness in Authorship Attribution (2025.acl-short)
Copied to clipboard
| Challenge: | Authorship misattribution can have profound consequences in real life . authors are considered as potential authors in forensic settings . |
| Approach: | They propose a measure to quantify the unfairness of authorship attribution systems . authors find that authors are more likely to be misattributed than others . |
| Outcome: | The proposed model shows that some authors are more likely to be misattributed than others. |
Latent Part-of-Speech Sequences for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for learning target side syntactic structure are greedy and only allow them to explore a limited portion of the latent space. |
| Approach: | They propose a new latent variable model, LaSyn, that captures the co-dependence between syntax and semantics while allowing for effective inference over the latent space. |
| Outcome: | The proposed model captures the co-dependence between syntax and semantics while allowing for efficient inference over the latent space. |
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models generate natural language reasoning steps or Chains-of-Thoughts when prompted appropriately. |
| Approach: | They propose a new approach that interleaves retrieval with steps (sentences) in a CoT and uses retrieved results to improve CoT. |
| Outcome: | The proposed approach improves retrieval and downstream QA significantly on four datasets. |
SpecNFS: A Challenge Dataset Towards Extracting Formal Models from Natural Language Specifications (2022.lrec-1)
Copied to clipboard
Sayontan Ghosh, Amanpreet Singh, Alex Merenstein, Wei Su, Scott A. Smolka, Erez Zadok, Niranjan Balasubramanian
| Challenge: | Existing methods for building formal semantic representations of specification texts are laborious and error-prone. |
| Approach: | They propose to use SpecIR to model sentences appearing in NFS specification documents as IF-THEN statements and introduce a representation language to parse them. |
| Outcome: | The proposed models achieve an F1 score of only 60.5 and 33.3 when using a state-of-the-art language model. |
IrEne: Interpretable Energy Prediction for Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. |
| Approach: | They propose an interpretable and extensible energy prediction system that predicts inference energy consumption of Transformer-based NLP models. |
| Outcome: | The proposed system predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. |
Residualized Factor Adaptation for Community Social Media Prediction Tasks (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to social media language capture only socio-demographic contexts, such as age, education rates, race, and gender. |
| Approach: | They propose a method which integrates community attributes and adapts linguistic features to community attributes. |
| Outcome: | The proposed model integrates community attributes and adapts linguistic features to community attributes. |
Don’t Let Discourse Confine Your Model: Sequence Perturbations for Improved Event Language Models (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to train event language models on text constrain them to follow discourse order of events. |
| Approach: | They propose a method to perturb event sequences so that they can relax model dependence on text order. |
| Outcome: | The proposed technique improves performance on applications and out-of-domain events data. |
SAGEViz: SchemA GEneration and Visualization (2023.emnlp-demo)
Copied to clipboard
Sugam Devare, Mahnaz Koupaee, Gautham Gunapati, Sayontan Ghosh, Sai Vallurupalli, Yash Kumar Lal, Francis Ferraro, Nathanael Chambers, Greg Durrett, Raymond Mooney, Katrin Erk, Niranjan Balasubramanian
| Challenge: | Schema induction involves creating a graph representation depicting how events unfold . supervised and few-shot approaches are not scalable and time-consuming . |
| Approach: | They propose a tool that utilizes human-AI collaboration to create and update complex schema graphs efficiently. |
| Outcome: | The proposed tool can generate schemas of better quality and be used by users in a variety of domains. |
PoMo: Generating Entity-Specific Post-Modifiers in Context (N19-1)
Copied to clipboard
Jun Seok Kang, Robert Logan, Zewei Chu, Yang Chen, Dheeru Dua, Kevin Gimpel, Sameer Singh, Niranjan Balasubramanian
| Challenge: | Using crowdsourcing, we show that contextual relevance is necessary for accurate post-modifier generation. |
| Approach: | They introduce entity post-modifier generation as an instance of a collaborative writing task . they build a post- modifier dataset from news articles that provides contextually relevant information about the target entity. |
| Outcome: | The proposed system can generate a post-modifier phrase that provides contextually relevant information about the target entity. |
SuMe: A Dataset Towards Summarizing Biomedical Mechanisms (2022.lrec-1)
Copied to clipboard
| Challenge: | Biomedical studies often examine how one entity affects another in a biological context. |
| Approach: | They propose a biomedical mechanism summarization task that pairs biomedically relevant texts with their summaries. |
| Outcome: | The proposed task improves performance but produces acceptable outputs in 32% of instances. |
Modeling Complex Event Scenarios via Simple Entity-focused Questions (2023.eacl-main)
Copied to clipboard
| Challenge: | Event schemas describe a sequence of events in a particular context, but they are difficult to model with standard event language models. |
| Approach: | They propose a question-guided generation framework that generates events as answers to questions about participants. |
| Outcome: | The proposed framework provides better coverage of participants, diverse events within a domain, comparable perplexities for modeling event sequences, and more effective control for interactive schema generation. |
On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformers (2021.findings-acl)
Copied to clipboard
Tianchu Ji, Shraddhan Jain, Michael Ferdman, Peter Milder, H. Andrew Schwartz, Niranjan Balasubramanian
| Challenge: | Recent work shows that attention can be pruned to zeros with minimal loss in accuracy. |
| Approach: | They propose a pruning technique which quantizes attention to a 3-bit format without retraining . they find that 80% of attention values can be pruned to zeros with minimal loss in accuracy . |
| Outcome: | The proposed approach produces only a few unique attention values with minimal loss in accuracy. |
NEUROSTRUCTURAL DECODING: Neural Text Generation with Structural Constraints (2023.acl-long)
Copied to clipboard
| Challenge: | Current approaches for conditional text generation focus on lexical constraints, but lack syntactic constraints to support complex semantic constraints. |
| Approach: | They propose a decoding algorithm that incorporates syntactic constraints to improve the quality of the generated text. |
| Outcome: | The proposed method improves on three different language generation tasks and shows improved lexical and syntactic metrics. |
POQue: Asking Participant-specific Outcome Questions for a Deeper Understanding of Complex Events (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models lag behind human performance in subtle ways in understanding complex situations, e.g., if the Argentine government yields to [IMF] pressure to rescind emergency legislation meant to protect ordinary families like the Brofmans. |
| Approach: | They propose to pre-identify a participant in a complex event and annotate their volitional engagement in causing the situation. |
| Outcome: | The proposed model can be used to infer the collective impact of salient events that make up a complex event, annotate volitional engagement of participants, and ground the outcome in state changes of the participants. |
Author’s Sentiment Prediction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on inferring author sentiment in news articles hasn't been done on this domain. |
| Approach: | They propose a crowd-sourced dataset that captures the sentiment of an author towards the main entity in a news article. |
| Outcome: | The proposed dataset performs the best amongst the baselines, but only achieves modest performance overall suggesting that fine-tuning document-level representations aloneisn’t adequate for this task. |
Hierarchical Modeling for User Personality Prediction: The Role of Message-Level Attention (2020.acl-main)
Copied to clipboard
| Challenge: | Language processing is increasingly finding use as a supplement for questionnaires to assess psychological attributes of consenting individuals, but most approaches neglect to consider whether all documents of an individual are equally informative. |
| Approach: | They propose a model that uses message-level attention to learn the relative weight of users’ social media posts for assessing their five factor personality traits. |
| Outcome: | The proposed model outperforms models with word-level attention and yields state-of-the-art accuracies for all five personality traits. |
Causal Graph based Event Reasoning using Semantic Relation Experts (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in event reasoning have limited ability to accurately identify causal connections between events. |
| Approach: | They propose a collaborative approach to generate correct graphs and graphs to assist reasoning . they propose 'a causal chain of events' task that requires a causal link between events . |
| Outcome: | The proposed approach achieves competitive results with state-of-the-art models on forecasting and next event prediction tasks. |
Teaching Broad Reasoning Skills for Multi-Step QA by Generating Hard Contexts (2022.emnlp-main)
Copied to clipboard
| Challenge: | Question-answering datasets require a broad set of reasoning skills. |
| Approach: | They use QDMR representations to programmatically create hard-to-cheat synthetic contexts for real questions in multi-step reasoning datasets. |
| Outcome: | The proposed model improves performance by 13 F1 points on 4 multi-step QA datasets and 21 points on more complex questions. |