Papers by Akshat Gupta

Conversational Semantic Parsing (2020.emnlp-main)

Copied to clipboard

Armen Aghajanyan, Jean Maillard, Akshat Shrivastava, Keith Diedrick, Michael Haeger, Haoran Li, Yashar Mehdad, Veselin Stoyanov, Anuj Kumar, Mike Lewis, Sonal Gupta

Natural Language Interfaces

Challenge:	Structured representations for task-oriented assistant systems are limited due to the limitations of the representation.
Approach:	They propose a semantic representation for task-oriented conversational systems that can represent co-reference and context carryover.
Outcome:	The proposed model improves the best results on ATIS, SNIPS, TOP and DSTC2 by up to 5 points for slot-carryover.

Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing (2023.eacl-main)

Copied to clipboard

Akshat Shrivastava, Shrey Desai, Anchit Gupta, Ali Elkahky, Aleksandr Livshits, Alexander Zotov, Ahmed Aly

Semantic Text Processing

Challenge:	Task-oriented semantic parsing models have achieved strong results in recent years, but they often face obstacles adapting to novel settings with distinct semantics and scarce data.
Approach:	They propose a scenario-based semantic parsing model which isolates coarse-grained and fine-grounded aspects of the task and solves them with off-the-shelf neural modules.
Outcome:	The proposed model outperforms previous approaches in high-resource, low-resourced, and multilingual settings, and is modular, differentiable, interpretable, and allows extra supervision from scenarios.

Evolutionary Strategies at Scale lead to Catastrophic Forgetting (2026.acl-short)

Copied to clipboard

Immanuel Abdi, Akshat Gupta, Micah Mok, Alex Lu, Nicholas Lee, Gopala Anumanchipalli

Language Models

Challenge:	ES has been shown to improve performance on specific tasks, but it is accompanied by significant forgetting of prior abilities.
Approach:	They propose to use Evolutionary Strategies to train gradient-free algorithms to improve performance.
Outcome:	The proposed algorithm achieves performance numbers closer to GRPO for math and reasoning tasks, but forgets prior abilities.

Muppet: Massive Multi-task Representations with Pre-Finetuning (2021.emnlp-main)

Copied to clipboard

Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, Sonal Gupta

Semantic Text Processing

Challenge:	Recent work shows gains from pre-training and fine-tuning that are multi-task . but it can be difficult to know which intermediate tasks will best transfer .
Approach:	They propose a large-scale learning stage for pre-finetuning between pre-training and fine-tun.
Outcome:	The proposed model improves performance on pretrained discriminators and generation models on a wide range of tasks while improving sample efficiency during fine-tuning.

Efficient Knowledge Editing via Minimal Precomputation (2025.acl-short)

Copied to clipboard

Akshat Gupta, Maochuan Lu, Thomas Hartvigsen, Gopala Anumanchipalli

Green & Sustainable NLP

Challenge:	Knowledge editing methods like MEMIT require a one-time but significant computational cost.
Approach:	They propose to pre-compute 44 million hidden vectors per edited layer . authors show that this precomputation step is unnecessary .
Outcome:	The proposed methods can be performed by pre-computing a small portion of 44 million hidden vectors.

Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing (2024.emnlp-main)

Copied to clipboard

Akshat Gupta, Sidharth Baskaran, Gopala Anumanchipalli

Challenge:	Recent work using Rank-One Model Editing (ROME) has shown that there are certain facts that the algorithm is unable to edit without breaking the model.
Approach:	They propose to use a model editing method called Rank-One Model Editing to make multiple edits to a single model without breaking it.
Outcome:	The proposed method improves generalization and locality of model editing and improves model collapse compared to the original implementation of ROME.

Geometric Interpretation of Layer Normalization and a Comparative Analysis with RMSNorm (2026.findings-eacl)

Copied to clipboard

Akshat Gupta, Atahan Ozdemir, Caoqinwei Gong, Gopala Anumanchipalli

Responsible & Trustworthy NLP

Challenge:	a novel geometric interpretation of LayerNorm is presented . layer normalization is a crucial yet often overlooked component of the transformer architecture .
Approach:	They propose a geometric interpretation of LayerNorm and explore how LayerNorm influences the norm and orientation of hidden vectors in the representation space.
Outcome:	The proposed interpretation of LayerNorm shows that it is redundant to remove a component along the uniform vector during training and inference.

A Unified Framework for Model Editing (2024.findings-emnlp)

Copied to clipboard

Akshat Gupta, Dev Sajnani, Gopala Anumanchipalli

Challenge:	EMMET is a new batched memory-editing algorithm for Transformers that can perform batched edits up to a batch-size of 10,000.
Approach:	They propose to unify ROME and MEMIT under a single umbrella to optimize for the preservation-memorization objective.
Outcome:	The proposed algorithms perform identically across multiple dimensions and are comparable in their optimization objective, performance and limitations.

Model Editing at Scale leads to Gradual and Catastrophic Forgetting (2024.findings-acl)

Copied to clipboard

Akshat Gupta, Anurag Rao, Gopala Anumanchipalli

Language Models

Challenge:	Existing model editing methods are evaluated using metrics for reliability, specificity and generalization over one or few edits.
Approach:	They evaluate model editing methods for three crucial properties - editing proficiency, fact forgetting and downstream performance.
Outcome:	The proposed methods are based on two state-of-the-art models - ROME and MEMIT.

Content Moderation for Evolving Policies using Binary Question Answering (2023.acl-industry)

Copied to clipboard

Sankha Subhra Mullick, Mohan Bhambhani, Suhit Sinha, Akshat Mathur, Somya Gupta, Jidnya Shah

Language Models

Challenge:	Social media platforms use content moderation to safeguard users from abuse, harassment, malicious attacks, spam, etc.
Approach:	They propose to model content moderation as a binary question answering problem where questions validate loosely coupled themes constituting a policy.
Outcome:	The proposed model improves recall at 95% precision on two proprietary datasets of social media posts and comments respectively annotated under curated Hate Speech and Commercial Spam policies.

SeqMMR: Sequential Model Merging and LLM Routing for Enhanced Batched Sequential Knowledge Editing (2025.findings-acl)

Copied to clipboard

Shanbao Qiao, Xuebing Liu, Akshat Gupta, Seung-Hoon Na

Semantic Text Processing

Challenge:	Existing research has demonstrated strong performance in single-instance or few-instantial sequential editing and one-time massive editing scenarios, but the batched sequential editing paradigm remains a significant challenge.
Approach:	They propose a framework for batched sequential knowledge editing that leverages SeqMMR and a model router to merge parameters from current batch-edited models with those of their predecessors.
Outcome:	The proposed framework iteratively merges parameters from current batch-edited models with those of their predecessors, ensuring that newly emerging knowledge is integrated while mitigating the forgetting of previously edited knowledge.

Weight Tying Biases Token Embeddings Towards the Output Space (2026.findings-acl)

Copied to clipboard

Antonio Lopardo, Avyukth Harish, Catherine Arnett, Akshat Gupta

Semantic Text Processing

Challenge:	Weight tying is a common practice in language model design, but its impact on learning embedding space remains unclear.
Approach:	They show that weight tying optimizes the embedding matrix for output prediction . they also show that tied embeddable matrices align more closely with output embedders .
Outcome:	The proposed weight tying approach harms performance at scale and has implications for training smaller LLMs.

Lifelong Knowledge Editing requires Better Regularization (2025.findings-emnlp)

Copied to clipboard

Akshat Gupta, Phudish Prateepamornkul, Maochuan Lu, Ahmed Alaa, Thomas Hartvigsen, Gopala Anumanchipalli

Semantic Text Processing

Challenge:	Knowledge editing is a promising way to improve factuality in large language models, but recent studies have shown significant model degradation during sequential editing.
Approach:	They formalize locate-then-edit methods as a two-step fine-tuning process . they show that model degradation occurs due to over-optimization of internal activations .
Outcome:	The proposed methods reduce time and improve factuality by 42-61%.

Disentangling Codemixing in Chats: The NUS ABC Codemixed Corpus (2026.findings-acl)

Copied to clipboard

Svetlana Churina, Akshat Gupta, Nur Insyirah Binte Imam Mujtahid, Kokil Jaidka

Challenge:	Existing studies on code-mixing have not been able to model human interactions in context.
Approach:	They propose to use a general-purpose code-mixing corpus to model human interactions and relationships in context while maintaining ethical standards.
Outcome:	The proposed corpus includes over 355,641 messages spanning various code-mixing patterns, with a primary focus on English, Mandarin, and other languages.