Papers by Pavan Kapanipathi
X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive Summarization (2022.emnlp-main)
Copied to clipboard
Subhajit Chaudhury, Sarathkrishna Swaminathan, Chulaka Gunasekara, Maxwell Crouse, Srinivas Ravishankar, Daiki Kimura, Keerthiram Murugesan, Ramón Fernandez Astudillo, Tahira Naseem, Pavan Kapanipathi, Alexander Gray
| Challenge: | Abstractive summarization models produce factually inconsistent summaries that are not supported by the original article. |
| Approach: | They propose a fact-aware filtering mechanism that improves the factuality of abstractive summarization models. |
| Outcome: | The proposed method improves the quality of training data and the factuality of generated summaries. |
Leveraging Abstract Meaning Representation for Knowledge Base Question Answering (2021.findings-acl)
Copied to clipboard
Pavan Kapanipathi, Ibrahim Abdelaziz, Srinivas Ravishankar, Salim Roukos, Alexander Gray, Ramón Fernandez Astudillo, Maria Chang, Cristina Cornelio, Saswati Dana, Achille Fokoue, Dinesh Garg, Alfio Gliozzo, Sairam Gurajada, Hima Karanam, Naweed Khan, Dinesh Khandelwal, Young-Suk Lee, Yunyao Li, Francois Luus, Ndivhuwo Makondo, Nandana Mihindukulasooriya, Tahira Naseem, Sumit Neelam, Lucian Popa, Revanth Gangi Reddy, Ryan Riegel, Gaetano Rossiello, Udit Sharma, G P Shrivatsa Bhargav, Mo Yu
| Challenge: | Existing approaches face challenges including complex question understanding and lack of large end-to-end training datasets. |
| Approach: | They propose a modular knowledge base question answering system that leverages AMR parses for task-independent question understanding. |
| Outcome: | The proposed system achieves state-of-the-art performance on two prominent KBQA datasets based on DBpedia. |
NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls (2025.emnlp-main)
Copied to clipboard
Kinjal Basu, Ibrahim Abdelaziz, Kiran Kate, Mayank Agarwal, Maxwell Crouse, Yara Rizk, Kelsey Bradford, Asim Munawar, Sadhana Kumaravel, Saurabh Goyal, Xin Wang, Luis A. Lastras, Pavan Kapanipathi
| Challenge: | Existing benchmarks and datasets for tool calling have lagged behind . nested sequencing is a common problem in LLMs, but it is not enough to evaluate them. |
| Approach: | They propose a benchmark to evaluate LLMs on nested sequences of API calls, i.e. sequences where the output of one API call is passed as input to a subsequent call. |
| Outcome: | The proposed model achieves a full sequence match accuracy of 28% and a win-rate of 60% on nested sequences of API calls. |
Looking Beyond Sentence-Level Natural Language Inference for Question Answering and Text Summarization (2021.naacl-main)
Copied to clipboard
Anshuman Mishra, Dhruvesh Patel, Aparna Vijayakumar, Xiang Lorraine Li, Pavan Kapanipathi, Kartik Talamadupula
| Challenge: | Large-scale, open Natural Language Inference datasets have catalyzed the development of NLI models that exhibit close to human-level performance, but the use of these models for other downstream NLP tasks has met with limited success. |
| Approach: | They use multiple-choice reading comprehension and checking factual correctness of textual summarization tasks to investigate potential reasons for this . authors leverage abundance of data from reading comprehension datasets into longer-premise NLI datasets to improve their models . |
| Outcome: | The proposed models outperform models trained on converted datasets due to the difference in premise lengths. |
Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks (2024.emnlp-industry)
Copied to clipboard
Ibrahim Abdelaziz, Kinjal Basu, Mayank Agarwal, Sadhana Kumaravel, Matthew Stallone, Rameswar Panda, Yara Rizk, G P Shrivatsa Bhargav, Maxwell Crouse, Chulaka Gunasekara, Shajith Ikbal, Sachindra Joshi, Hima Karanam, Vineet Kumar, Asim Munawar, Sumit Neelam, Dinesh Raghu, Udit Sharma, Adriana Soria, Dheeraj Sreedhar, Praveen Venkateswaran, Merve Unuvar, David Cox, Salim Roukos, Luis Lastras, Pavan Kapanipathi
| Challenge: | Existing research explores the use of Large Language Models (LLMs) as the backbone of agentic systems. |
| Approach: | They propose a model trained using a multi-task training approach on seven fundamental tasks encompassed in function calling that has better generalizability on multiple tasks across seven evaluation benchmarks. |
| Outcome: | The proposed model outperforms more than 15 other models on out-of-domain datasets and ranks among the top on the Berkeley Function Calling Leaderboard (BFCL). |
A Semantics-aware Transformer Model of Relation Linking for Knowledge Base Question Answering (2021.acl-short)
Copied to clipboard
Tahira Naseem, Srinivas Ravishankar, Nandana Mihindukulasooriya, Ibrahim Abdelaziz, Young-Suk Lee, Pavan Kapanipathi, Salim Roukos, Alfio Gliozzo, Alexander Gray
| Challenge: | Existing knowledge base question answering systems do not leverage the explicit semantic parse of the question text. |
| Approach: | They propose a transformer-based neural model that leverages the AMR semantic parse of a sentence. |
| Outcome: | The proposed model outperforms the state-of-the-art on 4 popular benchmark datasets. |
MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types (2023.findings-acl)
Copied to clipboard
Keerthiram Murugesan, Sarathkrishna Swaminathan, Soham Dan, Subhajit Chaudhury, Chulaka Gunasekara, Maxwell Crouse, Diwakar Mahajan, Ibrahim Abdelaziz, Achille Fokoue, Pavan Kapanipathi, Salim Roukos, Alexander Gray
| Challenge: | Existing evaluation metrics for machine text are inadequate to capture quality of text . a recent study has focused on task-specific evaluation metrics or on properties of machine-generated text based on mismatch errors . |
| Approach: | They propose a new evaluation scheme based on fine-grained mismatch errors . they propose 13 mismatch error types to guide the model for better prediction of human judgments . |
| Outcome: | The proposed evaluation scheme is based on mismatch errors in 7 NLP tasks . the mismatch error types guide the model for better prediction of human judgments . |
Reading Comprehension as Natural Language Inference:A Semantic Analysis (2020.starsem-1)
Copied to clipboard
Anshuman Mishra, Dhruvesh Patel, Aparna Vijayakumar, Xiang Li, Pavan Kapanipathi, Kartik Talamadupula
| Challenge: | In recent past, Natural language Inference (NLI) has gained significant attention, but its true impact has not been well studied. |
| Approach: | They propose to transform a large RACE dataset into an NLI model and compare it to a state-of-the-art model. |
| Outcome: | The proposed model outperforms the previous model on a question-answer concatenation form and a coherent entailment form. |
A Two-Stage Approach towards Generalization in Knowledge Base Question Answering (2022.findings-emnlp)
Copied to clipboard
Srinivas Ravishankar, Dung Thai, Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Tahira Naseem, Pavan Kapanipathi, Gaetano Rossiello, Achille Fokoue
| Challenge: | Existing approaches for Knowledge Base Question Answering focus on a specific knowledge base or evaluating it on underlying knowledge base requires non-trivial changes. |
| Approach: | They propose a framework that separates semantic parsing from knowledge base interaction . they propose KBQA framework that allows generalization across knowledge bases . |
| Outcome: | The proposed framework achieves comparable or state-of-the-art performance on datasets with a different knowledge base. |
An Interface for Annotating Science Questions (D18-2)
Copied to clipboard
Michael Boratko, Harshit Padigela, Divyendra Mikkilineni, Pritish Yuvraj, Rajarshi Das, Andrew McCallum, Maria Chang, Achille Fokoue, Pavan Kapanipathi, Nicholas Mattei, Ryan Musa, Kartik Talamadupula, Michael Witbrock
| Challenge: | a new interface for human annotation of science question-answer pairs with their knowledge and reasoning types is proposed . the interface is based on previous work on the ARC dataset, but does not provide clear definitions of these types of knowledge. |
| Approach: | They propose an interface for human annotation of science question-answer pairs with their respective knowledge and reasoning types. |
| Outcome: | The proposed interface improves the classification of science questions in a preliminary study involving 10 participants. |
Self-Supervised Rule Learning to Link Text Segments to Relational Elements of Structured Knowledge (2023.findings-emnlp)
Copied to clipboard
Shajith Ikbal, Udit Sharma, Hima Karanam, Sumit Neelam, Ronny Luss, Dheeraj Sreedhar, Pavan Kapanipathi, Naweed Khan, Kyle Erwin, Ndivhuwo Makondo, Ibrahim Abdelaziz, Achille Fokoue, Alexander Gray, Maxwell Crouse, Subhajit Chaudhury, Chitra Subramanian
| Challenge: | Various approaches have been tried to map predicate components of a natural language (NL) text segment onto their corresponding predicates within a knowledge base (KB). |
| Approach: | They propose a neuro-symbolic approach to self-learn rules that serve as interpretable knowledge to perform relation linking in knowledge base question answering systems. |
| Outcome: | The proposed approach achieves an average performance gain of 17% on CLUTRR and relation linking in a KBQA system. |
Logical Neural Networks for Knowledge Base Completion with Embeddings & Rules (2022.emnlp-main)
Copied to clipboard
Prithviraj Sen, Breno William Carvalho, Ibrahim Abdelaziz, Pavan Kapanipathi, Salim Roukos, Alexander Gray
| Challenge: | Knowledge base completion (KBC) is a human-interpretable dialect . rule-based KBC has a high quality but low accuracy . |
| Approach: | They propose to use logical neural networks to learn both kinds of rules in a common framework using gradient-based optimization. |
| Outcome: | The proposed method improves by 10% relative to SotA rule-based methods and by combining it with knowledge graph embeddings it achieves an additional 7.5% relative improvement. |
Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning (2023.acl-long)
Copied to clipboard
Subhajit Chaudhury, Sarathkrishna Swaminathan, Daiki Kimura, Prithviraj Sen, Keerthiram Murugesan, Rosario Uceda-Sosa, Michiaki Tatsubori, Achille Fokoue, Pavan Kapanipathi, Asim Munawar, Alexander Gray
| Challenge: | Existing text-based reinforcement learning agents use embeddings as representations for observation and are fed to an action scorer for predicting the next action. |
| Approach: | They propose a novel neurosymbolic agent that combines a semantic parser and a rule induction system to learn interpretable rules as policies. |
| Outcome: | The proposed method outperforms deep learning-based methods on established text-based game benchmarks on unobserved games and on unseen games. |
Zero-shot Entity Linking with Less Data (2022.findings-naacl)
Copied to clipboard
G P Shrivatsa Bhargav, Dinesh Khandelwal, Saswati Dana, Dinesh Garg, Pavan Kapanipathi, Salim Roukos, Alexander Gray, L Venkata Subramaniam
| Challenge: | Entity linking maps an entity mention in a natural language sentence to an entity in KB. |
| Approach: | They propose a neuro-symbolic, multi-task learning approach to bridge this gap by exploiting an auxiliary information about entity types. |
| Outcome: | The proposed approach achieves significantly higher performance on four different benchmark datasets when trained with just 0.01%, 0.1%, or 1% of the training data. |
SYGMA: A System for Generalizable and Modular Question Answering Over Knowledge Bases (2022.findings-emnlp)
Copied to clipboard
Sumit Neelam, Udit Sharma, Hima Karanam, Shajith Ikbal, Pavan Kapanipathi, Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Young-Suk Lee, Santosh Srivastava, Cezar Pendus, Saswati Dana, Dinesh Garg, Achille Fokoue, G P Shrivatsa Bhargav, Dinesh Khandelwal, Srinivas Ravishankar, Sairam Gurajada, Maria Chang, Rosario Uceda-Sosa, Salim Roukos, Alexander Gray, Guilherme Lima, Ryan Riegel, Francois Luus, L V Subramaniam
| Challenge: | Knowledge Base Question Answering (KBQA) systems have limited generalizability across knowledge bases and multiple reasoning types. |
| Approach: | They propose a modular approach for KBQA that is built on a framework adaptable to multiple knowledge bases and reasoning types. |
| Outcome: | The proposed approach is generalized across multiple knowledge bases and reasoning types. |
R2D2: Remembering, Replaying and Dynamic Decision Making with a Reflective Agentic Memory (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for web agents struggle with efficient navigation and action execution due to limited visibility and understanding of web structures. |
| Approach: | They propose a framework that integrates memory-enhanced navigation and reflective learning to improve web agents' performance. |
| Outcome: | The proposed framework shows significant improvements over existing methods, including 50% reduction in navigation errors and threefold increase in task completion rates. |
Efficient Text-based Reinforcement Learning by Jointly Leveraging State and Commonsense Graph Representations (2021.acl-short)
Copied to clipboard
Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Kartik Talamadupula, Mrinmaya Sachan, Murray Campbell
| Challenge: | Text-based games (TBGs) are useful benchmarks for evaluating progress in grounded language understanding and reinforcement learning (RL). |
| Approach: | They propose an agent that induces a graph representation of the game state and jointly grounds it with a commonsense knowledge from ConceptNet. |
| Outcome: | The proposed agent outperforms baseline agents in the proposed game . |
API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs (2024.acl-long)
Copied to clipboard
Kinjal Basu, Ibrahim Abdelaziz, Subhajit Chaudhury, Soham Dan, Maxwell Crouse, Asim Munawar, Vernon Austel, Sadhana Kumaravel, Vinod Muthusamy, Pavan Kapanipathi, Luis Lastras
| Challenge: | Existing methods to train and test large language models that involve calls to tools and APIs are lacking. |
| Approach: | They propose a large corpora for training and systematic testing of tool-augmented LLMs. |
| Outcome: | The proposed datasets mimic real-world scenarios involving API-tasks and slot filling. |
Laziness Is a Virtue When It Comes to Compositionality in Neural Semantic Parsing (2023.acl-long)
Copied to clipboard
Maxwell Crouse, Pavan Kapanipathi, Subhajit Chaudhury, Tahira Naseem, Ramon Fernandez Astudillo, Achille Fokoue, Tim Klinger
| Challenge: | Compositional generalization is a key feature of human intelligence and has been identified as a major point of weakness in neural methods for semantic parsing. |
| Approach: | They propose a neural parsing generation method that constructs logical forms from the bottom up, beginning from the logical form’s leaves. |
| Outcome: | The proposed method outperforms general-purpose parsers on a CFQ dataset and two other Text-to-SQL datasets while also being competitive with parser that have been tailored to each task. |