Papers by Pranoy Panda
HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to answer multi-hop questions are query-agnostic and the extracted facts are ambiguous as they lack context. |
| Approach: | They propose to use a knowledge graph to extract query-relevant information from unstructured text. |
| Outcome: | The proposed method achieves performance improvements on two popular datasets. |
Evaluating Compound AI Systems through Behaviors, Not Benchmarks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Compound AI (CAI) systems combine LLMs with retrievers and tools to enable information-seeking applications in the real-world. |
| Approach: | They propose a behavior-driven evaluation framework that generates test specifications aligned with real usage contexts. |
| Outcome: | The proposed framework generates test specifications aligned with real usage contexts and transforms them into concrete test cases. |
Adaptive LLM Routing under Budget Constraints (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized natural language processing, but their varying capabilities and costs pose challenges in practical applications. |
| Approach: | They propose to use LLM routing as a contextual bandit problem to enable adaptive decision-making . they use offline human preference data to refine the embeddings through online bandit feedback . |
| Outcome: | The proposed model is based on a contextual bandit problem enabling adaptive decision-making . it learns from offline human preference data and refines it through online bandit feedback. |