Papers by Pranav Bhagat
Evaluating Compound AI Systems through Behaviors, Not Benchmarks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Compound AI (CAI) systems combine LLMs with retrievers and tools to enable information-seeking applications in the real-world. |
| Approach: | They propose a behavior-driven evaluation framework that generates test specifications aligned with real usage contexts. |
| Outcome: | The proposed framework generates test specifications aligned with real usage contexts and transforms them into concrete test cases. |
COMPACT: Building Compliance Paralegals via Clause Graph Reasoning over Contracts (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing legal NLP benchmarks focus on single-clause tasks, such as ContractNLI and CUAD. |
| Approach: | They propose a framework that models cross-clause dependencies through structured clause graphs by extracting deontic-temporal entities from clauses and constructs typed relationship graphs capturing definitional dependencies, exception hierarchies, and temporal sequences. |
| Outcome: | The proposed framework extracts deontic-temporal entities from clauses and constructs typed relationship graphs capturing definitional dependencies, exception hierarchies, and temporal sequences. |