Papers by Ishaan Gupta
LLMs are Brittle to Simple Code Transformations: Introducing CETBench – A Benchmark for Code-Equivalence Checking (2026.findings-acl)
Copied to clipboard
| Challenge: | a new benchmarking tool for code equivalence checks the performance of LLMs. |
| Approach: | They propose a code-equivalence with transformations benchmark built from a repository of programs that may solve the same or different tasks. |
| Outcome: | The proposed approach boosts performance on the transformed pairs of programs. |
Which Modality should I use - Text, Motif, or Image? : Understanding Graphs with Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Current research typically employs limited setups with small real-world graphs. |
| Approach: | They propose a new approach to encoding a graph with diverse modalities, such as text, image, and motif, coupled with prompts to approximate a diagram’s global connectivity. |
| Outcome: | The proposed approach improves performance of LLMs in graph structure analysis by focusing on homophily, motif presence, and graph difficulty. |