Papers by Shweta Verma
A Critical Study of What Code-LLMs (Do Not) Learn (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models trained on code corpora have limitations such as suggesting codes with syntactic errors, variable misuse etc. |
| Approach: | They conduct a fine-grained analysis of attention maps and hidden representations of large-scale Large Language Models (cLLMs) trained on a large corpus of code and natural language -programming language pairs. |
| Outcome: | The proposed models encode relations among syntactic tokens and identifiers, but fail to encode relations between syntaktic token and identifier. |
CodeSSM: Towards State Space Models for Code Understanding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing transformers have limitations, such as quadratic complexity and high inference costs. |
| Approach: | They propose a state space model that is trained on code corpora to assess its effectiveness. |
| Outcome: | The proposed model reduces memory usage by up to 64% compared to transformers at a context length of 2048. |