Papers by Anwoy Chatterjee
POSIX: A Prompt Sensitivity Index For Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are sensitive to minor variations in prompts, such as spelling errors, alteration of wording or the prompt template. |
| Approach: | They propose a PrOmpt Sensitivity IndeX to measure prompt sensitivity . they use this to compare prompt sensitability of various open source LLMs . |
| Outcome: | The proposed method can measure and compare prompt sensitivity of open source LLMs. |
Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have transformed NLP with their remarkable In-context Learning capabilities. |
| Approach: | They propose to use large language models to generalize from labeled examples of predefined tasks to novel tasks . they use biological neurons and the Transformer architecture to study the potential for information sharing across tasks. |
| Outcome: | The proposed model can generalize from labeled examples of predefined tasks to novel tasks despite no examples from the target task in the context. |
Multilingual Language Models Encode Script Over Linguistic Structure (2026.acl-long)
Copied to clipboard
| Challenge: | a recent study suggests that multilingual language models organize representations around surface form, but the nature of this internal organization remains elusive. |
| Approach: | They analyze language-associated units across different model families and scales . romanization induces near-disjoint representations that align with neither native-script inputs nor English . |
| Outcome: | The results show that multilingual language models organize representations around surface form . romanization induces near-disjoint representations that align with neither native-script inputs nor English . |
Do You Know About My Nation? Investigating Multilingual Language Models’ Cultural Literacy Through Factual Knowledge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual question-answering benchmarks do not factor in regional diversity in the information they capture and tend to be Western-centric. |
| Approach: | They propose to benchmark eight standard multilingual LLMs on XNationQA and evaluate them using two novel transference metrics. |
| Outcome: | The proposed model shows greater knowledge of cultural information in English than in the dominant language of the respective culture. |