Papers by Kinshuk Vasisht
Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendations (2025.findings-naacl)
Copied to clipboard
| Challenge: | a large body of work examines language models for biases concerning gender, race, occupation and religion . however, the impact of the encoded geographical knowledge on real-world applications has not been documented . |
| Approach: | They examine large language models for two common scenarios that require geographical knowledge: travel recommendations and geo-anchored story generation. |
| Outcome: | The results show that the language models are biased against poorer countries and poorer socioeconomic conditions. |
Knowledge Graph Guided Evaluation of Abstention Techniques (2025.naacl-long)
Copied to clipboard
| Challenge: | Several prior studies test the safety promises of language models based on their effectiveness in blocking malicious requests. |
| Approach: | They create a benchmark based on benign concepts and ground them in a knowledge graph to evaluate abstention techniques. |
| Outcome: | The proposed framework causes models to abstain with over 80% abstention rates, but not as effective for descendants of the target concepts, where abstraction rates drop by 19%. |
Evaluating Reasoning Models for Queries with Presuppositions (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work notes that large language models fail to challenge erroneous assumptions and can reinforce users’ misinformed opinions. |
| Approach: | They construct queries with varying degrees of presuppositions spanning health, science, and general knowledge and evaluate several widely-deployed models. |
| Outcome: | The proposed models achieve higher accuracy but fail to challenge a large fraction of false presuppositions. |