Papers by Arjun Subramonian
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work analyzes, quantifies, and mitigates language model biases such as gender, race or religion-related stereotypes in static word embeddings and contextual representations. |
| Approach: | They explain the complexity of gender and language around it and examine how current representations perpetuate harms associated with binary gender. |
| Outcome: | The proposed model and dataset biases perpetuate harms associated with the treatment of gender as binary in English language technologies. |
Understanding “Democratization” in NLP and ML Research (2024.emnlp-main)
Copied to clipboard
| Challenge: | a large number of NLP and ML papers mention terms related to democracy . authors find that democratization is most frequently used to convey (ease of) access to or use of technologies without meaningfully engaging with theories of democratisation. |
| Approach: | They analyze papers using the term "democra*" to clarify how it is understood in NLP and ML . they find that democratization is most frequently used to convey (ease of) access to or use of technologies . |
| Outcome: | The authors analyze papers using the term "democra*" they find that democratization is most frequently used to convey (ease of) access to or use of technologies without meaningfully engaging with theories of democratisation. |
It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance (2023.findings-acl)
Copied to clipboard
| Challenge: | a meta-analysis and survey of practitioners reveal that benchmarks suffer from operationalization disagreements. |
| Approach: | They propose a taxonomy of disagreement to explain disagreements in NLP benchmarks . they propose defining how tasks are conceptualized and operationalizing benchmarks to document their limitations. |
| Outcome: | The proposed taxonomy identifies two types of disagreements among NLP practitioners . it shows that benchmarks are not clearly conceptualized and suffer from operationalization disagreements . |
SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models (2025.naacl-long)
Copied to clipboard
Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Nikita Nangia, Anaelia Ovalle, Giada Pistilli, Dragomir Radev, Beatrice Savoldi, Vipul Raheja, Jeremy Qin, Esther Ploeger, Arjun Subramonian, Kaustubh Dhole, Kaiser Sun, Amirbek Djanibekov, Jonibek Mansurov, Kayo Yin, Emilio Villa Cueva, Sagnik Mukherjee, Jerry Huang, Xudong Shen, Jay Gala, Hamdan Al-Ali, null Tair Djanibekov, Nurdaulet Mukhituly, Shangrui Nie, Shanya Sharma, Karolina Stanczak, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Marcelo Viridiano, Oskar Van Der Wal, Adina Yakefu, Aurélie Névéol, Mike Zhang, Sydney Zink, Zeerak Talat
| Challenge: | Large Language Models reproduce and exacerbate social biases present in training data, and resources to quantify this issue are limited. |
| Approach: | They propose a multilingual parallel dataset to examine culturally-specific stereotypes that may be learned by LLMs. |
| Outcome: | The proposed dataset includes stereotypes from 20 regions around the world and 16 languages, spanning multiple identity categories subject to discrimination worldwide. |