Papers by Luciana Benotti
Selectively Answering Visual Questions (2024.findings-acl)
Copied to clipboard
| Challenge: | Large multi-modal models (LMMs) are capable of visual question answering (VQA) with unprecedented accuracy. |
| Approach: | They propose a calibration score that can be used to quantify uncertainty in visual question answering models. |
| Outcome: | The proposed calibration score is better calibrated than in text-only models for in-context learning. |
A recipe for annotating grounded clarifications (2021.naacl-main)
Copied to clipboard
| Challenge: | In order to interpret communicative intents of an utterance, it needs to be grounded in world modalities. |
| Approach: | They propose a recipe for obtaining grounding annotations for dialogue clarification mechanisms that make explicit the process of interpreting communicative intents of an utterance. |
| Outcome: | The proposed method is based on the definitions of perceptual and collaborative grounding and on the classification of clarification phenomena. |
Adaptive Data Collection for Latin-American Community-sourced Evaluation of Stereotypes (LACES) (2026.findings-acl)
Copied to clipboard
Guido Ivetta, Pietro Palombini, Sofía Martinelli, Marcos J Gomez, M Emilia Echeveste, Sunipa Dev, Vinodkumar Prabhakaran, Luciana Benotti
| Challenge: | a geo-cultural gap in NLP evaluation hinders evaluation of societal biases . authors propose a new method to collect stereotypes from large language models . |
| Approach: | They propose a new method that integrates sourcing and validation of existing data into a single workflow. |
| Outcome: | The proposed method improves LACES by integrating new stereotype entries and validation of existing data. |
Region under Discussion for visual dialog (2021.emnlp-main)
Copied to clipboard
| Challenge: | Visual Dialog is assumed to require the dialog history to generate correct responses during a dialog. |
| Approach: | They propose an interpretable representation that visually grounds dialog history by constraining the image’s spatial features according to a semantic representation inspired by Question under Discussion. |
| Outcome: | The proposed representation constrains the image’s spatial features according to a semantic representation of the history inspired by the information structure notion of Question under Discussion. |
What kinds of errors do reference resolution models make and what can we learn from them? (2022.findings-naacl)
Copied to clipboard
| Challenge: | Referring resolution is the task of identifying the referent of a natural language expression. |
| Approach: | They propose a model that restores weakening of the spatial natural constraints on referring expressions by evaluating their performance on different datasets. |
| Outcome: | The proposed model shows improved performance on the most challenging kinds of referring expressions on different datasets. |
Ethics consideration sections in natural language processing papers (2022.emnlp-main)
Copied to clipboard
| Challenge: | a manual classification of ethics sections for ACL 2021 is presented . authors will be allowed extra space after the 8th page for a broader impact statement . |
| Approach: | They propose a manual classification of all ethical consideration sections for ACL 2021 . they also compare how many papers had an ethics consideration section per track . |
| Outcome: | The paper compares the number of papers with an ethics consideration section in ACL 2021 . it also examines obstacles to the discussion of ethical consideration sections . |
Navigating Ethical Challenges in NLP: Hands-on strategies for students and researchers (2025.acl-tutorials)
Copied to clipboard
Luciana Benotti, Fanny Ducel, Karën Fort, Guido Ivetta, Zhijing Jin, Min-Yen Kan, Seunghun J. Lee, Minzhi Li, Margot Mieskes, Adriana Pagano
| Challenge: | This tutorial will equip participants with basic guidelines for thinking deeply about ethical issues . participants will gain practical experience on when to flag a paper for ethics review . |
| Approach: | This tutorial will equip participants with basic guidelines for thinking deeply about ethical issues . participants will gain practical experience on when to flag a paper for ethics review . |
| Outcome: | This tutorial will equip participants with basic guidelines for thinking deeply about ethical issues . participants will gain practical experience on when to flag a paper for ethics review . |
HESEIA: A community-based dataset for evaluating social biases in large language models, co-designed in real school settings in Latin America (2025.emnlp-main)
Copied to clipboard
Guido Ivetta, Marcos J Gomez, Sofía Martinelli, Pietro Palombini, M Emilia Echeveste, Nair Carolina Mazzeo, Beatriz Busaniche, Luciana Benotti
| Challenge: | a dataset of 46,499 sentences created in a professional development course captures intersectional biases across multiple demographic axes and school subjects. |
| Approach: | They present a large-scale dataset of 46,499 sentences created in a professional development course . they show that the dataset contains more stereotypes unrecognized by current LLMs . |
| Outcome: | The proposed dataset captures intersectional biases across multiple demographic axes and school subjects. |
Grounding as a Collaborative Process (2021.eacl-main)
Copied to clipboard
| Challenge: | a new study shows that dialog requires a collaborative grounding approach to ground meaning . the problem is that dialog partners are not able to ground themselves . |
| Approach: | They argue that it is missing from current deep learning approaches to dialog . they argue that making mistakes and being able to recover from them is key . |
| Outcome: | The proposed model is based on the language acquisition and dialog systems literature . it shows that making mistakes and being able to recover from them is key . |
Understanding Ethics in NLP Authoring and Reviewing (2023.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will equip participants with basic guidelines for thinking deeply about ethical issues . |
| Approach: | This tutorial will equip participants with basic guidelines for thinking deeply about ethical issues . the methodology is interactive and participatory, including case studies and working in groups . |
| Outcome: | This tutorial will equip participants with basic guidelines for thinking deeply about ethical issues . the methodology is interactive and participatory, including case studies and working in groups. |
Your Stereotypical Mileage May Vary: Practical Challenges of Evaluating Biases in Multiple Languages and Cultural Contexts (2024.lrec-main)
Copied to clipboard
Karen Fort, Laura Alonso Alemany, Luciana Benotti, Julien Bezançon, Claudia Borg, Marthese Borg, Yongjian Chen, Fanny Ducel, Yoann Dupont, Guido Ivetta, Zhijian Li, Margot Mieskes, Marco Naguib, Yuyan Qian, Matteo Radaelli, Wolfgang S. Schmeisser-Nieto, Emma Raimundo Schulz, Thiziri Saci, Sarah Saidi, Javier Torroba Marchante, Shilin Xie, Sergio E. Zanotto, Aurélie Névéol
| Challenge: | Recent studies have identified a gap in the availability of tools and resources to study bias in languages other than English and social contexts outside the north of America. |
| Approach: | They use stereotypes to build a corpus of sentence pairs that cover biases in seven cultural contexts. |
| Outcome: | The proposed resource covers a wide range of languages and cultural settings . it favors sentences that express stereotypes in most bias categories . |