AI Sees Your Location—But With A Bias Toward The Wealthy World (2025.emnlp-main)
Copied to clipboard
| Challenge: | Visual Language Models (VLMs) have shown remarkable performance across various tasks, particularly in recognizing geographic information from images. |
| Approach: | They propose to use 1,200 images paired with detailed geographic metadata to evaluate VLMs' performance. |
| Outcome: | The models achieve 53.8% accuracy in city prediction, but exhibit significant biases in regional tasks. |
Similar Papers
AI Knows Where You Are: Exposure, Bias, and Inference in Multimodal Geolocation with KoreaGEO (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks show coarse granularity, linguistic bias, and a neglect of multimodal privacy risks. |
| Approach: | They propose a benchmark for visual-language models that analyzes social photos to assess location privacy risks. |
| Outcome: | The proposed benchmarks show coarse granularity, linguistic bias, and neglect of privacy risks. |
From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text (2025.emnlp-main)
Copied to clipboard
Ridwan Mahbub, Mohammed Saidul Islam, Mir Tafseer Nayeem, Md Tahmid Rahman Laskar, Mizanur Rahman, Shafiq Joty, Enamul Hoque
| Challenge: | Existing VLMs produce more positive descriptions for high-income countries compared to middle- or low-income nations, even when country attribution is the only variable changed. |
| Approach: | They propose to automate the process by generating textual summaries of charts using vision-language models to understand how a country’s economic status influences the sentiment of generated summary. |
| Outcome: | The proposed model amplifys geo-economic biases in 6,000 chart-country pairs from six widely used vision-language models to understand how a country’s economic status influences the sentiment of generated summaries. |
GeoRC: A Benchmark for Geolocation Reasoning Chains (2026.acl-long)
Copied to clipboard
Mohit Talreja, Joshua Diao, Jim James, Radu Casapu, Tejas Santanam, Ethan Mendes, Alan Ritter, Wei Xu, James Hays
| Challenge: | Vision Language Models (VLMs) are good at recognizing the global location of a photograph but are startlingly bad at explaining which image evidence led to their location prediction. |
| Approach: | They propose a benchmark for geolocation reasoning chains based on the global location prediction task in the popular GeoGuessr game. |
| Outcome: | The proposed benchmark compares LLM-as-a-judge and VLM-As-jumble strategies against human scoring. |
Measuring Geographic Performance Disparities of Offensive Language Classifiers (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work shows that text classifiers are biased regarding different languages and dialects. |
| Approach: | They propose to use a dataset to examine whether language, dialect, and topical content vary across geographical regions to address these gaps. |
| Outcome: | The proposed dataset includes 14 thousand examples across 15 cities and shows that current models do not generalize across locations. |
Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendations (2025.findings-naacl)
Copied to clipboard
| Challenge: | a large body of work examines language models for biases concerning gender, race, occupation and religion . however, the impact of the encoded geographical knowledge on real-world applications has not been documented . |
| Approach: | They examine large language models for two common scenarios that require geographical knowledge: travel recommendations and geo-anchored story generation. |
| Outcome: | The results show that the language models are biased against poorer countries and poorer socioeconomic conditions. |
Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have been proposed to augment LLMs with visual inputs. |
| Approach: | They propose large vision-Language Models to augment LLMs with visual inputs. |
| Outcome: | The proposed models condition generated text on both an input image and a visual prompt, enabling a variety of use cases such as visual question answering and multimodal chat. |
VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on VLM bias focus on portrait-style images and gender-occupation associations . existing studies ignore broader and more complex social stereotypes and their implied harm . |
| Approach: | They propose a large-scale VQA benchmark for evaluating bias in vision-language models . they use a question-answering framework that spans factuality, perception, stereotyping, and decision making . |
| Outcome: | The proposed framework examines bias in vision-language models using 30M+ images . findings reveal subtle, multifaceted, and surprising stereotypical patterns . |
Granular Privacy Control for Geolocation with Vision Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Vision Language Models (VLMs) are rapidly advancing in their capability to answer information-seeking questions. |
| Approach: | They develop a benchmark to evaluate the ability of VLMs to moderate geolocation dialogues with users. |
| Outcome: | a new benchmark evaluates the ability of VLMs to moderate geolocation conversations with users. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
Geographically-Informed Language Identification (2024.lrec-main)
Copied to clipboard
| Challenge: | a paper develops a method to identify languages based on geographic origin of text . the model is based in regions where languages are widely spoken and may occur anywhere . |
| Approach: | They propose to incorporate geographic information into a language identification model to ensure coverage of linguae francae regardless of location. |
| Outcome: | The proposed model includes 31 widely-spoken international languages . the model improves on social media data and improves performance on 916 languages compared to baseline models . |