Papers by Minje Choi
MM-SOC: Benchmarking Multimodal Large Language Models in Social Media Platforms (2024.findings-acl)
Copied to clipboard
| Challenge: | Social media platforms are hubs for multimodal information exchange, encompassing text, images, and videos, making it challenging for machines to comprehend the information or emotions associated with interactions in online spaces. |
| Approach: | They propose a benchmark to evaluate MLLMs' understanding of multimodal social media content and a large-scale YouTube tagging dataset to evaluate their performance. |
| Outcome: | The proposed model performs better in a zero-shot setting, suggesting potential improvements. |
Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs (2025.naacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) are widely used to simulate human responses, but their ability to account for demographic differences in subjective tasks remains uncertain. |
| Approach: | They evaluate large language models' ability to understand demographic differences in two subjective judgment tasks: politeness and offensiveness. |
| Outcome: | The proposed models perform better in politeness and offensiveness tasks, while sociodemographic prompting does not improve and worsens their ability to perceive language from sub-populations. |
You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments (2024.naacl-long)
Copied to clipboard
Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, David Jurgens
| Challenge: | Large Language Models (LLMs) are popular for research in social sciences . currently, prompting LLMs is insufficient to accurately and reliably capture model perceptions, and we discuss potential alternatives to improve this. |
| Approach: | They construct a dataset that contains 693 questions encompassing 39 different instruments of persona measurement on 115 persona axes and a set of questions containing minor variations. |
| Outcome: | The proposed model can generate answers and negate statements in a consistent and robust manner. |
Cross-Modal Projection in Multimodal LLMs Doesn’t Really Project Visual Attributes to Textual Space (2024.acl-short)
Copied to clipboard
| Challenge: | Existing multimodal large language models are limited to general-purpose multimodal tasks like question-answering on natural images. |
| Approach: | They propose to use cross-modal projection networks and a large language model to model domain-specific visual attributes of MLLMs. |
| Outcome: | The proposed models gain domain-specific visual capabilities when the projection is fine-tuned, but the updates do not extract relevant domain-specific visual attributes. |
Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks of social language are lacking for large language models. |
| Approach: | They propose a new benchmark that measures how well large language models understand social language by grouping 58 tasks into five categories: humor & sarcasm, offensiveness, sentiment & emotion, and trustworthiness. |
| Outcome: | The proposed model performs well at 58 tasks that are divided into five categories: humor & sarcasm, offensiveness, sentiment & emotion, and trustworthiness. |