Papers by Seunghyun Hwang
PR-MCS: Perturbation Robust Metric for MultiLingual Image Captioning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing image captioning metrics are vulnerable to lexical perturbations, but they are not robust to such perturbations. |
| Approach: | They propose a perturbation-robust multilingual CLIPScore which is a reference-free image captioning metric for multiple languages. |
| Outcome: | The proposed metric outperforms baseline metrics in capturing lexical noise of all various perturbation types in all five languages while maintaining a strong correlation with human judgments. |
Kiss up, Kick down: Exploring Behavioral Changes in Multi-modal Large Language Models with Assigned Visual Personas (2024.emnlp-main)
Copied to clipboard
Seungjong Sun, Eungu Lee, Seo Baek, Seunghyun Hwang, Wonbyung Lee, Dongyan Nan, Bernard Jansen, Jang Kim
| Challenge: | Large language models (LLMs) exhibit a high degree of alignment with human behavior based on their robust capabilities for natural language understanding and generation. |
| Approach: | They developed a dataset of 5K fictional avatar images for assignment as visual personas to large language models (LLMs) and analyzed their negotiation behaviors based on the visual traits depicted in these images. |
| Outcome: | The proposed model exhibited aggressive negotiation behaviors when the opponent’s image appeared less aggressive than their own, and less aggressive negotiation behavior when the opposing image appeared more aggressive. |
Spatial Dependency Parsing for Semi-Structured Document Information Extraction (2021.findings-acl)
Copied to clipboard
| Challenge: | Information extraction (IE) for semistructured document images is often considered as a sequence tagging problem . however, such a setup cannot handle complex spatial relationships and is not suitable for highly structured information. |
| Approach: | They propose a spatial dependency parsing problem that models complex spatial relationships . they evaluate it on receipts, name cards, forms, and invoices and compare it to other methods . |
| Outcome: | The proposed parser achieves similar or better performance on various kinds of documents compared to baselines including BERT-based IOB taggger. |