Papers by Chani Jung
FINEST: Improving LLM Responses to Sensitive Topics Through Fine-Grained Evaluation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing evaluation frameworks lack systematic methods to identify weaknesses in LLMs . Existing methods to evaluate LLM responses to sensitive topics are lacking . |
| Approach: | They propose a FINE-grained response evaluation taxonomy for sensitive topics that breaks down helpfulness and harmlessness into errors across three main categories: Content, Logic, and Appropriateness. |
| Outcome: | The proposed model outperforms refinement without guidance on Korean-sensitive questions . FINEST significantly improves the model responses across all three categories . |
RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs’ Contextual Sensitivity (2026.findings-acl)
Copied to clipboard
| Challenge: | a new benchmark measures the contextual sensitivity of large language models in role conflict scenarios . role conflicts are social dilemmas where multiple roles cannot be fulfilled simultaneously . authors: models are forced to arbitrate between dynamic contextual cues and learned preferences . |
| Approach: | They propose a benchmark to measure the contextual sensitivity of large language models in role conflict scenarios. |
| Outcome: | The proposed benchmark measures the contextual sensitivity of large language models in role conflict scenarios. |
Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing datasets for hate speech detection neglect the cultural diversity within a single language. |
| Approach: | They propose a CR**oss-cultural **E**nglish **Hate* speech dataset that uses culturally hateful keywords to identify posts from four countries plus the United States. |
| Outcome: | The proposed dataset shows that only 56.2% of the posts in CREHate achieve consensus among all countries, with the highest pairwise label difference rate of 26%. |
Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | While theory of mind (ToM) is naturally developed for humans in childhood, large language models (LLMs) exhibit inconsistency in ToM tasks, despite early reports of successful cases. |
| Approach: | They propose to evaluate human ToM precursors-perception inference and perception-to-belief inference-in large language models (LLMs) by annotating characters’ perceptions on ToMi and FANToM. |
| Outcome: | The proposed method significantly improves LLMs’ performance in false belief scenarios. |
Detecting Offensive Language in an Open Chatbot Platform (2024.lrec-main)
Copied to clipboard
Hyeonho Song, Jisu Hong, Chani Jung, Hyojin Chin, Mingi Shin, Yubin Choi, Junghoi Choi, Meeyoung Cha
| Challenge: | Existing efforts to automatically filter offensive language are vulnerable to users’ deliberate text manipulation tactics, such as misspelling words. |
| Approach: | They propose a contrastive learning model that embeds chat content with a random masking strategy to detect offensive language in open-domain chat conversations. |
| Outcome: | The proposed model outperforms existing models in detecting offensive language in open-domain chat conversations while also showing robustness against users’ deliberate text manipulation tactics when using offensive language. |