Papers by Jisu Shin
Does Rationale Quality Matter? Enhancing Mental Disorder Detection via Selective Reasoning Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | incorporating clinical symptom information into a model enhances domain expertise, improving its detection and interpretation performance. large language models are effective for generating explanatory rationales, but inconsistencies in relevance and domain alignment of LLM-generated rationale are challenging. |
| Approach: | They propose a framework that fine-tunes smaller language models with rationales exhibiting high domain relevance . they propose 'quality-focused' approach that selects rationale based on their alignment with clinical reasoning . |
| Outcome: | The proposed framework improves mental health detection and interpretation performance by ensuring high-quality rationales with domain relevance. |
Memorization or Reasoning? Exploring the Idiom Understanding of LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | idioms have long posed a challenge due to their unique linguistic properties, which set them apart from other common expressions. |
| Approach: | They propose to use a large-scale dataset of idioms in six languages to evaluate LLMs' idiomatic processing ability. |
| Outcome: | The proposed model integrates contextual cues and reasoning to improve idiom understanding in LLMs, suggesting that their performance is influenced by memorization and reasoning. |
Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static benchmarks for harmful content detection face limitations in scalability and diversity. |
| Approach: | They propose a framework for synthesizing harmful content using persona-guided large language model agents. |
| Outcome: | The proposed framework achieves a high success rate in harmful generation tests across multiple detection systems. |
ELF22: A Context-based Counter Trolling Dataset to Combat Internet Trolls (2022.lrec-1)
Copied to clipboard
| Challenge: | a new dataset aims to automate the method to counter trolls . trolleds cause psychological damage to individuals and increase social costs . |
| Approach: | They propose to use a dataset to generate counter responses by varying counter responses according to a given strategy. |
| Outcome: | The proposed method improves strategy-controlled sentence generation. |
Towards Effective Counter-Responses: Aligning Human Preferences with Strategies to Combat Online Trolling (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to counter trolling in online communities are not yet available to address the diversity of trolling behaviors. |
| Approach: | They propose a method for generating counter-responses to trolls by aligning these strategies with human preferences across different trolled contexts. |
| Outcome: | The proposed approach reduces negative effects of trolling and improves the online community environment. |
Ask LLMs Directly, “What shapes your bias?”: Measuring Social Bias in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to evaluate social bias in large language models have limitations . et al., 1995: stereotypes shape social perceptions without objective basis . |
| Approach: | They propose a method to intuitively quantify social perceptions and suggest metrics to evaluate biases within LLMs. |
| Outcome: | The proposed metrics capture the multi-dimensional aspects of social bias, the paper shows . they show that the proposed metrics can be used to evaluate bias in large language models . |
Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods struggle to capture subtle inconsistencies in large language models. |
| Approach: | They propose an atomic-level evaluation framework that quantifies persona fidelity at a finer granularity. |
| Outcome: | The proposed framework detects inconsistencies that prior evaluation methods overlook . it captures subtle deviations that real users would encounter . |
Distilling Long-CoT Reasoning through Collaborative Step-wise Multi-Teacher Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing curation-based approaches to inference are inefficient and fail to adapt dynamically, leading to redundant sampling and missed opportunities for complementary reasoning. |
| Approach: | They propose a collaborative multi-teacher decoding framework that performs step-wise reasoning synthesis guided by predictive perplexity–based scoring and beam search. |
| Outcome: | The proposed framework generates higher-quality reasoning data and achieves student-level results, demonstrating that fine-grained collaboration yields structured, efficient, and robust reasoning distillation. |
RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs’ Contextual Sensitivity (2026.findings-acl)
Copied to clipboard
| Challenge: | a new benchmark measures the contextual sensitivity of large language models in role conflict scenarios . role conflicts are social dilemmas where multiple roles cannot be fulfilled simultaneously . authors: models are forced to arbitrate between dynamic contextual cues and learned preferences . |
| Approach: | They propose a benchmark to measure the contextual sensitivity of large language models in role conflict scenarios. |
| Outcome: | The proposed benchmark measures the contextual sensitivity of large language models in role conflict scenarios. |
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning (2023.acl-long)
Copied to clipboard
Kyuyong Shin, Hanock Kwak, Wonjae Kim, Jisu Jeong, Seungjae Jung, Kyungmin Kim, Jung-Woo Ha, Sang-Woo Lee
| Challenge: | Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. |
| Approach: | They propose to use user behavior sequences as plain text to represent rich information in any domain or system without losing generality. |
| Outcome: | The proposed frameworks achieve excellent results on diverse recommendation tasks and can be used on unseen domains and services. |
A Simple and Flexible Modeling for Mental Disorder Detection by Learning from Clinical Questionnaires (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to detecting mental disorders lack domain-based interpretation . lack of quality data or complexity of models can cause problems . |
| Approach: | They propose a model that captures semantic meanings directly from social media and compares them to symptom-related descriptions. |
| Outcome: | The proposed model outperforms baselines on mental disorder detection tasks. |
Detecting Offensive Language in an Open Chatbot Platform (2024.lrec-main)
Copied to clipboard
Hyeonho Song, Jisu Hong, Chani Jung, Hyojin Chin, Mingi Shin, Yubin Choi, Junghoi Choi, Meeyoung Cha
| Challenge: | Existing efforts to automatically filter offensive language are vulnerable to users’ deliberate text manipulation tactics, such as misspelling words. |
| Approach: | They propose a contrastive learning model that embeds chat content with a random masking strategy to detect offensive language in open-domain chat conversations. |
| Outcome: | The proposed model outperforms existing models in detecting offensive language in open-domain chat conversations while also showing robustness against users’ deliberate text manipulation tactics when using offensive language. |
Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM Collectives (2026.acl-long)
Copied to clipboard
| Challenge: | Large language model (LLM) agents are increasingly acting as human delegates in multi-agent environments, where a representative agent integrates diverse peer perspectives to make a final decision. |
| Approach: | They define four key phenomena—social conformity, perceived expertise, dominant speaker effect, and rhetorical persuasion—and manipulate the number of adversaries, relative intelligence, argument length, and argumentative styles. |
| Outcome: | The results show that the reliability of the representative agent is undermined by the social context of its network. |