Papers by Sohee Yang
PiCSAR: Probabilistic Confidence Selection and Ranking for Reasoning Chains (2026.findings-acl)
Copied to clipboard
Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema, Sohee Yang, Wai-Chung Kwan, Xuanli He, Wenda Li, Pasquale Minervini, Eleonora Giunchiglia, Shay B Cohen
| Challenge: | Recent studies show that large reasoning models (LLMs) achieve strong performance on complex reasoning tasks. |
| Approach: | They propose a method that scores each candidate generation using the joint log-likelihood of the reasoning and final answer. |
| Outcome: | The proposed method outperforms baselines with 2x fewer samples in 20 out of 25 comparisons. |
Exploring the Practicality of Generative Retrieval on Dynamic Corpora (2024.emnlp-main)
Copied to clipboard
| Challenge: | a lack of comprehensive comparison between GR and Dual Encoders in IR systems is limiting . GR is more adaptable to evolving knowledge (4–11%), robust in learning knowledge with temporal information, and efficient in terms of inference FLOPs (x2), indexing time (x6) and storage footprint (x4) |
| Approach: | They propose to use autoregressive language models to perform information retrieval (IR) their results highlight the potential of GR for future use in practical IR systems . |
| Outcome: | The proposed model is more adaptable to evolving knowledge (4–11%), robust in learning knowledge with temporal information, efficient inference FLOPs (x2), indexing time (x6), and storage footprint (x4) compared to the most common model, Dual Encoder (DE). |
Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally. |
| Approach: | They propose a "back-patching" analysis method to solve multi-hop queries . they propose resolving the bridge entity into the bridge and the second hop into the target entity into latent steps. |
| Outcome: | The proposed method solves multi-hop queries that require two information extraction steps . it shows that the later layers lack the necessary knowledge to correctly generate the answer . |
Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts? (2025.findings-acl)
Copied to clipboard
| Challenge: | Latent multi-hop reasoning is a problem in Large Language Models that can develop shortcuts by encountering the head entity and answer entity in training sequences. |
| Approach: | They propose desiderata for shortcut-free evaluation of latent multi-hop reasoning ability . they exclude test queries where head and answer entities might have co-appeared . |
| Outcome: | The proposed model can latently recall and compose single-hop facts without shortcuts, but only for certain types of queries. |
Efficient Dialogue State Tracking by Selectively Overwriting Memory (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work in dialogue state tracking (DST) is inefficient in that it predicts dialogue state at every turn from scratch. |
| Approach: | They propose a method that selectively overwrites memory for dialogue state tracking by predicting dialogue state on each memory slot and overwriting it with new values. |
| Outcome: | The proposed model achieves state-of-the-art joint goal accuracy with 51.72% in MultiWOZ 2.0 and 53.01% in MultiWoz 2.1 in an open vocabulary-based DST setting. |
Single Ground Truth Is Not Enough: Adding Flexibility to Aspect-Based Sentiment Analysis Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis (ABSA) is a challenging task of extracting sentiments along with their corresponding aspects and opinion terms from text. |
| Approach: | They propose a pipeline that expands existing evaluation sets by adding alternative valid terms for aspect and opinion. |
| Outcome: | The proposed evaluation set uncovers the capabilities of large language models (LLMs) in ABSA tasks, which is concealed by the single-answer GT sets. |
Spatial Dependency Parsing for Semi-Structured Document Information Extraction (2021.findings-acl)
Copied to clipboard
| Challenge: | Information extraction (IE) for semistructured document images is often considered as a sequence tagging problem . however, such a setup cannot handle complex spatial relationships and is not suitable for highly structured information. |
| Approach: | They propose a spatial dependency parsing problem that models complex spatial relationships . they evaluate it on receipts, name cards, forms, and invoices and compare it to other methods . |
| Outcome: | The proposed parser achieves similar or better performance on various kinds of documents compared to baselines including BERT-based IOB taggger. |
How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent reasoning models show the ability to reflect, backtrack, and self-validate their reasoning . |
| Approach: | They investigate how well models identify and recover from unhelpful thoughts . they show that models are effective at identifying most unhelpfully thoughts - but struggle to recover from the same thoughts when they are injected into their thinking process . |
| Outcome: | The proposed model can identify unhelpful thoughts but struggle to recover from them . larger models struggle more than smaller models to recover short irrelevant thoughts . |
Unveiling the Response of Large Vision-Language Models to Visually Absent Tokens (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) generate contextually relevant responses by jointly interpreting visual and textual inputs. |
| Approach: | They propose a method to classify whether an input token is visually grounded by reinterpreting question prompts or replacing the detected absent tokens during generation. |
| Outcome: | The proposed method mitigates the models’ tendency to falsely presume the visual presence of text input and its generality across various LVLMs. |
Generative Multi-hop Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | A bi-encoder approach to text retrieval has limitations in multi-hop settings; the reformulated query gets longer as the number of hops increases, which further tightens the embedding bottleneck of the query vector. |
| Approach: | They propose an encoder-decoder model that performs multi-hop retrieval by simply generating the entire text sequences of the retrieval targets. |
| Outcome: | The proposed model achieves comparable or higher performance than bi-encoder models in five datasets while demonstrating superior GPU memory and storage footprint. |
Nonparametric Decoding for Generative Retrieval (2023.findings-acl)
Copied to clipboard
Hyunji Lee, JaeYoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vladimir Karpukhin, Yi Lu, Minjoon Seo
| Challenge: | Existing text retrieval models depend on the information encoded in its parameters without external memory, its information capacity is limited and fixed. |
| Approach: | They propose a nonparametric decoding approach which uses external memory instead of vanilla vocab embeddings as decoder voka embedds. |
| Outcome: | The proposed model can utilize parametric and nonparametric space. |
Do Large Language Models Latently Perform Multi-Hop Reasoning? (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of ‘Superstition’ is". |
| Approach: | They examine whether Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of ‘Superstition’ is". |
| Outcome: | The proposed model can latently perform multi-hop reasoning with complex prompts such as "The mother of the singer of ‘Superstition’ is". |
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models (2022.emnlp-main)
Copied to clipboard
Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Minjoon Seo
| Challenge: | Language Models (LMs) become outdated as the world changes, a phenomenon called temporal misalignment. |
| Approach: | They propose a lifelong benchmark that utilizes the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation. |
| Outcome: | The proposed benchmark can be trained on the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation. |
Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work shows that an adversary can extract training data from Pretrained Language Models including Personally Identifiable Information (PII) such as names, phone numbers, and email addresses. |
| Approach: | They propose to use knowledge unlearning to reduce privacy risks for LMs by performing gradient ascent on target token sequences instead of trying to unlearn all the data at once. |
| Outcome: | The proposed method can give a stronger empirical privacy guarantee in scenarios where the data vulnerable to extraction attacks are known a priori while being much more efficient and robust. |
Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Open-domain question answering is a task of finding answers to generic factoid questions. |
| Approach: | They propose to use a retrieve-and-read mechanism to drastically reduce the footprint of an open-domain QA system by up to 160x. |
| Outcome: | The proposed method achieves better accuracy than a parametric model with comparable docker-level system size. |