Papers by Justin Zhao
Hop, Union, Generate: Explainable Multi-hop Reasoning without Rationale Supervision (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods rely on supervision for both answers and rationales, but they have limited capacities in modeling interactions between sentences, let alone reasoning across multiple documents. |
| Approach: | They propose a principled, probabilistic approach for training explainable multi-hop question answering systems without rationale supervision. |
| Outcome: | The proposed method is more accurate at selecting rationales than previous methods while maintaining similar accuracy in predicting answers. |
Bayesian Active Learning with Gaussian Processes Guided by LLM Relevance Scoring for Dense Passage Retrieval (2026.findings-acl)
Copied to clipboard
Junyoung Kim, Anton Korikov, Jiazhou Liang, Justin Cui, Yifan Simon Liu, Qianfeng Wen, Mark Zhao, Scott Sanner
| Challenge: | Existing approaches passively rely on first-stage dense retrievers, which leads to two limitations: failing to retrieve relevant passages in semantically distinct clusters and failing to propagate relevance signals to the broader corpus. |
| Approach: | They propose a framework that propagates sparse LLM relevance signals across the embedding space to guide global exploration. |
| Outcome: | Experiments show that the proposed framework outperforms existing approaches under the same budget on all four datasets. |
Language Model Council: Democratically Benchmarking Foundation Models on Highly Subjective Tasks (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluations of Large Language Models (LLMs) rely on a single large model to score outputs from other LLMs, but this is prone to intra-model bias and many tasks may be too subjective for a one model to judge fairly. |
| Approach: | They propose a language model council where a group of LLMs collaborate to create tests, respond to them, and evaluate each other’s responses to produce a ranking in a democratic fashion. |
| Outcome: | The proposed model produces rankings that are more separable and robust than any individual LLM judge. |
Balanced Accuracy: The Right Metric for Evaluating LLM Judges - Explained through Youden’s J statistic (2026.eacl-industry)
Copied to clipboard
| Challenge: | False refusals and task pass rates are key to reliable evaluation of large language models. |
| Approach: | They propose a principled best practice for evaluating judges based on a golden set of judge-quality metrics. |
| Outcome: | The proposed method improves the quality of judge-quality metrics on a golden set. |
UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations (2024.naacl-long)
Copied to clipboard
Wenting Zhao, Justin Chiu, Jena Hwang, Faeze Brahman, Jack Hessel, Sanjiban Choudhury, Yejin Choi, Xiang Li, Alane Suhr
| Challenge: | Existing work evaluating commonsense reasoning focuses on making inferences about common, everyday situations. |
| Approach: | They propose to use an English language corpus to investigate commonsense reasoning . they characterize performance differences between human explainers and best-performing large language models . |
| Outcome: | The proposed method reduces the loss rate of human-written explanations on commonsense reasoning compared with the vanilla supervised fine-tuning approach . |
Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for abductive reasoning in natural language processing rely on manual supervision. |
| Approach: | They propose an approach for abductive commonsense reasoning that exploits the fact that only a subset of explanations is correct for a given context. |
| Outcome: | The proposed approach outperforms or is comparable to knowledge-augmented zero-shot methods on a diverse set of abductive reasoning datasets. |
Symbolic Planning and Code Generation for Grounded Dialogue (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models excel at processing and generating text and code, but lack a grounded task-oriented dialogue system that can handle grounding. |
| Approach: | They propose a modular and interpretable grounded dialogue system that integrates a reader and planner to convert partner utterances into executable code and a symbolic planner to determine the next appropriate response. |
| Outcome: | The proposed system outperforms the existing state-of-the-art on a one-common dialogue task and improves task success in human evaluations from 56% to 69% in the most challenging setting. |
Multimodal Item Scoring for Natural Language Recommendation via Gaussian Process Regression with LLM Relevance Judgments (2026.findings-acl)
Copied to clipboard
Yifan Simon Liu, Qianfeng Wen, Jiazhou Liang, Mark Zhao, Justin Cui, Anton Korikov, Armin Toroghi, Junyoung Kim, Scott Sanner
| Challenge: | Existing NLRec approaches use Dense Retrieval to compute item relevance scores . DR views the request as the sole relevance label, leading to a weak proxy for query relevance. |
| Approach: | They propose to use Gaussian Process Regression to model item relevance . they propose to combine LLM with a Gauss-based kernel to model multimodal relevance judging . |
| Outcome: | The proposed approach outperforms simpler unimodal kernels and baseline methods by up to 65% on four NLRec datasets and two LLM backbones. |