Papers by Moontae Lee

Code Models are Zero-shot Precondition Reasoners (2024.naacl-long)

Challenge:	Existing language models (LMs) can assign a high likelihood to incorrect steps . Existing models (LLMs), however, struggle with complex multi-step reasoning.
Approach:	They propose a stepwise decoding approach that steers the decoding process towards producing correct reasoning steps.
Outcome:	The proposed approach outperforms existing methods on math and symbolic reasoning tasks.

Copied to clipboard

Lajanugen Logeswaran, Sungryull Sohn, Yiwei Lyu, Anthony Liu, Dong-Ki Kim, Dongsub Shim, Moontae Lee, Honglak Lee

One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL (2025.acl-industry)

Challenge:	Existing methods to reason about action preconditions are lacking for agent to complete tasks.
Approach:	They propose a method to reason about action preconditions using pre-trained code models.
Outcome:	The proposed approach improves few-shot policy learning approaches across task-oriented dialog and embodied textworld benchmarks.

Copied to clipboard

Hyungjoo Chae, Dongjin Kang, Jihyuk Kim, Beong-woo Kwak, Sunghyun Park, Haeju Park, Jinyoung Yeo, Moontae Lee, Kyungjae Lee

From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning (2023.emnlp-main)

Challenge:	Existing large reasoning models are limited by their closed nature and high API costs and safety issues.
Approach:	They propose to build a long CoT dataset with existing short CoT LLMs that are not trained for inference-time scaling.
Outcome:	The proposed model achieves quality comparable to—or slightly below—R1 and is able to think longer and provide control over the thought budget to better manage the overthinking problem.

Copied to clipboard

Zheyuan Zhang, Shane Storks, Fengyuan Hu, Sungryull Sohn, Moontae Lee, Honglak Lee, Joyce Chai

Revisiting LLM Value Probing Strategies: Are They Robust and Expressive? (2025.emnlp-main)

Challenge:	Pre-trained language models have shown impressive performance in various language tasks, but are prone to spurious correlations and illusory information.
Approach:	They propose to use pre-trained language models to justify decisions with formalized, coherent reasoning chains.
Outcome:	The proposed strategies improve coherence of rationalizations yielding state-of-the-art results on Tiered Reasoning for Intuitive Physics (TRIP).

Copied to clipboard

Siqi Shen, Mehar Singh, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Rada Mihalcea

Responsible & Trustworthy NLP

Challenge:	Existing value probing methods that capture in-context information and predict models’ real-world actions are limited and lack systematic comparisons.
Approach:	They compare three widely used value probing methods: token likelihood, sequence perplexity, and text generation.
Outcome:	The proposed methods exhibit large variances under non-semantic perturbations in prompts and option formats, with sequence perplexity being the most robust overall.

Copied to clipboard

Moontae Lee, Sungjun Cho, David Bindel, David Mimno

Information Extraction & Text Mining

Challenge:	spectral topic models lack reliability in real data and lack of practical implementations.
Approach:	They propose to use a spectral topic inference method to infer correlations between topics in real data and a matrix-based approach to inference.
Outcome:	The proposed method outperforms tensor-based methods and probabilistic methods in real data and provides a complete guide to correlated topic modeling.

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models (2024.emnlp-main)

Copied to clipboard

Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo

Prospector: Improving LLM Agents with Self-Asking and Trajectory Ranking (2024.findings-emnlp)

Challenge:	Existing open-source evaluation paradigms lack flexibility and performance . language model-based evaluation is cheap and scalable, but it is difficult to evaluate .
Approach:	They propose a language model-based evaluation paradigm that uses a scalar indicator of quality to assess LM outputs.
Outcome:	The proposed language model-based evaluation model is more powerful than its predecessor.

Copied to clipboard

Byoungjip Kim, Youngsoo Jang, Lajanugen Logeswaran, Geon-Hyeong Kim, Yu Jin Kim, Honglak Lee, Moontae Lee

Challenge:	Existing LLMs are limited in their ability to incorporate feedback from an environment.
Approach:	They propose an LLM agent that consists of an Actor and a Critic.
Outcome:	The proposed agent outperforms existing LLMs on benchmark environments and shows that it can generate diverse trajectories and pick the most rewarding trajectory.

Few-shot Reranking for Multi-hop QA via Language Model Prompting (2023.acl-long)

Copied to clipboard

Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Lu Wang

Shifting from Ranking to Set Selection for Retrieval Augmented Generation (2025.acl-long)

Challenge:	Existing methods for multi-hop QA with open-domain questions require a large number of labeled question-document pairs for retrieval.
Approach:	They propose a language-based prompt for multi-hop path reranking that relies on language model prompting to generate a relevance score between a question and the path.
Outcome:	The proposed method yields strong retrieval performance on HotpotQA with only 128 training examples compared to state-of-the-art methods trained on thousands of examples.

Copied to clipboard

Dahyun Lee, Yongrae Jo, Haeju Park, Moontae Lee

Information Retrieval

Challenge:	Existing approaches primarily rerank top-k passages based on individual relevance, often failing to meet the information needs of complex queries in multi-hop question answering.
Approach:	They propose a set-wise passage selection approach and introduce SetR which explicitly identifies the information requirements of a query through Chain-of-Thought reasoning.
Outcome:	The proposed approach outperforms both proprietary LLM-based rerankers and open-source baselines in terms of answer correctness and retrieval quality.

Unsupervised Task Graph Generation from Instructional Video Transcripts (2023.findings-acl)

Copied to clipboard

Lajanugen Logeswaran, Sungryull Sohn, Yunseok Jang, Moontae Lee, Honglak Lee

Multimodality

Challenge:	Existing approaches to task graph generation use instruction-tuned language models to generate accurate task graphs.
Approach:	They propose an unsupervised approach that combines the reasoning capabilities of instruction-tuned language models with clustering and ranking components to generate accurate task graphs.
Outcome:	The proposed approach generates more accurate task graphs than a supervised learning approach on tasks from the ProceL and CrossTask datasets.

Few-shot Subgoal Planning with Language Models (2022.naacl-main)

Copied to clipboard

Lajanugen Logeswaran, Yao Fu, Moontae Lee, Honglak Lee

Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense (2024.naacl-long)

Challenge:	Pre-trained language models have shown successful progress in many text understanding benchmarks.
Approach:	They propose a strategy to re-rank language model predictions based on interaction and feedback from the environment.
Outcome:	The proposed approach shows competitive performance on subgoal prediction and task completion in the ALFRED benchmark compared to prior methods that assume more subgoals supervision.

Copied to clipboard

Siqi Shen, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Soujanya Poria, Rada Mihalcea

You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments (2024.naacl-long)

Challenge:	Large language models (LLMs) have demonstrated substantial commonsense understanding through numerous benchmark evaluations.
Approach:	They conduct a comprehensive examination of the capabilities and limitations of several state-of-the-art LLMs in the context of cultural commonsense tasks.
Outcome:	The language used to query the LLMs can impact their performance on cultural-related tasks.

Copied to clipboard

Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, David Jurgens

Semantic Skill Grounding for Embodied Instruction-Following in Cross-Domain Environments (2024.findings-acl)

Challenge:	Large Language Models (LLMs) are popular for research in social sciences . currently, prompting LLMs is insufficient to accurately and reliably capture model perceptions, and we discuss potential alternatives to improve this.
Approach:	They construct a dataset that contains 693 questions encompassing 39 different instruments of persona measurement on 115 persona axes and a set of questions containing minor variations.
Outcome:	The proposed model can generate answers and negate statements in a consistent and robust manner.

Copied to clipboard

Sangwoo Shin, SeungHyun Kim, Youngsoo Jang, Moontae Lee, Honguk Woo

On Sample-Efficient Code Generation (2023.emnlp-industry)

Challenge:	Existing frameworks for grounding pretrained language models as task planners are challenging due to their intricate entanglement with domain knowledge.
Approach:	They propose a framework that leverages the hierarchical nature of semantic skills to ground them in different domains.
Outcome:	The proposed framework is effective in 300 cross-domain EIF scenarios.

Copied to clipboard

Hojae Han, Yu Jin Kim, Byoungjip Kim, Youngwon Lee, Kyungjae Lee, Kyungmin Lee, Moontae Lee, Kyunghoon Bae, Seung-won Hwang

Programming Languages in NLP

Challenge:	Existing approaches to code generation rely on rejection sampling to generate multiple code snippets then select the best.
Approach:	They propose a framework that prioritizes sampling on test problems that models can solve.
Outcome:	The proposed framework reduces sampling costs while maintaining comparable code generation performance.

Learning to Explore and Select for Coverage-Conditioned Retrieval-Augmented Generation (2025.findings-naacl)

Copied to clipboard

Takyoung Kim, Kyungjae Lee, Young Rok Jang, Ji Yong Cho, Gangwoo Kim, Minseok Cho, Moontae Lee

IRPO: Implicit Policy Regularized Preference Optimization (2026.findings-eacl)

Challenge:	Recent advances of large language models (LLMs) have enabled them to provide long and detailed responses by leveraging their parametric knowledge.
Approach:	They introduce a dataset of 10K hierarchical sets of information-seeking subqueries that define structured boundaries for outline creation and evaluation in C2 scenarios.
Outcome:	The proposed model generates higher-quality outlines that better fulfill diverse user needs when trained with alignment techniques like DPO.

Copied to clipboard

Youngsoo Jang, Yu Jin Kim, Geon-Hyeong Kim, Honglak Lee, Moontae Lee

Efficiently Learning To Reason or Not to Reason: Root-token Policy Optimization for Adaptive Thinking (2026.acl-long)

Challenge:	Recent DPOs introduce additional hyperparameters, reducing feasibility for LLM fine-tuning.
Approach:	They propose an algorithm that regularizes the reward against a reference policy without extra hyperparameters to address suboptimal outcomes.
Outcome:	The proposed algorithm outperforms baseline algorithms with the same hyperparameter complexity while maintaining training simplicity.

Copied to clipboard

Taehyeon Kim, Hyunsoo Lee, Youngsoo Jang, Moontae Lee

Responsible & Trustworthy NLP

Challenge:	Large reasoning models (LRMs) externalize explicit reasoning traces before producing the answer, yet suffer from overthinking challenge.
Approach:	They propose a framework that enables large reasoning models to self-determine when to reason by training only the initial root token via group relative reward and group-wise advantages.
Outcome:	The proposed framework reduces training overhead and VRAM usage by focusing on the root token . it learns difficulty-aware adaptive thinking at just 2% of the training compute of prior methods.

Overlapping Context with Variable-Length Stride Increases Diversity when Training Large Language Model for Code (2025.acl-industry)

Copied to clipboard

Geonmo Gu, Jaeho Kwak, Haksoo Moon, Hyun Seung Shim, Yu Jin Kim, Byoungjip Kim, Moontae Lee, Hyejeong Jeon

Merging Generated and Retrieved Knowledge for Open-Domain QA (2023.emnlp-main)

Challenge:	Large language models for code (LLMs) are gaining more and more attention due to their wide applicability.
Approach:	They propose a method which extracts overlapping contexts from training data using variable-length stride.
Outcome:	The proposed method outperforms the conventional approach of controlling the number of epochs in terms of the pass@k rate.

Copied to clipboard

Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Lu Wang

Information Retrieval

Challenge:	Open-domain question answering systems often have retrieval modules but retrieving passages from external knowledge sources is known to suffer from insufficient knowledge coverage.
Approach:	They propose a Compatibility-Oriented knowledge Merging framework to leverage both sources of information by matching LLM-generated passages with retrieved counterparts into compatible pairs.
Outcome:	The proposed framework outperforms baselines on three out of four tested open-domain QA benchmarks.

The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models (2025.naacl-long)

Copied to clipboard

Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim, Dongkeun Yoon, Guijin Son, Yejin Cho, Sheikh Shafayat, Jinheon Baek, Sue Hyun Park, Hyeonbin Hwang, Jinkyung Jo, Hyowon Cho, Haebin Shin, Seongyun Lee, Hanseok Oh, Noah Lee, Namgyu Ho, Se June Joo, Miyoung Ko, Yoonjoo Lee, Hyungjoo Chae, Jamin Shin, Joel Jang, Seonghyeon Ye, Bill Yuchen Lin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo

When to Read Documents or QA History: On Unified and Selective Open-domain QA (2023.findings-acl)

Challenge:	a recent study evaluated language models using abstract evaluation criteria that lack the flexibility and granularity of human assessment.
Approach:	They propose a benchmark to evaluate nine distinct language models' capabilities . they use instance-specific evaluation criteria to mirror human evaluation .
Outcome:	The proposed benchmark evaluates nine distinct capabilities of language models across 77 tasks.

Copied to clipboard

Kyungjae Lee, Sang-eun Han, Seung-won Hwang, Moontae Lee

Natural Language Interfaces

Challenge:	Existing work aims to answer factoid questions from an open set of domains using knowledge sources.
Approach:	They propose to use QA-pair and document corpora to answer open-domain questions . they propose to apply natural follow-up to both models to find answers .
Outcome:	The proposed method is validated on natural questions and TriviaQA.

Adaptive Retrieval for Reasoning (2026.acl-long)

Copied to clipboard

Jongho Kim, Jaeyoung Kim, Jihyuk Kim, Yu Jin Kim, Seung-won Hwang, Moontae Lee

When ”A Helpful Assistant” Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models (2024.findings-emnlp)

Challenge:	Existing reasoning-based rerankers suffer from bounded recall.
Approach:	They propose a framework that leverages adaptive retrieval to ensure sufficient "bridge" documents are retrieved for reasoning-intensive retrieval.
Outcome:	The proposed method outperforms baselines on reasoning-intensive retrieval tasks by 5.6%pt.

Copied to clipboard

Mingqian Zheng, Jiaxin Pei, Lajanugen Logeswaran, Moontae Lee, David Jurgens

DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocode (2026.findings-acl)

Challenge:	Commercial AI systems often define the role of the LLM in system prompts.
Approach:	They conduct a systematic evaluation of personas in system prompts by adding 162 roles covering 6 types of interpersonal relationships and 8 domains of expertise.
Outcome:	The proposed model does not improve performance in the system prompt setting where no persona is added.

Copied to clipboard

Hojae Han, Jaejin Kim, Seung-won Hwang, Yu Jin Kim, Moontae Lee

Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2023.acl-long)

Challenge:	Recent studies have shown that test output prediction is difficult to achieve due to code errors.
Approach:	They propose a framework that grounds prediction on error-resilient pseudocode and simulates execution via LLM reasoning to overcome limitations of direct execution suffering from code errors.
Outcome:	The proposed framework improves Pass@1 on LiveCodeBench, BigCodeBech-Hard, DevEval and HumanEval(+) and improves on pass@1 by 13.6 pp.

Copied to clipboard

Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, Minjoon Seo

Small Language Models Need Strong Verifiers to Self-Correct Reasoning (2024.findings-acl)

Challenge:	Recent work shows that an adversary can extract training data from Pretrained Language Models including Personally Identifiable Information (PII) such as names, phone numbers, and email addresses.
Approach:	They propose to use knowledge unlearning to reduce privacy risks for LMs by performing gradient ascent on target token sequences instead of trying to unlearn all the data at once.
Outcome:	The proposed method can give a stronger empirical privacy guarantee in scenarios where the data vulnerable to extraction attacks are known a priori while being much more efficient and robust.

Copied to clipboard

Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, Lu Wang