Papers by Rujun Han
SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback (2026.findings-eacl)
Copied to clipboard
Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I-Hung Hsu, Jun Yan, Vishy Tirumalashetty, Eunsol Choi, Tomas Pfister, Chen-Yu Lee
| Challenge: | High-quality, complex question-answer pairs are pivotal for training and evaluating capable deep search agents. |
| Approach: | They propose a pipeline that generates high-quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. |
| Outcome: | The proposed pipeline generates high-quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. |
ACCENT: An Automatic Event Commonsense Evaluation Metric for Open-Domain Dialogue Systems (2023.acl-long)
Copied to clipboard
| Challenge: | evaluating commonsense in dialogue systems remains an open challenge . despite the success of open-domain dialogue systems, systems struggle to produce commonsensical responses as humans do. |
| Approach: | They propose an event commonsense evaluation metric empowered by commonsensence knowledge bases. |
| Outcome: | The proposed metric achieves higher correlations with human judgments than baselines. |
Reverse Thinking Makes LLMs Stronger Reasoners (2025.naacl-long)
Copied to clipboard
Justin Chen, Zifeng Wang, Hamid Palangi, Rujun Han, Sayna Ebrahimi, Long Le, Vincent Perot, Swaroop Mishra, Mohit Bansal, Chen-Yu Lee, Tomas Pfister
| Challenge: | Reverse-Enhanced Thinking (RevThink) is a framework for large language models to perform reverse thinking. |
| Approach: | They propose a framework for enhancing forward-backward reasoning by collecting data from a teacher model and employing three objectives to train a student model in a multi-task learning fashion. |
| Outcome: | The proposed framework outperforms a fine-tuning method trained on 10x more forward reasoning on 12 datasets covering commonsense, math, and logical reasoning. |
In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents (2025.acl-long)
Copied to clipboard
Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Rajan Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, Tomas Pfister
| Challenge: | Existing approaches to long-term dialogue memory management fail to capture the natural semantic structure of conversations, leading to fragmented and incomplete representations. |
| Approach: | They propose a mechanism that integrates forward- and backward-looking reflections into a personalized memory bank for effective future retrieval. |
| Outcome: | The proposed mechanism outperforms state-of-the-art benchmarks on a long-term dialogue memory model. |
ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PTLMs) have achieved noticeable success on many NLP tasks, but struggle for tasks that require event temporal reasoning. |
| Approach: | They propose a continual pre-training approach that equips PTLMs with targeted knowledge about event temporal relations by focusing on masked-out event and temporal indicators and discriminating sentences from their corrupted counterparts. |
| Outcome: | The proposed framework improves the PTLMs’ fine-tuning performances across five relation extraction and question answering tasks and achieves new or on-par state-of-the-art in most of our downstream tasks. |
Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction (D19-1)
Copied to clipboard
| Challenge: | Existing systems treat this task as a pipeline of two separate subtasks, i.e., event extraction and temporal relation classification. |
| Approach: | They propose a joint event and temporal relation extraction model with shared representation learning and structured prediction. |
| Outcome: | The proposed method improves both event extraction and temporal relation extraction over state-of-the-art systems. |
EventPlus: A Temporal Event Understanding Pipeline (2021.naacl-demos)
Copied to clipboard
Mingyu Derek Ma, Jiao Sun, Mu Yang, Kung-Hsiang Huang, Nuan Wen, Shikhar Singh, Rujun Han, Nanyun Peng
| Challenge: | Event information is a type of common sense knowledge that helps people understand how stories evolve and provides predictive hints for future events. |
| Approach: | They propose a temporal event understanding pipeline that integrates state-of-the-art components. |
| Outcome: | The proposed pipeline can be easily adapted to other domains, including biomedical domains. |
RobustQA: Benchmarking the Robustness of Domain Adaptation for Open-Domain Question Answering (2023.findings-acl)
Copied to clipboard
Rujun Han, Peng Qi, Yuhao Zhang, Lan Liu, Juliette Burger, William Yang Wang, Zhiheng Huang, Bing Xiang, Dan Roth
| Challenge: | Existing ODQA datasets consist mainly of Wikipedia corpus, and are insufficient to study models’ generalizability across diverse domains. |
| Approach: | They propose a benchmark to evaluate ODQA's domain robustness using Wikipedia corpus . they annotate QA pairs in retrieval datasets with rigorous quality control . |
| Outcome: | The proposed benchmark improves model performance on annotated QA pairs in retrieval datasets with rigorous quality control. |
Go Back in Time: Generating Flashbacks in Stories with Event Temporal Prompts (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing systems that generate *flashbacks* are monotonic and lack explicit guidance on how to insert them. |
| Approach: | They propose to use event temporal orders to encode events as temporal prompts . they leverage a Plan-and-Write framework enhanced by reinforcement learning to generate storylines . |
| Outcome: | The proposed method generates more interesting stories with *flashbacks* while maintaining textual diversity, fluency, and temporal coherence. |
ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. |
| Approach: | They propose a machine reading comprehension dataset that leverages natural language queries to reason about the five most common event semantic relations. |
| Outcome: | The proposed dataset shows that current SOTA systems achieve 22.1%, 63.3% and 83.5% for token-based exact-match, **F1** and event-based **HIT@1** scores. |
Conditional Word Embedding and Hypothesis Testing via Bayes-by-Backprop (D18-1)
Copied to clipboard
| Challenge: | Whether word's meaning varies across contexts has become a major focus of research in recent years. |
| Approach: | They propose a word embedding model that incorporates document covariates to estimate conditional word embeds. |
| Outcome: | The proposed model estimates word embedding distributions based on document covariates . if word embeds are statistically significant, hypothesis tests can be performed . |
RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering (2024.emnlp-main)
Copied to clipboard
Rujun Han, Yuhao Zhang, Peng Qi, Yumo Xu, Jenyuan Wang, Lan Liu, William Yang Wang, Bonan Min, Vittorio Castelli
| Challenge: | Existing datasets for question answering based on retrieval augmented generation (RAG-QA) are either constructed using a single source corpus or consist of short extractive answers, which fall short of evaluating large language model (LLM) based RAG-QA systems on cross-domain generalization. |
| Approach: | They propose a dataset that integrates short extractive answers from multiple documents into a single coherent narrative. |
| Outcome: | The proposed dataset integrates short extractive answers from multiple documents into a single coherent narrative, covering 26K queries and large corpora across seven different domains. |
TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current machine reading comprehension benchmarks have no questions that test temporal phenomena . a new study studies reading comprehension for temporal relations . |
| Approach: | They propose a reading comprehension benchmark built on news snippets and 21k human-generated questions querying temporal relationships. |
| Outcome: | The new reading comprehension benchmark TORQUE achieves an exact-match score of 51% on the test set . the benchmark is built on 3.2k news snippets with 21k human-generated questions . |
COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context (2026.acl-long)
Copied to clipboard
| Challenge: | Long-horizon tasks that require sustained reasoning and multiple tool interactions remain challenging for LLM agents. |
| Approach: | They propose a framework that separates tactical execution, strategic oversight, and context organization into three specialized components. |
| Outcome: | The proposed framework improves accuracy by 20% relative to baselines on GAIA, BrowseComp, and Humanity’s Last Exam tasks. |
Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language Models (2024.emnlp-main)
Copied to clipboard
Zhengxuan Wu, Yuhao Zhang, Peng Qi, Yumo Xu, Rujun Han, Yian Zhang, Jifan Chen, Bonan Min, Zhiheng Huang
| Challenge: | Modern language models fail to follow human instructions while being faithful . a trade-off exists between instruction following and faithfulness when training LMs . |
| Approach: | They propose a method that relies on Reject-sampling by Self-instruct with Continued Fine-tuning to train LMs to follow human instructions while being faithful. |
| Outcome: | The proposed method outperforms vanilla MTL with high-quality data, but with significantly smaller data. |
Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to extract event temporal relations from text data are limited by hard constraints and large datasets. |
| Approach: | They propose a framework that enhances deep neural network with distributional constraints constructed by probabilistic domain knowledge to improve the baseline neural network models. |
| Outcome: | The proposed framework improves baseline models with strong statistical significance on two widely used datasets in news and clinical domains. |
Modeling Context in Answer Sentence Selection Systems on a Latency Budget (2021.eacl-main)
Copied to clipboard
| Challenge: | Current AS2 models score question-answer pairs individually, ignoring any information from the document each potential answer was extracted from. |
| Approach: | They propose an approach to efficiently incorporate contextual information into AS2 models . they use unsupervised similarity techniques to extract relevant sentences from source document . |
| Outcome: | The proposed approach improves 6% to 11% over state-of-the-art in AS2 with minimal latency. |
CiteEval: Principle-Driven Citation Evaluation for Source Attribution (2025.acl-long)
Copied to clipboard
Yumo Xu, Peng Qi, Jifan Chen, Kunlun Liu, Rujun Han, Lan Liu, Bonan Min, Vittorio Castelli, Arshit Gupta, Zhiguo Wang
| Challenge: | Current evaluation frameworks rely on NLI to assess binary or ternary support from cited sources, which is suboptimal for citation evaluation. |
| Approach: | They propose a citation evaluation framework based on fine-grained citation ratings within a broad context and construct a multi-domain benchmark with high-quality human annotations. |
| Outcome: | The proposed framework provides a high-quality human annotation benchmark and a suite of model-based metrics that exhibit strong correlation with human judgments. |
Character-centric Story Visualization via Visual Planning and Token Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Story visualization is a task that requires machines to understand long text inputs and produce a globally consistent image sequence that illustrates the contents of the story. |
| Approach: | They propose to augment VQ-VAE with a text-to-visual-token (transformer) architecture to enable multiple image generation based on a complete story. |
| Outcome: | The proposed method excels at preserving characters and produces higher quality image sequences compared with baselines. |