Papers by Jiaming Han
Omni-RewardBench: Toward a Comprehensive Evaluation of Generative Reward Models Across Modalities (2026.acl-long)
Copied to clipboard
Chi-Min Chan, Yujin Zhou, Pengcheng Wen, Boqin Yin, Jiaming Ji, Juntao Dai, Wei Xue, Sirui Han, Yike Guo
| Challenge: | Existing evaluation benchmarks for ORMs are largely text-centric or limited to bimodal tasks . a new study examines the effectiveness of Omni-RewardBench for ORms across modalities . |
| Approach: | They propose a hybrid automatic-annotation and human-verification pipeline to construct high-quality evaluation data. |
| Outcome: | The proposed model is the first benchmark for comprehensive evaluation of ORMs across modalities. |
Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion (2022.findings-acl)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) aims to extract semantic relations among entity pairs in a document. |
| Approach: | They propose an evidence-enhanced framework that empowers document-level relation extraction (DocRE) Eider efficiently extracts evidence and effectively fuses extracted evidence in inference. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three benchmark datasets. |
SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery (2020.emnlp-main)
Copied to clipboard
| Challenge: | Entity set expansion and synonym discovery are two critical NLP tasks that are often performed separately, without exploring their interdependencies. |
| Approach: | They propose a framework that enables two tasks to mutually enhance each other by including popular entities’ infrequent synonyms into the set, which boosts set expansion recall. |
| Outcome: | The proposed framework can be used to enhance two NLP tasks by including popular entities’ infrequent synonyms into the set, which boosts set expansion recall. |
Probing Audio-Visual Reasoning in Multimodal Language Models through the Lens of Audio (2026.acl-long)
Copied to clipboard
Kaixiong Gong, Kaituo Feng, Bohao Li, Yibing Wang, Mofan Cheng, Shijia Yang, Jiaming Han, Benyou Wang, Yutong Bai, Zhuoran Yang, Xiangyu Yue
| Challenge: | Recent multimodal large language models lack robust audio-visual integration ability and performance on DeafTest is highly correlated with AV-Odyssey accuracy. |
| Approach: | They propose a benchmarking tool that integrates audio-visual reasoning with audio-video cues to infer solutions. |
| Outcome: | The proposed model performs well on DeafTest, but lacks audio perception in simple audio tasks. |
SafeMT: Multi-turn Safety for Multimodal Language Models (2026.acl-long)
Copied to clipboard
Han Zhu, Juntao Dai, Jiaming Ji, Haoran Li, Chengkun Cai, Pengcheng Wen, Chi-Min Chan, Boyuan Chen, Yaodong Yang, Sirui Han, Yike Guo
| Challenge: | Multi-turn dialogues pose a greater risk than single prompts, but existing safety benchmarks do not account for this situation. |
| Approach: | They propose a benchmark that features dialogues of varying lengths generated from harmful queries accompanied by images. |
| Outcome: | The proposed model reduces multi-turn Attack Success Rate (ASR) compared to existing guard models. |
Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Linguistic steganography studies how to hide secret messages in natural language cover texts. |
| Approach: | They propose a method which encodes secret messages using self-adjusting arithmetic coding based on a neural language model. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on four datasets by 15.3% and 38.9% in terms of bits/word and KL metrics. |
Boosting Policy and Process Reward Models with Monte Carlo Tree Search in Open-Domain QA (2025.findings-acl)
Copied to clipboard
Chi-Min Chan, Chunpu Xu, Junqi Zhu, Jiaming Ji, Donghai Hong, Pengcheng Wen, Chunyang Jiang, Zhen Ye, Yaodong Yang, Wei Xue, Sirui Han, Yike Guo
| Challenge: | Experimental results show that our approach can effectively improve the performance of both the policy model and the reward model. |
| Approach: | They propose to use Monte Carlo Tree Search for both policy model improvement and reward model improvement to bridge it to more subtle open-domain question answering. |
| Outcome: | The proposed approach surpasses existing methods for annotation and training data with fewer data points and achieves better performance in test-time scaling strategies. |
Benchmarking Fine-Grained Error Detection in Multimodal Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Process Reward Models (MPRMs) have emerged as a pivotal framework for enhancing the reasoning capabilities of Multimodal Large Language Models. |
| Approach: | They propose a benchmark specifically designed to evaluate MPRMs’ proficiency in detecting erroneous reasoning steps across diverse error categories. |
| Outcome: | The proposed model achieves up to 4.8% performance improvement through test-time scaling. |
Empower Entity Set Expansion via Language Model Probing (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for expanding seed entities with new entities belong to the same semantic class are difficult to implement and can lead to accumulative errors. |
| Approach: | They propose an iterative set expansion framework that leverages automatically generated class names to address the semantic drift issue. |
| Outcome: | The proposed framework generates high-quality class names and outperforms state-of-the-art methods significantly. |
TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names (2021.naacl-main)
Copied to clipboard
| Challenge: | Hierarchical multi-label text classification (HMTC) aims to assign each text document to a set of relevant classes from a taxonomy. |
| Approach: | They propose to conduct HMTC based on only class surface names as supervision signals to mimic human experts. |
| Outcome: | The proposed framework outperforms the best existing method by 25% on two challenging datasets. |
PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs (2024.findings-acl)
Copied to clipboard
Rongzhi Zhang, Jiaming Shen, Tianqi Liu, Haorui Wang, Zhen Qin, Feng Han, Jialu Liu, Simon Baumgartner, Michael Bendersky, Chao Zhang
| Challenge: | Knowledge distillation (KD) is a technique for transferring expertise from large teacher models to compact student models with reduced memory footprints and inference costs. |
| Approach: | They propose to transfer knowledge from large teacher models to compact student models by exploiting teacher-student capacity discrepancies to generate pseudo-preference pairs where teacher outputs are preferred over student outputs. |
| Outcome: | The proposed framework exploits teacher-student capacity discrepancy to generate pseudo-preference pairs where teacher outputs are preferred over student outputs. |
Phrase-aware Unsupervised Constituency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have achieved inspiring success in unsupervised grammar induction using masked language modeling (MLM) as the proxy task. |
| Approach: | They propose to regularize the parser with phrases extracted by an unsupervised phrase tagger to help the LM model quickly manage low-level structures. |
| Outcome: | The proposed method improves the identification of high-level structures using phrase-guided masking. |
Training ELECTRA Augmented with Multi-word Selection (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-training methods for NLP tasks require massive computation resources. |
| Approach: | They propose a method that trains a discriminator to detect replaced tokens and select original tokens from candidate sets. |
| Outcome: | The proposed method improves ELECTRA based on multi-task learning on GLUE and SQUAD datasets. |
LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoning (2025.acl-long)
Copied to clipboard
Weijie Shi, Han Zhu, Jiaming Ji, Mengze Li, Jipeng Zhang, Ruiyuan Zhang, Jia Zhu, Jiajie Xu, Sirui Han, Yike Guo
| Challenge: | Existing legal judgment prediction methods struggle with logical errors when conducting complex legal reasoning. |
| Approach: | They propose a method which enhances LJP reliability through step-wise verification and correction of the reasoning process. |
| Outcome: | The proposed model significantly improves concordance with court decisions from 72.37 to 80.27 on LLAMA-3.1-70B. |
PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference (2025.acl-long)
Copied to clipboard
Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Alex Qiu, Jiayi Zhou, Kaile Wang, Boxun Li, Sirui Han, Yike Guo, Yaodong Yang
| Challenge: | Using large-scale annotation data, large language models can generate noise, errors and biases, leading to unexpected behaviours. |
| Approach: | They propose a dataset to promote safety alignment in large language models . they separate helpfulness and harmlessness annotations for question-answering pairs . |
| Outcome: | The proposed dataset provides 44.6k prompts and 265k question-answer pairs with safety meta-labels for 19 harm categories and three severity levels, with answers generated by Llama-family models. |
Topic Taxonomy Expansion via Hierarchy-Aware Topic Phrase Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for topic taxonomies focus on frequent terms and local topic-subtopic relations, which leads to limited topic term coverage. |
| Approach: | They propose a framework for topic taxonomy expansion that directly generates topic-related terms belonging to new topics. |
| Outcome: | The proposed framework outperforms baseline methods on two real-world text corpora. |
Corpus-based Open-Domain Event Type Induction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing event extraction methods require predefined event types and their annotations to learn event extractors. |
| Approach: | They propose to represent each event type as a cluster of predicate sense, object head> pairs. |
| Outcome: | The proposed method can discover salient and high-quality event types on three datasets from different domains. |
SafeLawBench: Towards Safe Alignment of Large Language Models (2025.findings-acl)
Copied to clipboard
Chuxue Cao, Han Zhu, Jiaming Ji, Qichao Sun, Zhenghao Zhu, Wu Yinyu, Josef Dai, Yaodong Yang, Sirui Han, Yike Guo
| Challenge: | Recent studies indicate that large language models (LLMs) may exhibit risks, including threats to the protection of private data and the generation of hallucinations. |
| Approach: | They propose to evaluate LLMs from a legal perspective using the SafeLawBench benchmark. |
| Outcome: | The proposed framework categorizes safety risks into three levels based on legal standards and includes 24,860 multi-choice questions and 1,106 open-domain question-answering tasks. |
When Slower Isn’t Truer: Inverse Scaling Law of Truthfulness in Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
Sitong Fang, Wenjing Cao, Jiahao Li, Xuyao Wang, Chi-Min Chan, Sirui Han, Juntao Dai, Yike Guo, Yaodong Yang, Jiaming Ji
| Challenge: | a study of slow reasoning models for multimodal reasoning finds that they are more prone to fabricating plausible yet false details when confronted with incomplete or misleading visual inputs. |
| Approach: | They conduct the first systematic study of the inverse scaling law in slow-thinking paradigms for multimodal reasoning. |
| Outcome: | The findings suggest that slower reasoning models are more prone to fabricating false details . the study analyzed 5,000-sample hierarchical prompt dataset by 50 participants . |
End-to-End Reinforcement Learning for Automatic Taxonomy Induction (P18-1)
Copied to clipboard
| Challenge: | Existing methods for automating taxonomy induction often divide the problem into two subtasks . a novel end-to-end reinforcement learning approach is proposed to improve the accuracy of such methods. |
| Approach: | They propose an end-to-end reinforcement learning approach to automatic taxonomy induction from a set of terms. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on two public datasets of different domains. |
Benchmarking Multi-National Value Alignment for Large Language Models (2025.findings-acl)
Copied to clipboard
Chengyi Ju, Weijie Shi, Chengzhong Liu, Jiaming Ji, Jipeng Zhang, Ruiyuan Zhang, Jiajie Xu, Yaodong Yang, Sirui Han, Yike Guo
| Challenge: | Existing studies on large language models focus on ethical reviews, failing to capture the diversity of national values. |
| Approach: | They propose a national value extraction pipeline to efficiently construct value assessment datasets and a model-based model with instruction tagging to process raw data sources. |
| Outcome: | The proposed benchmark evaluates the alignment of LLMs with the values of five major nations: China, the United States, the UK, France, and Germany. |
FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation (2025.acl-long)
Copied to clipboard
Junyu Luo, Zhizhuo Kou, Liming Yang, Xiao Luo, Jinsheng Huang, Zhiping Xiao, Jingshu Peng, Chengzhong Liu, Jiaming Ji, Xuanzhe Liu, Sirui Han, Ming Zhang, Yike Guo
| Challenge: | Multimodal Large Language Models (MLLMs) have experienced rapid development in recent years, but there is a notable lack of effective and specialized multimodal evaluation datasets in the financial domain. |
| Approach: | They introduce FinMME, a multimodal large language model with 11,000 financial research samples and 20 annotators. |
| Outcome: | The proposed model performs better than state-of-the-art models, highlighting its challenging nature. |
Reward Generalization in RLHF: A Topological Perspective (2025.findings-acl)
Copied to clipboard
Tianyi Alex Qiu, Fanzhi Zeng, Jiaming Ji, Dong Yan, Kaile Wang, Jiayi Zhou, Yang Han, Josef Dai, Xuehai Pan, Yaodong Yang
| Challenge: | Existing alignment methods share a common topology of information flow, but their alternatives have not been thoroughly explored. |
| Approach: | They propose a theory of reward generalization in reinforcement learning from human feedback . they propose induced Bayesian networks to model the impact of dataset topologies on reward generalisation . |
| Outcome: | The proposed method achieves an average win rate of 65% on three NLP tasks. |