Papers by Jiahui Li
iPrOp: Interactive Prompt Optimization for Large Language Models with a Human in the Loop (2025.acl-srw)
Copied to clipboard
| Challenge: | Prompt engineering has made significant contributions to the era of large language models, yet its effectiveness depends on the skills of a prompt author. |
| Approach: | They propose a novel approach to prompt optimization that bridges manual prompt engineering and automatic prompt optimization by providing task-specific guidance. |
| Outcome: | The proposed approach bridges manual prompt engineering and automatic prompt optimization while offering users the flexibility to assess evolving prompts. |
LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Experts (2026.eacl-long)
Copied to clipboard
| Challenge: | LM-LEXICON is a definition modeling approach that integrates data clustering, semantic expert learning, and model merging. |
| Approach: | They propose a definition modeling approach that integrates data clustering, semantic expert learning, and model merging using a sparse mixture-of-experts architecture. |
| Outcome: | The proposed model outperforms existing methods on five widely used benchmarks and achieves a BLEU score of 7%. |
Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Despite advances in training Large Language Models, they remain vulnerable to jailbreak, an adversarial attack method. |
| Approach: | They propose an adversarial jailbreak algorithm that exploits the gradient information of the suffix tokens to accelerate the optimization process. |
| Outcome: | The proposed model achieves 1.5x speedup while maintaining high attack success rates. |
CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval (2026.acl-long)
Copied to clipboard
Jiahui Geng, Fengyu Cai, Shaobo Cui, Qing Li, Liangwei Chen, Chenyang Lyu, Haonan Li, Derui Zhu, Alexander Pretschner, Heinz Koeppl, Fakhri Karray
| Challenge: | Existing benchmarks focus on functional relevance while neglecting code quality. |
| Approach: | They propose a multilingual benchmark to evaluate quality-aware code retrieval . they include fine-grained quality annotations over 42,725 queries and 134,907 code snippets . |
| Outcome: | The proposed benchmarks show that state-of-the-art models fail to separate buggy or insecure code from robust counterparts. |
GER-LLM: Efficient and Effective Geospatial Entity Resolution with Large Language Model (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for integrating spatial data from diverse sources are limited by their reliance on large amounts of training data and their inability to incorporate commonsense knowledge. |
| Approach: | They propose a framework that integrates large language models into the GER pipeline. |
| Outcome: | The proposed framework improves on real-world geospatial datasets and shows that it is more efficient than state-of-the-art methods. |
Which Demographics do LLMs Default to During Annotation? (2025.acl-long)
Copied to clipboard
Johannes Schäfer, Aidan Combs, Christopher Bagdon, Jiahui Li, Nadine Probol, Lynn Greschner, Sean Papay, Yarik Menchaca Resendiz, Aswathy Velutharambath, Amelie Wuehrl, Sabine Weber, Roman Klinger
| Challenge: | Demographics and cultural background of annotators influence the labels they assign in text annotation. |
| Approach: | They examine the attributes of human annotators LLMs inherently mimic and compare them to demographic-conditioned prompts and placebo-conditioned ones. |
| Outcome: | The proposed model incorporates demographics and cultural background into the output of the large language models (LLMs) to evaluate which attributes of human annotators LLMs inherently mimic. |
SGPVT: Self-Generated Proximal Visual Tokens for Mitigating Proximal Collateral Damage in MLLM Unlearning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches focus on general utility metrics, overlooking the preservation of semantically related concepts. |
| Approach: | They propose a method that introduces self-generated proximal visual tokens to prevent forgetting vulnerability. |
| Outcome: | The proposed framework outperforms existing methods in preserving semantically related concepts while achieving effective target unlearning. |
Look Less, Reason More: Rollout-Guided Adaptive Pixel-Space Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown promise by incorporating pixel-level visual information into the reasoning process, enabling VLMs to access high-resolution visual details during their thought process. |
| Approach: | They propose a framework that dynamically determines necessary pixel-level operations based on the input query. |
| Outcome: | The proposed model achieves 73.4% accuracy on HR-Bench 4K while maintaining a tool usage ratio of only 20.1%, improving accuracy and reducing tool usage by 66.5% compared to the previous methods. |
Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language (2025.acl-long)
Copied to clipboard
Bo Zeng, Chenyang Lyu, Sinuo Liu, Mingyan Zeng, Minghao Wu, Xuanfan Ni, Tianqi Shi, Yu Zhao, Yefeng Liu, Chenyu Zhu, Ruizhe Li, Jiahui Geng, Qing Li, Yu Tong, Longyue Wang, Weihua Luo, Kaifu Zhang
| Challenge: | Existing datasets for instruction-following are monolingual and centered on English . existing data are unable to capture linguistic and cultural subtle differences . |
| Approach: | They propose an extension of IFEval to a localized multilingual version called Marco-Bench-MIF . their benchmark addresses linguistic constraints and cultural references via translation and verification . |
| Outcome: | The proposed extension of IFEval to a localized multilingual version covers 30 languages with varying levels of localization. |
Student Guides Teacher: Weak-to-Strong Inference via Spectral Orthogonal Exploration (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Large Language Models suffer from "Reasoning Collapse" on mathematical reasoning tasks where stochastic sampling produces lexical variations of the same erroneous logic rather than genuine semantic exploration. |
| Approach: | They propose a geometric inference framework that uses a spectral orthogonal probe to introduce semantically heterogeneous reasoning signals into the teacher's orthogonale complement of its dominant subspace. |
| Outcome: | The proposed framework improves accuracy and sampling efficiency over baseline methods on logic and code generation benchmarks. |
VisualEDU: A Benchmark for Assessing Coding and Visual Comprehension through Educational Problem-Solving Video Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | VisualEDU is a benchmark to evaluate VLMs' ability to produce coherent video from text . advanced proprietary models show promise, but struggle with increasing task complexity . |
| Approach: | VisualEDU is a benchmark to evaluate VLMs' ability to produce coherent video from text . it integrates meta-prompt learning, visual and code feedback, and a drawing toolkit to enhance output quality. |
| Outcome: | VisualEDU is a benchmark to evaluate VLMs' ability to produce coherent video from text . it integrates meta-prompt learning, visual and code feedback, and a drawing toolkit to improve output quality. |
Think Faster Than Words: Efficient LLM Chain-of-Thought Reasoning via Dynamic Shortcut Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods that prune or employ early stopping to reduce latency often compromise reasoning reliability. |
| Approach: | They propose a shortcut decoding framework that integrates probes over internal hidden states with step-level entropy to detect convergence of reasoning during generation and adaptively selects between a fast-exit path and a stability-verified path to remove redundant steps while preserving answer correctness. |
| Outcome: | The proposed framework reduces token usage by approximately 35% and maintains accuracy comparable to full CoT decoding. |
Pointing to a Llama and Call it a Camel: On the Sycophancy of Multimodal Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal large language models exhibit a pronounced form of visual sycophantic behavior when they process image inputs. |
| Approach: | They propose a technique that allows multimodal large language models to engage in reflective reasoning and determine whether a user’s instruction is misleading or corrective. |
| Outcome: | The proposed model resists misleading instructions but is stubborn even if it is wrong. |
Judge and Improve: Towards a Better Reasoning of Knowledge Graphs with Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to integrating graph and language models face two key limitations: achieving robust semantic alignment and ensuring interpretability in outputs. |
| Approach: | They propose a framework to integrate graph and language modalities while enhancing transparency. |
| Outcome: | Extensive experiments on three benchmark datasets show that the proposed framework surpasses existing methods in efficiency and generates outputs that are significantly more interpretable. |
RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results demonstrate the superiority of our approach to aligning large language models with human preferences. |
| Approach: | They propose a method that evaluates and assigns specific credit to each token using an off-the-shelf reward model. |
| Outcome: | The proposed method evaluates and assigns specific credit to each token using an off-the-shelf reward model. |
Multi-perspective Preference Alignment of LLMs for Programming-Community Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Extensive experiments on a high-quality, real-world PCQA dataset validate its accuracy and preference. |
| Approach: | They propose a multi-perspective preference alignment for programming-community question answering to generate user-centric responses. |
| Outcome: | Experiments on a high-quality, real-world PCQA dataset validate the proposed model's accuracy and preference. |
Learning to Edit: Aligning LLMs with Knowledge Editing (2024.acl-long)
Copied to clipboard
Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, Qun Liu, Wei Wang
| Challenge: | Existing knowledge editing techniques rely on memorizing updated knowledge, impeding LLMs from effectively combining the new knowledge with their inherent knowledge when answering questions. |
| Approach: | They propose a Learning to Edit framework that equips LLMs with the ability to apply updated knowledge to input questions through a two-phase process . |
| Outcome: | The proposed framework outperforms existing methods in knowledge editing tasks and compares it with four benchmarks and two LLM architectures. |
ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom (2025.emnlp-main)
Copied to clipboard
Jingqi Zhou, Sheng Wang, Jingwei Dong, Kai Liu, Lei Li, Jiahui Gao, Jiyue Jiang, Lingpeng Kong, Chuan Wu
| Challenge: | Large vision-language models often prioritize language knowledge over image information on visual reasoning tasks, incurring performance degradation. |
| Approach: | They propose a visual reasoning framework that decouples vision-reasoning capabilities and multi-run proactive perception. |
| Outcome: | The proposed framework outperforms existing models on benchmarks for open-source and closed-source models with 13.2% performance gain. |
Optimizing Language Models with Fair and Stable Reward Composition in Reinforcement Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has developed algorithms for reinforcement learning from human feedback and AI-generated feedback. |
| Approach: | They propose a method for reinforcement learning from human feedback and AI-generated feedback that incorporates weighting, ranking, and constraining to handle disparate rewards. |
| Outcome: | The proposed method reduces disparity and enhances stability among rewards . empirical results show that the proposed method is efficient and straightforward . |
MCP-Guard: A Multi-Stage Defense-in-Depth Framework for Securing Model Context Protocol in Agentic AI (2026.findings-acl)
Copied to clipboard
Wenpeng Xing, Zhonghao Qi, Yupeng Qin, Yilin Li, Caini Chang, Jiahui Yu, Changting Lin, Zhenzhen Xie, Meng Han
| Challenge: | Large Language Models (LLMs) are vulnerable to jailbreak, authors say . authors propose a robust, layered defense architecture designed for LLM–tool interactions . |
| Approach: | They propose a robust, layered defense architecture designed for LLM–tool interactions . they propose XCP-Guard, which employs a three-stage detection pipeline . |
| Outcome: | The proposed model achieves 96.01% accuracy in identifying adversarial prompts . the model is based on a three-stage detection pipeline that balances efficiency with accuracy . |
PREE: Towards Harmless and Adaptive Fingerprint Editing in Large Language Models via Knowledge Prefix Enhancement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing black-box fingerprinting techniques rely on overfitting high-perplexity trigger patterns . experimental results show that model editing in the fingerprint domain exhibits unique advantages . |
| Approach: | They propose a prefix-enhanced fingerprint editing framework that encodes copyright information into parameter offsets through dual-channel knowledge edit to achieve covert embedding of fingerprint features. |
| Outcome: | The proposed model editing framework achieves 90% trigger precision in mainstream architectures . the proposed model editor achieves the 90% accuracy in mainstream models . |
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios. |
| Approach: | They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset. |
| Outcome: | The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation. |
VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration (2025.findings-acl)
Copied to clipboard
Jiahui Geng, Qing Li, Zongxiong Chen, Yuxia Wang, Derui Zhu, Zhuohan Xie, Chenyang Lyu, Xiuying Chen, Preslav Nakov, Fakhri Karray
| Challenge: | Existing safety calibration methods focus on model undersafety, where the model responds to hazardous queries, while neglecting oversafetiness, where models refuse to answer safe queries. |
| Approach: | They propose safety calibration which addresses both undersafety and oversafetiness by comparing model responses to a novel dataset of 3,600 image-text pairs. |
| Outcome: | The proposed methods have been used to evaluate safety calibration across image-centric and text-centric scenarios. |
HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Hallucination is a significant challenge for large language models, but current methods struggle when non-factual information arises in the early or mid-sequence of outputs, reducing their reliability. |
| Approach: | They propose a method that captures the full dynamics of large language models by using neural differential equations to assess the truthfulness of statements. |
| Outcome: | The proposed method achieves 14% improvement in AUC-ROC on the True-False dataset compared to state-of-the-art methods. |
Mixture of insighTful Experts (MoTE): The Synergy of Reasoning Chains and Expert Mixtures in Self-Alignment (2025.acl-long)
Copied to clipboard
Zhili Liu, Yunhao Gou, Kai Chen, Lanqing Hong, Jiahui Gao, Fei Mi, Yu Zhang, Zhenguo Li, Xin Jiang, Qun Liu, James Kwok
| Challenge: | Recent studies show that reasoning abilities contribute significantly to model safety, while integrating Mixture-of-Experts (MoE) architectures can further enhance alignment. |
| Approach: | They propose a framework that synergistically combines reasoning chains and expert mixtures to improve self-alignment. |
| Outcome: | The proposed framework improves model safety, jailbreak resistance, and over-refusal capabilities, achieving performance comparable to OpenAI’s state-of-the-art o1 model. |
Reference-free Hallucination Detection for Large Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large vision-language models exhibit excellent ability in language understanding, question answering, and conversations of visual inputs, but they are prone to producing hallucinations. |
| Approach: | They propose to use supervised uncertainty quantification methods to detect hallucinations in large vision-language models. |
| Outcome: | The proposed methods outperform the others in detecting hallucinations on four representative LVLMs across two different tasks. |