Papers by Shilong Li
AGD: Adversarial Game Defense Against Jailbreak Attacks in Large Language Models (2025.acl-long)
Copied to clipboard
Shilong Pan, Zhiliang Tian, Zhen Huang, Wanlong Yu, Zhihua Wen, Xinwang Liu, Kai Lu, Minlie Huang, Dongsheng Li
| Challenge: | Existing defenses, including post-training alignment and prompt engineering, struggle with adaptability to out-of-distribution (OOD) attacks. |
| Approach: | They propose an adversarial game-based defense method that dynamically adjusts LLMs’ internal representations to achieve a balanced trade-off between helpfulness and harmlessness. |
| Outcome: | The proposed method improves LLMs’ safety over all baselines. |
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent (2024.findings-emnlp)
Copied to clipboard
Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, Yixin Wang
| Challenge: | Multi-modal Large Language Models (MLLMs) exhibit limited generality and often fall short when compared to specialized models. |
| Approach: | They propose a multi-modal medical agent that picks the most suitable medical tools based on user inputs. |
| Outcome: | The proposed agent performs better than open-source models and the closed-source model, GPT-4o. |
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents (2025.findings-acl)
Copied to clipboard
Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, Guohao Li
| Challenge: | Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexity of constructing tasks and evaluators. |
| Approach: | They propose a cross-environment agent benchmark framework that integrates graph-based evaluation and task generation methods. |
| Outcome: | The proposed framework supports multiple devices and can be easily extended to any environment with a Python interface. |
Clear Up Confusion: Advancing Cross-Domain Few-Shot Relation Extraction through Relation-Aware Prompt Learning (2024.naacl-short)
Copied to clipboard
Ge Bai, Chenji Lu, Daichi Guo, Shilong Li, Ying Liu, Zhang Zhang, Guanting Dong, Ruifang Liu, Sun Yong
| Challenge: | Existing approaches to few-shot Relation Extraction (RE) are prone to confusion when applying knowledge to a target domain with entirely new types of relations. |
| Approach: | They propose a relation-aware prompt learning method with pre-training to clear confusion by decomposing relation types through an innovative label prompt. |
| Outcome: | The proposed method outperforms previous sota methods and yields better results on cross-domain few-shot RE tasks. |
Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination Mitigation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for predicting hallucinations suffer from two drawbacks: Lack of scalable token-level rewards and Neglect of visual-anchored tokens. |
| Approach: | They propose a Token Preference Optimization model with self-calibrated rewards . they propose based on visual-anchored tokens and visual-aware training objective . |
| Outcome: | The proposed model improves hallucination performance by focusing on visual-anchored tokens without fine-grained annotations. |
POMP: Probability-driven Meta-graph Prompter for LLMs in Low-resource Unsupervised Neural Machine Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Low-resource languages (LRLs) face challenges in supervised neural machine translation due to limited parallel data. |
| Approach: | They propose a method that uses a dynamic graph to organize auxiliary languages in prompts to improve LRL translations. |
| Outcome: | The proposed method improves translation accuracy in low-resource languages (LRLs) using auxiliary language pairs and synthetic pseudo-parallel data. |
Always the Best Fit: Adaptive Domain Gap Filling from Causal Perspective for Few-Shot Relation Extraction (2023.findings-emnlp)
Copied to clipboard
Ge Bai, Chenji Lu, Jiaxiang Geng, Shilong Li, Yidong Shi, Xiyan Liu, Ying Liu, Zhang Zhang, Ruifang Liu
| Challenge: | Existing approaches to cross-domain relation extraction have been limited by domains . data bias between domains can be difficult to fill, especially in few-shot scenarios . |
| Approach: | They propose a framework to bridge the semantic gap caused by data bias between domains . they use syntactic structure, label distribution, and entities to calculate causal effects . |
| Outcome: | The proposed framework fills the domain gap and yields better results on the few-shot task. |
See the World, Discover Knowledge: A Chinese Factuality Evaluation for Large Vision Language Models (2025.findings-acl)
Copied to clipboard
Jihao Gu, Yingyao Wang, Pi Bu, Chen Wang, Ziming Wang, Tengtao Song, Donglai Wei, Jiale Yuan, Yingxiu Zhao, Yancheng He, Shilong Li, Jiaheng Liu, Meng Cao, Jun Song, Yingshui Tan, Xiang Li, Wenbo Su, Xiaoyong Zhu, Bo Zheng
| Challenge: | Existing models for large vision language models do not fully reflect their knowledge capacity and reliability, resulting in erroneous outputs that do not align with the image content or provide answers lacking knowledge evidence. |
| Approach: | They propose a Chinese-based benchmark for visual factuality across 8 major topics and 56 subtopics and a multi-hop question construction. |
| Outcome: | The proposed model decouples visual factuality into two parts: seeing the world and discovering knowledge. |
Both Matter: Enhancing the Emotional Intelligence of Large Language Models without Compromising the General Intelligence (2024.findings-acl)
Copied to clipboard
| Challenge: | Emotional Intelligence (EI) is a key concept in the field of human intelligence. |
| Approach: | They propose a method to enhance EI of large language models by naive fine-tuning on EI-related tasks. |
| Outcome: | The proposed method improves EI of two LLM-based assistants without compromising GI. |
MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for complex instruction-following with elaborate constraints rely on a weaker model, especially GPT-4, limiting their application. |
| Approach: | They propose a Multi-granularity Self-Contrastive Training framework to improve instruction alignment without relying on a stronger model. |
| Outcome: | The proposed framework improves instruction-following with elaborate constraints without external supervision on coarse and fine granularity. |
2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision (2025.findings-naacl)
Copied to clipboard
Shilong Li, Yancheng He, Hui Huang, Xingyuan Bu, Jiaheng Liu, Hangyu Guo, Weixun Wang, Jihao Gu, Wenbo Su, Bo Zheng
| Challenge: | Existing methods that optimize for scalar scores or ranking reward ignore multi-dimensional nature of human preferences. |
| Approach: | They propose to extend the preference of Direct Preference Optimization to two dimensions: segments and aspects. |
| Outcome: | The proposed framework decomposes the overall objective into multi-segment and multi-aspect objectives. |
AIR: Complex Instruction Generation via Automatic Iterative Refinement (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating complex instructions are resource-intensive and lack diversity. |
| Approach: | They propose a framework to generate complex instructions with constraints using a document-generated initial instruction and an iterative refinement framework to incorporate LLM-as-judge guidance. |
| Outcome: | The proposed framework significantly outperforms existing methods for generating complex instructions, and outperformed existing methods. |
DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models (2025.naacl-long)
Copied to clipboard
Jianyu Liu, Hangyu Guo, Ranjie Duan, Xingyuan Bu, Yancheng He, Shilong Li, Hui Huang, Jiaheng Liu, Yucheng Wang, Chenchen Jing, Xingwei Qu, Xiao Zhang, Pei Wang, Yanan Wu, Jihao Gu, Yangguang Li, Jianke Zhu
| Challenge: | Multimodal Large Language Models (MLLMs) pose unique safety challenges due to their integration of visual and textual data. |
| Approach: | They propose a method to disentangle risks through step-by-step reasoning within multimodal inputs. |
| Outcome: | The proposed approach improves safety alignment in MLLMs by fine-tuning and iterative Reinforcement Learning from AI feedback. |
Fusion Makes Perfection: An Efficient Multi-Grained Matching Approach for Zero-Shot Relation Extraction (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing methods to extract unseen relations require laborious manual annotation . a new approach uses fine-grained matching to reduce manual annotation cost . |
| Approach: | They propose an efficient multi-grained matching approach that uses virtual entity matching to reduce manual annotation cost. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods and achieves inference efficiency and accuracy in zero-shot relation extraction tasks. |
Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models (2025.acl-long)
Copied to clipboard
Yancheng He, Shilong Li, Jiaheng Liu, Yingshui Tan, Weixun Wang, Hui Huang, Xingyuan Bu, Hangyu Guo, Chengwei Hu, Boren Zheng, Zhuoran Lin, Dekai Sun, Zhicheng Zheng, Wenbo Su, Bo Zheng
| Challenge: | Current frontier models sometimes generate false outputs or answers that are not substantiated by evidence. |
| Approach: | They propose Chinese SimpleQA, a Chinese benchmark to evaluate LLMs' factuality . they focus on Chinese language over 6 major topics with 99 diverse subtopics . |
| Outcome: | The Chinese SimpleQA benchmark evaluates the factuality ability of LLMs . the questions and answers are short and easy-to-evaluate . |
GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models (2024.findings-emnlp)
Copied to clipboard
Shilong Li, Yancheng He, Hangyu Guo, Xingyuan Bu, Ge Bai, Jie Liu, Jiaheng Liu, Xingwei Qu, Yangguang Li, Wanli Ouyang, Wenbo Su, Bo Zheng
| Challenge: | Existing models for long contexts struggle to handle long inputs due to limited context window and memory usage. |
| Approach: | They propose a graph-based agent system that analyzes long texts into a graphical graph . GraphReader consistently outperforms GPT-4-128k across context lengths from 16k to 256k . |
| Outcome: | The proposed model outperforms existing models on four challenging benchmarks. |
Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning? (2025.acl-long)
Copied to clipboard
Yancheng He, Shilong Li, Jiaheng Liu, Weixun Wang, Xingyuan Bu, Ge Zhang, Z.y. Peng, Zhaoxiang Zhang, Zhicheng Zheng, Wenbo Su, Bo Zheng
| Challenge: | Recent advances in o1-like models have generated long Chain-of-Thought reasoning steps to improve the reasoning abilities of existing Large Language Models (LLMs). |
| Approach: | They propose a DeltaBench to analyze the quality and effectiveness of o1-like models and measure their ability to detect errors in long COT reasoning. |
| Outcome: | The proposed model can detect errors in long COT reasoning. |