Papers by Tianhao Li
Butterfly Effects in Toolchains: A Comprehensive Analysis of Failed Parameter Filling in LLM Tool-Agent Systems (2025.findings-emnlp)
Copied to clipboard
| Challenge: | emergence of tool agent paradigm has broadened capability boundaries of the Large Language Model (LLM) but effectiveness of tool agents limited due to parameter failure during execution . |
| Approach: | They propose a parameter failure taxonomy to investigate parameter failure . they propose suggestions for standardizing tool return formats and improving error feedback mechanisms . |
| Outcome: | The proposed model is based on a tool agent invocation chain and a mainstream tool agent . it shows that parameter name hallucination failure stems from inherent limitations . |
Differential Privacy for Text Analytics via Natural Text Sanitization (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing text sanitization mechanisms provide low utility, as cursed by the high-dimensional text representation. |
| Approach: | They propose to use sanitized texts to samaritize training data . they propose to retrain and fine-tune the senitization-aware language model . |
| Outcome: | The proposed approach enables privacypreserving natural language processing over the BERT language model with promising utility. |
Rethinking Smoothness for Fast and Adaptable Entity Alignment Decoding (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for integrating knowledge graphs rely on entity and relation embeddings . Fig. 1 shows how to decode knowledge graph in under 6 seconds . |
| Approach: | They propose a framework that only utilizes entity embeddings to decode knowledge graphs. |
| Outcome: | The proposed framework reconstructs KG representation by maximizing smoothness of entity embeddings. |
A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM𝛥 Integration into Upcycled MoE (2026.acl-long)
Copied to clipboard
Hao Zhou, Tianhao Li, Zhijun Wang, Shuaijie She, Linjuan Wu, Hao-Ran Wei, Baosong Yang, Jiajun Chen, Shujian Huang
| Challenge: | Large Language Models (LLMs) are expensive and require extensive Continued Pre-Training and data-intensive alignment to expand. |
| Approach: | They propose a method which upcycles a dense model into a Mixture-of-Experts architecture, allocating different experts to different languages. |
| Outcome: | Experiments show that the proposed model upcycles a dense model into a Mixture-of-Experts(MoE) architecture, allocating different experts to different languages. |
Soft Knowledge Prompt: Help External Knowledge Become a Better Teacher to Instruct LLM in Knowledge-based VQA (2024.acl-long)
Copied to clipboard
| Challenge: | Recent research focuses on improving prediction performance and reliability of LLM. |
| Approach: | They propose a method to actively extract valuable information from the knowledge to produce a latent vector as a soft prompt, which is fused with the image embedding to form a knowledge-enhanced context to instruct LLM. |
| Outcome: | The proposed method improves performance on knowledge-based VQA benchmarks. |
Large Language Models with Reinforcement Learning from Human Feedback Approach for Enhancing Explainable Sexism Detection (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have significantly improved text comprehension. |
| Approach: | They propose a Reinforcement Learning from Human Feedback (RLHF) based fine-tuning framework for sexism detection that leverages contextual learning to understand and apply instructions to new scenarios without additional training. |
| Outcome: | The proposed framework outperforms existing models on three EDOS tasks and scores 0.8681 on binary sexism detection, 0.6829 on category classification of sexists and 0.4722 on task C. |
MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues (2026.findings-acl)
Copied to clipboard
Yaning Pan, Qianqian Xie, Guohui Zhang, Zekun Moore Wang, Yongqian Wen, Yuanxing Zhang, Haoxuan Hu, Zhiyu Pan, Yibing Huang, Zhidong Gan, Yonghong Lin, An Ping, Shihao Li, Yanghai Wang, Tianhao Peng, Jiaheng Liu
| Challenge: | Existing evaluation benchmarks for Multimodal Large Language Models (MLLMs) focus on single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. |
| Approach: | They propose a video understanding benchmark for MLLMs in multi-turn dialogues that assesses six core competencies that focus on perceptivity and interactivity. |
| Outcome: | The MT-Video-Bench evaluates 1,000 multi-turn dialogues from diverse domains and reveals significant performance discrepancies and limitations in handling multi-turned video dialogues. |
Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training large language models require additional annotations to adjust to shifted distributions. |
| Approach: | They propose an algorithm that allows LLMs and reward models to update alternatively via a min-max game to improve their alignment. |
| Outcome: | The proposed framework improves existing alignment baselines in terms of LLM helpfulness and harmlessness. |
ChatSOP: An SOP-Guided MCTS Planning Framework for Controllable LLM Dialogue Agents (2025.acl-long)
Copied to clipboard
Zhigen Li, Jianxiang Peng, Yanmeng Wang, Yong Cao, Tianhao Shen, Minghui Zhang, Linxi Su, Shang Wu, Yihang Wu, YuQian Wang, Ye Wang, Wei Hu, Jianfeng Li, Shaojun Wang, Jing Xiao, Deyi Xiong
| Challenge: | Existing models that use Large Language Models (LLMs) show superior performance in various tasks, but lack of controllability leads to unfocused conversations or task failure. |
| Approach: | They propose a standard operating procedure (SOP) framework to regulate dialogue flow by integrating Chain of Thought reasoning and supervised fine-tuning for SOP prediction. |
| Outcome: | The proposed method achieves a 27.95% improvement in action accuracy compared to baseline models based on GPT-3.5 and also shows notable gains for open-source models. |
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2025.emnlp-main)
Copied to clipboard
Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, Huan Liu
| Challenge: | Recent advances in Large Language Models (LLMs) inspire the "LLM-as-a-judge" paradigm . traditional methods of assessment and evaluation fail in dynamic and open-ended scenarios . |
| Approach: | They propose a paradigm where LLMs are leveraged to perform scoring, ranking, or selection for machine learning evaluation scenarios. |
| Outcome: | The proposed model-based judgment and evaluation paradigms are based on large language models and are compared to the current model-driven evaluation paradigm. |
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)
Copied to clipboard
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)
Copied to clipboard
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, JoĂŁo Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Ĺ tajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
Unraveling the Mystery: Defending Against Jailbreak Attacks Via Unearthing Real Intention (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly vulnerable to elusive and implicit intentions, causing security risks and compromising user experience. |
| Approach: | They propose a method to detect and mitigate implicit jailbreak attacks using LLMs by unearthing real intentions and a greedy gradient-based algorithm to remove the least important parts of a sentence. |
| Outcome: | The proposed method reduces attacks success rate and Harmful Score while maintaining overall model performance. |
Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in LLMs (2026.eacl-long)
Copied to clipboard
Yujia Zheng, Tianhao Li, Haotian Huang, Tianyu Zeng, Jingyu Lu, Chuangxin Chu, Yuekai Huang, Ziyou Jiang, Qian Xiong, Yuyao Ge, Mingyang Li
| Challenge: | Existing studies treat prompts as flat text, overlooking their internal structure, and different components within a prompt contribute unequally to robustness. |
| Approach: | They propose a framework that decomposes prompts into functional components and a method that selectively modifies components to expose component-wise vulnerabilities. |
| Outcome: | The proposed framework exposes component-wise vulnerabilities while ensuring linguistic plausibility through perplexity-based filtering. |
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)
Copied to clipboard
Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, Zili Wang
| Challenge: | Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence. |
| Approach: | They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included . |
| Outcome: | The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model. |
Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing cross-lingual transfer are limited by parallel resources and lack linguistic and domain coverage. |
| Approach: | They propose a cross-lingual in-context pre-training approach that leverages semantically related bilingual Wikipedia documents to enhance cross-linguistic transfer. |
| Outcome: | The proposed approach improves multilingual performance on three models across six target languages. |
Automatic Marketing Theme and Commodity Construction System for E-commerce (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing recommendation system invites experts to write marketing themes and select relevant commodities, which suffer from difficulty in mass production, poor timeliness and low online indicators. |
| Approach: | They propose to use pretrained language model to generate marketing themes and commodity consistency module to select relevant commodities for the generative theme. |
| Outcome: | The proposed system can generate popular marketing themes and select relevant commodities automatically and improve theme online effectiveness compared with state-of-the-art methods. |
OAgents: An Empirical Study of Building Effective Agents (2025.findings-emnlp)
Copied to clipboard
He Zhu, Tianrui Qin, King Zhu, Heyuan Huang, Yeyi Guan, Jinxiang Xia, Hanhao Li, Yi Yao, Ningning Wang, Pai Liu, Tianhao Peng, Xin Gui, Li Xiaowan, Yuhui Liu, Xiangru Tang, Jian Yang, Ge Zhang, Xitong Gao, Yuchen Eleanor Jiang, Changwang Zhang, Jun Wang, Jiaheng Liu, Wangchunshu Zhou
| Challenge: | a recent study shows that agent research practices are far from standard, rigorous . lack of a standard evaluation protocol makes previous works not reproducible, authors say . |
| Approach: | They conduct an empirical study on the GAIA benchmark to investigate agent design choices . they find that lack of a standard evaluation protocol makes previous works not reproducible . |
| Outcome: | The proposed framework achieves state-of-the-art performance among open-source projects. |
GuideLLM: Exploring LLM-Guided Conversation with Applications in Autobiography Interviewing (2025.naacl-long)
Copied to clipboard
Jinhao Duan, Xinyu Zhao, Zhuoxuan Zhang, Eunhye Grace Ko, Lily Boddy, Chenan Wang, Tianhao Li, Alexander Rasgon, Junyuan Hong, Min Kyung Lee, Chenxi Yuan, Qi Long, Ying Ding, Tianlong Chen, Kaidi Xu
| Challenge: | Large Language Models (LLMs) have demonstrated their effectiveness in human-guided dialogues, but tasks in the real world are more complex and require greater autonomy from LLMs. |
| Approach: | They propose to characterize LLM-guided conversation into three fundamental components: Goal Navigation, Context Management, Empathetic Engagement and implement an interviewing environment for the evaluation of LLMs. |
| Outcome: | The proposed LLM outperforms baseline LLMs in interviewing quality and autobiography generation quality. |