Papers by Jiaqi Zhu
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection (2025.findings-acl)
Copied to clipboard
Jiaqi Li, Xinyi Dong, Yang Liu, Zhizhuo Yang, Quansen Wang, Xiaobo Wang, Song-Chun Zhu, Zixia Jia, Zilong Zheng
| Challenge: | ReflectEvo-460k is a large-scale, comprehensive, self-generated reflection dataset with broadened instructions and diverse multi-domain tasks. |
| Approach: | They propose a pipeline that iteratively generates self-reflection for self-training and a large-scale reflection dataset with broadened instructions and diverse multi-domain tasks. |
| Outcome: | The proposed pipeline improves Llama-3 reasoning ability by up to 71.2% and Mistral by upto 44.4%. |
Automatic and Reliable Evaluation for Academic Caption-to-Figure Generation with LMMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for evaluating text-to-image generation focus mostly on real-life images, which poses challenges for assessing academic figure generation given real scientific captions. |
| Approach: | They propose a dataset that first provides a Holistic Evaluation for Academic caption-to-Figure Generation (HE4AFG) they collect real figure captions from 8 scientific domains and generate 3,900 evaluation samples . |
| Outcome: | The proposed model provides high-quality human ratings in terms of three aspects—scientific aesthetic (SA), topic relevance (TR), and attribute correctness (AC). |
Learning to Think on Hypergraph: HyperCoT for Structure-Guided N-ary Knowledge Graph Completion (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to solve knowledge hypergraph link prediction problem are limited by their ability to generate chain-of-thought (CoT) representations. |
| Approach: | They propose a structure-aware approach that models multi-hop structural reasoning as a depth-sensitive progressive evidence accumulation process. |
| Outcome: | Experiments on three real-world datasets show that HyperCoT outperforms strong n-ary KGC baselines while yielding interpretable multi-hop reasoning traces. |
MedDialog: Large-scale Medical Dialogue Datasets (2020.emnlp-main)
Copied to clipboard
Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, Pengtao Xie
| Challenge: | telemedicine is a medical practice that provides patient care remotely using video conferencing tools. |
| Approach: | They build large-scale medical dialogue datasets to facilitate research . they pretrain several models on the Chinese MedDialog dataset and compare their performance . |
| Outcome: | The proposed datasets show that models trained on MedDialog can generate doctor-like medical dialogues. |
FaStFact: Faster, Stronger Long-Form Factuality Evaluations in LLMs (2025.findings-emnlp)
Copied to clipboard
Yingjia Wan, Haochen Tan, Xiao Zhu, Xinyu Zhou, Zhiwei Li, Qingsong Lv, Changxuan Sun, Jiaqi Zeng, Yi Xu, Jianqiao Lu, Yinhong Liu, Zhijiang Guo
| Challenge: | Prior evaluation pipelines fail to evaluate factuality of long-form LLMs due to inefficiency and costly human assessment. |
| Approach: | They propose a fast and strong evaluation pipeline that can evaluate factuality of long-form LLMs . they propose 'faStFact' to reduce cost of web searching and inference calling . |
| Outcome: | The proposed evaluation pipeline achieves highest alignment with human evaluation and efficiency among existing baselines. |
HMoE: Heterogeneous Mixture of Experts for Language Modeling (2025.emnlp-main)
Copied to clipboard
An Wang, Xingwu Sun, Ruobing Xie, Shuaipeng Li, Jiaqi Zhu, Zhen Yang, Pinxue Zhao, Weidong Han, Zhanhui Kang, Di Wang, Naoaki Okazaki, Cheng-zhong Xu
| Challenge: | Mixture of Experts (MoE) models use homogeneous experts with diverse capacities, resulting in a lack of expert specialization and parameter utilization. |
| Approach: | They propose a framework where experts differ in size and possess diverse capacities . they propose HMoE to encourage frequent activation of smaller experts . |
| Outcome: | The proposed framework outperforms homogeneous homogenous MoE models on evaluation benchmarks and achieves lower loss rate with fewer activated parameters. |
Debiasing In-Context Learning by Instructing LLMs How to Follow Demonstrations (2024.findings-acl)
Copied to clipboard
Lvxue Li, Jiaqi Chen, Xinyu Lu, Yaojie Lu, Hongyu Lin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Zhongyi Liu, Xianpei Han, Le Sun
| Challenge: | In-context learning (ICL) has gained considerable attention due to its data efficiency and task adaptability. |
| Approach: | They propose to de-biase demonstration bias in in-context learning by focusing on semantic ambiguity induced by demonstrations and reducing the semantic hazard. |
| Outcome: | The proposed methods significantly improve performance on six datasets. |
Translating Headers of Tabular Data: A Pilot Study of Schema Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Schema translation is not well studied in the community because of morphological difference and context difference between plain text and tabular data. |
| Approach: | They propose a schema translation model augmented with schema context . they model a target header and its context as a directed graph to represent their entities . |
| Outcome: | The proposed model outperforms state-of-the-art models on schema translation . it uses a graph to represent entity types and relations, and a relational-aware transformer . |
TARN-VIST: Topic Aware Reinforcement Network for Visual Storytelling (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for visual storytelling ignore latent topic information. |
| Approach: | They propose a topic-aware reinforcement network for VIsual StoryTelling that takes topic information into account to generate a coherent story. |
| Outcome: | The proposed method outperforms most of the competing models across multiple evaluation metrics. |
JurisBench: A Deep Benchmark for Assessing Large Language Models in Professional Legal Practice (2026.acl-long)
Copied to clipboard
Ziang Chen, Guannan Li, Fanlin Ji, Yipeng Kang, Jiaqi Li, Muhan Zhang, Yangtao Zhang, Li Tianjiao, Jiannan Wang, Xin Guo, Song-Chun Zhu, Bin Ling
| Challenge: | Existing legal benchmarks evaluate isolated tasks or exam-style questions, failing to capture the procedural interdependencies and adjudicative rigor inherent in professional practice. |
| Approach: | They propose a vertical, depth-oriented, domain-specific benchmark to evaluate Large Language Models (LLMs) in Chinese civil litigation. |
| Outcome: | The proposed benchmarks show that large language models exhibit an "illusion of competence" the results highlight a critical gap between fluent linguistic output and judicial reliability . |
SLiNT: Structure-aware Language Model with Injection and Contrastive Training for Knowledge Graph Completion (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) lack structural information and semantic context to infer missing entities . large language models often lack structural signals to infuse missing entities into knowledge graphs . |
| Approach: | a modular framework integrates structural information and semantic context into a frozen LLM backbone for link prediction. |
| Outcome: | a new framework integrates KG-derived structural information and semantic context to infer missing entities. |