Papers by Jiaqi Guo
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing Text-to-SQL parsers are vulnerable to perturbations in NL questions . we propose the Adversarial Table Perturbation (ATP) as a new attacking paradigm . |
| Approach: | They propose to use the Adversarial Table Perturbation to measure robustness of Text-to-SQL parsers against adversarial perturbations. |
| Outcome: | The proposed approach outperforms baseline methods in robustness evaluations on ADVETA and can be used in future projects. |
Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)
Copied to clipboard
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu
| Challenge: | Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature. |
| Approach: | They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management. |
| Outcome: | The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench. |
MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation (2026.acl-long)
Copied to clipboard
Jin Cui, Jiaqi Guo, Jiepeng Zhou, Ruixuan Yang, Jiayi Lu, Jiajun Xu, Jiangcheng Song, Boran Zhao, Pengju Ren
| Challenge: | Existing approaches restrict students to following a single golden rationale and treat different reasoning paths independently, causing suboptimal performance. |
| Approach: | They propose a capability-adaptive framework that transitions distillation from passive mimicry to active cognitive construction and employ a feedback-driven inertia calibration mechanism to align supervision with the student’s current adaptability. |
| Outcome: | Experiments show that the proposed framework achieves state-of-the-art performance on both in-distribution and out-of distribution benchmarks. |
Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding (2026.acl-long)
Copied to clipboard
Ke Ma, Jiaqi Tang, Bin Guo, Xueting Han, Ruonan Xu, Qingfeng He, Ziheng Wang, Xu Wang, Qifeng Chen, Zhiwen Yu, Yunhao Liu
| Challenge: | Existing methods for streaming video understanding are query-agnostic and implicitly model video evidence. |
| Approach: | They propose a framework that establishes explicit, structured alignment between the accumulated video evidence and the query’s expected response conditions via scene graphs. |
| Outcome: | The proposed model achieves more interpretable and accurate response timing decisions on both proactive and reactive tasks. |
Exploring the Secrets Behind the Learning Difficulty of Meaning Representations for Semantic Parsing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that the design of Meaning Representation (MR) greatly influences the final model performance of a neural semantic parser. |
| Approach: | They propose a data-aware metric called ISS to measure the final performance of MRs. |
| Outcome: | The proposed metric denoting incremental structural stability (ISS) of MRs can be used as an indicator for MR design to avoid the costly training-testing process. |
M2C: Towards Automatic Multimodal Manga Complement (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal manga analysis focuses on enhancing manga understanding with visual and textual features. |
| Approach: | They propose a task to enhance manga understanding with visual and textual features by providing a shared semantic space for vision and language understanding. |
| Outcome: | The proposed task provides a shared semantic space for vision and language understanding. |
Uncovering Main Causalities for Long-tailed Information Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Information Extraction (IE) aims to extract structural information from unstructured texts. |
| Approach: | They propose a framework that aims to uncover the main causalities behind data in the view of causal inference. |
| Outcome: | The proposed framework can detect the main causalities behind data in the view of causal inference. |
GeometryZero: Advancing Geometry Solving via Group Contrastive Policy Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for auxiliary construction training are expensive and underperform . Existing Corresponding Author training methods lack self-correction capabilities in reasoning chains. |
| Approach: | They propose a reinforcement learning framework that rewards auxiliary construction with geometric reasoning by grouping construction rewards with a Length Reward. |
| Outcome: | Experiments on Geometry3K and MathVista show that GeometryZero outperforms baselines on auxiliary constructions. |
FaStFact: Faster, Stronger Long-Form Factuality Evaluations in LLMs (2025.findings-emnlp)
Copied to clipboard
Yingjia Wan, Haochen Tan, Xiao Zhu, Xinyu Zhou, Zhiwei Li, Qingsong Lv, Changxuan Sun, Jiaqi Zeng, Yi Xu, Jianqiao Lu, Yinhong Liu, Zhijiang Guo
| Challenge: | Prior evaluation pipelines fail to evaluate factuality of long-form LLMs due to inefficiency and costly human assessment. |
| Approach: | They propose a fast and strong evaluation pipeline that can evaluate factuality of long-form LLMs . they propose 'faStFact' to reduce cost of web searching and inference calling . |
| Outcome: | The proposed evaluation pipeline achieves highest alignment with human evaluation and efficiency among existing baselines. |
HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation (2022.acl-long)
Copied to clipboard
Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, Dongmei Zhang
| Challenge: | Existing studies on table reasoning focus on flat tables and hierarchical tables . a new dataset, HiTab, aims to examine numerical reasoning over hierarchic tables based on hierarchically structured tables - a strong challenge for existing baselines and a valuable benchmark for future research. |
| Approach: | They propose a hierarchical question answering and natural language generation dataset to study hierarchic tables. |
| Outcome: | The proposed model shows that it is effective in QA and natural language generation over hierarchical tables. |
Translating Headers of Tabular Data: A Pilot Study of Schema Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Schema translation is not well studied in the community because of morphological difference and context difference between plain text and tabular data. |
| Approach: | They propose a schema translation model augmented with schema context . they model a target header and its context as a directed graph to represent their entities . |
| Outcome: | The proposed model outperforms state-of-the-art models on schema translation . it uses a graph to represent entity types and relations, and a relational-aware transformer . |
Chase: A Large-Scale and Pragmatic Chinese Dataset for Cross-Database Context-Dependent Text-to-SQL (2021.acl-long)
Copied to clipboard
| Challenge: | XDTS is a cross-database context-dependent text-to-sql problem with wide range of applications. |
| Approach: | They present a large-scale Chinese dataset for cross-database context-dependent Text-to-SQL . they find that only 35% of questions are context-independent and 28% of SQL queries are easy . |
| Outcome: | The proposed approach achieves an exact match accuracy of 40% over all questions and 16% over all question sequences. |
PEAP: Proactive Embodied Action Sequence Planning with Joint Understanding of Vision and Audio Perception (2026.acl-long)
Copied to clipboard
| Challenge: | Embodied action sequence planning focuses on the capability of embodied agents to implement action planning via environmental perception without explicit human instructions. |
| Approach: | They propose to use a multimodal dataset to evaluate the performance of multiple large language models to evaluate their models' environmental perception capabilities. |
| Outcome: | The proposed model shows that it lacks accurate environmental perception capabilities and that it can improve on the PEAP dataset. |
Weakly Supervised Semantic Parsing by Learning from Mistakes (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Weakly supervised semantic parsing requires searching consistent logical forms in a huge space and dealing with spurious logical form. |
| Approach: | They propose a learning framework that trains parsers via utterance-denotation pairs . they use utterrance-logical form pairs created from mistakes to bootstrap parser . |
| Outcome: | The proposed framework outperforms state-of-the-art methods on WikiSQL, TabFact and other datasets. |
JurisBench: A Deep Benchmark for Assessing Large Language Models in Professional Legal Practice (2026.acl-long)
Copied to clipboard
Ziang Chen, Guannan Li, Fanlin Ji, Yipeng Kang, Jiaqi Li, Muhan Zhang, Yangtao Zhang, Li Tianjiao, Jiannan Wang, Xin Guo, Song-Chun Zhu, Bin Ling
| Challenge: | Existing legal benchmarks evaluate isolated tasks or exam-style questions, failing to capture the procedural interdependencies and adjudicative rigor inherent in professional practice. |
| Approach: | They propose a vertical, depth-oriented, domain-specific benchmark to evaluate Large Language Models (LLMs) in Chinese civil litigation. |
| Outcome: | The proposed benchmarks show that large language models exhibit an "illusion of competence" the results highlight a critical gap between fluent linguistic output and judicial reliability . |
m3P: Towards Multimodal Multilingual Translation with Multimodal Prompt (2024.lrec-main)
Copied to clipboard
Jian Yang, Hongcheng Guo, Yuwei Yin, Jiaqi Bai, Bing Wang, Jiaheng Liu, Xinnian Liang, LinZheng Chai, Liqun Yang, Zhoujun Li
| Challenge: | Existing multimodal neural machine translation models focus on bilingual translation, but experimental results show that they outperform the text-only baselines and multilingual multimodal methods by a large margin. |
| Approach: | They propose a framework to leverage the multimodal prompt to guide the Multimodal Multilingual Neural Machine Translation (m3P) this framework aligns the representations of different languages with the same meaning and generates the conditional vision-language memory for translation. |
| Outcome: | The proposed framework outperforms previous text-only baselines and multilingual multimodal methods by a large margin. |
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation (2024.acl-long)
Copied to clipboard
| Challenge: | Weight quantization has emerged as a popular solution to reduce memory and computational demands. |
| Approach: | They propose a framework that synergizes Quantization-Aware Training (QAT) with Knowledge Distillation (KD) to boost the performance of LLMs at sub-4-bit. |
| Outcome: | The proposed framework outperforms existing QAT methods on language understanding and complex reasoning benchmarks on sub-4-bit models. |
SemRegex: A Semantics-Based Approach for Generating Regular Expressions from Natural Language Specifications (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to generate programs from natural language do not address program aliasing . semantically equivalent programs may have many syntactically different forms . |
| Approach: | They propose a semantics-based approach to generate regular expressions from natural language. |
| Outcome: | The proposed approach improves on three public datasets. |
Awakening Latent Grounding from Pretrained Language Models for Semantic Parsing (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent years pretrained language models (PLMs) have shown their power on modeling language . however, few efforts have been made to explore grounding capabilities of PLMs . |
| Approach: | They propose to use pretrained language models to explore syntactic structures . they propose to combine their approach with an erasingthen-awakening approach . their results show that the approach can awaken latent grounding, which is understandable to humans . |
| Outcome: | Empirical studies show that the proposed approach can awaken latent grounding . it shows great potential to benefit downstream semantic parsing models, it says . |
Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation (P19-1)
Copied to clipboard
| Challenge: | IRNet synthesizes SQL queries in an end-to-end manner, but it yields unsatisfactory performance on public benchmarks. |
| Approach: | They propose a neural approach called IRNet for complex and cross-domain Text-to-SQL. |
| Outcome: | IRNet achieves 46.7% accuracy on the Spider benchmark, a 19.5% improvement over state-of-the-art approaches. |
Benchmarking Meaning Representations in Neural Semantic Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on meaning representations is not comprehensively evaluated due to the lack of readily-available execution engines. |
| Approach: | They propose a unified benchmark on meaning representations by integrating existing semantic parsing datasets, completing the missing logical forms, and implementing the missing execution engines. |
| Outcome: | The proposed benchmark combines existing parsing datasets, completes missing logical forms, and implements missing execution engines. |