Papers by Xiaochen Gao
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)
Copied to clipboard
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
Towards Comprehensive Patent Approval Predictions:Beyond Traditional Document Classification (2022.acl-long)
Copied to clipboard
| Challenge: | a new framework for patent approval prediction is proposed to address this problem . novelty scores are based on comparing an application with millions of prior arts . |
| Approach: | They propose a framework that unifies the document classifier with handcrafted features, particularly time-dependent novelty scores. |
| Outcome: | The proposed framework unifies the document classifier with handcrafted features, particularly time-dependent novelty scores. |
ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format (2023.emnlp-demo)
Copied to clipboard
Qi Zhu, Christian Geishauser, Hsien-chin Lin, Carel van Niekerk, Baolin Peng, Zheng Zhang, Shutong Feng, Michael Heck, Nurul Lubis, Dazhen Wan, Xiaochen Zhu, Jianfeng Gao, Milica Gasic, Minlie Huang
| Challenge: | Existing tools for building TOD systems often lack a user-friendly interface . a toolkit with advanced, easily integrable modules is needed to bridge this gap . |
| Approach: | They propose a multifaceted dialogue system toolkit that integrates diverse datasets and models with a streamlined training process and in-depth evaluation tools. |
| Outcome: | The proposed toolkit combines RL and transfer learning to support the rapid development and evaluation of robust dialogue policies. |
Beyond Scaling: Predicting Patent Approval with Domain-specific Fine-grained Claim Dependency Graph (2024.acl-long)
Copied to clipboard
| Challenge: | Scaling up language models has demonstrated predictable improvement and unprecedented abilities in many language tasks. |
| Approach: | They propose a fine-grained cLAim depeNdency graph that captures the dependencies within the patent data and extends the embedding-based state-of-the-art (SOTA) they then explore prompt-based methods to harness proprietary LLMs' potential, but find the best results close to random guessing, underlining the ineffectiveness of model scaling-up. |
| Outcome: | The proposed graph methods outperform the standard model scaling methods in the patent approval prediction task and show that they are cost-effective. |
PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that our method outperforms full-model tuning in few-shot abstractive summarization tasks. |
| Approach: | They propose a soft prompts architecture with prompt pre-training and prompt fine-tuning paradigm to support few-shot abstractive summarization. |
| Outcome: | The proposed model outperforms Prompt Tuning and Profix-Tuning on CNN/DailyMail and XSum datasets and outperfies Profix Tuning by a large margin. |