Papers by Qianqian Xie
LongDocFACTScore: Evaluating the Factuality of Long Document Abstractive Summarisation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing metrics for text summarisation have restrictive token limits, limiting their effectiveness. |
| Approach: | They propose a human-annotated data set for evaluating automatic factuality metrics . they propose 'longDocFACTScore' framework which can be extended to any length document . |
| Outcome: | The proposed framework outperforms state-of-the-art metrics in evaluating long document summarisation data sets. |
SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for linking knowledge graphs lack contextual information in entity neighborhoods, which leads to false prediction results. |
| Approach: | They propose a Schema-augmented Multi-level contrastive LEarning framework to conduct knowledge graph link prediction using a knowledge graph schema. |
| Outcome: | The proposed framework is based on a knowledge graph schema and is compared against state-of-the-art datasets. |
Inductive Topic Variational Graph Auto-Encoder for Text Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for text classification do not assume explicit latent semantic structure of documents, making them less effective and difficult to interpret. |
| Approach: | They propose a model that integrates a topic model into variational graph-auto-encoder to capture hidden semantic information between documents and words. |
| Outcome: | The proposed model outperforms existing models on supervised and semi-supervised text classification and unsupervised representation learning. |
Graph Relational Topic Model with Higher-order Graph Attention Auto-encoders (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for learning low-dimensional representations of networked documents are largely ignored for document networks. |
| Approach: | They propose a graph relational topic model to explore document neighborhood information . the model can learn efficient networked document representations in the latent topic space . |
| Outcome: | The proposed model outperforms existing methods on unsupervised representation learning and other downstream tasks. |
EMPEC: A Comprehensive Benchmark for Evaluating Large Language Models Across Diverse Healthcare Professions (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) show their potential in accurately answering biomedical questions, yet current healthcare benchmarks primarily assess knowledge mastered by medical doctors, neglecting other essential professions. |
| Approach: | They evaluated 17 LLMs including proprietary and open-source models and found they struggled with specialized fields and alternative medicine. |
| Outcome: | The examinations for medical PErsonnel in Chinese (EMPEC) features 157,803 exam questions across 124 subjects and 20 healthcare professions. |
MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues (2026.findings-acl)
Copied to clipboard
Yaning Pan, Qianqian Xie, Guohui Zhang, Zekun Moore Wang, Yongqian Wen, Yuanxing Zhang, Haoxuan Hu, Zhiyu Pan, Yibing Huang, Zhidong Gan, Yonghong Lin, An Ping, Shihao Li, Yanghai Wang, Tianhao Peng, Jiaheng Liu
| Challenge: | Existing evaluation benchmarks for Multimodal Large Language Models (MLLMs) focus on single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. |
| Approach: | They propose a video understanding benchmark for MLLMs in multi-turn dialogues that assesses six core competencies that focus on perceptivity and interactivity. |
| Outcome: | The MT-Video-Bench evaluates 1,000 multi-turn dialogues from diverse domains and reveals significant performance discrepancies and limitations in handling multi-turned video dialogues. |
FLAG-TRADER: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading (2025.findings-acl)
Copied to clipboard
Guojun Xiong, Zhiyang Deng, Keyi Wang, Yupeng Cao, Haohang Li, Yangyang Yu, Xueqing Peng, Mingquan Lin, Kaleb E Smith, Xiao-Yang Liu, Jimin Huang, Sophia Ananiadou, Qianqian Xie
| Challenge: | Large language models (LLMs) have impressive reasoning capabilities in financial tasks, but struggle with multi-step, goal-oriented scenarios in interactive financial markets. |
| Approach: | They propose a framework that integrates large language models with gradient-driven reinforcement learning (RL) policy optimization. |
| Outcome: | The proposed framework improves performance in trading and other financial domain tasks. |
TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practice (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models excel in general domains but lack real-world practical capabilities. |
| Approach: | They propose a benchmark for Chinese taxation practice that combines 10 traditional application tasks with 3 pioneering real-world scenarios. |
| Outcome: | The proposed benchmark combines 10 traditional tasks with 3 pioneering real-world scenarios. |
INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent (2025.acl-long)
Copied to clipboard
Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.p. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow, Qianqian Xie
| Challenge: | Recent advances have underscored the potential of large language model (LLM)-based agents in financial decision-making. |
| Approach: | They propose to evaluate LLM agents using 13 different LLMs as backbone models across various market environments and tasks. |
| Outcome: | The proposed framework assesses the reasoning and decision-making capabilities of 13 different LLMs across various market environments and tasks. |
CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)
Copied to clipboard
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan
| Challenge: | Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages . |
| Approach: | They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models . |
| Outcome: | The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English . |
Human or LLM as Standardized Patients? A Comparative Study in Medical Education (2026.acl-long)
Copied to clipboard
| Challenge: | Standardized patients (VSPs) are indispensable for clinical skills training but remain expensive and difficult to scale. |
| Approach: | They propose a multi-agent VSP framework that separates case-grounded information disclosure from response generation to support stable, inquiry-conditioned patient behavior. |
| Outcome: | The proposed framework more closely matches human SP behavior than existing VSPs, particularly in case consistency and controlled disclosure. |
UCL-Bench: A Chinese User-Centric Legal Benchmark for Large Language Models (2025.findings-naacl)
Copied to clipboard
Ruoli Gan, Duanyu Feng, Chen Zhang, Zhihang Lin, Haochen Jia, Hao Wang, Zhenyang Cai, Lei Cui, Qianqian Xie, Jimin Huang, Benyou Wang
| Challenge: | Existing legal benchmarks focusing on knowledge and logic evaluate LLMs on various tasks in legal domain, but few have explored the practical application of LLM by actual users. |
| Approach: | They propose a Chinese user-centric legal benchmark that aims to assess the practical application of LLMs by real users. |
| Outcome: | The proposed model outperforms existing models on various tasks in legal domain but does not outperfect ChatGPT. |
Readability Controllable Biomedical Document Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing controllable summarization systems for biomedical documents have little attention to readability control, leaving users with incompatible summaries . |
| Approach: | They propose a task of readability controllable summarization for biomedical documents to generate summaries that are incompatible with users' levels of expertise. |
| Outcome: | The proposed model is based on pre-trained language models with prevalent controlling and generation techniques and evaluates the readability discrepancy between lay and technical summaries. |
GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to capture and integrate global semantic information are limited due to their limited ability to capture long-range dependencies. |
| Approach: | They propose a graph contrastive topic enhanced language model that integrates a neural topic model with a pre-trained language model to capture global contextual semantics. |
| Outcome: | The proposed model outperforms existing methods on general domain and biomedical datasets. |
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)
Copied to clipboard
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Towards Interpretable Mental Health Analysis with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on large language models lack adequate evaluations and prompting strategies for explainability. |
| Approach: | They evaluate the mental health analysis and emotional reasoning ability of large language models (LLMs) using 11 datasets across 5 tasks. |
| Outcome: | The proposed model shows strong in-context learning ability but still has a significant gap with advanced task-specific methods. |
UCFE: A User-Centric Financial Expertise Benchmark for Large Language Models (2025.findings-naacl)
Copied to clipboard
Yuzhe Yang, Yifei Zhang, Yan Hu, Yilin Guo, Ruoli Gan, Yueru He, Mingcong Lei, Xiao Zhang, Haining Wang, Qianqian Xie, Jimin Huang, Honghai Yu, Benyou Wang
| Challenge: | Recent advances in large language models (LLMs) have expanded their potential applications in finance. |
| Approach: | They propose a framework to evaluate the ability of large language models to handle financial tasks using human expert evaluations and task-specific interactions. |
| Outcome: | The proposed framework evaluates the ability of large language models to handle complex financial tasks and combines human expert evaluations with dynamic, task-specific interactions to simulate the complexities of evolving financial scenarios. |
HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy (2024.acl-long)
Copied to clipboard
Mengxi Xiao, Qianqian Xie, Ziyan Kuang, Zhicheng Liu, Kailai Yang, Min Peng, Weiguang Han, Jimin Huang
| Challenge: | Large Language Models (LLMs) can be used in psychotherapy to overcome challenges such as shame, distrust, and resource scarcity. |
| Approach: | They propose a cognitive reframing therapy method that uses empathetic dialogue to address deep-rooted negative thoughts and fosters rational, balanced perspectives. |
| Outcome: | The proposed model outperforms other models in terms of empathy, guidance, and logical coherence, demonstrating its effectiveness and potential positive impact on psychotherapy. |
Selective Preference Optimization via Token-Level Reward Function Estimation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for maximizing preference optimization on all available tokens are noisy and inefficient. |
| Approach: | They propose a selective alignment strategy that centers on efficient key token selection without strong, fine-grained supervision signals. |
| Outcome: | The proposed strategy outperforms baseline methods on three benchmarks with up to 60% reduction in training hours. |
Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have been widely used in NLP, but their social or cultural impact is under-explored. |
| Approach: | They build a dataset consisting of numerous **C**hinese **C*omical **C***rosstalk scripts, which is for a popular Chinese performing art called ‘Xiangsheng’ or ‘’ since 1800s. |
| Outcome: | The proposed approach can generate humor as humans do, but it is still in its infancy. |
RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Information (2025.acl-long)
Copied to clipboard
| Challenge: | Current methods for cross-domain misinformation detection focus on in-domain tasks and do not incorporate significant sentiment and emotion features. |
| Approach: | They propose a retrieval augmented (RAG) LLM framework that incorporates affective information into retrieval databases. |
| Outcome: | The proposed framework improves on three misinformation benchmarks. |
ELAINE-medLLM: Lightweight English Japanese Chinese Trilingual Large Language Model for Bio-medical Domain (2025.coling-main)
Copied to clipboard
Ken Yano, Zheheng Luo, Jimin Huang, Qianqian Xie, Masaki Asada, Chenhan Yuan, Kailai Yang, Makoto Miwa, Sophia Ananiadou, Jun’ichi Tsujii
| Challenge: | Existing bilingual or multilingual medical LLMs are limited in multilingual data and therefore perform poorly in non-English languages such as Japanese and Chinese. |
| Approach: | They propose to use a trilingual (English, Japanese, Chinese) large language model adapted for the bio-medical domain to harness the knowledge and abilities of the base model. |
| Outcome: | The proposed model can support English, Japanese, and Chinese and is adapted for a bio-medical domain. |
Neural Sparse Topical Coding (P18-1)
Copied to clipboard
| Challenge: | Topic models with sparsity enhancement are effective at learning discriminative and coherent latent topics of short texts. |
| Approach: | They propose a novel sparsity-enhanced topic model with back propagation that replaces the inference process with the back propagations, making it easy to explore extensions. |
| Outcome: | The proposed model outperforms existing methods on Web Snippet and 20Newsgroups datasets. |
Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance (2025.emnlp-main)
Copied to clipboard
Xueqing Peng, Triantafillos Papadopoulos, Efstathia Soufleri, Polydoros Giannouris, Ruoyu Xiang, Yan Wang, Lingfei Qian, Jimin Huang, Qianqian Xie, Sophia Ananiadou
| Challenge: | Greek is the dominant language of the world's merchant navy and is a key language for international trade. |
| Approach: | They propose to develop a Greek financial evaluation benchmark and a financial LLM fine-tuned on Greek-specific financial data to bridge this gap. |
| Outcome: | The proposed benchmarks surpass GPT-4 by 8.33%, GPT- 4o by 26.83%, and Deepseek-V3 by 67.74%. |
LAiW: A Chinese Legal Large Language Models Benchmark (2025.coling-main)
Copied to clipboard
Yongfu Dai, Duanyu Feng, Jimin Huang, Haochen Jia, Qianqian Xie, Yifang Zhang, Weiguang Han, Wei Tian, Hao Wang
| Challenge: | Xie et al., 2023) show that large language models (LLMs) can generate legal text, but lack the legal syllogism . legal experts are cautious about their practical application due to the opaque nature of the LLMs. |
| Approach: | They propose a Chinese legal LLM benchmark structured around the legal syllogism . they evaluate LLMs across three levels of capability, each reflecting a more complex stage of legal . |
| Outcome: | The proposed benchmark identifies that LLMs lack the legal syllogism, which hinders trust and understanding from legal experts. |