Papers by Yuan Qi

CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)

Challenge:	Recent studies have shown that large language models are useful, honest, harmless (HHH) however, RLHF requires high hardware resources and human efforts.
Approach:	They propose a framework that allows LLMs to align themselves with HHH . they use IF and reinforcement learning from human feedback to fine-tune their models .
Outcome:	The proposed framework achieves similar performance to RLHF and human-generated models with a minimal alignment tax.

Copied to clipboard

Shiting Huang, Zhen Fang, Zehui Chen, Siyu Yuan, Junjie Ye, Yu Zeng, Lin Chen, Qi Mao, Feng Zhao

ULMR: Unlearning Large Language Models via Negative Response and Model Parameter Average (2024.emnlp-industry)

Challenge:	a number of tools are used to perform complex tasks, but the tool utilization process can cause errors.
Approach:	They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks.
Outcome:	The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios.

Copied to clipboard

Shaojie Shi, Xiaoyu Tan, Xihe Qiu, Chao Qu, Kexin Nie, Yuan Cheng, Wei Chu, Xu Yinghui, Yuan Qi

On LLM-Based Scientific Inductive Reasoning Beyond Equations (2025.emnlp-main)

Challenge:	Large language models (LLMs) have attracted significant interest from the research community due to their broad applicability in many language-oriented tasks.
Approach:	They propose a framework which uses pre-training datasets to rewrite instructions and generate negative responses to preserve the performance of the original LLM.
Outcome:	The proposed framework can erase the pre-training data while maintaining the performance of the original model.

Copied to clipboard

Brian S. Lin, Jiaxin Yuan, Zihan Zhou, Shouli Wang, Shuo Wang, Cunliang Kong, Qi Shi, Yuxuan Li, Liner Yang, Zhiyuan Liu, Maosong Sun

Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Challenge:	Existing research on inductive reasoning models emphasizes rule design without grounding them in specific scenarios.
Approach:	They propose to use LLMs to learn underlying patterns from limited examples in entirely new environments.
Outcome:	The proposed benchmark evaluates the inductive reasoning abilities of large language models in scientific settings.

Copied to clipboard

Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong

Explainability & Interpretability in NLP

Challenge:	Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach:	They propose a survey structured around the pipeline to identify and improve MI models.
Outcome:	The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.

Infinity-Parser: Layout-Aware Reinforcement Learning with High-quality Document Parsing Dataset (2026.findings-acl)

Copied to clipboard

Baode Wang, Biao Wu, Weizhen Li, Meng Fang, Zuming Huang, Jun Huang, Yanjie Liang, Haozhe Wang, Ling Chen, Wei Chu, Yuan Qi

MCS: An In-battle Commentary System for MOBA Games (2022.coling-1)

Challenge:	Existing supervised fine-tuning methods struggle to generalize across document types, leading to poor performance.
Approach:	They propose layoutRL, a reinforcement learning framework that optimizes layout understanding through composite rewards integrating normalized edit distance, paragraph count accuracy, and reading order preservation.
Outcome:	The proposed model outperforms specialized document parsing systems and general-purpose vision-language models on a broad range of document types, languages, and structural complexities.

Copied to clipboard

Xiaofeng Qi, Chao Li, Zhongping Liang, Jigang Liu, Cheng Zhang, Yuanxin Wei, Lin Yuan, Guang Yang, Lanxiao Huang, Min Li

Challenge:	In-battle commentary is an important component of live streaming of e-sports competitions and is applicable to a wide range of scenarios like combat information analysis and live streaming.
Approach:	They propose a generative system for in-battle real-time commentary in mobile MOBA games and propose 'transform' method to convert match statistics and utterances into consistent encoding space.
Outcome:	The proposed system is based on real-time match statistics and events and can be used for live streaming, e-sports commentary and combat information analysis.

Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution (2021.acl-long)

Copied to clipboard

Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, Maosong Sun

Word-level Textual Adversarial Attacking as Combinatorial Optimization (2020.acl-main)

Challenge:	Recent studies show that neural natural language processing models are vulnerable to backdoor attacks.
Approach:	They propose to inject neural models with backdoors activated by word substitution . their results raise a serious alarm to the security of NLP models, they argue .
Outcome:	The proposed backdoors are activated by a learnable combination of word substitution and exhibit higher invisibility than previous methods.

Copied to clipboard

Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, Maosong Sun

ONION: A Simple and Effective Defense Against Textual Backdoor Attacks (2021.emnlp-main)

Challenge:	Existing word-level attack models are far from perfect because of unsuitable search space reduction methods and inefficient optimization algorithms.
Approach:	They propose a novel adversarial adversarialist model that incorporates word substitution and particle swarm optimization to solve two problems separately.
Outcome:	The proposed model achieves much higher success rates and crafts more high-quality adversarial examples as compared to baseline methods.

Copied to clipboard

Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, Maosong Sun

Leveraging Argumentation Knowledge Graph for Interactive Argument Pair Identification (2021.findings-acl)

Challenge:	Backdoor attacks can manipulate the output of deep neural networks and possess high insidiousness.
Approach:	They propose a textual backdoor defense based on outlier word detection that can handle all the textual attacks.
Outcome:	The proposed method can handle all the textual backdoor attack situations.

Copied to clipboard

Jian Yuan, Zhongyu Wei, Donghua Zhao, Qi Zhang, Changjian Jiang

Mitigating Hallucinations in Large Vision-Language Models by Self-Injecting Hallucinations (2025.findings-emnlp)

Challenge:	Existing researches focus on sentence matching but the interaction of opinions requires reasoning of knowledge, which is beyond textual information.
Approach:	They propose to leverage external knowledge to enhance the identification of interactive argument pairs by analyzing the discussion thread of the target topic in an online forum.
Outcome:	The proposed model achieves state-of-the-art in the benchmark dataset.

Copied to clipboard

Yifan Lu, Ziqi Zhang, Chunfeng Yuan, Jun Gao, Congxuan Zhang, Xiaojuan Qi, Bing Li, Weiming Hu

Evaluating LLMs’ Assessment of Mixed-Context Hallucination Through the Lens of Summarization (2025.findings-acl)

Challenge:	Existing methods for hallucination mitigation are based on external dependency and require external annotations or auxiliary models for preference data collection.
Approach:	a new method is proposed to help model-generated hallucinations without external dependencies.
Outcome:	a new method that self-injects hallucinations into a generated response improves halluuutations mitigation.

Copied to clipboard

Siya Qi, Rui Cao, Yulan He, Zheng Yuan

Text Generation

Challenge:	Large language models generate coherent text and follow instructions across diverse tasks, but a critical challenge in scaling LLM applications is hallucination, where the generated content lacks factual grounding or deviates from the intended discourse context.
Approach:	They use summarization as a representative task to evaluate LLMs' capability in detecting mixed-context hallucinations, specifically distinguishing between factual and non-factual hallucinos.
Outcome:	The proposed model distinguishes between factual and non-factual hallucinations, and their performance bottlenecks.

TEF: Causality-Aware Taxonomy Expansion via Front-Door Criterion (2025.coling-main)

Copied to clipboard

Yuan Meng, Songlin Zhai, Yuxin Zhang, Zhongjian Hu, Guilin Qi

PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering (2026.acl-long)

Challenge:	Existing research still faces spurious query-anchor matching due to unobserved factors.
Approach:	They propose a model that uses the front-door criteria to decompose the expansion process into a parser module and a connector to isolate confounding effects.
Outcome:	Extensive experiments on three benchmarks validate the effectiveness of the proposed model.

Copied to clipboard

Xiangfeng Wang, Hangyu Guo, Yanlin Lai, Mitt Huang, Liang Zhao, Chengyuan Yao, Yinmin Zhang, Qi Han, null Xiaoxiaoren, Chun Yuan, Tong Xu, Zheng Ge, Xiangyu Zhang, Daxin Jiang

OpenAttack: An Open-source Textual Adversarial Attack Toolkit (2021.acl-demo)

Challenge:	Current outcome-centric verification paradigms neglect potential errors in the derivation process.
Approach:	They propose a process-aware RLVR training paradigm utilizing verifiers selected via PRIME.
Outcome:	The proposed approach outperforms the baseline verification paradigm on AIME24, AIME25, and Beyond-AIME models.

Copied to clipboard

Guoyang Zeng, Fanchao Qi, Qianrui Zhou, Tingji Zhang, Zixian Ma, Bairu Hou, Yuan Zang, Zhiyuan Liu, Maosong Sun

PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matching (2023.emnlp-industry)

Challenge:	Various attack models are distinct and implemented with different programming frameworks and settings, which hinders quick utilization and fair comparison of attack models.
Approach:	They propose an open-source textual adversarial attack toolkit to solve these issues by combining 15 typical attack models into one toolkit.
Outcome:	The proposed toolkit supports all attack types, multilinguality, and parallel processing.

Copied to clipboard

Zhenting Qi, Xiaoyu Tan, Shaojie Shi, Chao Qu, Yinghui Xu, Yuan Qi

ChemAmp: Amplified Chemistry Tools via Composable Agents (2026.findings-acl)

Challenge:	Low-Rank Adaptation (LoRA) has been used to adapt Large Language Models to a variety of tasks, but it requires substantial computational resources to perform.
Approach:	They propose a low-rank adaptive learning approach that leverages LoRA's in-context learning capability through prompt matching via reinforcement learning in resource-constrained environments.
Outcome:	The proposed model improves LoRA performance on evaluation metrics and utilises consumer-grade GPU resources.

Copied to clipboard

Zhucong Li, Powei Chang, Jin Xiao, Zhijian Zhou, Qianyu He, Jiaqing Liang, Fenglei Cao, Xu Yinghui, Yuan Qi

Challenge:	LLM-based agents are powerful tools for automating complex scientific workflows, especially in chemistry, but their single-task performance is limited by tool constraints.
Approach:	They propose a framework that optimizes the collective capabilities of specialized tools by dynamic coordination within individual tasks.
Outcome:	The proposed framework outperforms chemistry-specialized models, generalist LLMs, and agent systems with tool orchestration.

SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)

Copied to clipboard

Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang

Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biases (2025.acl-long)

Challenge:	Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases.
Approach:	They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs.
Outcome:	The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs.

Copied to clipboard

Rena Wei Gao, Xuetong Wu, Tatsuki Kuribayashi, Mingrui Ye, Siya Qi, Carsten Roever, Yuanxing Liu, Zheng Yuan, Jey Han Lau

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) can simulate non-native-like English use observed in human second language (L2) learners interfered with by their native first language (N1) knowledge.
Approach:	They use large language models to simulate non-native-like English use observed in human second language (L2) learners, and then compare their outputs to real L2 learner data.
Outcome:	The proposed models replicate L1-dependent patterns observed in human second language (L2) learners, with distinct influences from various languages.

DiffuVST: Narrating Fictional Scenes with Global-History-Guided Denoising Models (2023.findings-emnlp)

Copied to clipboard

Shengguang Wu, Mei Yuan, Qi Su

Unified Dual-view Cognitive Model for Interpretable Claim Verification (2021.acl-long)

Challenge:	Existing methods for visual storytelling suffer from low inference speed and are not well-suited for synthetic scenes.
Approach:	They propose a diffusion-based system that generates visual descriptions as a single conditional denoising process.
Outcome:	The proposed system improves inter-sentence coherence and image-to-text fidelity.

Copied to clipboard

Lianwei Wu, Yuan Rao, Yuqian Lan, Ling Sun, Zhaoyin Qi

Question Directed Graph Attention Network for Numerical Reasoning over Text (2020.emnlp-main)

Challenge:	Existing studies constructing direct interactions between the claim and each single user response to capture evidence have shown remarkable success in interpretable claim verification.
Approach:	They propose a Dual-view model based on the views of Collective and Individual Cognition (CICD) that captures word-level semantics based . on individual cognition, they adjust the proportion between them to generate global evidence.
Outcome:	The proposed model is based on the views of collective and individual cognition and achieves state-of-the-art performance on three benchmark datasets.

Copied to clipboard

Kunlong Chen, Weidi Xu, Xingyi Cheng, Zou Xiaochuan, Yuyu Zhang, Le Song, Taifeng Wang, Yuan Qi, Wei Chu

Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models (2024.emnlp-main)

Challenge:	Numerical reasoning requires both natural language understanding and arithmetic computation.
Approach:	They propose a graph representation for the context of the passage and question needed for numerical reasoning.
Outcome:	The proposed model achieves remarkable results in benchmark datasets such as DROP.

Copied to clipboard

Yu Yuan, Lili Zhao, Kai Zhang, Guangting Zheng, Qi Liu

R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities in various tasks, but may rely on dataset biases as shortcuts for prediction.
Approach:	They propose to use a test suite to evaluate the impact of shortcuts on LLMs' performance.
Outcome:	The proposed test suite incorporates six shortcut types, five evaluation metrics, and four prompting strategies.

Copied to clipboard

Yuan Li, Qi Luo, Xiaonan Li, Bufan Li, Qinyuan Cheng, Bo Wang, Yining Zheng, Yuxin Wang, Zhangyue Yin, Xipeng Qiu

Unveiling the Deficiencies of Pre-trained Text-and-Layout Models in Real-world Visually-rich Document Information Extraction (2026.findings-eacl)

Challenge:	RAG systems that integrate external knowledge with Large Language Models often become bottlenecks due to their limited parameters compared to LLMs and their inability to perform step-by-step reasoning.
Approach:	They propose a model that integrates external knowledge with Large Language Models to enhance factual correctness and mitigate hallucination.
Outcome:	The proposed model outperforms baselines and can transfer well to different retrievers.

Copied to clipboard

Chong Zhang, Yixi Zhao, Yulu Xie, Chenshu Yuan, Yi Tu, Ya Guo, Mingxu Chai, Ziyu Shen, Yue Zhang, Qi Zhang

Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding (2024.emnlp-main)

Challenge:	PTLMs have shown remarkable success in multiple information extraction tasks . however, their performance in real-world scenarios falls short of expectations .
Approach:	They propose to use an entity-centric dataset to evaluate PTLMs' performance . they find that inadequate annotations in benchmark datasets lead to spurious correlations .
Outcome:	The proposed dataset disentangles the falsely-coupled segment and entity annotations that arises from the block-level annotation of FUNSD.

Copied to clipboard

Chong Zhang, Yi Tu, Yixi Zhao, Chenshu Yuan, Huan Chen, Yue Zhang, Mingxu Chai, Ya Guo, Huijia Zhu, Qi Zhang, Tao Gui

Parameter-Aware Contrastive Knowledge Editing: Tracing and Rectifying based on Critical Transmission Paths (2025.acl-long)

Challenge:	Existing models of layout reading order do not convey the complete reading order information in the layout.
Approach:	They propose to model layout reading order as ordering relations over layout elements . they propose a reading-order-relation-enhancing pipeline to improve model performance .
Outcome:	The proposed model outperforms existing models on a visual-rich document dataset and on eight cross-domain VrD-IE/QA tasks without targeted optimization.

Copied to clipboard

Songlin Zhai, Yuan Meng, Yuxin Zhang, Guilin Qi

CENTAUR: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inference (2025.acl-long)

Challenge:	Large language models (LLMs) encode vast amounts of knowledge in their parameters, but the acquired knowledge can be incorrect or outdated over time, necessitating rectification after pre-training.
Approach:	They propose a method that captures key information flows that influence model predictions . they propose 'critical transmission paths' to improve model editing .
Outcome:	The proposed method improves on two prominent datasets and three widely used LLMs.

Copied to clipboard

Jinglong Luo, Guanzhong Chen, Yehong Zhang, Shiyu Liu, Hui Wang, Yue Yu, Xun Zhou, Yuan Qi, Zenglin Xu

SaFER: A Robust and Efficient Framework for Fine-tuning BERT-based Classifier with Noisy Labels (2023.acl-industry)

Challenge:	Existing privacy-preserving Transformer Inference frameworks suffer from high computational overhead and performance losses.
Approach:	They propose a framework that integrates random permutations and SMPC to address the "impossible trinity" CENTAUR resists diverse data reconstruction attacks and boosts inference speed by 5.030.4 times .
Outcome:	CENTAUR achieves an unprecedented balance between privacy, efficiency, and performance.

Copied to clipboard

Zhenting Qi, Xiaoyu Tan, Chao Qu, Yinghui Xu, Yuan Qi

OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)

Challenge:	Existing noise-handling methods could not improve performance of BERT on noisy datasets . existing methods could only improve performance on noisy data, authors say .
Approach:	They propose a fine-tuning framework for BERT-based text classifiers that combats label noises without access to clean data for training or validation.
Outcome:	The proposed framework achieves superior performance on multiple text classification benchmarks.

Copied to clipboard

Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, Zili Wang

SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check (2020.acl-main)

Challenge:	Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence.
Approach:	They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included .
Outcome:	The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model.

Copied to clipboard

Xingyi Cheng, Weidi Xu, Kunlong Chen, Shaohua Jiang, Feng Wang, Taifeng Wang, Wei Chu, Yuan Qi

Towards Better Entity Linking with Multi-View Enhanced Distillation (2023.acl-long)

Challenge:	Existing methods to detect and correct spelling errors in Chinese take external input or just heuristic rules.
Approach:	They propose to incorporate phonological and visual similarity knowledge into Chinese language models by using a specialized graph convolutional network.
Outcome:	The proposed method outperforms existing models on three human-annotated datasets.

Copied to clipboard

Yi Liu, Yuan Tian, Jianxun Lian, Xinlong Wang, Yanan Cao, Fang Fang, Wen Zhang, Haizhen Huang, Weiwei Deng, Qi Zhang

Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning (2026.acl-long)

Challenge:	Entity linking is a fundamental task in Natural Language Processing (NLP), connecting mentions within unstructured contexts to their corresponding entities in a Knowledge Base (KB).
Approach:	They propose a dual-encoder framework that can efficiently match mentions to two-encoding frameworks by a global-view.
Outcome:	The proposed framework achieves state-of-the-art on several entity linking benchmarks.

Copied to clipboard

Shuyao Xu, Cheng Peng, Jiangxuan Long, Weidi Xu, Wei Chu, Yuan Qi

Try to Substitute: An Unsupervised Chinese Word Sense Disambiguation Method Based on HowNet (2020.coling-main)

Challenge:	Recent advances in model distillation show that data from advanced reasoning models can effectively train smaller student models.
Approach:	They propose a method to use both positive and negative distilled reasoning traces to maximize LLM reasoning performance in offline settings.
Outcome:	The proposed model outperforms existing methods in the distillation context.

Copied to clipboard

Bairu Hou, Fanchao Qi, Yuan Zang, Xurui Zhang, Zhiyuan Liu, Maosong Sun

Rethinking Expert Trajectory Utilization in LLM Post-training for Mathematical Reasoning (2026.acl-long)

Challenge:	Existing unsupervised methods for word sense disambiguation cannot work for HowNet-based WSD because of its uniqueness.
Approach:	They propose a method which exploits the masked language model task of pre-trained language models to conduct word sense disambiguation using a lexical knowledge base as the sense inventory.
Outcome:	The proposed method achieves significantly better performance than baseline methods.

Copied to clipboard

Bowen Ding, Yuhan Chen, Jiayang Lyu, Jiyao Yuan, Qi Zhu, Shuangshuang Tian, Dantong Zhu, Futing Wang, Heyuan Deng, Fei Mi, Lifeng Shang, Tao Lin

GuideLLM: Exploring LLM-Guided Conversation with Applications in Autobiography Interviewing (2025.naacl-long)

Challenge:	evolving generic Large Language Models into specialized Large Reasoning Models requires effective post-training.
Approach:	They propose a plasticity-ceiling framework to harness expert trajectories . they establish the Sequential SFT-then-RL pipeline as the superior standard .
Outcome:	The proposed framework overcomes stability and premature convergence deficits in synchronized approaches.

Copied to clipboard

Jinhao Duan, Xinyu Zhao, Zhuoxuan Zhang, Eunhye Grace Ko, Lily Boddy, Chenan Wang, Tianhao Li, Alexander Rasgon, Junyuan Hong, Min Kyung Lee, Chenxi Yuan, Qi Long, Ying Ding, Tianlong Chen, Kaidi Xu

Natural Language Interfaces

Challenge:	Large Language Models (LLMs) have demonstrated their effectiveness in human-guided dialogues, but tasks in the real world are more complex and require greater autonomy from LLMs.
Approach:	They propose to characterize LLM-guided conversation into three fundamental components: Goal Navigation, Context Management, Empathetic Engagement and implement an interviewing environment for the evaluation of LLMs.
Outcome:	The proposed LLM outperforms baseline LLMs in interviewing quality and autobiography generation quality.

Few-shot Intent Classification and Slot Filling with Retrieved Examples (2021.naacl-main)

Copied to clipboard

Dian Yu, Luheng He, Yuan Zhang, Xinya Du, Panupong Pasupat, Qi Li

Information Retrieval

Challenge:	Existing methods for few-shot learning are based on labeled examples, but they are non-trivial . few-sshot learning is challenging due to the imbalance in the amount of data between the source and target domains.
Approach:	They propose retrieval-based methods for intent classification and slot filling tasks . they use a batch-softmax objective to learn similar contextualized representations for spans .
Outcome:	The proposed method outperforms previous systems on the CLINC and SNIPS benchmarks.

ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)

Copied to clipboard

Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen

QA-Driven Zero-shot Slot Filling with Weak Supervision Pretraining (2021.acl-short)

Challenge:	Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models.
Approach:	They propose a dataset that provides rigorous evaluation of multi-hop tool use.
Outcome:	The proposed model achieves 49.04% accuracy across five model families.

Copied to clipboard

Xinya Du, Luheng He, Qi Li, Dian Yu, Panupong Pasupat, Yuan Zhang

PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models (2024.acl-long)

Challenge:	Existing methods to predict slots and their values do not encode enough semantic information, limiting the models’ zero-shot capability.
Approach:	They propose a QA-driven slot filling model which extracts slot-filler spans from utterances with a span-based QA model.
Outcome:	The proposed model outperforms baselines by over 5% on the SNIPS benchmark.

Copied to clipboard

Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan Yao, Yangqiu Song

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Challenge:	generative large language models (LLMs) exhibit surprising capability and integrate previous tasks into a unified text generation formulation.
Approach:	They propose a privacy evaluation benchmark to quantify the privacy leakage of language models.
Outcome:	The proposed benchmark compares PPLMs with different privacy implementations to find out how privacy leakage is handled.

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang