Papers by Shi Qiu
Self-Criticism: Aligning Large Language Models with their Understanding of Helpfulness, Honesty, and Harmlessness (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models are useful, honest, harmless (HHH) however, RLHF requires high hardware resources and human efforts. |
| Approach: | They propose a framework that allows LLMs to align themselves with HHH . they use IF and reinforcement learning from human feedback to fine-tune their models . |
| Outcome: | The proposed framework achieves similar performance to RLHF and human-generated models with a minimal alignment tax. |
Improving Image Captioning with Better Use of Caption (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to image captioning focus on visual attention, but many do not. |
| Approach: | They propose a framework that explores semantics available in captions and leverages that to enhance both image representation and caption generation. |
| Outcome: | The proposed framework outperforms baselines on the MSCOCO dataset and is state-of-the-art under a wide range of evaluation metrics. |
ULMR: Unlearning Large Language Models via Negative Response and Model Parameter Average (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) have attracted significant interest from the research community due to their broad applicability in many language-oriented tasks. |
| Approach: | They propose a framework which uses pre-training datasets to rewrite instructions and generate negative responses to preserve the performance of the original LLM. |
| Outcome: | The proposed framework can erase the pre-training data while maintaining the performance of the original model. |
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
Latent Inter-User Difference Modeling for LLM Personalization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly integrated into users’ daily lives, leading to a growing demand for personalized outputs. |
| Approach: | They propose a framework that models inter-user differences in the latent space instead of relying on language-based prompts. |
| Outcome: | The proposed framework outperforms baseline methods on personalized review generation. |
HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static compression methods suffer from coarse-grained caching and high I/O overhead. |
| Approach: | They propose a training-free dynamic compression framework that uses a sparse attention mechanism to categorize attention heads based on stability and similarity. |
| Outcome: | The proposed framework achieves state-of-the-art performance on long-context benchmarks and accelerates decoding by up to 3 compared to the original model with a 224K context. |
Minos: A Multimodal Evaluation Model for Bidirectional Generation Between Image and Text (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation models struggle to achieve consistent performance across image-to-text (I2T) and text-to image (T2I) tasks. |
| Approach: | They construct a multimodal evaluation model using a large multimodal dataset and rigorous quality control strategies to train it. |
| Outcome: | The proposed model achieves state-of-the-art evaluation performance across 16 out-of domain datasets covering both I2T and T2I tasks among all open-source multimodal evaluation models and remain competitive with closed-source models. |
Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing (2022.emnlp-main)
Copied to clipboard
Linlu Qiu, Peter Shaw, Panupong Pasupat, Tianze Shi, Jonathan Herzig, Emily Pitler, Fei Sha, Kristina Toutanova
| Challenge: | Pre-trained language models struggle on out-of-distribution compositional generalization . recent work shows considerable improvements on many NLP tasks from model scaling . |
| Approach: | They evaluate encoder-decoder models up to 11B parameters and decoder-only models up 540B parameters . they compare scaling curves for fine-tuning, prompt tuning, and in-context learning methods . |
| Outcome: | The proposed scaling methods improve compositional generalization on many tasks . fine-tuning generally has flat or negative scaling curves on out-of-distribution compositional . larger models are better at modeling the syntax of the output space, the study finds . |
PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)
Copied to clipboard
Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei
| Challenge: | Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning. |
| Approach: | They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios. |
| Outcome: | The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics. |
PersLEARN: Research Training through the Lens of Perspective Cultivation (2023.acl-demo)
Copied to clipboard
Yu-Zhe Shi, Shiqian Li, Xinyi Niu, Qiao Xu, Jiawen Liu, Yifan Xu, Shiyu Gu, Bingru He, Xinyang Li, Xinyu Zhao, Zijian Zhao, Yidong Lyu, Zhen Li, Sijia Liu, Lin Qiu, Jinhao Ji, Lecheng Ruan, Yuxi Ma, Wenjuan Han, Yixin Zhu
| Challenge: | PersLEARN is a tool designed to facilitate the cultivation of scientific perspectives . junior researchers struggle to identify the perspectives reflected in the literature and struggle to develop their own viewpoints. |
| Approach: | They propose a tool to facilitate the cultivation of scientific perspectives by interacting with a prompt-based model and allowing students to develop their own perspectives explicitly. |
| Outcome: | The proposed tool outperforms baseline approaches across multiple domains of literature from different perspectives. |
CORAL: Learning Consistent Representations across Multi-step Training with Lighter Speculative Drafter (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods that focus on training and inference suffer from misalignment . speculative decoding is a powerful technique that accelerates large language models . |
| Approach: | They propose a framework that improves both accuracy and efficiency in speculative drafting by using cross-step representational alignment. |
| Outcome: | The proposed framework outperforms existing methods on three LLM families and three benchmark datasets. |
Structured Attention for Unsupervised Dialogue Structure Induction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using structured attention, a model can learn dialogue structure in unsupervised fashion. |
| Approach: | They propose to incorporate structured attention layers into a Variational Recurrent Neural Network model with discrete latent states to learn dialogue structure in an unsupervised fashion. |
| Outcome: | The proposed model learns semantic structures similar to templates used to generate a dialogue corpus on two-party datasets and on multi-party dialogues, disentangling dialogues without human annotation. |
GLIMPSE: Do Large Vision-Language Models Truly Think With Videos or Just Glimpse at Them? (2025.emnlp-main)
Copied to clipboard
Yiyang Zhou, Linjie Li, Shi Qiu, Zhengyuan Yang, Yuyang Zhao, Siwei Han, Yangfan He, Kangqi Li, Haonian Ji, Zihao Zhao, Haibo Tong, Lijuan Wang, Huaxiu Yao
| Challenge: | Existing video benchmarks often resemble image-based questions with scans of only a few key frames, without deep temporal reasoning. |
| Approach: | They propose a video benchmark to assess whether large vision-language models can genuinely think with videos rather than perform superficial frame-level analysis. |
| Outcome: | The proposed benchmark consists of 3,269 videos and over 4,342 highly visual-centric questions across 11 categories, including Trajectory Analysis, Temporal Reasoning, and Forensics Detection. |
Failure makes the agent stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactions (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to self-reflection rely on heuristic prompting or unidirectional reasoning traces. |
| Approach: | They propose a structured reflection method that transforms the "from error to repair" process into a first-class, controllable, and trainable action. |
| Outcome: | The proposed method improves multi-turn tool-call success rates and error recovery while reducing redundant calls. |
Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge-enhanced pre-trained language models (PLMs) introduce redundant factual knowledge from knowledge bases and require complex modules. |
| Approach: | They propose a knowledge prompting-based PLM framework that incorporates factual knowledge into PLMs. |
| Outcome: | The proposed framework can be flexibly combined with existing mainstream PLMs. |
Pre-training with Meta Learning for Chinese Word Segmentation (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained models are beneficial to Chinese Word Segmentation (CWS). However, these models lack task-specific prior segmentation knowledge. |
| Approach: | They propose a pre-trained Chinese word segmentation model MetaSeg which incorporates meta learning into a multi-criteria pre-training task. |
| Outcome: | Empirical results show that MetaSeg can achieve new state-of-the-art performance on twelve widely-used CWS datasets and significantly improve model performance in low-resource settings. |
KECP: Knowledge Enhanced Contrastive Prompting for Few-shot Extractive Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Extractive Question Answering (EQA) is one of the most essential tasks in Machine Reading Comprehension (MRC). |
| Approach: | They propose a framework that transforms extractive question answering into a non-autoregressive Masked Language Modeling (MLM) generation problem. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches in few-shot learning scenarios by a large margin. |
Calibrating the Confidence of Large Language Models by Eliciting Fidelity (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models with RLHF and RLAIF have good alignment but exhibit overconfidence post-alignment. |
| Approach: | They propose a plug-and-play method to estimate the confidence of large language models. |
| Outcome: | The proposed method has shown good calibration performance on 6 RLHF-LMs on four MCQA datasets. |
ACSE: An Ancient Character Semantic-Aware Embedding for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on pre-Qin documents are insufficient to understand ancient characters . ancient characters have a low level of digitization and training corpora are extremely scarce . |
| Approach: | They propose a semantic-aware embedding for ancient Chinese characters that integrates glyphs and lexicality into modern Chinese semantic space. |
| Outcome: | The proposed model integrates glyph and lexicality of ancient characters and maps them to the modern Chinese semantic space. |
How to Mitigate Overfitting in Weak-to-strong Generalization? (2025.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that weak-to-strong generalization significantly improves PGR compared to naive weak- to-strong . superalignment refers to how humans can align models on tasks beyond human ability to evaluate . |
| Approach: | They propose a framework that elicits the capabilities of strong models through weak supervisors . they propose 'superalignment' to ensure that strong models align with supervisors' intentions . |
| Outcome: | The proposed framework significantly improves quality of supervision signals and quality of input questions compared to naive weak-to-strong generalization . |