Papers by Xin Shang

Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification (2025.acl-long)

Challenge:	Experimental results show that pretrained language models generate inconsistent factual knowledge in many conversational tasks.
Approach:	They propose a method which explicitly introduces extended feedforward networks (FFNs) in Transformers to enhance factual knowledge expressions given the specific patterns of knowledge-grounded dialogue inputs.
Outcome:	The proposed methods improve the factual expression capability of feedforward networks (FFNs) in knowledge-grounded dialogue systems by knowledge enhancement and alignment respectively.

Copied to clipboard

Chengwu Liu, Ye Yuan, Yichun Yin, Yan Xu, Xin Xu, Zaoyu Chen, Yasheng Wang, Lifeng Shang, Qun Liu, Ming Zhang

Decomposable Neural Paraphrase Generation (P19-1)

Challenge:	Chain-of-Thought prompting is a de facto method to elicit reasoning capabilities from large language models (LLMs).
Approach:	They propose a step-aware formal verification framework Safe to address hallucinations in CoT prompting . they propose 'formal step' as a benchmark for step correctness theorem proving with 30,809 formal statements.
Outcome:	The proposed framework shows significant performance improvement while offering interpretable and verifiable evidence.

Copied to clipboard

Zichao Li, Xin Jiang, Lifeng Shang, Qun Liu

Text Generation

Challenge:	Existing models learn to generate paraphrases by mapping a sequence to another, with each word processed and generated in a uniform way.
Approach:	They propose a Transformer-based model that can learn and generate paraphrases at different levels of granularity in a disentangled way.
Outcome:	The proposed model achieves competitive in-domain performance compared to state-of-the-art models and significantly better performance when adapting to a new domain.

How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis (2022.findings-acl)

Copied to clipboard

Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks (2022.emnlp-main)

Challenge:	Recent studies show that pre-trained language models can fill in the missing factual words in cloze-style prompts such as ”Dante was born in [MASK]” .
Approach:	They propose to quantitatively measure and evaluate the word-level patterns that PLMs depend on to generate the missing factual words.
Outcome:	The proposed model fills in the missing factual words in cloze-style prompts by relying on effective clues or shortcut patterns.

Copied to clipboard

Zhongwei Wan, Yichun Yin, Wei Zhang, Jiaxin Shi, Lifeng Shang, Guangyong Chen, Xin Jiang, Qun Liu

mCLIP: Multilingual CLIP via Cross-lingual Transfer (2023.acl-long)

Challenge:	Existing domain-adaptive pre-training (DAPT) models tend to forget the general knowledge acquired by general PLMs, leading to catastrophic forgetting and sub-optimal performance.
Approach:	They propose a framework which augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge.
Outcome:	The proposed framework augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge.

Copied to clipboard

Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, Wenping Wang

Multilinguality

Challenge:	Existing multilingual vision-language pretrained models are biased towards English due to the lack of sufficient non-English image-text pairs.
Approach:	They propose to train a retrieval-efficient dual-stream multilingual VLP model by aligning CLIP model and a multilingual text encoder through a novel Triangle Cross-modal Knowledge Distillation method.
Outcome:	Empirical results show that mCLIP achieves new state-of-the-art performance for both zero-shot and finetuned multilingual image-text retrieval tasks.

ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch (2026.acl-long)

Copied to clipboard

Zheng Liu, Honglin Lin, Xiaoyang Wang, Xin Gao, Yu Li, Mengzhang Cai, Yun Zhu, Zhanping Zhong, Qizhi Pei, Zhuoshi Pan, Xiaoran Shang, Conghui He, Bin Cui, Wentao Zhang, Lijun Wu

Multimodality

Challenge:	Existing open-source vision language models lack high-quality training data for chart reasoning . current models are simplistic and repetitive, while associated QA pairs are prone to hallucinations .
Approach:	They propose a framework to synthesize complex charts and reliable reasoning data from scratch.
Outcome:	Experimental results show that ChartVerse-8B surpasses existing models in QA and difficulty . lack of high-quality training data hampers development of open-source models .

Subtle Errors in Reasoning: Preference Learning via Error-injected Self-editing (2025.acl-long)

Copied to clipboard

Kaishuai Xu, Tiezheng Yu, Wenjun Hou, Yi Cheng, Chak Tou Leong, Liangyou Li, Xin Jiang, Lifeng Shang, Qun Liu, Wenjie Li

MINER: Multi-Interest Matching Network for News Recommendation (2022.findings-acl)

Challenge:	Existing studies to improve mathematical ability typically involve applying preference learning to step-wise solution pairs, but they overlook critical subtle errors.
Approach:	They propose a preference learning framework that injects predefined subtle errors into pivotal tokens to construct hard pairs for error mitigation.
Outcome:	Extensive experiments show that the proposed framework improves on Qwen2-7B-Instruct and MATH with 4.5K training samples.

Copied to clipboard

Jian Li, Jieming Zhu, Qiwei Bi, Guohao Cai, Lifeng Shang, Zhenhua Dong, Xin Jiang, Qun Liu

Representation Learning

Challenge:	Existing methods learn a single user embedding from user’s historical behaviors to represent the reading interest.
Approach:	They propose a poly attention scheme to learn multiple interest vectors for each user, which encodes the different aspects of user interest.
Outcome:	The proposed approach significantly outperforms existing state-of-the-art methods on the MIND news recommendation benchmark.

LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling (2022.emnlp-main)

Copied to clipboard

Dongsheng Chen, Chaofan Tao, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu

MTRec: Multi-Task Learning over BERT for News Recommendation (2022.findings-acl)

Challenge:	Recent large-scale video-language pre-trained models have shown appealing performance on downstream tasks.
Approach:	They propose a video-text model that adapts a pre-trained image-language model into a text-based model without heavy pre-training.
Outcome:	The proposed model outperforms existing models on video-text retrieval and video question answering tasks without heavy pre-training.

Copied to clipboard

Qiwei Bi, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Hanfang Yang

PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (2026.acl-long)

Challenge:	Existing news recommendation methods learn news representations solely based on news titles. Existing methods only utilize title information and neglect other valuable news information such as categories and entities.
Approach:	They propose a multi-task method to incorporate multi-field information into BERT, which improves its news encoding capability.
Outcome:	Extensive experiments on the MIND news recommendation benchmark show the proposed method is effective.

Copied to clipboard

Jingcheng Hu, Yinmin Zhang, Shijie Shang, Xiaobo Yang, Yue Peng, Zhewei Huang, Hebin Zhou, Xin Wu, Jie Cheng, Fanqi Wan, Xiangwen Kong, Chengyuan Yao, Kaiwen Yan, Ailin Huang, Hongyu Zhou, Qi Han, Zheng Ge, Xiangyu Zhang, Heung-Yeung Shum

Compression of Generative Pre-trained Language Models via Quantization (2022.acl-long)

Challenge:	Parallel Coordinated Reasoning (PaCoRe) overcomes a central limitation of contemporary language models: their inability to scale test-time compute (TTC) far beyond sequential reasoning under a fixed context window.
Approach:	They propose a training-and-inference framework to overcome a central limitation of language models: their inability to scale test-time compute (TTC) under a fixed context window.
Outcome:	The proposed model scales to multi-million-token effective TTC without exceeding context limits.

Copied to clipboard

Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong

GhostBERT: Generate More Features with Cheap Operations for BERT (2021.acl-long)

Challenge:	Existing methods to compress generative pre-trained language models fail on generative tasks due to homogeneous word embeddings and limited memory.
Approach:	They propose a token-level contrastive distillation method to learn distinguishable word embeddings and a module-wise dynamic scaling method to make quantizers adaptive to different modules.
Outcome:	The proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin.

Copied to clipboard

Zhiqi Huang, Lu Hou, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu

AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models (2023.acl-short)

Challenge:	Existing studies show that some parameters in pre-trained language models can be pruned away without severe accuracy degradation.
Approach:	They propose a method which generates more features with very cheap operations from the remaining features and can be applied to unpruned BERT models to enhance their performance.
Outcome:	Empirical results on the GLUE benchmark on three backbone models (i.e., BERT, RoBERTa and ELECTRA) verify the efficacy of the proposed method.

Copied to clipboard

Siheng Li, Cheng Yang, Yichun Yin, Xinyu Zhu, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang

Dialogue Systems & Conversational Agents

Challenge:	Existing research on information-seeking conversations is stymied by the lack of training data.
Approach:	They propose to use autoconv for synthetic conversation generation to capture the characteristics of the information-seeking process and fine tune an LLM with a few human conversations to generate synthetic conversations with high quality.
Outcome:	The proposed model improves on two commonly-used datasets and alleviates the dependence on human annotation.

Can Language Models Follow Multiple Turns of Entangled Instructions? (2025.findings-emnlp)

Copied to clipboard

Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji

Controlled Text Generation Using Dictionary Prior in Variational Autoencoders (2022.findings-acl)

Challenge:	Despite of significant achievements in improving instruction-following capabilities of large language models, the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge.
Approach:	They construct multi-turn instruction with 1.1K high-quality multi-turned conversations using the human-in-the-loop approach and examine their capabilities.
Outcome:	The proposed model shows that it is difficult to integrate multiple turns and balance competing objectives when instructions intersect or conflict.

Copied to clipboard

Xianghong Fang, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Dit-Yan Yeung

DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling (2021.emnlp-main)

Challenge:	Variational autoencoders (VAEs) have been widely applied in text generation tasks, but they suffer from insufficient representation capacity and poor controllability.
Approach:	They propose a data-driven prior that has expressivity and controllability.
Outcome:	The proposed prior enjoys expressivity and controllability and can be used in language modeling and controlled text generation.

Copied to clipboard

Baojun Wang, Zhao Zhang, Kun Xu, Guang-Yuan Hao, Yuyang Zhang, Lifeng Shang, Linlin Li, Xiao Chen, Xin Jiang, Qun Liu

CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation (2024.acl-long)

Challenge:	Existing approaches to integrate lexical knowledge into deep learning models are limited by large-scale dynamic lexicons.
Approach:	They propose a plug-in lexicon incorporation approach for BERT based sequence labeling tasks . they adopt word-agnostic tag embeddings to avoid re-training the representation .
Outcome:	The proposed framework achieves new SOTA even with large scale lexicons, the authors show . they adopt word-agnostic tag embeddings to avoid re-training the representation .

Copied to clipboard

Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, Rui Yan

Dialogue Systems & Conversational Agents

Challenge:	CharacterEval is a benchmark for comprehensive RPCA assessment in Chinese . authors show that Chinese LLMs exhibit more promising capabilities than GPT-4 in role-playing conversation.
Approach:	They propose a Chinese benchmark for comprehensive RPCA assessment . they use a dataset of Chinese role-playing dialogues and character profiles .
Outcome:	The proposed benchmark demonstrates that Chinese LLMs exhibit more promising capabilities than GPT-4 in Chinese role-playing conversation.

Read before Generate! Faithful Long Form Question Answering with Machine Reading (2022.findings-acl)

Copied to clipboard

Dan Su, Xiaoguang Li, Jindi Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

Natural Language Interfaces

Challenge:	Long-form question answering (LFQA) generates a paragraph-length answer for a given question.
Approach:	They propose a framework that jointly models answer generation and machine reading.
Outcome:	The proposed model generates a more factually accurate answer from millions of documents retrieved from a large dataset.

UCS: Estimating Unseen Coverage for Improved In-Context Learning (2026.findings-acl)

Copied to clipboard

Jiayi Xin, Xiang Li, Evan Qiang, Weiqing He, Tianqi Shang, Weijie J Su, Qi Long

Text Clustering

Challenge:	Existing selection methods prioritize heuristic notions of relevance or diversity and provide limited insight into the coverage of a demonstration set.
Approach:	They propose a training-free, subset-level coverage prior that is unrevealed by a model-consistent embedding and a Smoothed Good-Turing estimator to estimate the number of unrevelled clusters within a candidate subset.
Outcome:	Experiments on multiple intent-classification and reasoning benchmarks show that augmenting strong baselines with UCS improves ICL accuracy by 2-6% under the same selection budget.

BinaryBERT: Pushing the Limit of BERT Quantization (2021.acl-long)

Copied to clipboard

Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King

Stepwise Reasoning Checkpoint Analysis: A Test Time Scaling Method to Enhance LLMs’ Reasoning (2025.emnlp-main)

Challenge:	Recent pre-trained language models have achieved remarkable performance improvement in various tasks, but the improvement generally comes at the cost of increasing model size and computation.
Approach:	They propose a binary quantization technique which initializes binaryBERT by splitting from a ternary network.
Outcome:	The proposed model achieves state-of-the-art performance on the GLUE and SQUAD benchmarks while being 24x smaller.

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Yufei Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge (2025.acl-long)

Challenge:	Existing methods that use Chain-of-Thought suffer from path homogenization and inefficient use of intermediate results.
Approach:	They propose a framework that introduces checkpoints between reasoning steps to reduce path homogenization and create fault-tolerant mechanisms.
Outcome:	The proposed framework reduces path homogenization and creates fault-tolerant mechanism by utilizing high-quality intermediate results.

Copied to clipboard

Qiyuan Zhang, Yufei Wang, Yuxin Jiang, Liangyou Li, Chuhan Wu, Yasheng Wang, Xin Jiang, Lifeng Shang, Ruiming Tang, Fuyuan Lyu, Chen Ma

Learning to Edit: Aligning LLMs with Knowledge Editing (2024.acl-long)

Challenge:	Existing methods rely on majority voting or criteria expansion to capture detailed and detailed details, often leading to incomplete outcomes.
Approach:	They propose a method which introduces additional crowd responses to compare with the candidate responses, thereby exposing deeper and more comprehensive details within the candidate answers.
Outcome:	Experiments show that the proposed method improves evaluation reliability and achieves an average gain of 6.7% across five benchmarks.

Copied to clipboard

Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, Qun Liu, Wei Wang

Challenge:	Existing knowledge editing techniques rely on memorizing updated knowledge, impeding LLMs from effectively combining the new knowledge with their inherent knowledge when answering questions.
Approach:	They propose a Learning to Edit framework that equips LLMs with the ability to apply updated knowledge to input questions through a two-phase process .
Outcome:	The proposed framework outperforms existing methods in knowledge editing tasks and compares it with four benchmarks and two LLM architectures.

Generate & Rank: A Multi-task Framework for Math Word Problems (2021.findings-emnlp)

Copied to clipboard

Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, Qun Liu

Gradually Excavating External Knowledge for Implicit Complex Question Answering (2023.findings-emnlp)

Challenge:	Existing studies formalize MWP as a generation task but mathematical expressions are prone to minor mistakes.
Approach:	They propose a ranking task for math word problem (MWP) that learns from its own mistakes and distinguishes between correct and incorrect expressions.
Outcome:	The proposed model outperforms baselines on the classical Math23k dataset and is 7% higher than the state-of-the-art.

Copied to clipboard

Chang Liu, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, Edmund Lam, Ngai Wong

Prompt-Based Length Controlled Generation with Multiple Control Types (2024.findings-acl)

Challenge:	Large language models (LLMs) have gained attention for their human-comparable capabilities but they may not solve open-domain implicit questions due to out-of-date domain knowledge, one-shot generation and restricted comprehensiveness.
Approach:	They propose a gradual knowledge excavation framework for open-domain complex question answering using extrinsic knowledge and historical knowledge.
Outcome:	The proposed framework achieves 78.17% accuracy with less than 6% parameters of its competitors, setting new SOTA in the 10B LLM class.

Copied to clipboard

Renlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang, Qun Liu

Low-Resource NLP

Challenge:	Existing length control methods focus on a simple control type of “equal to” a target length.
Approach:	They propose a prompt-based method to achieve length controlled generation under different control types with high accuracy by using reinforcement learning and sample filtering with the reward signal given by rule-based reward models.
Outcome:	The proposed method significantly improves the accuracy of prompt-based length control on popular summarization datasets like CNNDM and NYT under multiple control types.

Pre-training Language Models with Deterministic Factual Knowledge (2022.emnlp-main)

Copied to clipboard

Shaobo Li, Xiaoguang Li, Lifeng Shang, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

TernaryBERT: Distillation-aware Ultra-low Bit BERT (2020.emnlp-main)

Challenge:	Existing studies show that Pre-trained Language Models fail to capture factual knowledge robustly.
Approach:	They propose to let PLMs learn the deterministic relationship between context and masked content to improve their ability to capture factual knowledge.
Outcome:	The proposed methods improve accuracy and consistency of factual knowledge capturing and boost performance of other knowledge-intensive tasks.

Copied to clipboard

Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu

ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis (2025.naacl-long)

Challenge:	Transformer-based pre-training models like BERT are computationally expensive and limited to resource-constrained devices.
Approach:	They propose a method which ternarizes the weights in a fine-tuned BERT model.
Outcome:	The proposed method outperforms the other methods on the GLUE and SQUAD benchmarks while being 14.9x smaller.

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) can be enhanced by using supervised fine-tuning . however, access to fine-timing data can be limited.
Approach:	They propose a Graph-based Sampling strategy and a Planned-generation strategy to enhance the coherence between dialogues by using 8,000 synthetic dialogues.
Outcome:	The proposed model achieves tool-calling performance comparable to or surpassing GPT-4 while maintaining strong general capabilities.

FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models (2024.acl-long)

Copied to clipboard

Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, Wei Wang

Paraphrase Generation with Deep Reinforcement Learning (D18-1)

Challenge:	Existing benchmarks focus on evaluating pure response quality, rather than assessing whether the response follows constraints stated in the instruction.
Approach:	They propose a Multi-level Fine-grained Constraints Following Benchmark for Large Language Models that adds a single constraint to the initial instruction at each increased level.
Outcome:	The proposed model can follow instructions with more constraints, and is deemed to have better instruction-following ability.

Copied to clipboard

Zichao Li, Xin Jiang, Lifeng Shang, Hang Li

Text Generation

Challenge:	Paraphrase generation is an important but challenging task in natural language processing . traditional symbolic approaches to paraphrase generation include rule-based methods, thesaurus-based approaches and statistical machine translation (SMT)
Approach:	They propose a deep reinforcement learning approach to automatic paraphrase generation . they propose supervised learning and reinforcement learning for evaluators .
Outcome:	The proposed framework outperforms state-of-the-art methods in paraphrase generation on two datasets.

bert2BERT: Towards Reusable Pretrained Language Models (2022.acl-long)

Copied to clipboard

Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen, Zhiyuan Liu, Qun Liu

Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation (2022.findings-acl)

Challenge:	Pre-training large language models can be expensive and wasteful.
Approach:	They propose a method which can transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and a two-stage learning method to further accelerate the pre-training.
Outcome:	The proposed method can transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and significantly improve the pre-training efficiency of the large model.

Copied to clipboard

Wenliang Dai, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

Responsible & Trustworthy NLP

Challenge:	Recent large-scale vision-language pre-training models are powerful in multimodal classification and retrieval tasks.
Approach:	They propose to augment a vision-language pre-training model with a textual pre-trained language model . the model achieves 44.5% zero-shot accuracy on multimodal generation tasks .
Outcome:	The proposed model achieves 44.5% zero-shot accuracy on open-ended visual question answering and image captioning tasks.

Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step (2025.findings-naacl)

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Yufei Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework (2025.findings-acl)

Challenge:	Current research found the issue of Early Answering in large language models where the models already have an answer before generating the Chain-of-Thought (CoT).
Approach:	They propose a method to probe changes in confidence during the model’s reasoning and prioritize answers with correct reasoning among multiple candidates.
Outcome:	The proposed method reveals that in a significant number of question-answer cases, CoT appears to be unnecessary and this necessity correlates with the simplicity of the task, defined by the reasoning steps required.

Copied to clipboard

Kaishuai Xu, Tiezheng Yu, Yi Cheng, Wenjun Hou, Liangyou Li, Xin Jiang, Lifeng Shang, Qun Liu, Wenjie Li

Robustness in NLP

Challenge:	Existing methods for fine-tuning open-source LLMs are limited to text-based analysis under predefined general criteria.
Approach:	They propose a framework that fine-tunes LLMs to replicate the evaluation explanations and judgments of proprietary models.
Outcome:	The proposed evaluation framework outperforms existing fine-tuned evaluation methods in effectiveness and robustness.

TinyBERT: Distilling BERT for Natural Language Understanding (2020.findings-emnlp)

Copied to clipboard

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu

Improving Unsupervised Question Answering via Summarization-Informed Question Generation (2021.emnlp-main)

Challenge:	Pre-trained language models are computationally expensive and difficult to efficiently execute on resource-restricted devices.
Approach:	They propose a Transformer distillation method that performs Transformer distillations at pre-training and task-specific learning stages.
Outcome:	The proposed method accelerates inference and reduces model size while maintaining accuracy.

Copied to clipboard

Chenyang Lyu, Lifeng Shang, Yvette Graham, Jennifer Foster, Xin Jiang, Qun Liu

Text Generation

Challenge:	Question Generation (QG) is the production of meaningful questions given a set of input passages and corresponding answers.
Approach:	They propose a method which uses questions generated heuristically from news summaries as a source of training data for a QG system.
Outcome:	The proposed method outperforms previous unsupervised models on three in-domain datasets and three out-of-domain ones.

Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios (2024.findings-acl)

Copied to clipboard

Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruifeng Xu, Qun Liu

WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning (2024.acl-long)

Challenge:	Existing benchmarks focus on simple synthesized queries that do not reflect real-world complexity, thereby offering limited perspectives in evaluating tool utilization.
Approach:	They propose a benchmark to evaluate LLMs’ ability in tool utilization within real-world scenarios.
Outcome:	The proposed benchmark improves LLMs’ ability in tool utilization within real-world scenarios and eliminates the restriction of pre-defined toolset.

Copied to clipboard

Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang, Can Xu, Yishujie Zhao, Wenxiang Hu, Qiufeng Yin

CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark (2022.acl-long)

Challenge:	Recent work shows that Code Large Language Models can address a wide range of code-related tasks.
Approach:	They propose a method to generate widespread and versatile instruction data from open source code datasets and use it to train code-related models.
Outcome:	The proposed model outperforms open-source models in generalization ability across code-related tasks.

Copied to clipboard

Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen

Challenge:	a new benchmark for biomedical language understanding is being developed in Chinese . most benchmarks are limited to English, which makes it difficult to replicate success in other languages.
Approach:	They propose to use Chinese biomedical language understanding evaluation benchmarks to evaluate Chinese models.
Outcome:	The proposed benchmarks show that the current models perform worse than the human ceiling.

NewsDialogues: Towards Proactive News Grounded Conversation (2023.findings-acl)

Copied to clipboard

Siheng Li, Yichun Yin, Cheng Yang, Wangjie Jiang, Yiwei Li, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang

Natural Language Interfaces

Challenge:	Hot news is one of the most popular topics in daily conversations.
Approach:	They propose a task where a dialogue system can lead the conversation based on key topics of the news.
Outcome:	The proposed method can lead conversations based on key topics of the news . it can also be used in information-seeking and chit-chat scenarios .

Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction (2025.findings-acl)

Copied to clipboard

Yuxin Jiang, Yufei Wang, Chuhan Wu, Xinyi Dai, Yan Xu, Weinan Gan, Yasheng Wang, Xin Jiang, Lifeng Shang, Ruiming Tang, Wei Wang

Low-Resource NLP

Challenge:	Existing methods for generating and curating high-quality instruction-tuning data rely heavily on the quality of seed data or strong assumptions about the structure and content of web documents.
Approach:	They propose a fully automated framework for synthesizing high-quality instruction-tuning (IT) data directly from raw web documents with minimal assumptions.
Outcome:	The proposed framework outperforms state-of-the-art baselines by 16.65% across four instruction-following benchmarks.

MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models (2024.emnlp-main)

Copied to clipboard

Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models (2021.acl-long)

Challenge:	Existing evaluation frameworks focus on single-turn evaluations, overlooking the models’ capabilities in multi-turn interactions.
Approach:	They propose a benchmark to evaluate the multi-turn conversational abilities of large language models (LLMs) by analyzing human-LLM conversations and constructing multi-turned queries for each category using GPT-4.
Outcome:	The proposed model outperforms open-source models in multi-turn tasks while retaining and recalling historical information.

Copied to clipboard

Yichun Yin, Cheng Chen, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu

Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering (2022.acl-long)

Challenge:	Pre-trained language models (PLMs) have achieved great success in natural language processing.
Approach:	They propose a method that automatically searches architecture hyper-parameters in BERT . they use one-shot learning and the search space to provide an adaptive development way .
Outcome:	The proposed method outperforms both the baseline and distillation-based methods on GLUE and SQUAD benchmarks.

Copied to clipboard

Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, Hao Jiang, Zhao Cao, Fan Yu, Xin Jiang, Qun Liu, Lei Chen