Papers by Lifeng Liu

Safe: Enhancing Mathematical Reasoning in Large Language Models via Retrospective Step-aware Formal Verification (2025.acl-long)

Challenge:	Experimental results show that pretrained language models generate inconsistent factual knowledge in many conversational tasks.
Approach:	They propose a method which explicitly introduces extended feedforward networks (FFNs) in Transformers to enhance factual knowledge expressions given the specific patterns of knowledge-grounded dialogue inputs.
Outcome:	The proposed methods improve the factual expression capability of feedforward networks (FFNs) in knowledge-grounded dialogue systems by knowledge enhancement and alignment respectively.

Copied to clipboard

Chengwu Liu, Ye Yuan, Yichun Yin, Yan Xu, Xin Xu, Zaoyu Chen, Yasheng Wang, Lifeng Shang, Qun Liu, Ming Zhang

Decomposable Neural Paraphrase Generation (P19-1)

Challenge:	Chain-of-Thought prompting is a de facto method to elicit reasoning capabilities from large language models (LLMs).
Approach:	They propose a step-aware formal verification framework Safe to address hallucinations in CoT prompting . they propose 'formal step' as a benchmark for step correctness theorem proving with 30,809 formal statements.
Outcome:	The proposed framework shows significant performance improvement while offering interpretable and verifiable evidence.

Copied to clipboard

Zichao Li, Xin Jiang, Lifeng Shang, Qun Liu

Text Generation

Challenge:	Existing models learn to generate paraphrases by mapping a sequence to another, with each word processed and generated in a uniform way.
Approach:	They propose a Transformer-based model that can learn and generate paraphrases at different levels of granularity in a disentangled way.
Outcome:	The proposed model achieves competitive in-domain performance compared to state-of-the-art models and significantly better performance when adapting to a new domain.

How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis (2022.findings-acl)

Copied to clipboard

Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks (2022.emnlp-main)

Challenge:	Recent studies show that pre-trained language models can fill in the missing factual words in cloze-style prompts such as ”Dante was born in [MASK]” .
Approach:	They propose to quantitatively measure and evaluate the word-level patterns that PLMs depend on to generate the missing factual words.
Outcome:	The proposed model fills in the missing factual words in cloze-style prompts by relying on effective clues or shortcut patterns.

Copied to clipboard

Zhongwei Wan, Yichun Yin, Wei Zhang, Jiaxin Shi, Lifeng Shang, Guangyong Chen, Xin Jiang, Qun Liu

mCLIP: Multilingual CLIP via Cross-lingual Transfer (2023.acl-long)

Challenge:	Existing domain-adaptive pre-training (DAPT) models tend to forget the general knowledge acquired by general PLMs, leading to catastrophic forgetting and sub-optimal performance.
Approach:	They propose a framework which augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge.
Outcome:	The proposed framework augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge.

Copied to clipboard

Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, Wenping Wang

Multilinguality

Challenge:	Existing multilingual vision-language pretrained models are biased towards English due to the lack of sufficient non-English image-text pairs.
Approach:	They propose to train a retrieval-efficient dual-stream multilingual VLP model by aligning CLIP model and a multilingual text encoder through a novel Triangle Cross-modal Knowledge Distillation method.
Outcome:	Empirical results show that mCLIP achieves new state-of-the-art performance for both zero-shot and finetuned multilingual image-text retrieval tasks.

Subtle Errors in Reasoning: Preference Learning via Error-injected Self-editing (2025.acl-long)

Copied to clipboard

Kaishuai Xu, Tiezheng Yu, Wenjun Hou, Yi Cheng, Chak Tou Leong, Liangyou Li, Xin Jiang, Lifeng Shang, Qun Liu, Wenjie Li

MINER: Multi-Interest Matching Network for News Recommendation (2022.findings-acl)

Challenge:	Existing studies to improve mathematical ability typically involve applying preference learning to step-wise solution pairs, but they overlook critical subtle errors.
Approach:	They propose a preference learning framework that injects predefined subtle errors into pivotal tokens to construct hard pairs for error mitigation.
Outcome:	Extensive experiments show that the proposed framework improves on Qwen2-7B-Instruct and MATH with 4.5K training samples.

Copied to clipboard

Jian Li, Jieming Zhu, Qiwei Bi, Guohao Cai, Lifeng Shang, Zhenhua Dong, Xin Jiang, Qun Liu

Representation Learning

Challenge:	Existing methods learn a single user embedding from user’s historical behaviors to represent the reading interest.
Approach:	They propose a poly attention scheme to learn multiple interest vectors for each user, which encodes the different aspects of user interest.
Outcome:	The proposed approach significantly outperforms existing state-of-the-art methods on the MIND news recommendation benchmark.

LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling (2022.emnlp-main)

Copied to clipboard

Dongsheng Chen, Chaofan Tao, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu

A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering (2021.acl-long)

Challenge:	Recent large-scale video-language pre-trained models have shown appealing performance on downstream tasks.
Approach:	They propose a video-text model that adapts a pre-trained image-language model into a text-based model without heavy pre-training.
Outcome:	The proposed model outperforms existing models on video-text retrieval and video question answering tasks without heavy pre-training.

Copied to clipboard

Zhihong Shao, Lifeng Shang, Qun Liu, Minlie Huang

Natural Language Interfaces

Challenge:	Weakly supervised question answering usually has only final answers as supervision signals while correct solutions are not provided.
Approach:	They propose to explicitly exploit the semantic correlations between question-answer pairs and predicted answers by maximizing mutual information between question and answer pairs.
Outcome:	The proposed method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions.

MTRec: Multi-Task Learning over BERT for News Recommendation (2022.findings-acl)

Copied to clipboard

Qiwei Bi, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Hanfang Yang

Compression of Generative Pre-trained Language Models via Quantization (2022.acl-long)

Challenge:	Existing news recommendation methods learn news representations solely based on news titles. Existing methods only utilize title information and neglect other valuable news information such as categories and entities.
Approach:	They propose a multi-task method to incorporate multi-field information into BERT, which improves its news encoding capability.
Outcome:	Extensive experiments on the MIND news recommendation benchmark show the proposed method is effective.

Copied to clipboard

Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong

GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification (P19-1)

Challenge:	Existing methods to compress generative pre-trained language models fail on generative tasks due to homogeneous word embeddings and limited memory.
Approach:	They propose a token-level contrastive distillation method to learn distinguishable word embeddings and a module-wise dynamic scaling method to make quantizers adaptive to different modules.
Outcome:	The proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin.

Copied to clipboard

Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun

GhostBERT: Generate More Features with Cheap Operations for BERT (2021.acl-long)

Challenge:	Existing methods to extract information from evidence are unable to grasp relational and logical information among the evidence.
Approach:	They propose a graph-based evidence aggregating and reasoning framework to integrate evidence from multiple pieces of evidence.
Outcome:	The proposed framework achieves significant performance improvements on a large-scale benchmark dataset.

Copied to clipboard

Zhiqi Huang, Lu Hou, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu

AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models (2023.acl-short)

Challenge:	Existing studies show that some parameters in pre-trained language models can be pruned away without severe accuracy degradation.
Approach:	They propose a method which generates more features with very cheap operations from the remaining features and can be applied to unpruned BERT models to enhance their performance.
Outcome:	Empirical results on the GLUE benchmark on three backbone models (i.e., BERT, RoBERTa and ELECTRA) verify the efficacy of the proposed method.

Copied to clipboard

Siheng Li, Cheng Yang, Yichun Yin, Xinyu Zhu, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang

Dialogue Systems & Conversational Agents

Challenge:	Existing research on information-seeking conversations is stymied by the lack of training data.
Approach:	They propose to use autoconv for synthetic conversation generation to capture the characteristics of the information-seeking process and fine tune an LLM with a few human conversations to generate synthetic conversations with high quality.
Outcome:	The proposed model improves on two commonly-used datasets and alleviates the dependence on human annotation.

Controlled Text Generation Using Dictionary Prior in Variational Autoencoders (2022.findings-acl)

Copied to clipboard

Xianghong Fang, Jian Li, Lifeng Shang, Xin Jiang, Qun Liu, Dit-Yan Yeung

More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression (2025.findings-emnlp)

Challenge:	Variational autoencoders (VAEs) have been widely applied in text generation tasks, but they suffer from insufficient representation capacity and poor controllability.
Approach:	They propose a data-driven prior that has expressivity and controllability.
Outcome:	The proposed prior enjoys expressivity and controllability and can be used in language modeling and controlled text generation.

Copied to clipboard

Jiebin Zhang, Dawei Zhu, Yifan Song, Wenhao Wu, Chuqiao Kuang, Xiaoguang Li, Lifeng Shang, Qun Liu, Sujian Li

Challenge:	storing more tokens in the KV cache at lower precision can enhance the long-context performance of large language models.
Approach:	They propose a token-precision trade-off strategy to optimize KV cache compression . they also propose storing more tokens in the KV at lower precision .
Outcome:	The proposed method achieves an optimal point within the Information Bottleneck compared to standalone KV pruning or KV quantization.

DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling (2021.emnlp-main)

Copied to clipboard

Baojun Wang, Zhao Zhang, Kun Xu, Guang-Yuan Hao, Yuyang Zhang, Lifeng Shang, Linlin Li, Xiao Chen, Xin Jiang, Qun Liu

Read before Generate! Faithful Long Form Question Answering with Machine Reading (2022.findings-acl)

Challenge:	Existing approaches to integrate lexical knowledge into deep learning models are limited by large-scale dynamic lexicons.
Approach:	They propose a plug-in lexicon incorporation approach for BERT based sequence labeling tasks . they adopt word-agnostic tag embeddings to avoid re-training the representation .
Outcome:	The proposed framework achieves new SOTA even with large scale lexicons, the authors show . they adopt word-agnostic tag embeddings to avoid re-training the representation .

Copied to clipboard

Dan Su, Xiaoguang Li, Jindi Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

Natural Language Interfaces

Challenge:	Long-form question answering (LFQA) generates a paragraph-length answer for a given question.
Approach:	They propose a framework that jointly models answer generation and machine reading.
Outcome:	The proposed model generates a more factually accurate answer from millions of documents retrieved from a large dataset.

BinaryBERT: Pushing the Limit of BERT Quantization (2021.acl-long)

Copied to clipboard

Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King

Stepwise Reasoning Checkpoint Analysis: A Test Time Scaling Method to Enhance LLMs’ Reasoning (2025.emnlp-main)

Challenge:	Recent pre-trained language models have achieved remarkable performance improvement in various tasks, but the improvement generally comes at the cost of increasing model size and computation.
Approach:	They propose a binary quantization technique which initializes binaryBERT by splitting from a ternary network.
Outcome:	The proposed model achieves state-of-the-art performance on the GLUE and SQUAD benchmarks while being 24x smaller.

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Yufei Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Learning to Edit: Aligning LLMs with Knowledge Editing (2024.acl-long)

Challenge:	Existing methods that use Chain-of-Thought suffer from path homogenization and inefficient use of intermediate results.
Approach:	They propose a framework that introduces checkpoints between reasoning steps to reduce path homogenization and create fault-tolerant mechanisms.
Outcome:	The proposed framework reduces path homogenization and creates fault-tolerant mechanism by utilizing high-quality intermediate results.

Copied to clipboard

Yuxin Jiang, Yufei Wang, Chuhan Wu, Wanjun Zhong, Xingshan Zeng, Jiahui Gao, Liangyou Li, Xin Jiang, Lifeng Shang, Ruiming Tang, Qun Liu, Wei Wang

Challenge:	Existing knowledge editing techniques rely on memorizing updated knowledge, impeding LLMs from effectively combining the new knowledge with their inherent knowledge when answering questions.
Approach:	They propose a Learning to Edit framework that equips LLMs with the ability to apply updated knowledge to input questions through a two-phase process .
Outcome:	The proposed framework outperforms existing methods in knowledge editing tasks and compares it with four benchmarks and two LLM architectures.

Generate & Rank: A Multi-task Framework for Math Word Problems (2021.findings-emnlp)

Copied to clipboard

Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, Qun Liu

Gradually Excavating External Knowledge for Implicit Complex Question Answering (2023.findings-emnlp)

Challenge:	Existing studies formalize MWP as a generation task but mathematical expressions are prone to minor mistakes.
Approach:	They propose a ranking task for math word problem (MWP) that learns from its own mistakes and distinguishes between correct and incorrect expressions.
Outcome:	The proposed model outperforms baselines on the classical Math23k dataset and is 7% higher than the state-of-the-art.

Copied to clipboard

Chang Liu, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, Edmund Lam, Ngai Wong

Prompt-Based Length Controlled Generation with Multiple Control Types (2024.findings-acl)

Challenge:	Large language models (LLMs) have gained attention for their human-comparable capabilities but they may not solve open-domain implicit questions due to out-of-date domain knowledge, one-shot generation and restricted comprehensiveness.
Approach:	They propose a gradual knowledge excavation framework for open-domain complex question answering using extrinsic knowledge and historical knowledge.
Outcome:	The proposed framework achieves 78.17% accuracy with less than 6% parameters of its competitors, setting new SOTA in the 10B LLM class.

Copied to clipboard

Renlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang, Qun Liu

Low-Resource NLP

Challenge:	Existing length control methods focus on a simple control type of “equal to” a target length.
Approach:	They propose a prompt-based method to achieve length controlled generation under different control types with high accuracy by using reinforcement learning and sample filtering with the reward signal given by rule-based reward models.
Outcome:	The proposed method significantly improves the accuracy of prompt-based length control on popular summarization datasets like CNNDM and NYT under multiple control types.

NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit (P19-3)

Copied to clipboard

Liqun Liu, Funan Mu, Pengyu Li, Xin Mu, Jing Tang, Xingsheng Ai, Ran Fu, Lifeng Wang, Xing Zhou

Information Extraction & Text Mining

Challenge:	NeuralClassifier is a toolkit for hierarchical multi-label text classification.
Approach:	They propose a toolkit for neural hierarchical multi-label text classification . they use a variety of text encoders to implement the model .
Outcome:	The proposed model achieves comparable performance with reported results in the literature.

Dynamic Augmentation Data Selection for Few-shot Text Classification (2022.findings-emnlp)

Copied to clipboard

Guangliang Liu, Lifeng Jin, Owen Yuan, Jiayu Zhou

Pre-training Language Models with Deterministic Factual Knowledge (2022.emnlp-main)

Challenge:	Data augmentation is a popular method for fine-tuning pre-trained language models to increase model robustness and performance.
Approach:	They propose a dynamic data selection method to select effective augmentation data from different augmentation sources according to the model’s learning stage by identifying a set of augmentation samples that optimally facilitates the learning process of the most current model.
Outcome:	The proposed method outperforms strong baselines on a variety of sentence classification tasks.

Copied to clipboard

Shaobo Li, Xiaoguang Li, Lifeng Shang, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

TernaryBERT: Distillation-aware Ultra-low Bit BERT (2020.emnlp-main)

Challenge:	Existing studies show that Pre-trained Language Models fail to capture factual knowledge robustly.
Approach:	They propose to let PLMs learn the deterministic relationship between context and masked content to improve their ability to capture factual knowledge.
Outcome:	The proposed methods improve accuracy and consistency of factual knowledge capturing and boost performance of other knowledge-intensive tasks.

Copied to clipboard

Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu

ProxyQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models (2024.acl-long)

Challenge:	Transformer-based pre-training models like BERT are computationally expensive and limited to resource-constrained devices.
Approach:	They propose a method which ternarizes the weights in a fine-tuned BERT model.
Outcome:	The proposed method outperforms the other methods on the GLUE and SQUAD benchmarks while being 14.9x smaller.

Copied to clipboard

Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, Linqi Song

ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis (2025.naacl-long)

Challenge:	Existing evaluation methods for large language models are labor-intensive and lack efficiency.
Approach:	They propose a framework dedicated to assessing long-text generation that includes in-depth human-curated meta-questions spanning various domains . they use a set of proxy-quests with pre-annotated answers to assess the content's quality by incorporating the generated texts as contextual background.
Outcome:	The proposed framework assesses the quality of long-text content by matching it with references through human evaluation or automated metrics.

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Dialogue Systems & Conversational Agents

Challenge:	Large Language Models (LLMs) can be enhanced by using supervised fine-tuning . however, access to fine-timing data can be limited.
Approach:	They propose a Graph-based Sampling strategy and a Planned-generation strategy to enhance the coherence between dialogues by using 8,000 synthetic dialogues.
Outcome:	The proposed model achieves tool-calling performance comparable to or surpassing GPT-4 while maintaining strong general capabilities.

FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models (2024.acl-long)

Copied to clipboard

Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, Wei Wang

On the Robustness of Editing Large Language Models (2024.emnlp-main)

Challenge:	Existing benchmarks focus on evaluating pure response quality, rather than assessing whether the response follows constraints stated in the instruction.
Approach:	They propose a Multi-level Fine-grained Constraints Following Benchmark for Large Language Models that adds a single constraint to the initial instruction at each increased level.
Outcome:	The proposed model can follow instructions with more constraints, and is deemed to have better instruction-following ability.

Copied to clipboard

Xinbei Ma, Tianjie Ju, Jiyang Qiu, Zhuosheng Zhang, Hai Zhao, Lifeng Liu, Yulong Wang

bert2BERT: Towards Reusable Pretrained Language Models (2022.acl-long)

Challenge:	Existing studies have exhibited impressive success and significant potential.
Approach:	They propose to modify the knowledge memory with minimum computational cost while preserving the performance on the retained knowledge.
Outcome:	The proposed methods avoid retraining to update the model parameters and have demonstrated promising performance and efficiency.

Copied to clipboard

Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen, Zhiyuan Liu, Qun Liu

Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation (2022.findings-acl)

Challenge:	Pre-training large language models can be expensive and wasteful.
Approach:	They propose a method which can transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and a two-stage learning method to further accelerate the pre-training.
Outcome:	The proposed method can transfer the knowledge of an existing smaller pre-trained model to a large model through parameter initialization and significantly improve the pre-training efficiency of the large model.

Copied to clipboard

Wenliang Dai, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step (2025.findings-naacl)

Challenge:	Recent large-scale vision-language pre-training models are powerful in multimodal classification and retrieval tasks.
Approach:	They propose to augment a vision-language pre-training model with a textual pre-trained language model . the model achieves 44.5% zero-shot accuracy on multimodal generation tasks .
Outcome:	The proposed model achieves 44.5% zero-shot accuracy on open-ended visual question answering and image captioning tasks.

Copied to clipboard

Zezhong Wang, Xingshan Zeng, Weiwen Liu, Yufei Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework (2025.findings-acl)

Challenge:	Current research found the issue of Early Answering in large language models where the models already have an answer before generating the Chain-of-Thought (CoT).
Approach:	They propose a method to probe changes in confidence during the model’s reasoning and prioritize answers with correct reasoning among multiple candidates.
Outcome:	The proposed method reveals that in a significant number of question-answer cases, CoT appears to be unnecessary and this necessity correlates with the simplicity of the task, defined by the reasoning steps required.

Copied to clipboard

Kaishuai Xu, Tiezheng Yu, Yi Cheng, Wenjun Hou, Liangyou Li, Xin Jiang, Lifeng Shang, Qun Liu, Wenjie Li

Robustness in NLP

Challenge:	Existing methods for fine-tuning open-source LLMs are limited to text-based analysis under predefined general criteria.
Approach:	They propose a framework that fine-tunes LLMs to replicate the evaluation explanations and judgments of proprietary models.
Outcome:	The proposed evaluation framework outperforms existing fine-tuned evaluation methods in effectiveness and robustness.

Discover and Prove: An Open-source Agentic Framework for Hard Mode Automated Theorem Proving in Lean 4 (2026.acl-long)

Copied to clipboard

Chengwu Liu, Yichun Yin, Ye Yuan, Jiaxuan Xie, Botao Li, Siqi Li, Jianhao Shen, Yan Xu, Lifeng Shang, Ming Zhang

TinyBERT: Distilling BERT for Natural Language Understanding (2020.findings-emnlp)

Challenge:	Existing approaches to solving mathematical problems fall into two broad categories: informal methods and formal methods.
Approach:	They propose to use LLM natural-language reasoning to discover answers . they introduce Discover And Prove framework that rewrites Hard Mode statements into Easy Mode ones for existing ATP provers.
Outcome:	The proposed framework can be used to prove hard mode statements on ATP benchmarks.

Copied to clipboard

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu

Improving Unsupervised Question Answering via Summarization-Informed Question Generation (2021.emnlp-main)

Challenge:	Pre-trained language models are computationally expensive and difficult to efficiently execute on resource-restricted devices.
Approach:	They propose a Transformer distillation method that performs Transformer distillations at pre-training and task-specific learning stages.
Outcome:	The proposed method accelerates inference and reduces model size while maintaining accuracy.

Copied to clipboard

Chenyang Lyu, Lifeng Shang, Yvette Graham, Jennifer Foster, Xin Jiang, Qun Liu

Text Generation

Challenge:	Question Generation (QG) is the production of meaningful questions given a set of input passages and corresponding answers.
Approach:	They propose a method which uses questions generated heuristically from news summaries as a source of training data for a QG system.
Outcome:	The proposed method outperforms previous unsupervised models on three in-domain datasets and three out-of-domain ones.

Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios (2024.findings-acl)

Copied to clipboard

Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruifeng Xu, Qun Liu

NewsDialogues: Towards Proactive News Grounded Conversation (2023.findings-acl)

Challenge:	Existing benchmarks focus on simple synthesized queries that do not reflect real-world complexity, thereby offering limited perspectives in evaluating tool utilization.
Approach:	They propose a benchmark to evaluate LLMs’ ability in tool utilization within real-world scenarios.
Outcome:	The proposed benchmark improves LLMs’ ability in tool utilization within real-world scenarios and eliminates the restriction of pre-defined toolset.

Copied to clipboard

Siheng Li, Yichun Yin, Cheng Yang, Wangjie Jiang, Yiwei Li, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang

Natural Language Interfaces

Challenge:	Hot news is one of the most popular topics in daily conversations.
Approach:	They propose a task where a dialogue system can lead the conversation based on key topics of the news.
Outcome:	The proposed method can lead conversations based on key topics of the news . it can also be used in information-seeking and chit-chat scenarios .

MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models (2024.emnlp-main)

Copied to clipboard

Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models (2021.acl-long)

Challenge:	Existing evaluation frameworks focus on single-turn evaluations, overlooking the models’ capabilities in multi-turn interactions.
Approach:	They propose a benchmark to evaluate the multi-turn conversational abilities of large language models (LLMs) by analyzing human-LLM conversations and constructing multi-turned queries for each category using GPT-4.
Outcome:	The proposed model outperforms open-source models in multi-turn tasks while retaining and recalling historical information.

Copied to clipboard

Yichun Yin, Cheng Chen, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu

Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering (2022.acl-long)

Challenge:	Pre-trained language models (PLMs) have achieved great success in natural language processing.
Approach:	They propose a method that automatically searches architecture hyper-parameters in BERT . they use one-shot learning and the search space to provide an adaptive development way .
Outcome:	The proposed method outperforms both the baseline and distillation-based methods on GLUE and SQUAD benchmarks.

Copied to clipboard

Jiawei Zhou, Xiaoguang Li, Lifeng Shang, Lan Luo, Ke Zhan, Enrui Hu, Xinyu Zhang, Hao Jiang, Zhao Cao, Fan Yu, Xin Jiang, Qun Liu, Lei Chen

LDEDE: LRP-Driven Efficient Detection and Editing Framework for LLM Privacy Neurons (2026.findings-acl)

Challenge:	Existing methods to train dense passage retrieval have a large data gap between upstream and downstream relevance.
Approach:	They propose a method to pre-train the dense retriever with the text relevance induced by hyperlinks within Web documents.
Outcome:	The proposed method outperforms existing methods under different scenarios and in the open-domain question answering domain.

Copied to clipboard

Zhao Zhengyuan, Cao Lifeng, null Sunhaodong, Shi Haotian, Du Xuehui, Liu Aodi, Niu Lanjie, Yang Xiaocheng