Papers by Chengyu Wang

Thinking with DistilQwen: A Tale of Four Distilled Reasoning and Reward Model Series (2025.emnlp-industry)

Challenge:	Instruction tuning aims to align large language models (LLMs) with open-domain instructions and human-preferred responses.
Approach:	They propose a multi-round distillation framework that uses an oracle LLM to select instructions that are difficult for a student LLM.
Outcome:	The proposed framework outperforms large language models and user-tuned models on several widely recognized benchmarks and multiple student LLMs.

Copied to clipboard

Wenrui Cai, Chengyu Wang, Junbing Yan, Jun Huang, Xiangzhong Fang

Green & Sustainable NLP

Challenge:	In the rapidly evolving landscape of large language models, the need for efficient reasoning models has become increasingly urgent.
Approach:	They extend the Qwen model family by introducing four model series specifically designed for industrial applications.
Outcome:	The proposed models outperform previous models in multiple benchmarks and provide scalable training and inference functionality on the Alibaba Cloud PAI platform.

PAI-Diffusion: Constructing and Serving a Family of Open Chinese Diffusion Models for Text-to-image Synthesis on the Cloud (2024.acl-demos)

Copied to clipboard

Chengyu Wang, Zhongjie Duan, Bingyan Liu, Xinyi Zou, Cen Chen, Kui Jia, Jun Huang

Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehension (2023.findings-emnlp)

Challenge:	Existing diffusion models fail to address the challenges of generating high-quality images from textual descriptions due to its large vocabulary size and complex character relationships.
Approach:	They propose a framework that integrates Chinese diffusion models with Alibaba Cloud's Platform for AI and enables the generation of contextually relevant images.
Outcome:	The proposed framework integrates with Alibaba Cloud’s Platform for AI, providing accessible and scalable solutions.

Copied to clipboard

Tingfeng Cao, Chengyu Wang, Chuanqi Tan, Jun Huang, Jinhui Zhu

Multilinguality

Challenge:	X-STA is a new approach for cross-lingual machine reading comprehension . the variation of answer span positions in different languages makes it difficult to transfer knowledge across languages.
Approach:	They propose a method that leverages an attentive teacher to subtly transfer the answer spans of the source language to the answer output space of the target.
Outcome:	The proposed method outperforms state-of-the-art approaches on three multi-lingual datasets.

DAFNet: Dynamic Auxiliary Fusion for Sequential Model Editing in Large Language Models (2024.findings-acl)

Copied to clipboard

Taolin Zhang, Qizhou Chen, Dongyang Li, Chengyu Wang, Xiaofeng He, Longtao Huang, Hui Xue’, Jun Huang

HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction (2022.findings-acl)

Challenge:	Large language models (LLMs) have shown impressive results, but still suffer from hallucination, i.e., the generation of false information.
Approach:	They propose a task of sequential model editing that aims to rectify mistakes continuously.
Outcome:	The proposed method significantly outperforms baselines in single-turn and sequential editing.

Copied to clipboard

Dongyang Li, Taolin Zhang, Nan Hu, Chengyu Wang, Xiaofeng He

Taming "Zombie" Agents: A Markov State-Aware Framework for Resilient Multi-Agent Evolution (2026.acl-long)

Challenge:	Existing approaches of distantly supervised relation extraction (DSRE) focus on sentence-level or bag-level de-noising, neglecting the explicit interaction with cross levels.
Approach:	They propose a hierarchical contrastive learning framework for distantly supervised relation extraction to reduce noisy sentences.
Outcome:	The proposed framework outperforms baselines in various mainstream DSRE datasets.

Copied to clipboard

Taolin Zhang, Pukun Zhao, Qizhou Chen, Jiuheng Wan, Chen Chen, Xiaofeng He, Chengyu Wang, Richang Hong

KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning (2024.lrec-main)

Challenge:	Existing approaches to improve efficiency of multi-agent systems rely on aggressive graph topology evolution . however, such hard pruning overlooks the potential for "zombie" agents to recover and contribute in subsequent discussion rounds.
Approach:	They propose a Markov state-aware framework for resilient multi-agent evolution that manages agent collaboration through soft state transitions.
Outcome:	The proposed framework outperforms baselines and significantly reduces token consumption through state-aware agent scheduling.

Copied to clipboard

Dongyang Li, Taolin Zhang, Longtao Huang, Chengyu Wang, Xiaofeng He, Hui Xue

Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training (2022.emnlp-industry)

Challenge:	General pre-trained language models (PLMs) leverage relation triples from knowledge graphs (KGs) and integrate external data sources into language models via self-supervised learning.
Approach:	They propose to learn Knowledge-Enhanced language representations with Hierarchical Reinforcement Learning (KEHRL) to detect positions for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge.
Outcome:	The proposed model can detect essential positions in texts for knowledge injection and integrate external knowledge into the model to avoid injecting inaccurate or irrelevant knowledge.

Copied to clipboard

Taolin Zhang, Junwei Dong, Jianing Wang, Chengyu Wang, Ang Wang, Yinghui Liu, Jun Huang, Yong Li, Xiaofeng He

Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning (2024.findings-naacl)

Challenge:	Existing knowledge-enhanced pre-trained language models (KEPLMs) can capture internal knowledge, but can't understand external background knowledge.
Approach:	They propose to use Chinese knowledge-enhanced pre-trained language models to improve context-aware representations via learning from structured relations in knowledge bases.
Outcome:	Experiments show that Chinese knowledge-enhanced pre-trained language models outperform strong baselines over various benchmark NLP tasks and in different model sizes.

Copied to clipboard

Jianing Wang, Chengyu Wang, Chuanqi Tan, Jun Huang, Ming Gao

Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources (2021.findings-acl)

Challenge:	Existing studies have explored multiple aspects that affect the performance of large language models (LLMs) such as input-output mapping, extensive data resources, and the ability to train on labeled examples.
Approach:	They propose a framework that injects knowledge into LLMs during continual self-supervised pre-training and judiciously selects examples with high knowledge relevance.
Outcome:	The proposed framework outperforms baseline models and improves by more than 13% and 7% on text classification and question-answering tasks.

Copied to clipboard

Taolin Zhang, Chengyu Wang, Minghui Qiu, Bite Yang, Zerui Cai, Xiaofeng He, Jun Huang

Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)

Challenge:	MRC is a popular task in NLP, aiming to understand a passage and answer the relevant questions.
Approach:	They propose a multi-target machine learning task for the medical domain that predicts answers to medical questions and corresponding support sentences from medical information sources simultaneously.
Outcome:	The proposed model outperforms baselines by fusing context-aware and knowledge-awful token representations.

Copied to clipboard

Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao

Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment (2025.emnlp-main)

Challenge:	Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks.
Approach:	They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks.
Outcome:	Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks.

Copied to clipboard

Wenrui Cai, Chengyu Wang, Junbing Yan, Jun Huang, Xiangzhong Fang

UnClE: Explicitly Leveraging Semantic Similarity to Reduce the Parameters of Word Embeddings (2021.findings-emnlp)

Challenge:	Existing methods to distill chain-of-thought (CoT) results from large language reasoning models (LRMs) to small models are ineffective and require substantial amount of annotated data.
Approach:	They propose a Critique-Rethink-Verify system for training small language reasoning models that can be critiquized according to the cognitive capabilities of smaller models.
Outcome:	The proposed system outperforms other methods on challenging reasoning benchmarks.

Copied to clipboard

Zhi Li, Yuchen Zhai, Chengyu Wang, Minghui Qiu, Kailiang Li, Yin Zhang

TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models (2024.lrec-main)

Challenge:	Existing methods to reduce word embedding parameters ignore semantic information . existing methods do not consider semantic information, allowing for performance degradation .
Approach:	They propose a method that leverages semantic similarity with weight sharing to reduce dimensionality of word embeddings.
Outcome:	The proposed method reduces word embedding parameters by more than 11x on a standard English-German dataset.

Copied to clipboard

Junbing Yan, Chengyu Wang, Taolin Zhang, Xiaofeng He, Jun Huang, Wei Zhang, Longtao Huang, Hui Xue

Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification (2023.emnlp-main)

Challenge:	Existing methods for incorporating external knowledge into language models do not prioritize learning embeddings for entity-related tokens.
Approach:	They propose a framework for incorporating external knowledge into pre-training models that utilize entity-related tokens.
Outcome:	The proposed framework reduces pre-training time by 50% and outperforms other KEPLMs in knowledge probing tasks and multiple knowledge-aware language understanding tasks.

Copied to clipboard

Chengyu Dong, Zihan Wang, Jingbo Shang

Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining (2020.emnlp-main)

Challenge:	Recent advances in weakly supervised text classification focus on designing sophisticated methods to turn high-level human heuristics into quality pseudo-labels.
Approach:	They propose to use a seed matching-based method to generate quality pseudo-labels by deleting the seed words present in the matched input text.
Outcome:	The proposed method can be improved significantly by deleting the seed words in the matched input text with a high deletion ratio.

Copied to clipboard

Chengyu Wang, Minghui Qiu, Jun Huang, Xiaofeng He

Text Grafting: Near-Distribution Weak Supervision for Minority Classes in Text Classification (2024.emnlp-main)

Challenge:	Pre-trained neural language models improve learning for various NLP tasks by fine-tuning them on task-specific training sets.
Approach:	They propose a meta-learning procedure to fine-tune neural language models on task-specific training sets.
Outcome:	The proposed procedure solves a group of similar NLP tasks on a text mining dataset.

Copied to clipboard

Letian Peng, Yi Gu, Chengyu Dong, Zihan Wang, Jingbo Shang

Information Extraction & Text Mining

Challenge:	Recent work generates pseudo labels by mining texts similar to the class names from the raw corpus, but there is a high risk that LLMs cannot generate in-distribution data, leading to ungeneralizable classifiers.
Approach:	They propose to use LLMs to generate pseudo labels by mining masked templates from corpus . they then use state-of-the-art LLM to synthesize near-distribution texts falling into minority classes .
Outcome:	The proposed framework improves on the previous methods for extremely weak-supervised text classification.

Training Language Models to Generate Text with Citations via Fine-grained Rewards (2024.acl-long)

Copied to clipboard

Chengyu Huang, Zeqiu Wu, Yushi Hu, Wenya Wang

QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs (2026.findings-acl)

Challenge:	Recent Large Language Models (LLMs) are prone to hallucination and their outputs often contain incorrect or unverifiable claims.
Approach:	They propose a training framework using fine-grained rewards to teach LLMs to generate highly supportive and relevant citations while ensuring the correctness of their responses.
Outcome:	The proposed training framework outperforms existing methods on QA datasets and surpasses GPT-3.5-turbo on LLaMA-2-7B.

Copied to clipboard

Taolin Zhang, Haidong Kang, Dongyang Li, Qizhou Chen, Xiaofeng He, Chengyu Wang, Richang Hong

TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification (2021.emnlp-main)

Challenge:	Recent studies have shown that large language models (LLMs) can be effective for correcting factual inaccuracies but can still suffer from hallucinations.
Approach:	They propose a queue-based self-correction framework that addresses parameter bias during sequential model editing.
Outcome:	The proposed framework outperforms baseline models while maintaining competitive performance in single-turn editing.

Copied to clipboard

Chengyu Wang, Jianing Wang, Minghui Qiu, Jun Huang, Ming Gao

VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Models (2024.emnlp-main)

Challenge:	Recent studies show that prompts improve performance of large pre-trained language models for few-shot text classification.
Approach:	They propose a prompt-based framework for few-shot learning that captures cross-task transferable knowledge and uses two de-biasing techniques to make it more task-agnostic and unbiased .
Outcome:	The proposed framework outperforms strong baselines over multiple NLP tasks and datasets.

Copied to clipboard

Jiapeng Wang, Chengyu Wang, Kunzhe Huang, Jun Huang, Lianwen Jin

Building a Family of Data Augmentation Models for Low-cost LLM Fine-tuning on the Cloud (2025.coling-industry)

Challenge:	Existing studies have shown that CLIP models with short summary texts cannot process extensive textual descriptions due to its text encoder's reliance on positional embeddings with length 77.
Approach:	They propose a Contrastive Language-Image Pre-training (CLIP) model which aims to unleash the long-description understanding capability of video CLIP models.
Outcome:	The proposed model can learn the distribution of feature space while expanding the long description capability.

Copied to clipboard

Chengyu Wang, Yuanhao Yue, Jun Huang, Peng Wang

Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning (2024.emnlp-main)

Challenge:	Existing models for learning large language models are expensive and difficult to build and fine-tune.
Approach:	They propose a family of data augmentation models to improve model fine-tuning efficiency . they leverage powerful LLMs to expand, refine and re-write instructions and responses .
Outcome:	The proposed models improve the efficiency of model fine-tuning by leveraging small datasets and quality assessment techniques.

Copied to clipboard

Qizhou Chen, Taolin Zhang, Xiaofeng He, Dongyang Li, Chengyu Wang, Longtao Huang, Hui Xue’

ARTIST: A Transformer-based Chinese Text-to-Image Synthesizer Digesting Linguistic and World Knowledge (2022.findings-emnlp)

Challenge:	Existing methods to correct outdated or erroneous knowledge in large language models (LLMs) are slow and cumbersome, resulting in catastrophic knowledge forgetting and degradation of model performance.
Approach:	They propose a RetriEval-augmented ContInuous Prompt lEarning method that converts knowledge statements into short and informative continuous prompts, prefixed to the LLM’s input query embedding.
Outcome:	The proposed method improves the performance of large language models (LLMs) while maintaining the overall performance of the model.

Copied to clipboard

Tingting Liu, Chengyu Wang, Xiangru Zhu, Lei Li, Minghui Qiu, Jun Huang, Ming Gao, Yanghua Xiao

A Customized Text Sanitization Mechanism with Differential Privacy (2023.findings-acl)

Challenge:	Text-to-Image Synthesis (TIS) is a popular task to convert natural language texts into realistic images.
Approach:	They propose a transformer-based Chinese text-to-image synthesizer for high-resolution image generation that incorporates linguistic and relational knowledge facts into the model to ensure better performance without the usage of ultra-large models.
Outcome:	The proposed model outperforms existing models in Chinese with linguistic and relational knowledge facts.

Copied to clipboard

Sai Chen, Fengran Mo, Yanhao Wang, Cen Chen, Jian-Yun Nie, Chengyu Wang, Jamie Cui

From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models (2023.findings-emnlp)

Challenge:	Existing methods to sanitize texts subject to differential privacy do not work for non-metric semantic similarity measures.
Approach:	They propose a customized text sanitization mechanism based on a metric local differential privacy definition.
Outcome:	The proposed mechanism achieves better privacy-utility trade-offs than existing mechanisms on benchmark datasets.

Copied to clipboard

Yan Junbing, Chengyu Wang, Taolin Zhang, Xiaofeng He, Jun Huang, Wei Zhang

Evaluating the Smooth Control of Attribute Intensity in Text Generation with LLMs (2024.findings-acl)

Challenge:	Existing methods to solve complex logical reasoning problems are cumbersome for language models.
Approach:	They propose to use iterative methodology to construct a cognitive tree using language models . they propose to generate multiple responses by utilizing in-context examples .
Outcome:	The proposed model achieves a performance level comparable to that of GPT-3.5 . the proposed model contains fewer parameters than 5% of the model with 175B parameters .

Copied to clipboard

Shang Zhou, Feng Yao, Chengyu Dong, Zihan Wang, Jingbo Shang

On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Models (2024.acl-short)

Challenge:	Controllable text generation is increasingly tailored to individual preferences.
Approach:	They propose to evaluate the attribute intensity of text generated by large language models on five different attributes for error, variation of the generated sentence's intensities and relevance to the generation questions.
Outcome:	The proposed methods are based on Elo rating system and GPT4 and are able to be trained without training.

Copied to clipboard

Dongyang Li, Junbing Yan, Taolin Zhang, Chengyu Wang, Xiaofeng He, Longtao Huang, Hui Xue’, Jun Huang

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models (2025.emnlp-demos)

Challenge:	Existing RAG methods focus on improving the task performance, without fine-grained process of knowledge.
Approach:	They propose a method that detects long-tail knowledge in large language models by analyzing retrieved documents and enhancing queries indiscriminately with retrieved information.
Outcome:	The proposed method achieves over 4x speedup in average inference time and consistent performance improvement in downstream tasks compared to existing pipelines.

Copied to clipboard

Chengyu Wang, Junbing Yan, Wenrui Cai, Yuanhao Yue, Jun Huang

BiRRE: Learning Bidirectional Residual Relation Embeddings for Supervised Hypernymy Detection (2020.acl-main)

Challenge:	In this paper, we present EasyDistill, a comprehensive toolkit designed for effective black-box and white-box knowledge distillation (KD) of large language models.
Approach:	They propose a toolkit for effective black-box and white-box knowledge distillation (KD) of large language models (LLMs).
Outcome:	The framework offers data synthesis, supervised fine-tuning, ranking optimization, and reinforcement learning techniques specifically tailored for KD scenarios.

Copied to clipboard

Chengyu Wang, Xiaofeng He

Information Retrieval

Challenge:	supervised hypernymy detection has been studied under various frameworks . supervised classifiers are more likely to suffer from "lexical memorization"
Approach:	They propose a representation learning framework called Bidirectional Residual Relation Embeddings to model the possibility of a term being mapped to another in the embedding space by hypernymy relations.
Outcome:	The proposed model outperforms baselines over evaluation frameworks.

Learning Knowledge-Enhanced Contextual Language Representations for Domain Natural Language Understanding (2023.emnlp-main)

Copied to clipboard

Taolin Zhang, Ruyao Xu, Chengyu Wang, Zhongjie Duan, Cen Chen, Minghui Qiu, Dawei Cheng, Xiaofeng He, Weining Qian

TaiChi: Improving the Robustness of NLP Models by Seeking Common Ground While Reserving Differences (2024.lrec-main)

Challenge:	Existing methods for pre-training KEPLMs with relational triples are difficult to adapt to close domains due to the lack of sufficient domain graph semantics.
Approach:	They propose a Knowledge-enhanced language representation learning framework for various closed domains that captures the implicit graph structure among the entities.
Outcome:	The proposed framework outperforms existing methods for pre-training KEPLMs in closed domains significantly.

Copied to clipboard

Huimin Chen, Chengyu Wang, Yanhao Wang, Cen Chen, Yinggui Wang

Robustness in NLP

Challenge:	Pre-trained Language Models are vulnerable to adversarial examples that introduce human-imperceptible perturbations to clean examples to deceive the models.
Approach:	They propose a Siamese network-based approach to teach adversarial models to focus on similarities . they propose combining two sub-networks sharing the same structure but trained on clean and adversarials .
Outcome:	The proposed approach reduces the differences between clean and adversarial samples and focuses more on similarities.

EasyNLP: A Comprehensive and Easy-to-use Toolkit for Natural Language Processing (2022.emnlp-demos)

Copied to clipboard

Chengyu Wang, Minghui Qiu, Taolin Zhang, Tingting Liu, Lei Li, Jianing Wang, Ming Wang, Jun Huang, Wei Lin

Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding (2023.findings-emnlp)

Challenge:	Pre-Trained Models (PTMs) have reshaped the development of natural language processing (NLP) but it is not easy to obtain high-performing PTMs without a large amount of labeled training data and deploy them online with fast inference speed.
Approach:	They propose to make it easy to build NLP applications with knowledge-enhanced pre-training and knowledge distillation.
Outcome:	EasyNLP supports a comprehensive suite of NLP algorithms and features knowledge-enhanced pre-training, knowledge distillation and few-shot learning functionalities.

Copied to clipboard

Jianing Wang, Qiushi Sun, Nuo Chen, Chengyu Wang, Jun Huang, Ming Gao, Xiang Li

Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning (2023.acl-short)

Challenge:	Existing methods for pre-trained language models rely on noisy data, which can be expensive if all parameters are updated.
Approach:	They propose a self-training framework that incorporates Monte Carlo dropouts into the model and judiciously selects reliable pseudo-labeled examples based on confidence and certainty.
Outcome:	The proposed framework improves performance and efficiency over multiple tasks over multiple datasets.

Copied to clipboard

Zhen-Ru Zhang, Chuanqi Tan, Haiyang Xu, Chengyu Wang, Jun Huang, Songfang Huang

Rapid Diffusion: Building Domain-Specific Text-to-Image Synthesizers with Fast Inference Speed (2023.acl-industry)

Challenge:	Parameter-efficient fine-tuning only optimizes a few task-specific parameters with frozen pre-trained model.
Approach:	They propose to optimize a prefix vector inserted into Transformer layers to optimize the prefix . they propose to use a gate mechanism to adjust the prefixed to each layer .
Outcome:	The proposed approach improves on the SuperGLUE and NER datasets.

Copied to clipboard

Bingyan Liu, Weifeng Lin, Zhongjie Duan, Chengyu Wang, Wu Ziheng, Zhang Zipeng, Kui Jia, Lianwen Jin, Cen Chen, Jun Huang

DiffChat: Learning to Chat with Text-to-Image Synthesis Models for Interactive Image Creation (2024.findings-acl)

Challenge:	Text-to-Image Synthesis (TIS) aims to generate images based on textual inputs . but, current diffusion-based models lack entity knowledge and low inference speed .
Approach:	They propose a framework for training and deploying latent diffusion models with rich entity knowledge injected and optimized networks.
Outcome:	The proposed framework improves image quality and inference speed and can be used in industrial applications.

Copied to clipboard

Jiapeng Wang, Chengyu Wang, Tingfeng Cao, Jun Huang, Lianwen Jin

BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering (2025.acl-long)

Challenge:	a novel method to align Large Language Models to "chat" with prompt-as-input Text-to-Image Synthesis models is proposed . a user-specified instruction can be used to create a high quality image .
Approach:	They propose a method to align Large Language Models to "chat" with prompt-as-input Text-to-Image Synthesis models for interactive image creation.
Outcome:	The proposed method can exhibit superior performance than baseline models and strong competitors based on automatic and human evaluations.

Copied to clipboard

Taolin Zhang, Dongyang Li, Qizhou Chen, Chengyu Wang, Xiaofeng He

Question Answering

Challenge:	Existing studies on multi-hop question answering employ specific methods regardless of question types . complexity of multihop question answerrs often exceeds knowledge boundaries of LLMs .
Approach:	They propose a framework that uses chain-of-thought prompting to prompt LLMs to answer multi-hop questions.
Outcome:	The proposed framework outperforms baseline models in multi-hop QA scenarios.

HER: Human-like Reasoning and Reinforcement Learning for LLM Role-playing (2026.findings-acl)

Copied to clipboard

Chengyu Du, Xintao Wang, Aili Chen, Weiyuan Li, Rui Xu, Junteng Liu, Zishan Huang, Rong Tian, Zijun Sun, Yuhao Li, Liheng Feng, Deming Ding, Pengyu Zhao, Yanghua Xiao

Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains (2021.acl-long)

Challenge:	Existing models for LLM role-playing lack high-quality datasets with explicit reasoning traces and reliable reward signals aligned with human preferences.
Approach:	They propose a unified framework for cognitive-level persona simulation that strictly distinguishes characters’ first-person thinking processes from LLMs’ third-person reasoning.
Outcome:	The proposed framework outperforms the Qwen3-32B baseline model and achieves a 30.26% and 14.97% performance on the minimax benchmarks.

Copied to clipboard

Haojie Pan, Chengyu Wang, Minghui Qiu, Yichang Zhang, Yaliang Li, Jun Huang

SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining (2021.acl-long)

Challenge:	Pre-trained language models have been successful in NLP tasks, but their large size and long inference time limit their deployment in real-time applications.
Approach:	They propose a meta-teacher model that captures transferable knowledge across domains and passes it to students.
Outcome:	The proposed model can distill large teacher models into small student models with guidance from the meta-teacher.

Copied to clipboard

Taolin Zhang, Zerui Cai, Chengyu Wang, Minghui Qiu, Bite Yang, Xiaofeng He

Exploratory Neural Relation Classification for Domain Knowledge Acquisition (C18-1)

Challenge:	Existing knowledge-based PLMs are based on linked-entity information, but they only use linked-enemy information as auxiliary information.
Approach:	They propose to integrate semantic knowledge from neighbours of linked-entity into a medical PLM that integrates heterogeneous-entities into the homogeneously neighbouring entity structure.
Outcome:	Experiments show that SMedBERT outperforms baselines in knowledge-intensive Chinese medical tasks.

Copied to clipboard

Yan Fan, Chengyu Wang, Xiaofeng He

Information Extraction & Text Mining

Challenge:	Existing methods for relation classification are limited and lack of low-frequency relations in specific domains.
Approach:	They propose a method to learn a classifier on pre-defined relations and discover new relations expressed in texts.
Outcome:	The proposed method can classify entities into a finite set of relations and discover relations with high precision and recall.

Mock Worlds, Real Skills: Building Small Agentic Language Models with Synthetic Tasks, Simulated Environments, and Rubric-Based Rewards (2026.acl-long)

Copied to clipboard

Yuanjie Lyu, Chengyu Wang, Lei Shen, Jun Huang, Tong Xu

KECP: Knowledge Enhanced Contrastive Prompting for Few-shot Extractive Question Answering (2022.emnlp-main)

Challenge:	Existing agentic training data are narrow in task variety and easily solved . real-world APIs lack diversity and are unstable for large-scale reinforcement learning rollout processes.
Approach:	They propose a framework that synthesizes diverse tool-use training data and simulates complete environments.
Outcome:	The proposed framework synthesizes diverse tool-use training data and simulates complete environments.

Copied to clipboard

Jianing Wang, Chengyu Wang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Jun Huang, Ming Gao

Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations (2026.acl-long)

Challenge:	Extractive Question Answering (EQA) is one of the most essential tasks in Machine Reading Comprehension (MRC).
Approach:	They propose a framework that transforms extractive question answering into a non-autoregressive Masked Language Modeling (MLM) generation problem.
Outcome:	The proposed framework outperforms state-of-the-art approaches in few-shot learning scenarios by a large margin.

Copied to clipboard

Wenrui Cai, Chengyu Wang, Junbing Yan, Jun Huang, Xiangzhong Fang

FashionKLIP: Enhancing E-Commerce Image-Text Retrieval with Fashion Multi-Modal Conceptual Knowledge Graph (2023.acl-industry)

Challenge:	Existing resources often fail to provide extensive reasoning problems with coherent CoT processes distilled from multiple teacher models.
Approach:	They propose a large-scale dataset featuring 2 million CoT processes generated by multiple powerful LRMs.
Outcome:	The proposed dataset features 2 million CoT processes and is validated by multiple powerful LRMs.

Copied to clipboard

Xiaodan Wang, Chengyu Wang, Lei Li, Zhixu Li, Ben Chen, Linbo Jin, Jun Huang, Yanghua Xiao, Ming Gao

CocaCLIP: Exploring Distillation of Fully-Connected Knowledge Interaction Graph for Lightweight Text-Image Retrieval (2023.acl-industry)

Challenge:	Recent advances in visual-language pre-trained (VLP) models have greatly improved cross-modal retrieval performance . however, the fine-grained interactions between objects from different modalities are far from well-established . e-commerce domain lacks sufficient training data and fine-granular cross-modulal knowledge .
Approach:	They propose a visual-language pre-trained (VLP) image-text retrieval model that integrates cross-modal knowledge into the model to improve performance.
Outcome:	The proposed model improves performance on e-commerce image-text retrieval task by a large margin.

Copied to clipboard

Jiapeng Wang, Chengyu Wang, Xiaodan Wang, Jun Huang, Lianwen Jin

XtremeCLIP: Extremely Parameter-efficient Tuning for Low-resource Vision Language Understanding (2023.findings-acl)

Challenge:	Existing methods for text-image retrieval are limited to edge devices and real-time situations due to the substantial indexing and inference time.
Approach:	They propose a fully-Connected knowledge interaction graph technique for cross-modal pre-training distillation.
Outcome:	The proposed method achieves SOTA performances on the widely-used Flickr30K and MSCOCO benchmarks under the lightweight setting.

Copied to clipboard

Moming Tang, Chengyu Wang, Jianing Wang, Chuanqi Tan, Songfang Huang, Cen Chen, Weining Qian

DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models (2025.acl-industry)

Challenge:	Existing approaches to fine-tune visual-language understanding (VLU) require tasks-specific designs and sufficient training data.
Approach:	They propose a simple yet efficient paradigm for low-resource Visual Language Understanding (VLU) they reformulate a series of VLU tasks as an open-book affinity-matching problem.
Outcome:	The proposed framework outperforms baselines in low-resource settings.

Copied to clipboard

Chengyu Wang, Junbing Yan, Yuanhao Yue, Jun Huang

“Average” Approximates “First Principal Component”? An Empirical Analysis on Representations from Neural Language Models (2021.emnlp-main)

Challenge:	Existing studies on distilled lightweight LLMs have focused on transferring knowledge from a larger model (the teacher) to a smaller model (sector).
Approach:	They propose a family of distilled, lightweight LLMs derived from Qwen2.5 models.
Outcome:	Experimental results show that the distilled models have significantly stronger instruction-following capabilities than the original models.

Copied to clipboard

Zihan Wang, Chengyu Dong, Jingbo Shang

Meta Distant Transfer Learning for Pre-trained Language Models (2021.emnlp-main)

Challenge:	Contextualized representations have been used in various NLP tasks, but their nature remains a mystery.
Approach:	They propose to use a property to estimate the power of contextualized representations . they show that the average representation shares almost the same direction as the first principal component .
Outcome:	The proposed representations share the same direction as the first principal component . the results suggest that the property is intrinsic to the distribution of representations .

Copied to clipboard

Chengyu Wang, Haojie Pan, Minghui Qiu, Jun Huang, Fei Yang, Yin Zhang

MedCoach: Enhancing Medical Reasoning in LLMs via Knowledge Graph-Augmented Chain-of-Thought Distillation (2026.findings-acl)

Challenge:	Notable PLMs are available for text classification tasks, but performance of PLM on downstream tasks may be limited by the availability of training set.
Approach:	They propose a meta-learning framework to learn the transferable knowledge across tasks using PLMs.
Outcome:	The proposed framework outperforms baselines on seven datasets and is task-agnostic and unbiased.

Copied to clipboard

Chuan Li, Ye Lyu, Chengyu Wang, Mingyuan Fan, Cen Chen

AMATA: Adaptive Multi-Agent Trajectory Alignment for Knowledge-Intensive Question Answering (2026.acl-long)

Challenge:	Existing methods for training specialized reasoning models for the medical domain are limited due to the scarcity of high-quality, large-scale Chain-of-Thought (CoT) data.
Approach:	They propose a framework that introduces a dedicated coach role to guide the student model through question decomposition.
Outcome:	The proposed framework smooths the learning curve in medical reasoning by facilitating domain adaptation before advancing to complex long-chain reasoning.

Copied to clipboard

Taolin Zhang, Dongyang Li, Chen Chen, Qizhou Chen, Jiuheng Wan, Xiaofeng He, Chengyu Wang, Richang Hong

Question Answering

Challenge:	Empirical results show that AMATA outperforms baseline approaches, knowledge-augmented frameworks, and LLMs on knowledge-intensive QA benchmarks.
Approach:	They propose an Adaptive Multi-Agent Trajectory Alignment framework that integrates external knowledge to improve response interpretability and factual grounding.
Outcome:	The proposed framework outperforms baseline approaches, knowledge-augmented frameworks, and LLM-based trajectory systems on five established knowledge-intensive QA benchmarks.

AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use (2026.acl-industry)

Copied to clipboard

Yuanjie Lyu, Chengyu Wang, Haonan Zheng, Yuanhao Yue, Junbing Yan, Ming Wang, Jun Huang

UniPSDA: Unsupervised Pseudo Semantic Data Augmentation for Zero-Shot Cross-Lingual Natural Language Understanding (2024.lrec-main)

Challenge:	Modern industrial applications increasingly demand language models capable of multi-step reasoning and tool use in real-world settings.
Approach:	They propose a model family that trains via multi-round reinforcement learning on synthetic data and open-source data.
Outcome:	The proposed model train on synthetic and open-source data achieves strong performance on multiple agentic benchmarks and in an industrial agent system.

Copied to clipboard

Dongyang Li, Taolin Zhang, Jiali Deng, Longtao Huang, Chengyu Wang, Xiaofeng He, Hui Xue

Low-Resource NLP

Challenge:	Existing studies rely on shallow unsupervised data generated by token surface matching regardless of global context-aware semantics of the surrounding text tokens.
Approach:	They propose an Unsupervised Pseudo Semantic Data Augmentation mechanism to enrich training data without human intervention.
Outcome:	The proposed model improves on general zero-shot cross-lingual understanding tasks on different languages without human intervention.

SphereRE: Distinguishing Lexical Relations with Hyperspherical Relation Embeddings (P19-1)

Copied to clipboard

Chengyu Wang, Xiaofeng He, Aoying Zhou

Representation Learning

Challenge:	Lexical relations are relations between terms in lexicons.
Approach:	They propose a neural representation learning model to distinguish lexical relations among term pairs based on hyperspherical relation embeddings.
Outcome:	The proposed model outperforms state-of-the-art models on several benchmarks.

BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis (2023.emnlp-industry)

Copied to clipboard

Tingfeng Cao, Chengyu Wang, Bingyan Liu, Ziheng Wu, Jinhui Zhu, Jun Huang