Papers by Du Liang

Enhancing Medical Dialogue Generation through Knowledge Refinement and Dynamic Prompt Adjustment (2025.findings-acl)

Challenge:	Existing zero-shot quantization methods are based on overfitting problem in adversarial learning process, leading to sub-optimal performance.
Approach:	They propose a zero-shot sharpness-aware quantization framework for the quantization of various PLMs by optimizing a minimax problem.
Outcome:	The proposed framework can achieve significant performance gains on discriminative and generative PLMs.

Copied to clipboard

Hongda Sun, Jiaren Peng, Wenzhong Yang, Liang He, Bo Du, Rui Yan

Dialogue Systems & Conversational Agents

Challenge:	Medical dialogue systems (MDS) struggle to identify relevant medical knowledge and generate accurate responses.
Approach:	They propose a medical dialogue system that integrates knowledge refining and dynamic prompt adjustment to improve medical knowledge and accuracy.
Outcome:	The proposed system outperforms state-of-the-art systems in both generation quality and medical entity accuracy.

Revisiting Knowledge Distillation for Autoregressive Language Models (2024.acl-long)

Copied to clipboard

Qihuang Zhong, Liang Ding, Li Shen, Juhua Liu, Bo Du, Dacheng Tao

MathPrompter: Mathematical Reasoning using Large Language Models (2023.acl-industry)

Challenge:	Autoregressive language models (LMs) are expensive and memory intensive, preventing the development of industrial applications.
Approach:	They propose an adaptive teaching approach to improve the KD of autoregressive language models by distilling knowledge into a small student model.
Outcome:	The proposed method can achieve consistent and significant performance gains across all model types and sizes.

Copied to clipboard

Shima Imani, Liang Du, Harsh Shrivastava

AdaV: Adaptive Text-visual Redirection for Vision-Language Models (2025.findings-acl)

Challenge:	Recent advances in natural language processing (NLP) can be attributed to massive scaling of Large Language Models (LLMs).
Approach:	They propose a technique that improves performance of Large Language Models (LLMs) on arithmetic problems along with increased reliance in the predictions.
Outcome:	The proposed technique improves performance on arithmetic problems and increases confidence in the output results.

Copied to clipboard

Jiayi Han, Liang Du, Yiwen Wu, Guanming Liang, Xiangguo Zhou, Weibo Zheng, Donghong Han, Zixun Sun

Visual Data in NLP

Challenge:	Vision-language models often generate excessive visual tokens, leading to poor performance . a novel training-free visual token pruning method is proposed to improve performance despite the computational cost associated with VLMs.
Approach:	They propose a training-free visual token pruning method that reduces biased token pruning . they plan to open-source the code upon publication .
Outcome:	The proposed method reduces biased token pruning and enhances model robustness with limited visual token budget.

MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models (2025.findings-acl)

Copied to clipboard

Boyang Xue, Hongru Wang, Rui Wang, Sheng Wang, Zezhong Wang, Yiming Du, Bin Liang, Wenxuan Zhang, Kam-Fai Wong

Revisiting Token Dropping Strategy in Efficient BERT Pretraining (2023.acl-long)

Challenge:	Existing studies on LLM confidence estimations in languages other than English have been limited to English.
Approach:	They propose to use question-related language to prompt LLMs to assess their confidence in large language models.
Outcome:	The proposed model improves on question-related language prompts for LS tasks, while English exhibits notable linguistic dominance in confidence estimations.

Copied to clipboard

Qihuang Zhong, Liang Ding, Juhua Liu, Xuebo Liu, Min Zhang, Bo Du, Dacheng Tao

From Domains to Instances: Dual-Granularity Data Synthesis for LLM Unlearning (2026.findings-acl)

Challenge:	Token dropping is a recently-proposed strategy to speed up the pretraining of masked language models, such as BERT.
Approach:	They propose a semantic-consistent learning method to improve token dropping by skipping the computation of a subset of input tokens at several middle layers.
Outcome:	The proposed method achieves consistent and significant performance gains across all tasks and model sizes.

Copied to clipboard

Xiaoyu Xu, Minxin Du, Zitong LI, Zi Liang, Zhibiao Guo, Zhang Shiyu, Peizhao Hu, Qingqing Ye, Haibo Hu

Challenge:	Currently, the evaluation of unlearning is limited due to the lack of granularity in the model.
Approach:	They propose a framework for synthesizing high-quality forget sets that exploits the target model per se to elicit data that matches its internal knowledge distribution through seed-guided and adversarial prompting.
Outcome:	The proposed framework achieves a superior balance of relevance, diversity, and efficiency across benchmarks.

Jointly Learning Aspect-Focused and Inter-Aspect Relations with Graph Convolutional Networks for Aspect Sentiment Analysis (2020.coling-main)

Copied to clipboard

Bin Liang, Rongdi Yin, Lin Gui, Jiachen Du, Ruifeng Xu

Sentiment Analysis

Challenge:	Existing methods for aspect sentiment analysis do not include explicit sentiment expressions.
Approach:	They propose to construct a heterogeneous graph by leveraging aspect-focused and inter-aspect contextual dependencies for the specific aspect.
Outcome:	The proposed model outperforms state-of-the-art methods on four benchmark datasets and significantly boosts performance in comparison with BERT.

SEMGraph: Incorporating Sentiment Knowledge and Eye Movement into Graph Model for Sentiment Analysis (2022.emnlp-main)

Copied to clipboard

Bingbing Wang, Bin Liang, Jiachen Du, Min Yang, Ruifeng Xu

Context-aware Embedding for Targeted Aspect-based Sentiment Analysis (P19-1)

Challenge:	Existing research on sentiment analysis based on eye movement signals has been attributed importance.
Approach:	They propose a linguistic probing eye movement paradigm to extract eye movement features based on the relationship between linguistic features and human reading behavior.
Outcome:	The proposed graph architecture achieves state-of-the-art performance on two sentiment analysis datasets with eye movement signals and three sentiment analysis data without eye movement signal.

Copied to clipboard

Bin Liang, Jiachen Du, Ruifeng Xu, Binyang Li, Hejiao Huang

Representation Learning

Challenge:	Existing methods do not specifically pre-train reasonable embeddings for targets and aspects in TABSA.
Approach:	They propose to refine the embeddings of targets and aspects using a sparse coefficient vector . this allows the embeds to be refined from highly correlative words instead of context-independent vectors .
Outcome:	Experiments show that the proposed method improves on two benchmark datasets.

Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond (2025.acl-industry)

Copied to clipboard

Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Tanglifu Tanglifu, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, Xiangzheng Zhang

DoTAT: A Domain-oriented Text Annotation Tool (2022.acl-demo)

Challenge:	Experimental results show that opensource curriculum training is more effective when distinct datasets are available for different training stages.
Approach:	They propose an opensource suite for training long reasoning models using publicdata and models.
Outcome:	The proposed model outperforms DeepSeek-R1-DistillQwen-32B models in math reasoning.

Copied to clipboard

Yupian Lin, Tong Ruan, Ming Liang, Tingting Cai, Wen Du, Yi Wang

Challenge:	DoTAT is a domain-oriented text annotation tool that can reduce the time for event annotation by 19.7% . the tool supports multi-person collaborative process with automatically merging and review .
Approach:	They propose a domain-oriented text annotation tool called DoTAT . it provides multi-person collaborative process with automatic merging and review .
Outcome:	The proposed tool can reduce the time for event annotation by 19.7% compared with existing tools.

A Generative Model for End-to-End Argument Mining with Reconstructed Positional Encoding and Constrained Pointer Mechanism (2022.emnlp-main)

Copied to clipboard

Jianzhu Bao, Yuhang He, Yang Sun, Bin Liang, Jiachen Du, Bing Qin, Min Yang, Ruifeng Xu

Data-Efficient Selection via Grammatical Complexity in Continual Pre-training of Domain-Specific LLMs (2025.emnlp-main)

Challenge:	Argument mining (AM) is a challenging task as it requires recognizing complex argumentation structures involving multiple subtasks.
Approach:	They propose a generative framework where expected outputs of AM are framed as a simple target sequence.
Outcome:	The proposed framework achieves state-of-the-art on two AM benchmarks.

Copied to clipboard

Yizhou Ying, Geng Zhang, Cui Danxin, Chengyu Du, Guanglei Yue, Sihang Jiang, Jiaqing Liang, Yifei Fu, Hailin Hu, Yanghua Xiao

Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion (2025.acl-long)

Challenge:	Existing data selection strategies for continual pre-training of large language models often rely on scarce labeled data or computationally expensive LLMs.
Approach:	They propose an annotation-independent data selection framework for CPT that evaluates grammatical complexity using lexical diversity and syntactic complexity.
Outcome:	The proposed framework outperforms baselines on a financial dataset and surpasses full-data training by 1.7% using only 20% of the data.

Copied to clipboard

Jianqing Zhu, Huang Huang, Zhihang Lin, Juhao Liang, Zhengyang Tang, Khalid Almubarak, Mosen Alharthi, Bang An, Juncai He, Xiangbo Wu, Fei Yu, Junying Chen, Ma Zhuoheng, Yuhao Du, He Zhang, Saied Alshahrani, Emad A. Alghamdi, Lian Zhang, Ruoyu Sun, Haizhou Li, Benyou Wang, Jinchao Xu

From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalities (2025.findings-acl)

Challenge:	In the evolving landscape of large language models, the predominant focus has been on English and Chinese.
Approach:	They propose to utilize Arabic-specific vocabulary in the tokenizer to accelerate decoding.
Outcome:	The proposed model achieves decent performance comparable to the best Arabic LLMs across various Arabic benchmarks.

Copied to clipboard

Shixin Jiang, Jiafeng Liang, Jiyuan Wang, Xuan Dong, Heng Chang, Weijiang Yu, Jinhua Du, Ming Liu, Bing Qin

Hawkes based Representation Learning for Reasoning over Scale-free Community-structured Temporal Knowledge Graphs (2025.coling-main)

Challenge:	MLLMs are able to integrate multiple modalities into a single model to tackle complex tasks in real-world scenarios.
Approach:	They propose a comprehensive survey of Omni-MLLMs to address the challenges and opportunities of multimodal modeling.
Outcome:	The proposed model can integrate multiple modalities into a single model and provide novel perspectives.

Copied to clipboard

Yuwei Du, Xinyue Liu, Wenxin Liang, Linlin Zong, Xianchao Zhang

Your Reasoning Model is Secretly a Reward Model - Optimization-Free Verification from Experience (2026.acl-long)

Challenge:	Temporal knowledge graph reasoning is a useful tool for many practical tasks.
Approach:	They propose a Hawkes process-based Evolutional Representation Learning Network model which learns structural information and evolutional patterns of a TKG simultaneously.
Outcome:	The proposed model learns structural information and evolutional patterns of a TKG simultaneously, considering the characteristics of real-world networks: community structure, scale-free and temporal decaying.

Copied to clipboard

Zhenwen Liang, Ruosen Li, Yujun Zhou, Linfeng Song, Dian Yu, Xinya Du, Haitao Mi, Dong Yu

KaFT: Knowledge-aware Fine-tuning for Boosting LLMs’ Domain-specific Question-Answering Performance (2025.findings-acl)

Challenge:	Existing verifiers operate on the surface text or on confidence proxies derived from token probabilities, which can be brittle.
Approach:	They propose a training-free, non-parametric verifier that summarizes each reasoning trace by an activation delta and compares it to two class centroids computed from labeled experience.
Outcome:	The proposed model improves selection and reranking on large and less-calibrated models.

Copied to clipboard

Qihuang Zhong, Liang Ding, Xiantao Cai, Juhua Liu, Bo Du, Dacheng Tao

Question Answering

Challenge:	Recent literature reveals that supervised fine-tuning (SFT) is suboptimal for domain-specific question-answering tasks.
Approach:	They propose a query diversification strategy for robust conflict detection and a knowledge-aware fine-tuning approach to effectively boost LLMs’ performance.
Outcome:	The proposed approach improves the model generalization and alleviates the hallucination.

Agent Trading Arena: A Study on Numerical Understanding in LLM-Based Agents (2025.findings-emnlp)

Copied to clipboard

Tianmi Ma, Jiawei Du, Wenxin Huang, Wenjie Wang, Liang Xie, Xian Zhong, Joey Tianyi Zhou

ZhuJiu-Knowledge: A Fairer Platform for Evaluating Multiple Knowledge Types in Large Language Models (2024.naacl-demo)

Challenge:	Existing approaches to large language models are limited to historical backtesting and static data.
Approach:	a new large-language model is developed to simulate real-time trading in a virtual stock market . the agent trading arena simulates real-world bid-ask interactions and provides real-life trading scenarios .
Outcome:	The Agent Trading Arena simulates real-world market conditions and directly impacts price dynamics.

Copied to clipboard

Pengfan Du, Sirui Liang, Baoli Zhang, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao

NL ⇒ Schedule: Evaluate Multitask Scheduling Capability of Large Language Models (2026.acl-long)

Challenge:	evaluating the knowledge of large language models (LLMs) is crucial, and rapid advancement in large language modeling has heightened the importance of model evaluations.
Approach:	They propose a fairer benchmark for evaluating multiple knowledge types of LLMs by focusing on commonsense knowledge, world knowledge, and language knowledge.
Outcome:	The proposed framework evaluates 14 current mainstream LLMs and provides a detailed discussion and analysis of their results.

Copied to clipboard

Wenrui Liao, Weihong Du, Yi Li, Hongru Liang, Wenqiang Lei

BAR: A Backward Reasoning based Agent for Complex Minecraft Tasks (2025.findings-acl)

Challenge:	Existing methods for scheduling from natural language descriptions rely on experts with limited scheduling skills and domain knowledge.
Approach:	They propose a model to generate a feasible schedule from natural language descriptions.
Outcome:	The proposed framework achieves more robust performance than six state-of-the-art LLM+solver methods.

Copied to clipboard

Weihong Du, Wenrui Liao, Binyu Yan, Hongru Liang, Anthony G Cohn, Wenqiang Lei

EventWeave: A Dynamic Framework for Capturing Core and Supporting Events in Dialogue Systems (2026.acl-long)

Challenge:	Existing studies focus on forward reasoning based planning, but this paradigm doesn't work well for complex tasks.
Approach:	They propose to decompose a task into easily executed steps by planning and use a backward reasoning based agent to make the planning starting from the terminal state.
Outcome:	The proposed model outperforms existing methods and the proposed modules in a virtual environment that simulates complex tasks based on real-world scenarios.

Copied to clipboard

Zhengyi Zhao, Shubo Zhang, Yiming Du, Bin Liang, Baojun Wang, Zhongyang Li, Binyang Li, Kam-Fai Wong

Natural Language Interfaces

Challenge:	Existing dialogue systems process conversational turns in isolation, overlooking event structures that guide natural interactions.
Approach:	They propose a framework that explicitly models relationships between conversational events to generate more contextually appropriate dialogue responses.
Outcome:	Experiments on three dialogue datasets show that the proposed approach produces more natural responses while requiring less computational overhead.

ROSE Doesn’t Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding (2024.findings-acl)

Copied to clipboard

Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

Can MLLMs Understand the Deep Implication Behind Chinese Images? (2025.acl-long)

Challenge:	Existing methods for aligning LLMs output with expected safety require substantial training efforts and expensive computational resources.
Approach:	They propose a method to directly boost the safety of existing instruction-tuned large language models without additional training.
Outcome:	The proposed method improves safety of instruction-tuned large language models without training and requires expensive computational resources.

Copied to clipboard

Chenhao Zhang, Xi Feng, Yuelin Bai, Xeron Du, Jinchang Hou, Kaixin Deng, Guangzeng Han, Qinrui Li, Bingli Wang, Jiaheng Liu, Xingwei Qu, Yifei Zhang, Qixuan Zhao, Yiming Liang, Ziqiang Liu, Feiteng Fang, Min Yang, Wenhao Huang, Chenghua Lin, Ge Zhang, Shiwen Ni

Invocation Refiner: A Plug-and-Play Module for Rectifying LLM Tool Invocations (2026.findings-acl)

Challenge:	MLLMs perform poorly on traditional culture images, indicating limitations in understanding high-level semantics and lacking a deep knowledge base of Chinese traditional culture.
Approach:	They propose to use Chinese images to assess MLLMs' higher-order perception and understanding of Chinese visual content.
Outcome:	The proposed model incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, to ensure the authenticity of the Chinese context.

Copied to clipboard

Qirui Jiao, Dian Jiao, Nan Du, Ying Shen, Liang Lin

SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture (2025.naacl-long)

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities in Tool-Integrated Reasoning (TIR) however, the practical application is often hindered by frequent errors in tool invocations, such as incorrect tool names, invalid parameters, wrong tool-call order, or malformed invocation formats.
Approach:	They propose a specialized post-processing module that performs independent reasoning on the input of a frozen upstream LLM and an advanced RL algorithm to improve the tool-use reliability of base LLMs.
Outcome:	The proposed module improves task completion rates and invocation accuracy over the raw outputs of various upstream LLMs on a diverse set of tool-use and reasoning benchmarks.

Copied to clipboard

Jiayi Han, Liang Du, Hongwei Du, Xiangguo Zhou, Yiwen Wu, Yuanfang Zhang, Weibo Zheng, Donghong Han

Green & Sustainable NLP

Challenge:	balancing the training budget, downstream performance, and general capabilities of large language models remains a challenge in many applications.
Approach:	They propose a mixture of expert framework based on Soft LoRA and Identity Mixture . SLIM allows dynamic routing between LoRA adapters and identity layers .
Outcome:	The proposed framework reduces training cost while maintaining general capabilities . it can be open-sourced upon publication.

AdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling (2026.acl-long)

Copied to clipboard

Yongliang Miao, Yangyang Liang, Mengnan Du

Challenge:	Existing reward models rely on a static pooling strategy to condense sequences into scalar scores, which is ill-suited for fine-grained discrimination.
Approach:	They propose a framework that jointly adapts representation and aggregation to address these limitations by integrating a static inductive bias with a representational mismatch.
Outcome:	Experiments on RM-Bench and JudgeBench show that AdaJudge outperforms strong off-the-shelf reward models and traditional pooling baselines.

PAGED: A Benchmark for Procedural Graphs Extraction from Documents (2024.acl-long)

Copied to clipboard

Weihong Du, Wenrui Liao, Hongru Liang, Wenqiang Lei

COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)

Challenge:	Existing methods for extraction of procedural graphs from documents are not solving the task well.
Approach:	They propose a benchmark to test automatic extraction of procedural graphs from documents . they involve three advanced large language models and enhance them with a novel self-refine strategy .
Outcome:	The proposed benchmark systematically examines the progress of current methods and explores the potential of emerging large language models (LLMs) on this task.

Copied to clipboard

Siwei Wu, JinCheng Ren, Xeron Du, Shuyue Guo, Xingwei Qu, Yiming Liang, Jie Liu, Yunwen Li, Tyler Loakman, Tianyu Zheng, Boyu Feng, Huaqing Yuan, Zili Wang, Jiaheng Liu, Wenhao Huang, Chenglin Cai, Haoran Que, Jian Yang, Yuelin Bai, Zekun Moore Wang, Zhouliang Yu, Qunshu Lin, Ding Pan, Yuchen Eleanor Jiang, Tiannan Wang, Wangchunshu Zhou, Shenzhi Wang, Xingyuan Bu, Minghao Liu, Guoyin Wang, Ge Zhang, Chenghua Lin

Efficient Hyperparameter Optimization for LLM Reinforcement Learning (2026.acl-long)

Challenge:	Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation.
Approach:	They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets.
Outcome:	The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark.

Copied to clipboard

Minping Chen, Bowen Xiao, Du Liang, Chuxuan Zeng, Zeyi Wen

Green & Sustainable NLP

Challenge:	Existing hyperparameter optimization methods are inefficient in reinforcement learning due to model scale and resource-intensive training cycles.
Approach:	They propose a hyperparameter optimization method that adapts both model size and training budget as fidelity.
Outcome:	The proposed method significantly improves the computational efficiency of each trial (up to 14.9) over existing HPO methods.

Flexibly Utilize Memory for Long-Term Conversation via a Fragment-then-Compose Framework (2025.emnlp-main)

Copied to clipboard

Cai Ke, Yiming Du, Bin Liang, Yifan Xiang, Lin Gui, Zhongyang Li, Baojun Wang, Yue Yu, Hui Wang, Kam-Fai Wong, Ruifeng Xu

Dialogue Systems & Conversational Agents

Challenge:	Large language models extract useful information from conversation history to enhance the response in long-term conversations.
Approach:	They propose a Fragment-then-Compose framework to optimize memory utilization for long-term open-domain conversation.
Outcome:	The proposed framework can be used to extract useful information from conversation history . it can be adapted to different situations and improve response generation .

Knowing-how & Knowing-that: A New Task for Machine Comprehension of User Manuals (2023.findings-acl)

Copied to clipboard

Hongru Liang, Jia Liu, Weihong Du, Dingnan Jin, Wenqiang Lei, Zujie Wen, Jiancheng Lv

Natural Language Interfaces

Challenge:	Existing methods for machine reading comprehension of user manuals have trouble answering complex questions.
Approach:	They propose a knowing-how & knowing-that task that requires the model to answer factoid-style, procedure-style and inconsistent questions about user manuals.
Outcome:	The proposed model can answer factoid-style, procedure-style and inconsistent questions about user manuals.

Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL (2024.findings-emnlp)

Copied to clipboard

Qihuang Zhong, Kunfeng Chen, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

Programming Languages in NLP

Challenge:	Existing text-to-SQL LLMs are computationally expensive and difficult to deploy in real-world applications.
Approach:	They propose to distill a larger teacher model into a smaller student model by using imperfect data to improve the KD.
Outcome:	The proposed method achieves the best tradeoff between performance and efficiency on 5 text-to-SQL benchmarks.

CARE: A Clue-guided Assistant for CSRs to Read User Manuals (2024.acl-long)

Copied to clipboard

Weihong Du, Jia Liu, Zujie Wen, Dingnan Jin, Hongru Liang, Wenqiang Lei

Challenge:	Current solutions don't fit the online custom service scenarios well due to the lack of attention to user questions and possible responses.
Approach:	They propose to build a clue-guided assistant for customer service representations (CSRs) that can provide accurate responses and explicitly show explainable paths about how to arrive at these responses.
Outcome:	The proposed assistant can reduce CSRs' reading burden and keep high service quality, in particular with >35% decrease in time spent and keeping a >0.75 ICC score.

Quantification of Large Language Model Distillation (2025.acl-long)

Copied to clipboard

Sunbowen Lee, Junting Zhou, Chang Ao, Kaige Li, Xeron Du, Sirui He, Haihong Wu, Tianci Liu, Jiaheng Liu, Hamid Alinejad-Rokny, Min Yang, Yitao Liang, Zhoufutu Wen, Shiwen Ni

Responsible & Trustworthy NLP

Challenge:	Existing studies have revealed the robustness degra-dation caused by data distillation.
Approach:	They propose a framework to evaluate and quantify model distillation . they aim to identify identity cognition contradictions and analyse multi-granularity response similarities across models to measure the extent of homogenization.
Outcome:	The proposed framework addresses two key aspects: (1) Identifying identity cognition contradictions to assess discrepancies in how models perceive and represent identity-related information; (2) Analyzing multi-granularity response similarities across models to measure the extent of homogenization.

Multilingual Molecular Representation Learning via Contrastive Pre-training (2022.acl-long)

Copied to clipboard

Zhihui Guo, Pramod Sharma, Andy Martinez, Liang Du, Robin Abraham

Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers (2024.acl-long)

Challenge:	Molecular representation learning is an essential role in cheminformatics, but language model-based approaches focus on local features, hence they may not capture global information.
Approach:	They propose a multilingual molecular embedding generation approach that uses two different languages to train a given molecule.
Outcome:	The proposed approach is pre-trained using SMILES and IUPAC as two different languages on large-scale molecules.

Copied to clipboard

Jiawen Xie, Pengyu Cheng, Xiao Liang, Yong Dai, Nan Du

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)

Challenge:	Existing transformer-based models struggle with long-sequence processing due to computational costs . a framework to enhance long-content processing of transformers is proposed .
Approach:	They propose a framework to enhance long-sequence processing of transformers by three steps . they demonstrate that the framework significantly outperforms prior long-quence processors .
Outcome:	The proposed framework outperforms baseline models on long-sequence summarization and reading comprehension tasks.

Copied to clipboard

Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang

Self-Evolution Learning for Discriminative Language Model Pretraining (2023.findings-acl)

Challenge:	Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns.
Approach:	They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users.
Outcome:	The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks.

Copied to clipboard

Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models (2022.findings-emnlp)

Challenge:	Random masking does not consider the importance of the different words in the sentence meaning, e.g., entity-level masking requires expensive prior knowledge and generally does not use existing model weights.
Approach:	They propose a token masking and learning method that uses a random masking strategy to learn the under-explored tokens.
Outcome:	The proposed method improves linguistic knowledge learning and generalization on 10 tasks.

Copied to clipboard

Qihuang Zhong, Liang Ding, Li Shen, Peng Mi, Juhua Liu, Bo Du, Dacheng Tao

UNIVID: Unified Vision-Language Model for Video Moderation (2026.acl-industry)

Challenge:	Existing methods for fine-tuning pretrained language models suffer from poor generalization . however, they add a perturbation to each model parameter equally, which is sub-optimal .
Approach:	They propose a sharpness-aware minimization optimization procedure that introduces a Fisher mask to improve the efficiency of SAM.
Outcome:	The proposed method outperforms the vanilla sharpness-aware minimization method on GLUE and SuperGLUE benchmarks.

Copied to clipboard

Kejuan Yang, Yizhuo Zhang, Mingyuan Du, Yue Zhang, Dixin Zheng, Kaili Zhao, Yang Xiao, Hanzhong Liang, Kenan Xiao