Papers by Deli Chen

Group, Extract and Aggregate: Summarizing a Large Amount of Finance News for Forex Movement Prediction (D19-51)

Copied to clipboard

Deli Chen, Shuming Ma, Keiko Harimoto, Ruihan Bao, Qi Su, Xu Sun

Challenge:	Existing studies on forex prediction ignore related text completely and focus on forex trade data only, which loses important semantic information.
Approach:	They propose a BERT-based Hierarchical Aggregation Model to summarize forex news . they group news from different aspects and extract the most crucial news in each group .
Outcome:	The proposed model outperforms baseline methods and grouping methods and summarizes the influence patterns for forex trading.

Diffusion Theory as a Scalpel: Detecting and Purifying Poisonous Dimensions in Pre-trained Language Models Caused by Backdoor or Bias (2023.findings-acl)

Copied to clipboard

Zhiyuan Zhang, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

Language Models

Challenge:	Existing methods to fine-tune pre-trained language models (PLMs) are not safe, since the fine-uning process is invisible to the user.
Approach:	They propose a technique to study the dynamic process of fine-tuning for finding poisonous dimensions using diffusion theory.
Outcome:	The proposed approach can detect poisonous dimensions with abnormal dynamics, purify them and fine-tune them on a clean dataset.

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models (2024.acl-long)

Copied to clipboard

Damai Dai, Chengqi Deng, Chenggang Zhao, R.x. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.k. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, Wenfeng Liang

Language Models

Challenge:	Mixture-of-Experts (MoE) architectures face challenges in ensuring expert specialization . despite the promising performance, scaling language models to an extremely large scale is associated with exceedingly high computational costs.
Approach:	They propose an architecture that allows for ultimate expert specialization by segmenting experts into mN ones and activating mK from them.
Outcome:	The proposed architecture achieves comparable performance with GShard with 2B parameters and computation.

Incorporating Fine-grained Events in Stock Movement Prediction (D19-51)

Copied to clipboard

Deli Chen, Yanyan Zou, Keiko Harimoto, Ruihan Bao, Xuancheng Ren, Xu Sun

Information Extraction & Text Mining

Challenge:	Existing studies mainly adopt coarse-grained events, which loses the specific semantic information of diverse event types.
Approach:	They propose to use a finance event dictionary to extract fine-grained events from finance news to train a neural model that uses the extracted events as the distant supervised label to train stock prediction.
Outcome:	The proposed method outperforms baselines and has good generalizability.

Leveraging Word-Formation Knowledge for Chinese Word Sense Disambiguation (2021.findings-emnlp)

Copied to clipboard

Hua Zheng, Lei Li, Damai Dai, Deli Chen, Tianyu Liu, Xu Sun, Yang Liu

Semantic Text Processing

Challenge:	Word sense disambiguation (WSD) methods have not explored word-formations in parataxis languages like Chinese.
Approach:	They propose to leverage word-formation knowledge to enhance Chinese WSD by incorporating word-forms into sense disambiguation models.
Outcome:	The proposed model improves on baselines in Chinese word sense disambiguation (WSD) with word-formation knowledge, the results show.

FineReason: Evaluating and Improving LLMs’ Deliberate Reasoning through Reflective Puzzle Solving (2025.acl-long)

Copied to clipboard

Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Chaoqun Liu, Lidong Bing, Deli Zhao, Anh Tuan Luu, Yu Rong

Reasoning

Challenge:	Recent advances in large language models (LLMs) highlight an important shift from the “System 1” way of quick reactions to the “system 2” style of reflection-and-correction problem solving.
Approach:	They propose a logic-puzzle benchmark for systematic evaluation of large language models' reasoning capabilities that decomposes each puzzle into atomic steps.
Outcome:	The proposed model improves on state checking and state transition tasks and demonstrates gains in reasoning by up to 5.1%.

Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning (2023.emnlp-main)

Copied to clipboard

Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun

Semantic Text Processing

Challenge:	In-context learning (ICL) is a promising capability for large language models (LLMs) but its underlying mechanism remains unexplored.
Approach:	They propose a demonstration compression technique to expedite inference and an analysis framework for diagnosing ICL errors in GPT2-XL.
Outcome:	The proposed method improves ICL performance and expedites inference.

GeoPQA: Bridging the Visual Perception Gap in MLLMs for Geometric Reasoning (2025.findings-emnlp)

Copied to clipboard

Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Deli Zhao, Anh Tuan Luu, Yu Rong

Multimodality

Challenge:	Recent advances in reinforcement learning (RL) have enhanced the reasoning abilities of large language models, but the impact on multimodal LLMs is limited.
Approach:	They propose a two-stage RL framework that enhances visual perception and fosters reasoning capabilities.
Outcome:	The proposed framework improves geometric reasoning by 9.7% and problem-solving by 9.1% compared to direct reasoning training approach.

Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations (2024.acl-long)

Copied to clipboard

Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, Zhifang Sui

Numerical Reasoning

Challenge:	Existing methods for process-oriented math reward models rely on manual annotation.
Approach:	They propose a process-oriented math process reward model called Math-shepherd which assigns a reward score to each step of math problem solutions.
Outcome:	The proposed model breaks the bottleneck of manual supervision in two scenarios.

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models (2024.emnlp-main)

Copied to clipboard

Zihan Wang, Deli Chen, Damai Dai, Runxin Xu, Zhuoshu Li, Yu Wu

Semantic Text Processing

Challenge:	Existing studies on parameter-efficient fine-tuning (PEFT) for dense-architecture LLMs are lacking.
Approach:	They propose an expert-specialized fine-tuning method that tunes the experts most relevant to downstream tasks while freezing the other experts.
Outcome:	The proposed method matches or surpasses full-parameter fine-tuning.

CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade (2021.findings-emnlp)

Copied to clipboard

Lei Li, Yankai Lin, Deli Chen, Shuhuai Ren, Peng Li, Jie Zhou, Xu Sun

Language Models

Challenge:	Experimental results show that CascadeBERT can achieve an overall 15% improvement under 4x speed-up compared with existing dynamic early exiting methods on six classification tasks.
Approach:	They propose a framework which emits predictions in internal layers without passing through the entire model.
Outcome:	The proposed framework can achieve 15% improvement under 4x speed-up compared with existing methods on six classification tasks yielding more calibrated and accurate predictions.