Papers by Zhefeng Wang
FISTAPruner: Layer-wise Post-training Pruning for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing pruning methods require inefficient retraining for billion-scale LLMs or rely on heuristicically designed metrics to determine pruning masks, leading to performance degradation. |
| Approach: | They propose a convex optimization model that induces sparsity in large language models by leveraging FISTA. |
| Outcome: | The proposed method can remove 50% of model parameters while retaining 98.6% and 95.6% of the zero-shot performance. |
An In-depth Study on Internal Structure of Chinese Words (2021.acl-long)
Copied to clipboard
Chen Gong, Saihao Huang, Houquan Zhou, Zhenghua Li, Min Zhang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan
| Challenge: | Unlike English letters, Chinese characters have rich and specific meanings. |
| Approach: | They propose to model Chinese words' internal structures as dependency trees with 11 labels for distinguishing syntactic relationships. |
| Outcome: | The proposed model of Chinese word-internal structures shows it can be used to parse sentences . it shows that the model can be applied to a sentence-level task with a competitive dependency parser. |
Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework (2023.acl-long)
Copied to clipboard
| Challenge: | Current evaluations of FEC models that depend on factuality metrics are not reliable and detailed enough. |
| Approach: | They propose a fine-grained evaluation framework that automatically evaluates FEC models on different error categories. |
| Outcome: | The proposed evaluation framework compares models on different error categories and finds the best training modes and significant differences in the performance of existing models. |
HacRED: A Large-Scale Relation Extraction Dataset Toward Hard Cases in Practical Applications (2021.findings-acl)
Copied to clipboard
Qiao Cheng, Juntao Liu, Xiaoye Qu, Jin Zhao, Jiaqing Liang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan, Yanghua Xiao
| Challenge: | Relation extraction (RE) is an essential topic in natural language processing and has attracted extensive attention. |
| Approach: | They propose a case-oriented construction framework to build a hard case relation extraction dataset with 65,225 relational facts annotated from 9,231 documents. |
| Outcome: | The proposed model achieves a high 96% F1 score on data quality and is far lower than humans. |
Delving Deep into Regularity: A Simple but Effective Method for Chinese Named Entity Recognition (2022.findings-naacl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a system for identifying text spans pertaining to specific entity types. |
| Approach: | They propose a method to investigate the regularity of Chinese NER's entity mentions by a regularity-aware module and a periodicity-gnostic module. |
| Outcome: | The proposed model significantly outperforms previous state-of-the-art methods on three benchmark datasets and a practical medical dataset. |
Accurate KV Cache Quantization with Outlier Tokens Tracing (2025.acl-long)
Copied to clipboard
Yi Su, Yuechi Zhou, Quantong Qiu, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, Min Zhang
| Challenge: | Large Language Models (LLMs) require substantial computational resources during deployment. |
| Approach: | They propose a method to identify outlier tokens and exclude them from quantization . they find that the method can deliver a 6.4 times reduction in memory usage and a 2.5 times increase in throughput . |
| Outcome: | The proposed method delivers a 6.4 times reduction in memory usage and a 2.5 times increase in throughput under 2-bit quantization. |
Adaptive Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization (2024.findings-emnlp)
Copied to clipboard
Yixin Ji, Yang Xiang, Juntao Li, Qingrong Xia, Zi Ye, Xinyu Duan, Zhefeng Wang, Kehai Chen, Min Zhang
| Challenge: | Large language models require a balance between efficiency and performance. |
| Approach: | They propose a low-rank compression technique that reduces non-essential parameters by decomposing weight matrices into products of two low-ranked matrici. |
| Outcome: | The proposed method outperforms existing pruning and low-rank compression techniques in maintaining model performance at the same compression ratio. |
Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to accelerate autoregressive generation of large language models require training costs. |
| Approach: | They propose a training-free alignment-augmented speculative decoding algorithm . it leverages the output distribution obtained in the prefilling phase to provide more aligned draft candidates . |
| Outcome: | The proposed method increases the average generation score by 3.3 points for the LLaMA3 model. |
Punctuation-Steered Representation Fine-Tuning (2026.acl-short)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning (PeFT) are limited due to their prohibitive size and computational demands. |
| Approach: | They propose a method that fine-tunes punctuation representations to achieve performance improvements. |
| Outcome: | The proposed method improves performance by altering the representation space alone . but it results in suboptimal performance due to the effects of the method on the output . |
OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure (2025.tacl-1)
Copied to clipboard
| Challenge: | Autoregressive language models generate one token in one step, limiting inference efficiency . Existing methods do not adapt to different situations to maximize acceptance length . speculative decoding has shown great potential for lossless acceleration . |
| Approach: | They propose an algorithm to construct adaptive and scalable draft trees for autoregressive language models. |
| Outcome: | Experimental results show that OPT-Tree outperforms existing draft trees and achieves speed-up ratio of up to 3.2 compared with autoregressive decoding. |
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Processing (2022.emnlp-main)
Copied to clipboard
Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, Ahmad Rashid, Khalil Bibi, Mehdi Rezagholizadeh, Chao Xing, Yasheng Wang, Xinyu Duan, Zhefeng Wang, Baoxing Huai, Xin Jiang, Qun Liu, Phillippe Langlais
| Challenge: | Existing pre-trained language models are not well-explored and are not reproducible in the literature. |
| Approach: | They propose to improve existing Arabic language pre-trained language models using a more methodical approach. |
| Outcome: | The proposed models outperform existing models on ALUE, a leaderboard-powered benchmark for Arabic NLU and NLG tasks. |
Mirror: A Universal Framework for Various Information Extraction Tasks (2023.emnlp-main)
Copied to clipboard
Tong Zhu, Junfei Ren, Zijian Yu, Mengsong Wu, Guoliang Zhang, Xiaoye Qu, Wenliang Chen, Zhefeng Wang, Baoxing Huai, Min Zhang
| Challenge: | Recent studies often formulate IE tasks as a triplet extraction problem, but this paradigm does not support multi-span and n-ary extraction, leading to weak versatility. |
| Approach: | They propose a multi-span cyclic graph extraction problem and a non-autoregressive graph decoding algorithm to extract all spans in a single step. |
| Outcome: | The proposed model outperforms or reaches competitive performance with SOTA systems under few-shot and zero-shot settings and it is compatible with 57 datasets. |
High-order Joint Constituency and Dependency Parsing (2024.lrec-main)
Copied to clipboard
| Challenge: | Syntactic parsing aims to reveal how sentences are syntactically structured. |
| Approach: | They propose to produce compatible constituency and dependency trees simultaneously for input sentences . they adopt a much more efficient decoding algorithm and explore joint modeling at training phase . |
| Outcome: | The proposed model significantly improves matching ratio of whole trees compared to separate models . the proposed model adopts a much more efficient decoding algorithm . |
Improving Chinese Named Entity Recognition with Multi-grained Words and Part-of-Speech Tags via Joint Modeling (2024.lrec-main)
Copied to clipboard
| Challenge: | Named entity recognition (CNER) is a fundamental task in natural language processing (NLP). |
| Approach: | They propose a tree parsing approach for jointly modeling Chinese named entity recognition (CNER) with multi-grained word segmentation (MWS) and POS tagging tasks. |
| Outcome: | The proposed approach achieves better or comparable performance with current methods. |
APGN: Adversarial and Parameter Generation Networks for Multi-Source Cross-Domain Dependency Parsing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for dependency parsing use labeled training data for several fixed domains, but performance drops when labeles only exist for several out-domains. |
| Approach: | They propose a model for multi-source cross-domain dependency parsing that uses a parameter generation network and adversarial network for learning domain-invariant representations. |
| Outcome: | The proposed model improves cross-domain parsing performance by about 2 points over strong BERT-enhanced baselines over a recently released dataset for multi-domain dependency parse. |
CopyNE: Better Contextual ASR by Copying Named Entities (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to transcribe contextual named entities (NEs) treat entities as tokens and generate them token-by-token, which may result in incomplete transcriptions of entities. |
| Approach: | They propose a mechanism that can copy entities from the NE dictionary and reduce errors during entity transcription. |
| Outcome: | The proposed mechanism can copy entities from the NE dictionary, reducing errors during entity transcription, ensuring the completeness of the entity. |
AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing (2023.findings-acl)
Copied to clipboard
Asaad Alghamdi, Xinyu Duan, Wei Jiang, Zhenhai Wang, Yimeng Wu, Qingrong Xia, Zhefeng Wang, Yi Zheng, Mehdi Rezagholizadeh, Baoxing Huai, Peilun Cheng, Abbas Ghaddar
| Challenge: | Developing monolingual large Pre-trained Language Models (PLMs) is shown to be very successful in handling different tasks in Natural Language Processing (NLP). |
| Approach: | They present AraMUS, the largest Arabic PLM with 11B parameters trained on 529GB of high-quality Arabic textual data. |
| Outcome: | The proposed model achieves state-of-the-art performance on a diverse set of Arabic classification and generative tasks. |
A High Precision Pipeline for Financial Knowledge Graph Construction (2020.coling-main)
Copied to clipboard
Sarah Elhammadi, Laks V.S. Lakshmanan, Raymond Ng, Michael Simpson, Baoxing Huai, Zhefeng Wang, Lanjun Wang
| Challenge: | Knowledge graphs are a standard for structured knowledge representation in the Semantic Web. |
| Approach: | They propose to extract financial news articles into a knowledge graph by using a financial dictionary. |
| Outcome: | The proposed pipeline extracts 342,000 financial news articles with a precision of 78% at the top-100 extractions. |