Papers by Wei Ge

Automatic Data Visualization Generation from Chinese Natural Language Questions (2024.lrec-main)

Challenge:	Extensive experiments show EdgeFormer can effectively outperform previous parameter-efficient Transformer baselines and achieve competitive results under both the computation and memory constraints.
Approach:	They propose a parameter-efficient Transformer for on-device seq2seq generation that uses two novel principles for cost-effective parameterization.
Outcome:	Extensive experiments show that EdgeFormer outperforms the previous parameter-efficient Transformers and achieves competitive results under both the computation and memory constraints.

Copied to clipboard

Yan Ge, Victor Junqiu Wei, Yuanfeng Song, Jason Chen Zhang, Raymond Chi-Wing Wong

Challenge:	Existing studies on data visualization generation from natural languages have not been conducted on Chinese Text-to-Vis.
Approach:	They propose to generate a Chinese text-to-vis dataset using a multilingual encoder and a cross-lingual ability.
Outcome:	The proposed dataset is challenging and deserves further research.

Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression (2021.emnlp-main)

Copied to clipboard

Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge (2021.naacl-main)

Challenge:	Recent studies on compression of pretrained language models usually use preserved accuracy as the metric for evaluation.
Approach:	They propose two new metrics that measure how closely a compressed model mimics the original model.
Outcome:	The proposed metrics measure how closely a compressed model (i.e., student) mimics the original model (e.g., teacher).

Copied to clipboard

Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

UOR: Universal Backdoor Attacks on Pre-trained Language Models (2024.findings-acl)

Challenge:	Cant is important for understanding advertising, comedies and dogwhistle politics . currently, there are very few resources available for the research of cant .
Approach:	They propose a large and diverse dataset for creating and understanding cant from a computational linguistics perspective.
Outcome:	The proposed dataset can be used to test word embedding similarity and pretrained language models.

Copied to clipboard

Wei Du, Peixuan Li, Haodong Zhao, Tianjie Ju, Ge Ren, Gongshen Liu

Backdoor NLP Models via AI-Generated Text (2024.lrec-main)

Challenge:	Existing methods to attack pre-trained language models rely on manual selection of triggers and backdoor representations.
Approach:	They propose a backdoor attack method that turns manual selection into automatic optimization . they propose to use poisoned contrastive learning to learn more uniform backdoor representations .
Outcome:	The proposed method achieves better attack performance on text classification tasks compared to manual methods.

Copied to clipboard

Wei Du, Tianjie Ju, Ge Ren, GaoLei Li, Gongshen Liu

Robustness in NLP

Challenge:	Existing attacks disregard fluency and semantic fidelity of poisoned text, rendering it easily detectable.
Approach:	They propose to use AI-generated poisoned text to attack NLP models by establishing covert associations between trigger patterns and target labels without affecting normal accuracy.
Outcome:	The proposed method achieves effective attacks while maintaining fluency and semantic similarity across all scenarios.

Low-code LLM: Graphical User Interface over Large Language Models (2024.naacl-demo)

Copied to clipboard

Yuzhe Cai, Shaoguang Mao, Wenshan Wu, Zehua Wang, Yaobo Liang, Tao Ge, Chenfei Wu, WangYou WangYou, Ting Song, Yan Xia, Nan Duan, Furu Wei

Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding (2021.acl-long)

Challenge:	Low-code LLM is a visual programming interface that allows users to incorporate their ideas into the process without writing trivial prompts.
Approach:	They propose a human-LLM interaction framework that incorporates low-code visual programming interactions to achieve more controllable and stable responses.
Outcome:	The proposed framework enables users to incorporate ideas into the process without writing trivial prompts.

Copied to clipboard

Xin Sun, Tao Ge, Furu Wei, Houfeng Wang

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction (2020.emnlp-main)

Challenge:	Existing approaches to improve online inference efficiency of the Transformer for instantaneous Grammatical Error Correction (GEC) are sequenceto-sequence (seq2sequ) and sequenceto sequence (saq2eq)
Approach:	They propose a novel approach to improve the online inference efficiency of the Transformer model for instantaneous Grammatical Error Correction (GEC) it aggressively decodes as many tokens as possible in parallel instead of always decoding only one token in each step to improve computational parallelism.
Outcome:	The proposed approach can achieve state-of-the-art results in English and Chinese benchmarks with 10x speedup over the Transformer-big model.

Copied to clipboard

Mengyun Chen, Tao Ge, Xingxing Zhang, Furu Wei, Ming Zhou

Green & Sustainable NLP

Challenge:	Existing methods to improve the efficiency of GEC are not efficient enough for GEC.
Approach:	They propose a language-independent approach to improve the efficiency of GEC by dividing the task into two subtasks: ESD and ESC.
Outcome:	The proposed approach performs comparably to conventional seq2seq approaches in English and Chinese GEC benchmarks with less than 50% time cost for inference.

Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration (2024.naacl-long)

Copied to clipboard

Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, Heng Ji

Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correction (2024.findings-acl)

Challenge:	Existing work on LLMs that only enhance reasoning abilities, but which lack factual hallucination and slow-thinking capabilities, argues that SPP is a cognitive synergist.
Approach:	They propose a Solo Performance Prompting (SPP) that transforms a single LLM into a cognitive synergist by engaging in multi-turn self-collaboration with multiple personas.
Outcome:	The proposed model reduces factual hallucination and maintains strong reasoning abilities on three challenging tasks .

Copied to clipboard

Dingyao Yu, Yang An, Wei Ye, Xiongfeng Xiao, Shaoguang Mao, Tao Ge, Shikun Zhang

MdEval: Massively Multilingual Code Debugging (2026.findings-acl)

Challenge:	Chinese Spelling Correction (CSC) lacks large-scale high-quality corpora due to labor-intensive labeling of spelling errors in real-life writing or typing scenarios.
Approach:	They propose to use OCR/ASR-based generation to refine Chinese Spelling Correction models on random replacement-based corpora and filter them based on prediction confidence.
Outcome:	The proposed model outperforms existing models on three widely-used benchmarks while significantly alleviating over-correction.

Copied to clipboard

Shukai Liu, Linzheng Chai, Jian Yang, Jiajun Shi, He Zhu, Liran Wang, Jin Ke, Wei Zhang, Hualei Zhu, Shuyue Guo, Tao Sun, Jiaheng Liu, Yunlong Duan, Yu Hao, Liqun Yang, Guanglin Niu, Ge Zhang, Zhoujun Li

ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)

Challenge:	Existing benchmarks primarily focus on Python and are limited in terms of language diversity.
Approach:	They propose a multilingual debugging benchmark that includes 3.9K test samples of 20 programming languages and introduces the debug instruction corpora MdEval-Instruct by injecting bugs into the correct multilingual queries and solutions.
Outcome:	The proposed benchmark includes 3.9K test samples of 20 programming languages and covers the automated program repair task, bug localization task, and bug identification task.

Copied to clipboard

Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo

Improving Grammatical Error Correction with Machine Translation Pairs (2020.findings-emnlp)

Challenge:	Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task.
Approach:	They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5.
Outcome:	The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei, Ming Zhou

Machine Translation

Challenge:	Existing methods to generate error-corrected sentence pairs for improving grammatical error correction are not available.
Approach:	They propose a method to generate error-corrected sentence pairs for improving grammatical error correction based on machine translation models of different qualities .
Outcome:	The proposed method can generate multiple error-corrected sentence pairs from Chinese to English text.

Learn to Memorize: Scalable Continual Learning in Semiparametric Models with Mixture-of-Neighbors Induction Memory (2025.acl-long)

Copied to clipboard

Guangyue Peng, Tao Ge, Wen Luo, Wei Li, Houfeng Wang

BERT-of-Theseus: Compressing BERT by Progressive Module Replacing (2020.emnlp-main)

Challenge:	Semiparametric language models (LMs) use static storage, which lacks learning capability and is disconnected from the internal information flow of the parametric models.
Approach:	They reconceptualize the non-parametric memory represented by kNN-LM as a learnable Mixture-of-Neighbors Induction Memory (MoNIM) this synergizes the induction capabilities of attention heads with the memorization strength of feed-forward networks .
Outcome:	The proposed model is a learnable Mixture-of-neighbors induction memory (MoNIM) it synergizes the induction capabilities of attention heads with the memorization strength of feed-forward networks (FFNs).

Copied to clipboard

Canwen Xu, Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou

Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study (P19-1)

Challenge:	a novel approach to compress neural networks by progressive module replacement is proposed . a number of techniques have been proposed to compress pretraining and fine-tuning models .
Approach:	They propose a model compression approach that divides BERT into modules and builds their compact substitutes.
Outcome:	The proposed approach outperforms existing knowledge distillation approaches on GLUE benchmark . it is based on a model that divides the original BERT into several modules and builds their substitutes .

Copied to clipboard

Tao Ge, Xingxing Zhang, Furu Wei, Ming Zhou

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting (2021.emnlp-main)

Challenge:	Sequence-to-sequence (seq2sequ) models have a weakness: they cannot always generate sentences without grammatical errors.
Approach:	They propose to use automatic grammatical error correction to improve seq2seq models . they conduct experiments on machine translation, formality style transfer, sentence compression and simplification .
Outcome:	The proposed system can improve grammaticality of generated text and improve formal style tasks.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Canwen Xu, Ke Xu, Furu Wei

Text Generation

Challenge:	Existing text infilling objectives for pretrained language models require self-supervision by masking out tokens or spans in text.
Approach:	They propose to extend text infilling to a self-supervised sequence-to-sequence (Seq2Sequen) task.
Outcome:	The proposed task improves the model's performance on various natural language generation tasks.

Plug and Play Knowledge Distillation for kNN-LM with External Logits (2022.aacl-short)

Copied to clipboard

Xuyang Jin, Tao Ge, Furu Wei

M3TQA: Massively Multilingual Multitask Table Question Answering (2026.findings-acl)

Challenge:	Despite the promising evaluation results by knowledge distillation (KD) in natural language understanding (NLU) and sequence-to-sequence (seq2sequ) tasks, KD for causal language modeling (LM) remains a challenge.
Approach:	They propose to use external logits to improve a student's kNN-LM by leveraging teacher's knowledge at test time.
Outcome:	The proposed method improves a student's kNN-LM in multiple language modeling datasets and improves perplexity.

Copied to clipboard

Daixin Shu, Jian Yang, Zhenhe Wu, Xianjie Wu, Xianfu Cheng, Guan Xiangyuan, Yanghai Wang, Pengfei Wu, Tingyang Yang, Hualei Zhu, Wei Zhang, Ge Zhang, Jiaheng Liu, Zhoujun Li

Structured Data in NLP

Challenge:	Existing multilingual table benchmarks suffer from geolinguistic imbalance - overrepresenting certain languages and lacking sufficient scale for rigorous cross-lingual analysis.
Approach:	They propose a framework for massively multilingual table question answering that includes tables expanded to 97 languages from Chinese and English sources.
Outcome:	Experiments on state-of-the-art LLMs show that synthetically generated training data significantly boosts performance, especially for low-resource languages.

MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization (2023.findings-emnlp)

Copied to clipboard

Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, Yanghua Xiao

ALYMPICS: LLM Agents Meet Game Theory (2025.coling-main)

Challenge:	Existing research emphasizes the importance of adapting prompts to specific tasks, rather than specific LLMs.
Approach:	They propose a model-adaptive prompt optimizer method that optimizes original prompts for each LLM in downstream tasks.
Outcome:	The proposed method can optimize prompts for an LLM in downstream tasks.

Copied to clipboard

Shaoguang Mao, Yuzhe Cai, Yan Xia, Wenshan Wu, Xun Wang, Fengyi Wang, Qiang Guan, Tao Ge, Furu Wei

Linguistics & Cognitive NLP

Challenge:	Alympics provides a framework for simulating human-like strategic interactions with Large Language Model (LLM) agents.
Approach:	They propose a framework utilizing Large Language Models (LLM) agents for empirical game theory research.
Outcome:	The proposed framework can be used to study human-like strategic interactions with large language model (LLM) agents in a game on the multi-round auction of scarce survival resources.

Targeted Exploration via Unified Entropy Control for Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Chen Wang, Lai Wei, Yanzhi Zhang, Chenyang Shao, Zedong Dan, Weiran Huang, Ge Lan, Yue Wang

E-ViC: Reasoning Beyond Text via Embodied Visual Chain for Spatial Intelligence (2026.acl-long)

Challenge:	Existing methods for group relative policy optimization suffer from entropy collapse . Existing exploration methods introduce additional bias or variance during exploration, making it difficult to maintain stability.
Approach:	They propose a framework that provides targeted mechanisms for exploration and stabilization.
Outcome:	The proposed framework expands search space on difficult prompts while preventing entropy growth uncontrollably.

Copied to clipboard

Junbo Qi, Yi Zhang, Hanchu Ni, Che Liu, Zhimin Yao, Ruilin Yang, Xiancong Ren, Liangjian Wen, Wei Ge, Yuya Ieiri, Osamu Yoshie, Yong Dai, Xiaozhu Ju

Evaluating the Expressive Appropriateness of Speech in Rich Contexts (2026.acl-long)

Challenge:	Existing Vision-Language Models (VLMs) lack spatial reasoning, despite text-based CoTs . e-ViC reframes spatial intelligence as a verifiable, tool-using capability, argues a new study.
Approach:	They propose a framework that moves reasoning beyond text into the visual domain . they ground reasoning in pixel-level interactions to enable human-like "look-and-confirm" strategies .
Outcome:	The proposed framework outperforms existing Vision-Language Models with an average gain of 10.1%.

Copied to clipboard

Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, Yuheng Lu, Guanrou Yang, Xuanchen Li, Hexin Liu, Chunyu Qiang, Cheng Gong, Yifan Yang, Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, Xie Chen, Longbiao Wang, Jianwu Dang

Speech & Audio in NLP

Challenge:	Existing methods for evaluating expressive speech focus on word accuracy, naturalness, signal quality, or emotional intensity at the utterance level.
Approach:	They propose a framework for Evaluating Expressive Appropriateness in speech that assesses whether a speech sample aligns with the underlying communicative intent implied by its discourse-level narrative context.
Outcome:	The proposed framework outperforms existing speech evaluation and analysis systems on a human-annotated test set.

Boosting Event Extraction with Denoised Structure-to-Text Augmentation (2023.findings-acl)

Copied to clipboard

Bo Wang, Heyan Huang, Xiaochi Wei, Ge Shi, Xiao Liu, Chong Feng, Tong Zhou, Shuaiqiang Wang, Dawei Yin

Information Extraction & Text Mining

Challenge:	Existing methods for event extraction neglect grammatical incorrectness, structure misalignment, and semantic drifting . et al., 2004; Ahn, 2006) show that the proposed method generates more diverse text representations for event extracting compared with the state-of-the-art.
Approach:	They propose a framework for event extraction that generates additional training data and iteratively selects the effective subset from the generated training data.
Outcome:	The proposed method generates more diverse representations of training data and achieves comparable results with the state-of-the-art.

SCALE: Synergized Collaboration of Asymmetric Language Translation Engines (2024.findings-acl)

Copied to clipboard

Xin Cheng, Xun Wang, Tao Ge, Si-Qing Chen, Furu Wei, Dongyan Zhao, Rui Yan

Multilinguality

Challenge:	In this paper, we introduce SCALE, a collaborative framework that connects a compact Specialized Translation Model (STM) and a general-purpose Large Language Model (LLM) as one unified translation engine.
Approach:	They propose a collaborative framework that connects a Specialized Translation Model (STM) and a general-purpose Large Language Model (LLM) as one unified translation engine.
Outcome:	The proposed framework outperforms both LLMs and supervised models in high-resource or challenging low-resourced settings.

K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning (2025.naacl-long)

Copied to clipboard

Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Yan Xia, Man Lan, Furu Wei

Scheduled DropHead: A Regularization Method for Transformer Models (2020.findings-emnlp)

Challenge:	Strategic reasoning requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments.
Approach:	They propose a framework that enables Large Language Models to achieve varying levels of strategic depth by recursive mechanisms that allow agents to form higher order beliefs about others' beliefs.
Outcome:	The proposed framework enables LLMs to achieve varying levels of strategic depth, allowing agents to form higher order beliefs—beliefs about others’ beliefs.

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Furu Wei, Ming Zhou, Ke Xu

OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)

Challenge:	DropHead is a structured dropout method for regularizing multi-head attention . DropHed drops entire attention heads during training to prevent overfitting .
Approach:	They propose a structured dropout method specifically designed for regularizing multi-head attention mechanism . DropHead drops entire attention heads during training to prevent overfitting .
Outcome:	The proposed method can improve transformer models by 0.9 BLEU score on translation task and around 1.0 accuracy for various text classification tasks.

Copied to clipboard

Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, Zili Wang

EventWiki: A Knowledge Base of Major Events (L18-1)

Challenge:	Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence.
Approach:	They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included .
Outcome:	The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model.

Copied to clipboard

Tao Ge, Lei Cui, Baobao Chang, Zhifang Sui, Furu Wei, Ming Zhou

Knowledge Representation

Challenge:	Existing knowledge bases focus on static entities such as people, locations and organizations.
Approach:	They propose a new knowledge base resource called EventWiki which concentrates on major events . they show that EventWiki is a very useful resource for information extraction regarding events in NLP .
Outcome:	The proposed resource is the first knowledge base resource of major events.

Fine-grained Coordinated Cross-lingual Text Stream Alignment for Endless Language Knowledge Acquisition (D18-1)

Copied to clipboard

Tao Ge, Qing Dou, Heng Ji, Lei Cui, Baobao Chang, Zhifang Sui, Furu Wei, Ming Zhou

Multilinguality

Challenge:	a novel information network decipherment paradigm is proposed for fine-grained coordinated cross-lingual text stream alignment.
Approach:	They propose to use Burst Information Networks as media to represent text streams . they propose a simple yet effective information network decipherment algorithm with diverse clues .
Outcome:	The proposed approach outperforms existing approaches on bilingual lexicon extraction from coordinated text streams and can harvest high-quality alignments from large amounts of streaming data for endless language knowledge mining.

Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation (2023.findings-emnlp)

Copied to clipboard

Heming Xia, Tao Ge, Peiyi Wang, Si-Qing Chen, Furu Wei, Zhifang Sui

Green & Sustainable NLP

Challenge:	Experimental results show draft-then-verify paradigm can achieve around 5x speedup for the popular Transformer architectures with comparable generation quality to beam search decoding.
Approach:	They propose to use Spec-Drafter and Spec Verification to accelerate autoregressive (AR) decoding by combining a model optimized for efficient and accurate drafting and a reliable method for verifying the drafted tokens efficiently.
Outcome:	The proposed method achieves 5x speedup on seq2seq tasks with comparable generation quality to beam search decoding, refreshing the impression that draft-then-verify paradigm introduces only 1.4x2x speed up.

UnihanLM: Coarse-to-Fine Chinese-Japanese Language Model Pretraining with the Unihan Database (2020.aacl-main)

Copied to clipboard

Canwen Xu, Tao Ge, Chenliang Li, Furu Wei

Fluency Boost Learning and Inference for Neural Grammatical Error Correction (P18-1)

Challenge:	Chinese and Japanese share many characters with similar surface morphology.
Approach:	They propose a Chinese-Japanese pretrained masked language model with a coarse-to-fine training approach to exploit the shared knowledge across the languages.
Outcome:	The proposed model is effective on mono- and cross-lingual Chinese and Japanese tasks.

Copied to clipboard

Tao Ge, Furu Wei, Ming Zhou

Syntactic Text Processing

Challenge:	Seq2seq models for grammatical error correction (GEC) have two limitations: (1) a seq2q model may not be well generalized with only limited error-corrected data; (2) a model may fail to completely correct a sentence with multiple errors through normal seq1sequeq inference.
Approach:	They propose a fluency boost learning and inference mechanism to improve the performance of seq2seq models for grammatical error correction (GEC) by generating fluency-boost sentence pairs during training.
Outcome:	Experiments show that the proposed model improves on both CoNLL-2014 and JFLEG benchmark datasets.

BERT-based Lexical Substitution (P19-1)

Copied to clipboard

Wangchunshu Zhou, Tao Ge, Ke Xu, Furu Wei, Ming Zhou