BPE

Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective (2026.findings-acl)

Challenge:	Pre-trained language models (PLMs) construct word representations at subword level with Byte-Pair Encoding (BPE) or its variations . but these methods split a word into subword units and make it incomplete and fragile .
Approach:	They propose a character-aware pre-trained language model to tackle OOV problems . they construct contextual word embedding for each token from sequential character representations .
Outcome:	The proposed model improves on the existing models on multiple NLP benchmarks.

Copied to clipboard

Meifang Chen, Zhe Yang, Huang Nianchen, Yizhan Huang, Yichen LI, Zihan Li, Michael R. Lyu

Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation (D19-55)

Challenge:	Code Large Language Models (CLLMs) are reshaping how software is built, maintained, and evolved.
Approach:	They propose to use BPE tokenization to inadvertently leak code secrets . they propose to mitigate the gibberish bias by using a newer tokenizer .
Outcome:	The proposed model is based on a novel method that can be used to detect and mitigate gibberish bias in CLLMs.

Copied to clipboard

Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, Marjan Ghazvininejad

Robustness in NLP

Challenge:	Recent machine translation methods are highly sensitive to orthographical variations such as spelling errors.
Approach:	They propose to train machine translation models with random synthetic noise at training time . they focus on translation performance on natural typos, and show robustness to such noise .
Outcome:	The proposed method significantly improves translation models on natural typos without accessing natural noise data or distribution.

Peek2: Regex-free Byte-level Byte-Pair Encoding Pretokenizer for LLM Inference on Edge Devices (2026.acl-srw)

Copied to clipboard

Liu Zai, Iraklis A. Klampanos

BERTSeg: BERT Based Unsupervised Subword Segmentation for Neural Machine Translation (2022.aacl-short)

Challenge:	Pretokenization is a crucial, sequential pass in Byte-level BPE tokenizers . little work has been done to optimize it for edge-side inference .
Approach:	They propose a pretokenization algorithm with linear time complexity and trivial memory usage that is suited for edge scenarios.
Outcome:	The proposed pretokenization algorithm improves microbenchmarking throughput by 2.48 and delivers 1.14 improvement in overall throughput across the entire Byte-level BPE encoding process.

Copied to clipboard

Haiyue Song, Raj Dabre, Zhuoyuan Mao, Chenhui Chu, Sadao Kurohashi

Facebook AI’s WAT19 Myanmar-English Translation Task Submission (D19-52)

Challenge:	Existing subword segmenters are frequency-based without semantics information or neural-based but trained on parallel corpora.
Approach:	They propose an unsupervised neural subword segmenter for neural machine translation that utilizes contextualized semantic embeddings of words from characterBERT and maximizes the generation probability of subword segments.
Outcome:	The proposed method improves translation performance on ALT, IWSLT15 Vi->En, WMT16 Ro->En and WMT15 Fi->En datasets.

Copied to clipboard

Peng-Jen Chen, Jiajun Shen, Matthew Le, Vishrav Chaudhary, Ahmed El-Kishky, Guillaume Wenzek, Myle Ott, Marc’Aurelio Ranzato

Morphology Matters: A Multilingual Language Modeling Analysis (2021.tacl-1)

Challenge:	Using back-translation, we can improve generalization by using noisy channel re-ranking and ensembling.
Approach:	They propose to use BPE-based transformer models to leverage monolingual data to improve generalization and use noisy channel re-ranking and ensembling to improve results.
Outcome:	The proposed system improves on the baseline system trained exclusively on the provided small parallel dataset, and the human evaluation and BLEU score are higher.

Copied to clipboard

Hyunji Hayley Park, Katherine J. Zhang, Coleman Haley, Kenneth Steimel, Han Liu, Lane Schwartz

Self-Vocabularizing Training for Neural Machine Translation (2025.naacl-srw)

Challenge:	Existing studies on inflectional morphology disagree on whether or not it makes languages harder to model.
Approach:	They propose to use a corpus of 145 Bible translations in 92 languages to investigate whether inflectional morphology makes languages harder to model.
Outcome:	The proposed model trains with linguistically motivated subword segmentation strategies and reduces the impact of morphology on language modeling.

Copied to clipboard

Pin-Jie Lin, Ernie Chang, Yangyang Shi, Vikas Chandra

An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks (2020.aacl-main)

Challenge:	Past vocabulary learning techniques identify relevant vocabulary before training, relying on corpus statistics or frequency counts without considering contextual information or the model's ability to represent it.
Approach:	They propose a method that self-vocabularizes a smaller, more optimal vocabulary by pairing source sentences with the model's predictions to define a new vocabulary.
Outcome:	The proposed method produces a 1.49 BLEU improvement in the simulated model and an increase in unique token usage and a 6–8% reduction in vocabulary size.

Copied to clipboard

Kyubyong Park, Joohong Lee, Seongbo Jang, Dawoon Jung

Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages (2022.acl-long)

Challenge:	Traditionally, tokenization is the very first step in most text processing works.
Approach:	They propose to use morphological segmentation followed by BPE for Korean NLP tasks . they empirically examine what is the best tokenization strategy for Korean to/from English .
Outcome:	The proposed approach is best for Korean to/from English machine translation and natural language understanding tasks.

Copied to clipboard

Vaidehi Patil, Partha Talukdar, Sunita Sarawagi

Low-Resource NLP

Challenge:	Pre-trained multilingual models have shown great potential for zero-shot cross-lingual transfer to low web-resource languages (LRLs).
Approach:	They propose a vocabulary generation algorithm which enhances lexical overlap across related languages by generating a token that increases the representation of LRLs.
Outcome:	The proposed approach improves cross-lingual transfer accuracy without reducing HRL representation and accuracy.

Using Neural Machine Translation Methods for Sign Language Translation (2022.acl-srw)

Copied to clipboard

Galina Angelova, Eleftherios Avramidis, Sebastian Möller

The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translation (2021.eacl-srw)

Challenge:	Sign languages are the main medium of exchanging information for the deaf and hard of hearing.
Approach:	They propose to use two NMT architectures to train models on parallel German Sign Language corpora . they achieve substantial improvement in BLEU scores for the models trained on the two corporales .
Outcome:	The proposed models achieve significant improvements on the two corpora trained on the german sign language . the proposed models outperform the models trained on both corporales .

Copied to clipboard

Jonne Saleva, Constantine Lignos

A Formal Perspective on Byte-Pair Encoding (2023.findings-acl)

Challenge:	Current NMT systems typically operate at the level of subwords, causing problems of vocabulary sparsity.
Approach:	They compare subword segmentation methods with morphologically-based methods in a low-resource setting . they find that no consistent and reliable differences emerge between the methods .
Outcome:	The proposed methods outperform BPE in a low-resource translation setting.

Copied to clipboard

Vilém Zouhar, Clara Meister, Juan Gastaldi, Li Du, Tim Vieira, Mrinmaya Sachan, Ryan Cotterell

Challenge:	Byte-Pair Encoding (BPE) is a popular algorithm used for tokenizing data in NLP, but the underlying optimization problem that BPE seeks to solve has not yet been laid down.
Approach:	They propose an algorithm which is a 1/sigma*(1-e(-sigma))-approximation of an optimal merge sequence.
Outcome:	The proposed algorithm improves the runtime complexity from O(NM) to O(N log M) and the lower bound of the approximation is approx0.37.

Tokenization Is More Than Compression (2024.emnlp-main)

Copied to clipboard

Craig Schmidt, Varshini Reddy, Haoran Zhang, Alec Alameddine, Omri Uzan, Yuval Pinter, Chris Tanner

Segmenting Numerical Substitution Ciphers (2022.emnlp-main)

Challenge:	Existing tokenization approaches like Byte-Pair Encoding (BPE) have been suggested that their effectiveness stems from their ability to condense text into a relatively small number of tokens.
Approach:	They propose a tokenizer that segments a document’s text into the minimum number of tokens for a given vocabulary and propose fewer tokens to improve downstream performance.
Outcome:	The proposed tokenizers can initialize vocabulary construction and pre-tokenization, and the results show that fewer tokens lead to better performance.

Copied to clipboard

Nada Aldarrab, Jonathan May

Sailor: Open Language Models for South-East Asia (2024.emnlp-demo)

Challenge:	Existing methods for deciphering historical substitution ciphers are difficult to crack . cipheries that are not segmented are still difficult to deciphere .
Approach:	They propose automatic methods to segment historical substitution ciphers using BPE and unigram language models.
Outcome:	The proposed methods achieve an average segmentation error of 2% on 100 monoalphabetic ciphers and 27% on 3 real historical homophonic cipheries.

Copied to clipboard

Longxu Dou, Qian Liu, Guangtao Zeng, Jia Guo, Jiahui Zhou, Xin Mao, Ziqi Jin, Wei Lu, Min Lin

Explicit Cross-lingual Pre-training for Unsupervised Machine Translation (D19-1)

Challenge:	Large language models (LLMs) rely on English data for training, but are often not comparable across other languages.
Approach:	They propose to develop a family of open language models for SEA languages . they use BPE dropout, aggressive data cleaning and deduplication to improve model robustness .
Outcome:	The proposed models perform well across four benchmarks, including commonsense reasoning, question answering, reading comprehension and examination.

Copied to clipboard

Shuo Ren, Yu Wu, Shujie Liu, Ming Zhou, Shuai Ma

A Systematic Study of Leveraging Subword Information for Learning Word Representations (N19-1)

Challenge:	Existing approaches to build initial unsupervised machine translation models with cross-lingual n-gram embeddings are inexplicit and limited.
Approach:	They propose a cross-lingual pre-training method that incorporates cross-linguistic training signals into pre-trained models by randomly choosing source n-grams in the input text stream.
Outcome:	The proposed method significantly improves the performance of unsupervised machine translation.

Copied to clipboard

Yi Zhu, Ivan Vulić, Anna Korhonen

Representation Learning

Challenge:	Existing word representation models for morphologically rich languages use subword-level information, but their systematic comparative analysis across typologically diverse languages and tasks is still missing.
Approach:	They propose a framework for learning subword-informed word representations that allows for easy experimentation with different segmentation and composition components.
Outcome:	The proposed framework allows for easy experimentation with different segmentation and composition components, as well as advanced techniques based on position embeddings and self-attention.

Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion (2025.acl-long)

Copied to clipboard

Jianqing Zhu, Huang Huang, Zhihang Lin, Juhao Liang, Zhengyang Tang, Khalid Almubarak, Mosen Alharthi, Bang An, Juncai He, Xiangbo Wu, Fei Yu, Junying Chen, Ma Zhuoheng, Yuhao Du, He Zhang, Saied Alshahrani, Emad A. Alghamdi, Lian Zhang, Ruoyu Sun, Haizhou Li, Benyou Wang, Jinchao Xu

SelectNoise: Unsupervised Noise Injection to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages (2023.findings-emnlp)

Challenge:	In the evolving landscape of large language models, the predominant focus has been on English and Chinese.
Approach:	They propose to utilize Arabic-specific vocabulary in the tokenizer to accelerate decoding.
Outcome:	The proposed model achieves decent performance comparable to the best Arabic LLMs across various Arabic benchmarks.

Copied to clipboard

Maharaj Brahma, Kaushal Maurya, Maunendra Desarkar

Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies (2024.findings-naacl)

Challenge:	Currently, MT systems for low-resource languages lack parallel data and monolingual data.
Approach:	They propose an unsupervised approach to generate noisy HRLs training data by selective candidate extraction and noise injection.
Outcome:	The proposed model outperforms strong baselines on 12 ELRLs in a zero-shot setting .

Copied to clipboard

Anaelia Ovalle, Ninareh Mehrabi, Palash Goyal, Jwala Dhamala, Kai-Wei Chang, Richard Zemel, Aram Galstyan, Yuval Pinter, Rahul Gupta

AdaptBPE: From General Purpose to Specialized Tokenizers (2026.eacl-long)

Challenge:	a recent study documented the harmful limitations of gender binary-centric large language models . data scarcity is a known culprit, but the precise mechanisms through which scarcity affects this behavior remain underexplored.
Approach:	They propose to use BPE tokenization to enforce consistent tokenization across gendered pronouns to improve neopronoun proficiency.
Outcome:	The proposed methods outperform finetuning with standard BPE, and improve neopronoun proficiency.

Copied to clipboard

Vijini Pilana Liyanage, François Yvon

Green & Sustainable NLP

Challenge:	Subword tokenization methods impact performance and efficiency of large language models . generic tokens can incur inefficiencies when applying the model to specific domains or languages .
Approach:	They propose a subword tokenization technique that selectively replaces low-utility tokens with more relevant ones based on their frequency in an adaptation corpus.
Outcome:	The proposed method compresses test corpora more effectively than baselines using the same vocabulary size.

Byte Pair Encoding for Symbolic Music (2023.emnlp-main)

Copied to clipboard

Nathan Fradet, Nicolas Gutowski, Fabien Chhel, Jean-Pierre Briot

Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance (2024.findings-acl)

Challenge:	Recent research has focused on reducing the overall sequence length by merging embeddings or combining tokens.
Approach:	They propose to convert music into a sequence of discrete tokens using Byte Pair Encoding . they propose to combine tokens with embeddings to improve the results .
Outcome:	The proposed method reduces the sequence length while increasing the vocabulary size.

Copied to clipboard

Omer Goldman, Avi Caciularu, Matan Eyal, Kris Cao, Idan Szpektor, Reut Tsarfaty

Length-aware Byte Pair Encoding for Mitigating Over-segmentation in Korean Machine Translation (2024.findings-acl)

Challenge:	Despite being the cornerstone of BPE, the importance of compression in the tokenization process is still unclear.
Approach:	They argue for the theoretical importance of compression in the tokenization process . they also demonstrate the empirical importance of compressing tokenizers for downstream success of pre-trained language models.
Outcome:	The proposed method can be viewed as 0-gram language modeling where equal probability is assigned to all tokens.

Copied to clipboard

Jungseob Lee, Hyeonseok Moon, Seungjun Lee, Chanjun Park, Sugyeong Eo, Hyunwoong Ko, Jaehyung Seo, Seungyoon Lee, Heuiseok Lim

Char2Subword: Extending the Subword Embedding Space Using Robust Character Compositionality (2021.findings-emnlp)

Challenge:	Byte Pair Encoding (BPE) is an effective approach in machine translation across several languages, but it is prone to over-segmentation in Korean, an agglutinative and morphologically rich language.
Approach:	They propose a new method that incorporates long words into the Korean vocabulary by strategically preserving morphological information and reducing semantic confusion.
Outcome:	The proposed method outperforms BPE and surpasses state-of-the-art morpheme-aware tokenization methods.

Copied to clipboard

Gustavo Aguilar, Bryan McCann, Tong Niu, Nazneen Rajani, Nitish Shirish Keskar, Thamar Solorio

Investigating the Effectiveness of BPE: The Power of Shorter Sequences (D19-1)

Challenge:	Byte-pair encoding (BPE) is a ubiquitous algorithm in the tokenization process of language models but is only based on pre-training data statistics.
Approach:	They propose a character-based subword module that learns the subword embedding table in pre-trained language models like BERT.
Outcome:	The proposed method significantly improves the performance on the social media linguistic code-switching evaluation (LinCE) benchmark.

Copied to clipboard

Matthias Gallé

A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation (2024.findings-naacl)

Challenge:	Byte-Pair Encoding (BPE) is an unsupervised sub-word tokenization technique, but its reasons for its effectiveness are not well understood.
Approach:	They link BPE to the broader family of dictionary-based compression algorithms and compare it with other members of this family.
Outcome:	The proposed method is compared with dictionary-based compression algorithms and improves on a fixed vocabulary size budget.

Copied to clipboard

Francois Meyer, Jan Buys

Multi-Granularity Contrasting for Cross-Lingual Pre-Training (2021.findings-acl)

Challenge:	Multilingual modelling can improve machine translation for low-resource languages, partly through shared subword representations.
Approach:	They propose to use subword regularisation to promote synergy and BPE to facilitate cross-lingual transfer.
Outcome:	The proposed methods promote synergy and prevent interference across different linguistic typologies.

Copied to clipboard

Shicheng Li, Pengcheng Yang, Fuli Luo, Jun Xie

One Size Does Not Fit All: Comparing NMT Representations of Different Granularities (N19-1)

Challenge:	Existing approaches to pre-training focus on embedding alignment, but they neglect the modeling of bidirectional contexts.
Approach:	They propose a framework to learn languageuniversal representations using multi-granularity contrasting framework . they encode semantic equivalents from different languages into similar representations .
Outcome:	The proposed framework can achieve significant performance gains in machine translation and cross-lingual language understanding.

Copied to clipboard

Nadir Durrani, Fahim Dalvi, Hassan Sajjad, Yonatan Belinkov, Preslav Nakov

Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning (2025.emnlp-main)

Challenge:	Recent work has shown that contextualized word representations are a viable alternative to simple word prediction tasks.
Approach:	They propose to use subword units and characters to model morphology, syntax, and semantics instead of word embeddings.
Outcome:	The proposed representations are better for modeling syntax and more robust to noisy input.

Copied to clipboard

Gagan Bhatia, Maxime Peyrard, Wei Zhao

Reasoning

Challenge:	a tokeniser that splits "2025-03-14" into "20", "25", "-0", "3", "-1", "4" obscures temporal cues and obscures structure . excessive fragmentation correlates with accuracy drops of up to 10 points on uncommon dates .
Approach:	They propose a date fragmentation ratio measure that measures how faithfully a tokeniser preserves multi-digit date components.
Outcome:	The proposed method shows that excessive fragmentation correlates with accuracy drops of up to 10 points on uncommon dates like historical and futuristic dates.

BPE-Dropout: Simple and Effective Subword Regularization (2020.acl-main)

Copied to clipboard

Ivan Provilkov, Dmitrii Emelianenko, Elena Voita

Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation (2023.findings-acl)

Challenge:	Subword segmentation is widely used to address the open vocabulary problem in machine translation.
Approach:	They propose a method that stochastically corrupts the segmentation procedure of BPE and produces multiple segmentations within the same fixed BPE framework.
Outcome:	The proposed method produces multiple segmentations within the same fixed BPE framework.

Copied to clipboard

Francois Meyer, Jan Buys

Enhancing Character-Level Understanding in LLMs through Token Internal Structure Learning (2025.acl-long)

Challenge:	Subword segmenters are used in neural machine translation, but are not used in high-resource settings.
Approach:	They propose a subword segmental machine translation (SSMT) that unifies subword and MT in a single trainable model.
Outcome:	The proposed model improves chrF scores for morphologically rich agglutinative languages and is more robust on a test set constructed for evaluating morphology generalisations.

Copied to clipboard

Zhu Xu, Zhiqiang Zhao, Zihan Zhang, Yuchi Liu, Quanwei Shen, Fei Liu, Yu Kuang, Jian He, Conglin Liu

Challenge:	Large language models (LLMs) use tokenization methods but often obscure internal character structures within tokens.
Approach:	They propose a method that improves models’ ability to capture character positions within tokens by training them on reverse character prediction tasks using the tokenizer’s vocabulary.
Outcome:	Experiments show that the proposed method improves position prediction accuracy in large language models, enabling more precise identification of target characters in original text.

ReBPE: Iteratively Improving the Internal Structure of a Structured Tokeniser by Mining its Internal Structure (2026.findings-eacl)

Copied to clipboard

Thomas Bauwens, Miryam de Lhoneux

CE-VDG: Counterfactual Entropy-based Bias Reduction for Video-grounded Dialogue Generation (2024.lrec-main)

Challenge:	Recent work explores pruning merges from BPE subword tokenisers using corpus data as a signal for which merges to prune.
Approach:	They propose a pruning algorithm that inspects the effects left by pruning . they propose reification of the tokenisers and a new pruning algorithm .
Outcome:	The proposed algorithm outperforms the original BPE-knockout algorithm on alignment in all 14 languages tested by over 11% F1 on average.

Copied to clipboard

Hongcheng Liu, Pingjie Wang, Zhiyuan Zhu, Yanfeng Wang, Yu Wang

Dialogue Systems & Conversational Agents

Challenge:	Existing methods to reduce question-related bias in video-grounded dialogue generation (VDG) however, the dataset often contains inherent bias, which can cause VDG models to learn spurious correlations between questions and answers.
Approach:	They propose to extend the counterfactual reasoning from the information entropy perspective to the generative task, which can effectively reduce the question-related bias in the auto-regressive generation task.
Outcome:	The proposed method can reduce question-related bias in the auto-regressive generation task by using counterfactual entropy as an external loss.

Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation (2020.acl-main)

Copied to clipboard

Xuanli He, Gholamreza Haffari, Mohammad Norouzi

APIRecX: Cross-Library API Recommendation via Pre-Trained Language Model (2021.emnlp-main)

Challenge:	Empirical results on machine translation suggest that DPE is effective for segmenting output sentences.
Approach:	They propose a new algorithm for tokenizing sentences into subword units . they propose enabling exact log marginal likelihood estimation and exact MAP inference .
Outcome:	The proposed algorithm improves on machine translation datasets and on a large dataset.

Copied to clipboard

Yuning Kang, Zan Wang, Hongyu Zhang, Junjie Chen, Hanmo You

BanglaByT5: Byte-Level Modelling for Bangla (2025.findings-emnlp)

Challenge:	API recommendation tools can help programmers use APIs by recommending which APIs to be used next given the APIs that have been written.
Approach:	They propose a cross-library API recommendation approach that uses BPE to split API calls in each sequence and pre-train a GPT based language model.
Outcome:	The proposed APIRecX can recommend APIs that are previously regarded as OOV . it can migrate knowledge of existing libraries to a new library and recommend API that is previously viewed as OVO .

Copied to clipboard

Pramit Bhattacharyya, Arnab Bhattacharya

From characters to words: the turning point of BPE merges (2021.eacl-main)

Challenge:	Large language models (LLMs) have achievedremarkable success across various natural lan-guage processing tasks.
Approach:	They propose a byte-level encoder-decoder model specifically tailored for Bangla.
Outcome:	The proposed model outperforms existing models in gen-erative and classification tasks and surpasses several multilingual and larger models.

Copied to clipboard

Ximena Gutierrez-Vasques, Christian Bentz, Olga Sozinova, Tanja Samardzic

How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation (2021.findings-acl)

Challenge:	morphological complexity is still a major challenge for NLP and the study of language.
Approach:	They perform a cross-linguistic comparison following incremental merges of BPE for 47 diverse languages.
Outcome:	The results show that language distributions are similar under specific levels of tokenization.

Copied to clipboard

Marco Gaido, Beatrice Savoldi, Luisa Bentivogli, Matteo Negri, Marco Turchi

BPE-knockout: Pruning Pre-existing BPE Tokenisers with Backwards-compatible Morphological Semi-supervision (2024.naacl-long)

Challenge:	Existing methods for subword splitting penalize the representation of feminine linguistic markings.
Approach:	They propose a method that preserves subword splitting while leveraging character-based segmentation to properly translate gender.
Outcome:	The proposed approach preserves BPE overall translation quality while leveraging the higher ability of character-based segmentation to properly translate gender.

Copied to clipboard

Thomas Bauwens, Pieter Delobelle

Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pretrained Models (2026.findings-eacl)

Challenge:	Subword tokenisation is a mainstay in natural language processing (NLP) it allows the representation of an infinite space of text with a finite set of units.
Approach:	They propose to use byte-pair encoding to represent an infinite space of text with a finite set of units by removing subwords from the BPE vocabulary without impeding further use of merges that relied on them.
Outcome:	The proposed method improves token-based tasks in English, Dutch and German while saving training costs.

Copied to clipboard

Taido Purason, Pavel Chizhov, Ivan P. Yamshchikov, Mark Fishel

Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization (2026.acl-long)

Challenge:	Extending existing vocabulary is a widely used step in adapting pre-trained language models to new domains or languages.
Approach:	They propose to extend a pre-trained tokenizer by continuing the BPE merge learning process on new data.
Outcome:	The proposed method improves tokenization efficiency and improves model utilization.

Copied to clipboard

Negar Foroutan, Clara Meister, Debjit Paul, Joel Niklaus, Sina Ahmadi, Antoine Bosselut, Rico Sennrich

The Challenges of Optimizing Machine Translation for Low Resource Cross-Language Information Retrieval (D19-1)

Challenge:	Tokenization is the first step of most NLP pipelines.
Approach:	They propose a parity-aware byte pair encoder that maximizes the compression gain of the currently worst-compressed language for cross-lingual parity.
Outcome:	a new algorithm reduces tokenization inequality by 89% compared to classical BPE . the proposed algorithm is based on a fair-max rule that maximizes the compression gain of the currently worst-compressed language .

Copied to clipboard

Constantine Lignos, Daniel Cohen, Yen-Chieh Lien, Pratik Mehta, W. Bruce Croft, Scott Miller

BERT for Monolingual and Cross-Lingual Reverse Dictionary (2020.findings-emnlp)

Challenge:	Existing studies do not investigate the effectiveness of MT metrics in predicting performance of downstream IR models.
Approach:	They examine the relationship between MT performance and IR quality in a CLIR-based system . they find that the choice of IR collection can significantly affect MT tuning decisions .
Outcome:	The proposed model can predict CLIR performance better from MT quality, the authors show . the proposed model is based on a BLEU-based model with a bag of words constraint .

Copied to clipboard

Hang Yan, Xiaonan Li, Xipeng Qiu, Bocao Deng

Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models (2026.eacl-long)

Challenge:	Existing methods to find the proper word for a word are based on the surface form of words, but they cannot extract the semantic meaning.
Approach:	They propose a method to make BERT generate the target word for this task . cross-lingual reverse dictionary is the task to find the proper target word .
Outcome:	The proposed method can generate the target word for cross-lingual reverse dictionary task even without the parallel corpus.

Copied to clipboard

Sawsan Alqahtani, Mir Tafseer Nayeem, Md Tahmid Rahman Laskar, Tasnim Mohiuddin, M Saiful Bari

Functional Lexicon in Subword Tokenization (2025.naacl-long)

Challenge:	Subword tokenization approaches misalign with linguistic structure and waste capacity across languages and domains.
Approach:	They argue for a context-aware framework that integrates tokenizer and model co-design . they argue that tokenization should be treated as a core design problem, not an afterthought .
Outcome:	The proposed framework integrates tokenizer and model co-design, guided by linguistic, domain, and deployment considerations.

Copied to clipboard

Zachary William Hopton, Yves Scherrer, Tanja Samardzic

Egalitarian Language Representation in Language Models: It All Begins with Tokenizers (2025.coling-main)

Challenge:	Function units are hard to map across languages, while being the most frequent tokens.
Approach:	They analyze subword tokens in terms of their productivity and try to find thresholds that best distinguish function from content tokens.
Outcome:	The proposed method can be used to identify functional lexical units in low-resource languages with minimal annotated data.

Copied to clipboard

Menan Velayuthan, Kengatharaiyer Sarveswaran

Byte Pair Encoding is Suboptimal for Language Model Pretraining (2020.findings-emnlp)

Challenge:	Tokenizers influence how language is represented in large language models . pre-tokenization choices can be problematic for some languages .
Approach:	They propose a tokenization algorithm that incorporates graphemes to improve tokenization . they validate this algorithm with Tamil, Sinhala, and Hindi scripts .
Outcome:	The proposed method outperforms tokenizers on Tamil, Sinhala, and Hindi scripts.

Copied to clipboard

Kaj Bostrom, Greg Durrett

Neural Machine Translation for Low-Resourced Indian Languages (2020.lrec-1)

Challenge:	Subword tokenization is a popular language model that can be used to segment text.
Approach:	They analyze differences between byte-pair encoding (BPE) and unigram LM tokenization methods to find subword units that align more closely with morphology.
Outcome:	The proposed method recovers subword units that align more closely with morphology and avoids problems stemming from BPE’s greedy construction procedure.

Copied to clipboard

Himanshu Choudhary, Shivansh Rao, Rajesh Rohilla

Empowering Backbone Models for Visual Text Generation with Input Granularity Control and Glyph-Aware Training (2024.emnlp-main)

Challenge:	Neural machine translation (NMT) is an effective way to convert text to a different language without human involvement.
Approach:	They propose to use multihead self-attention along with pre-trained Byte-Pair-Encoded (BPE) and MultiBPE embeddings to develop an efficient machine translation system.
Outcome:	The proposed system outperforms Google translator and the existing translators on two of the most morphological rich Indian languages.

Copied to clipboard

Wenbo Li, Guohao Li, Zhibin Lan, Xue Xu, Wanru Zhuang, Jiachen Liu, Xinyan Xiao, Jinsong Su

Visual Data in NLP

Challenge:	Existing text-to-image models struggle to generate images with legible visual texts . current models lack support for Chinese texts, misspelling, and lack of diversity .
Approach:	They propose to empower backbone models to generate visual texts in Chinese and English . they propose to augment conventional training objective with glyph-aware training losses .
Outcome:	The proposed methods can generate visual texts in English and Chinese while maintaining image generation quality.

Effects of sub-word segmentation on performance of transformer language models (2023.emnlp-main)

Copied to clipboard

Jue Hou, Anisia Katinskaia, Anh-Duc Vu, Roman Yangarber

BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages (L18-1)

Challenge:	Language models are a fundamental task in natural language processing, but few studies focus on the effect of sub-word segmentation on the performance of models.
Approach:	They compare GPT and BERT models trained with statistical segmentation algorithm BPE to unsupervised morphological segmentation algorithms Morfessor and StateMorph.
Outcome:	The proposed model trains for several languages and compares them with two unsupervised morphological segmentation algorithms.

Copied to clipboard

Benjamin Heinzerling, Michael Strube

Subword Segmental Language Modelling for Nguni Languages (2022.findings-emnlp)

Challenge:	In an evaluation using fine-grained entity typing as testbed, BPEmb performs competitively . pre-trained subword embeddings for BPE units are commonly available .
Approach:	They present a collection of pre-trained subword embeddings in 275 languages . they use fine-grained entity typing as testbed to evaluate BPEmb .
Outcome:	The proposed method performs better than other methods, but requires less resources and no tokenization.

Copied to clipboard

Francois Meyer, Jan Buys

From Characters to Tokens: Dynamic Grouping with Hierarchical BPE (2025.findings-emnlp)

Challenge:	Subword segmentation is a standard practice in NLP, but is viewed as a preprocessing step for low-resource languages with complex morphologies.
Approach:	They propose a subword segmental language model that learns how to segment words while being trained for autoregressive language modelling.
Outcome:	The proposed model outperforms existing models on unsupervised morphological segmentation and outperfies standard subword segmenters on all 4 languages.

Copied to clipboard

Rares Dolga, Lucas Maystre, Tudor Berariu, David Barber

Green & Sustainable NLP

Challenge:	Subword tokenization methods suffer from inefficiencies in representing rare words and require large embedding matrices.
Approach:	They propose a dynamic character grouping method that leverages existing BPE tokenization without adding additional models.
Outcome:	The proposed method matches or exceeds existing entropy- and whitespace-based patching strategies while maintaining a compact vocabulary.

Distributional Properties of Subword Regularization (2024.emnlp-main)

Copied to clipboard

Marco Cognetta, Vilém Zouhar, Naoaki Okazaki

Lattice Transformer for Speech Translation (P19-1)

Challenge:	Subword regularization reduces the dependency on exact tokenizations, augments training corpus, and exposes model to unique contexts during training.
Approach:	They propose an algorithm to uniformly sample subword tokenizations to replace stochastic variants that are biased towards a small set of tokenization per word.
Outcome:	The proposed algorithm reduces the dependency on exact tokenizations and augments the training corpus.

Copied to clipboard

Pei Zhang, Niyu Ge, Boxing Chen, Kai Fan

The Learnability of the Annotated Input in NMT Replicating (Vanmassenhove and Way, 2018) with OpenNMT (2020.lrec-1)

Challenge:	Recent advances in sequence modeling have highlighted the strengths of the transformer architecture.
Approach:	They propose a general lattice transformer for speech translation where the input is the output of the automatic speech recognition (ASR) they propose 'controllable' lattica attention mechanism to consume latent representations.
Outcome:	The proposed model outperforms baseline and lattice LSTM on the Chinese-English translation task.

Copied to clipboard

Nicolas Ballier, Nabil Amari, Laure Merat, Jean-Baptiste Yunès

MoVoC: Morphology-Aware Subword Construction for Ge’ez Script Languages (2025.findings-emnlp)

Challenge:	reproducibility of experiments is a key issue in Neural Networks, which are fed with variable samples of training data.
Approach:	They reproduce some of the experiments related to neural network training for Machine Translation as reported in . they annotated a sample from the EN-FR and EN-DE Europarl with syntactic and semantic annotations to train neural networks with the Nematus Neural Machine Translation toolkit.
Outcome:	The results obtained were lower than the original paper, but on a more limited set of annotations.

Copied to clipboard

Hailay Kidu Teklehaymanot, Dren Fazlija, Wolfgang Nejdl

Interpreting Topic Models in Byte-Pair Encoding Space (2025.coling-main)

Challenge:	Subword-based tokenization methods fail to preserve morphological boundaries, a limitation especially pronounced in low-resource, morphology complex languages such as those written in the Ge‘ez script.
Approach:	They propose a tokenizer that integrates supervised morphological analysis into the subword vocabulary and propose morpheme-based tokenization with Byte Pair Encoding (BPE) tokens.
Outcome:	The proposed tokenizer preserves morphological integrity while maintaining lexical meaning.

Copied to clipboard

Jia Peng Lim, Hady Lauw

Explainability & Interpretability in NLP

Challenge:	Byte-pair encoding (BPE) is a popular method of tokenizing valid words onto a token space V b with a predetermined fixed size, and handling out-of-vocabulary words, breaking words into smaller tokens.
Approach:	They propose to interpret the recovery of valid words from these tokens as a ranking problem and apply existing evaluation measures to topic sets.
Outcome:	The proposed model interprets the recovery of valid words from these tokens as a ranking problem and applies existing evaluation measures.

How Important Is Tokenization in French Medical Masked Language Models? (2024.lrec-main)

Copied to clipboard

Yanis Labrak, Adrien Bazoge, Béatrice Daille, Mickael Rouvier, Richard Dufour

Impacts of Misspelled Queries on Translation and Product Search (2024.acl-long)

Challenge:	Word tokenization into subword units has become the prevailing standard in the field of natural language processing (NLP) over recent years . the precise factors contributing to its success remain unclear .
Approach:	They propose a tokenization strategy that integrates morpheme-enriched word segmentation into existing tokenization methods.
Outcome:	The proposed tokenization strategy outperforms character and word tokenization but the precise factors contributing to its success remain unclear.

Copied to clipboard

Greg Hanneman, Natawut Monaikul, Taichi Nakatani

Words, Subwords, and Morphemes: What Really Matters in the Surprisal-Reading Time Relationship? (2023.findings-emnlp)

Challenge:	In end-to-end e-commerce, the inclusion of a dedicated spelling correction model, and the augmentation of that model’s training data with language-relevant phenomena, each improve robustness and consistency of search results.
Approach:	They first analyze the spelling-robustness of a population of machine translation systems and then apply them to a multilingual e-commerce setting to test whether spelling variations affect MT output and user behavior.
Outcome:	The proposed model reduces the number of BPE operations and improves spelling-robustness in six languages.

Copied to clipboard

Sathvik Nair, Philip Resnik

Optimized Tokenization for Transcribed Error Correction (2023.emnlp-main)

Challenge:	Existing studies using LLMs on psycholinguistic data have gone unverified . a growing body of research is using word-level prediction as a computational proxy .
Approach:	They compare morphological, morphologic, and BPE tokenization estimates with reading time data.
Outcome:	The proposed method could be used to evaluate morphological prediction.

Copied to clipboard

Tomer Wullach, Shlomo Chazan

Multimodality

Challenge:	transcribed-like data is often used to correct recurring errors, but training with synthetic data is difficult.
Approach:	They propose to use synthetic transcribed-like data to train error correction models . they show that synthetic data outperforms the common approach of random perturbations .
Outcome:	The proposed method outperforms the common method using random perturbations in transcribed data and language-specific adjustments to the vocabulary of a BPE tokenizer.

FLEXITOKENS: Flexible Tokenization for Evolving Language Models (2026.findings-acl)

Copied to clipboard

Abraham Toluwase Owodunni, Orevaoghene Ahia, Sachin Kumar

Finding the Optimal Byte-Pair Encoding Merge Operations for Neural Machine Translation in a Low-Resource Setting (2024.findings-emnlp)

Challenge:	Widely used subword tokenizers overfragment sequences in unseen domains, languages, and scripts . inefficient tokenizer models can cause overfragments in out-of-distribution domains if not trained properly .
Approach:	They propose a byte-level LM with learnable tokenizers to make tokenization adaptive . they propose 'flexitoken' which enables significantly greater flexibility during adaptation .
Outcome:	The proposed method significantly reduces token overfragmentation and improves on multilingual benchmarks and domains.

Copied to clipboard

Kristine Adlaon, Nelson Marcos

Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language Models (2024.findings-emnlp)

Challenge:	Using different byte pair encoder configurations, we can improve neural machine translation performance for low-resource languages.
Approach:	They investigate the impact of different Byte Pair Encoding configurations on neural machine translation performance for the Filipino-Cebuano language pair across various text domains.
Outcome:	The proposed methods show that smaller BPE configurations yield higher BLEU scores, indicating improved translation quality through finer tokenization granularity . larger BPE setups and the absence of BPE result in lower BLUE scores, suggesting a decline in translation quality due to coarser tokenisation.

Copied to clipboard

Gunjan Balde, Soumyadeep Roy, Mainack Mondal, Niloy Ganguly

Tokenization with Factorized Subword Encoding (2023.findings-acl)

Challenge:	Current vocabulary adaptation approaches append the target domainspecific vocabulary (V DOMAIN) at the end of the PLM vocabulary.
Approach:	They propose a vocabulary adaptation scheme that appends a target domain-specific vocabulary (V DOMAIN) at the end of the PLM vocabulary.
Outcome:	The proposed approach improves by 3.57% (in terms of accuracy) and 1.87% (royal-L) over various classification and summarization tasks.

Copied to clipboard

David Samuel, Lilja Øvrelid

Improbable Bigrams Expose Vulnerabilities of Incomplete Tokens in Byte-Level Tokenizers (2025.emnlp-main)

Challenge:	Subword tokenization methods are often used to project subwords onto triplets . a typical tokenizer consists of 10 000s of subword mapped onto a single index .
Approach:	They propose a subword tokenization method that factorizes subwords onto triplets using a VQ-VAE model.
Outcome:	The proposed tokenization method is more appropriate and robust for morphological tasks than the commonly used byte-pair encoding (BPE) tokenization algorithm.

Copied to clipboard

Eugene Jang, Kimin Lee, Jin-Woo Chung, Keuntae Park, Seungwon Shin

BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training (2024.emnlp-main)

Challenge:	Recent studies have revealed that tokenizers can be exploited to elicit unwanted behavior.
Approach:	They propose to exploit incomplete tokens with stray bytes to exploit their dependency . they propose to use improbable bigrams to exploit the dependency of their adjacent tokens .
Outcome:	The proposed tokenizers can be exploited to elicit unwanted behavior in language models.

Copied to clipboard

Pavel Chizhov, Catherine Arnett, Elizaveta Korotkova, Ivan Yamshchikov

Unsupervised Morphological Tree Tokenizer (2025.findings-acl)

Challenge:	Tokenization is a relatively understudied area, but it can greatly impact model performance and efficiency.
Approach:	They propose a modified BPE tokenizer that removes merges that leave intermediate "junk" tokens from the vocabulary.
Outcome:	The proposed method improves vocabulary efficiency, eliminates under-trained tokens, and does not compromise text compression.

Copied to clipboard

Qingyang Zhu, Xiang Hu, Pengyu Ji, Wei Wu, Kewei Tu

Splintering Nonconcatenative Languages for Better Tokenization (2025.findings-acl)

Challenge:	Conventional statistical tokenizers often disrupt constituent boundaries within words, thereby corrupting semantic information.
Approach:	They propose a method that uses morphological structure guidance to induce character-level structures of words by training a deep model.
Outcome:	Empirical results show that the proposed method retains complete morphemes and outperforms existing methods on morphological segmentation and language modeling tasks.

Copied to clipboard

Bar Gazit, Shaltiel Shmidman, Avi Shmidman, Yuval Pinter

R-BPE: Improving BPE-Tokenizers with Token Reuse (2025.emnlp-main)

Challenge:	Subword tokenizers are not effective in nonconcatenative languages such as Hebrew and Arabic, where morphology is encoded in root-template patterns, or Malay and Georgian, where split affixes are common.
Approach:	They propose a pre-processing step which rearranges text into a linear form that better represents nonconcatenative morphologies.
Outcome:	The proposed method is based on measures evaluating token vocabularies in Hebrew, Arabic, and Malay, and on BERT-architecture models trained for Hebrew.

Copied to clipboard

Nancy Hamdan, Osama Rakan Al Mraikhat, Fadi Zaraket

Exploiting Phonetics and Glyph Representation at Radical-level for Classical Chinese Understanding (2025.findings-acl)

Challenge:	Large pretrained language models prioritize high-resource languages in their vocabularies, leaving others with poor coverage.
Approach:	They propose a framework that reuses existing tokenizers and creates ID-based maps to resolve the new tokens of the chosen language.
Outcome:	The proposed framework reduces subword fertility by 24.4% on Arabic models and preserves performance on EnglishMMLU.

Copied to clipboard

Junyi Xiang, Maofu Liu

Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries (2025.findings-acl)

Challenge:	Existing approaches in classical Chinese understanding have integrated heterogeneous linguistic knowledge, spanning lexicalsemantic relationships.
Approach:	They propose a radical-level phonetics and glyph representation enhanced Chinese model with powerful fine-grained semantic modeling capabilities.
Outcome:	The proposed model establishes robust representations through rules-based radical decomposition and bype pair encoder (BPE) based radical aggregated for structural pattern recognition, phonetic-glyph semantic mapping, and dynamic semantic fusion.

Copied to clipboard

Haruki Sakajo, Yusuke Ide, Justin Vasselli, Yusuke Sakai, Yingtao Tian, Hidetaka Kamigaito, Taro Watanabe

Two Counterexamples to Tokenization and the Noiseless Channel (2024.lrec-main)

Challenge:	Existing approaches to cross-lingual vocabulary transfer face challenges when dealing with low-resource languages.
Approach:	They propose a dictionary-based crosslingual vocabulary transfer method that leverages bilingual dictionaries, which are available for many languages thanks to descriptive linguists.
Outcome:	The proposed method outperforms existing methods for low-resource languages.

Copied to clipboard

Marco Cognetta, Vilém Zouhar, Sangwhan Moon, Naoaki Okazaki

Unmasking Biases: Exploring Gender Bias in English-Catalan Machine Translation through Tokenization Analysis and Novel Dataset (2024.lrec-main)

Challenge:	Nevertheless, Rényi efficiency is not perfect and the metric is difficult to evaluate because training multiple tokenizers can be prohibitively expensive and takes days or weeks.
Approach:	They propose to use Rényi efficiency as an intrinsic mechanism to evaluate a tokenizer for NLP tasks without the expensive step of training multiple models with different tokenizers.
Outcome:	The proposed metric is better correlated to downstream model performance than a percentile frequency metric.

Copied to clipboard

Audrey Mash, Carlos Escolano, Aleix Sant, Maite Melero, Francesca de Luca Fornaciari

Exploring morphology-aware tokenization: A case study on Spanish language modeling (2025.emnlp-main)

Challenge:	a new dataset focuses on gender-neutral terms that necessitate gendered translations in Catalan.
Approach:	They propose to use a new dataset to evaluate gender bias in machine translation . they train four MT systems using different tokenization techniques .
Outcome:	The proposed dataset focuses on gender-neutral terms necessitating gendered translations in Catalan.

Copied to clipboard

Alba Táboas García, Piotr Przybyła, Leo Wanner

Scaling Laws or Threshold Effects: Exploring the Optimal Vocabulary Size for Balancing Performance and Efficiency in Low-Resource Languages (2026.findings-acl)

Challenge:	a recent study shows that subword tokenization improves performance of neural language models.
Approach:	They propose a linguistically grounded approach to train a tokenizer on morphologically segmented data.
Outcome:	The proposed tokenizer improves on a Spanish language model with morphological information.

Copied to clipboard

Ao Han, Andong Chen, Yuan Sun, Xiaobing Zhao

Multilingual Tokenization through the Lens of Indian Languages: Challenges and Insights (2026.findings-acl)

Challenge:	vocab expansion scaling laws are well-established for high-resource languages, but they remain unverified in low-resourced settings.
Approach:	They propose to scale trilingual vocabulary for languages with 140 to 195,000 tokens . they find that BBPE follows a "decline-then-rise" pattern, whereas BPE improves monotonically .
Outcome:	The proposed configuration reduces pre-training duration by over 71% across 1.5B to 8B models while improving downstream performance.

Copied to clipboard

Maharaj Brahma, N J Karthika, Rajat Verma, Nagasai Saketh Naidu, Rohit Saluja, Maunendra Sankar Desarkar, Ganesh Ramakrishnan

Train It and Forget It: Merge Lists are Unnecessary for BPE Inference in Language Models (2025.emnlp-main)

Challenge:	Existing tokenizers are often skewed towards high-resource languages limiting their effectiveness for linguistically diverse and morphologically rich languages.
Approach:	They evaluate multilingual tokenization across 17 Indic languages spanning 11 scripts and two language families.
Outcome:	The proposed method improves tokenization quality and vocabulary size in 17 languages . poor tokenization can lead to increase in sequence lengths, fragment meaningful units, weaken model's ability to capture linguistic structure and semantics.

Copied to clipboard

Tomohiro Sawada, Kartik Goyal

From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution (2026.acl-long)

Challenge:	Existing work shows that byte-pair encoding (BPE) tokenization uses a learned merge list to iteratively combine subword units into tokens during inference time.
Approach:	They propose to use a standard byte-pair encoding algorithm to pair a learned token vocabulary with a detailed merge list to compress text.
Outcome:	The proposed algorithms differ from the encoding process during training and show that the targetted deviation from merge lists exhibits significant degradation in language model performance.

Copied to clipboard

Pavel Chizhov, Egor Bogomolov, Ivan P. Yamshchikov

MorphBPE: Morphology-Aware Tokenization for Efficient LLM Training (2026.findings-acl)

Challenge:	Currently, subword tokenization is the most common approach for vocabulary building in large models.
Approach:	They propose to regularize training and minimize overfitting by using source-attributed BPE . they find that undertrained tokens are prone to producing unused, unusable tokens .
Outcome:	The proposed techniques reduce the number of under-trained tokens while maintaining the same inference procedure as with regular BPE.

Copied to clipboard

Ehsaneddin Asgari, Yassine El Kheir, MohammadAli SadraeiJavaheri, Ali Nazari

MUTANT: A Recipe for Multilingual Tokenizer Design (2026.acl-long)

Challenge:	Tokenization is a key design choice in modern NLP systems and a critical bottleneck for multilingual Large Language Models.
Approach:	They propose a tokenization extension that constrains merge operations to respect morpheme boundaries while preserving inference.
Outcome:	The proposed tokenization improves morphological coherence and language model cross-entropy in four languages.

Copied to clipboard

Souvik Rana, Ashish Kulkarni, Arul Menezes, Chandra Khatri, Shubham Agarwal

TokDrift: When LLM Speaks in Subwords but Code Speaks in Grammar (2026.acl-long)

Challenge:	Subword tokenization schemes such as Byte Pair Encoding (BPE) are widely adopted, but their effectiveness in multilingual settings remains understudied.
Approach:	They propose a multilingual tokenizer that produces linguistically coherent tokens for multilingual LLMs.
Outcome:	The proposed tokenizer improves fertility score by 39.5% over LLaMA4 and 18% over Sutra.

Copied to clipboard

Yinxi Li, Yuntian Deng, Pengyu Nie