Papers by Pavel Chizhov
From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution (2026.acl-long)
Copied to clipboard
| Challenge: | Currently, subword tokenization is the most common approach for vocabulary building in large models. |
| Approach: | They propose to regularize training and minimize overfitting by using source-attributed BPE . they find that undertrained tokens are prone to producing unused, unusable tokens . |
| Outcome: | The proposed techniques reduce the number of under-trained tokens while maintaining the same inference procedure as with regular BPE. |
Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pretrained Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Extending existing vocabulary is a widely used step in adapting pre-trained language models to new domains or languages. |
| Approach: | They propose to extend a pre-trained tokenizer by continuing the BPE merge learning process on new data. |
| Outcome: | The proposed method improves tokenization efficiency and improves model utilization. |
Model in Distress: Sentiment Analysis on French Synthetic Social Media (2026.findings-acl)
Copied to clipboard
Pierre-Carl Langlais, Pavel Chizhov, Yannick Detrois, Carlos Rosas Hinostroza, Ivan P. Yamshchikov, Bastien Perroy
| Challenge: | Large companies and services handle a substantial volume of reviews and social media mentions. |
| Approach: | They propose a generalizable pipeline for automated customer feedback analysis using backtranslation and fine-tuned models to generate 1.7 million tweets from a small seed corpus, complemented by synthetic reasoning traces. |
| Outcome: | The proposed pipeline generates 1.7 million synthetic tweets from a small seed corpus, complemented by synthetic reasoning traces. |
BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training (2024.emnlp-main)
Copied to clipboard
| Challenge: | Tokenization is a relatively understudied area, but it can greatly impact model performance and efficiency. |
| Approach: | They propose a modified BPE tokenizer that removes merges that leave intermediate "junk" tokens from the vocabulary. |
| Outcome: | The proposed method improves vocabulary efficiency, eliminates under-trained tokens, and does not compromise text compression. |