Adapting Large Language Models for Character-based Augmentative and Alternative Communication (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Most character language models predict subword tokens of variable length . |
| Approach: | They propose to use large pretrained character language models to make accurate character predictions. |
| Outcome: | The proposed method produces more accurate character predictions than classification models and n-gram models. |
Similar Papers
Language Adaptation of Large Language Models: An Empirical Study on LLaMA2 (2025.coling-main)
Copied to clipboard
| Challenge: | Popularity of Large Language Models (LLMs) has seen a skyrocketing increase in recent years. |
| Approach: | They present a systematic review of the language adaptation process for Large Language Models including vocabulary expansion, continued pre-training, and instruction fine-tuning. |
| Outcome: | The proposed model is based on empirical studies conducted on LLaMA2 and discussions on various settings affecting the model's capabilities. |
Data and Model Centric Approaches for Expansion of Large Language Models to New languages (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Existing LLMs mainly support English alongside a handful of high resource languages . this leaves a major gap for most low-resource languages despite increasing pace of research . |
| Approach: | This tutorial examines approaches to expand the language coverage of LLMs . they look at tokenizer training, pre-training, instruction tuning, alignment, evaluation, etc. |
| Outcome: | This tutorial examines approaches to expand the language coverage of LLMs . it provides an efficient and viable path to bring LLM technologies to low-resource languages . |
Adaptation of Large Language Models (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | a tutorial on adaptation of large language models addresses the growing demand for models that go beyond static capabilities. |
| Approach: | This tutorial will provide an overview of dynamic, domain-specific, and task-adaptive LLM adaptation techniques. |
| Outcome: | This tutorial will outline dynamic, domain-specific, and task-adaptive LLM adaptation techniques. |
Learn Your Tokens: Word-Pooled Tokenization for Language Modeling (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Language models typically tokenize text into subwords, using a deterministic, hand-engineered heuristic of combining characters into longer surface-level strings such as ‘ing’ or whole words. |
| Approach: | They propose a 'learn your tokens' scheme which pooles bytes/characters into word representations and decodes individual characters/bytes per word in parallel. |
| Outcome: | The proposed tokenizer outperforms subword models and byte/character models over the word boundary and outperformed on rare words by a factor of 30! |
When Large Language Models Meet Speech: A Survey on Integration Approaches (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models have spurred interest in expanding their application beyond text-based tasks. |
| Approach: | They propose to categorize the integration of speech with LLMs into three main approaches . they demonstrate how these methods are applied across various speech-related applications . |
| Outcome: | The proposed methods are applied across speech-related applications and highlight the challenges in this field to offer inspiration for future research. |
AdaptBPE: From General Purpose to Specialized Tokenizers (2026.eacl-long)
Copied to clipboard
| Challenge: | Subword tokenization methods impact performance and efficiency of large language models . generic tokens can incur inefficiencies when applying the model to specific domains or languages . |
| Approach: | They propose a subword tokenization technique that selectively replaces low-utility tokens with more relevant ones based on their frequency in an adaptation corpus. |
| Outcome: | The proposed method compresses test corpora more effectively than baselines using the same vocabulary size. |
Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean (2024.lrec-main)
Copied to clipboard
ChangSu Choi, Yongbin Jeong, Seoyoon Park, Inho Won, HyeonSeok Lim, SangMin Kim, Yejee Kang, Chanhyuk Yoon, Jaewan Park, Yiseul Lee, HyeJin Lee, Younggyun Hahm, Hansaem Kim, KyungTae Lim
| Challenge: | Large language models (LLMs) use pretraining to predict the subsequent word, but less-resourced languages are being overlooked. |
| Approach: | They propose to expand the MLLM vocabularies to enhance expressiveness and use bilingual data for pretraining to align the high- and less-resourced languages. |
| Outcome: | The proposed model outperforms existing models in qualitative analyses compared to Korean monolingual models. |
Empowering Large Language Models for Textual Data Augmentation (2024.findings-acl)
Copied to clipboard
| Challenge: | True. True. False |
| Approach: | False slants are proposed to generate a large pool of augmentation instructions and select the most suitable task-informed instructions. |
| Outcome: | False omissions: the proposed approach consistently generates augmented data with better quality compared to non-LLM and LLM-based data augmentation methods. |
From Characters to Words: Hierarchical Pre-trained Language Model for Open-vocabulary Language Understanding (2023.acl-long)
Copied to clipboard
| Challenge: | Current models for natural language understanding require a preprocessing step to convert raw text into discrete tokens. |
| Approach: | They propose a hierarchical open-vocabulary language model that adopts a shallow Transformer architecture to learn word representations from their characters and a deep inter-word Transformer module that contextualizes each word representation by attending to the entire word sequence. |
| Outcome: | The proposed model outperforms baselines on various downstream tasks and is robust to textual corruption and domain shift. |
Exploring Design Choices for Building Language-Specific LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work focused on building multilingual models that cover a broad spectrum of languages. |
| Approach: | They conduct systematic experiments on how design choices impact the adapted LLM, both in terms of efficiency and end task performance. |
| Outcome: | The proposed model performs better on English-centric models than multilingual models despite poor performance on low-resource languages. |