Papers by Jimin Hong
Accelerating Multilingual Language Model for Excessively Tokenized Languages (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown a significant degree of multilingual proficiency on a variety of tasks in multiple languages. |
| Approach: | They propose a framework to fine-tune a language model head and fine-track it while preserving its performance. |
| Outcome: | The proposed framework increases the generation speed by 1.7 while maintaining the performance of pre-trained multilingual models on target monolingual tasks. |
AVocaDo: Strategy for Adapting Vocabulary to Downstream Domain (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to fine-tune a language model with a large corpus in a general domain are suboptimal for downstream data when domain discrepancy exists. |
| Approach: | They propose to consider the pretrained vocabulary as an optimizable parameter . they add domain specific vocabulary based on a tokenization statistic . their method achieved consistent performance improvements on diverse domains . |
| Outcome: | The proposed method achieves consistent performance improvements on diverse domains. |
Reweighting Strategy Based on Synthetic Data Identification for Sentence Similarity (2022.coling-1)
Copied to clipboard
| Challenge: | obtaining large amounts of human-annotated datasets to train a sentence embedding model is difficult and expensive. |
| Approach: | They propose to train a classifier that identifies machine-written sentences and then use it to train an embedding model on synthetic data. |
| Outcome: | The proposed method outperforms baselines on four real-world datasets and generalizes well. |
Fˆ2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmax (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text generation do not fully reflect the rich diversity of human language. |
| Approach: | They propose to use F2-Softmax and MefMax to train a balanced frequency distribution using a frequency class-based method. |
| Outcome: | The proposed methods improve the diversity and quality of generated texts. |