Papers by Junyeop Lee
Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to character-level language modeling have suffered from high learning complexity caused by inherently long character sequences. |
| Approach: | They propose a method that efficiently reduces the computational cost and parameter size of Transformer by splitting feature space into multiple groups, factorizing the calculation paths, and reducing computations for the group interaction. |
| Outcome: | The proposed model reduces the computational cost and parameter size of Transformer on two benchmark tasks, enwik8 and text8, and it performs well. |