Papers with PESC
Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated considerable proficiency in general natural language processing tasks. |
| Approach: | They propose a parameter-efficient sparsity crafting method which crafts dense models into sparse models using the mixture-of-experts architecture. |
| Outcome: | The proposed method significantly reduces computational costs and GPU memory requirements, while maintaining the quality of approximation in function space. |
Towards More Realistic Chinese Spell Checking with New Benchmark and Specialized Expert Model (2024.lrec-main)
Copied to clipboard
Yue Wang, Zilong Zheng, Juntao Li, Zhihui Liu, Jinxiong Chang, Qishen Zhang, Zhongyi Liu, Guannan Zhang, Min Zhang
| Challenge: | Large Language Models (LLMs) have been gaining attention for their ability to perform a wide range of open-domain tasks . however, the performance of LLMs has yet to be comprehensively evaluated in realistic scenarios . |
| Approach: | They propose a task to evaluate the performance of Large Language Models (LLMs) they propose RCSC task to convert Chinese text into correct text . |
| Outcome: | The proposed task evaluates the performance of existing methods in Chinese text . the realistic Chinese spell checker can achieve state-of-the-art performance on the task . |