Papers by Sehoon Kim
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement (2024.findings-acl)
Copied to clipboard
Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipalli, Michael Mahoney, Kurt Keutzer, Amir Gholami
| Challenge: | Pretrained large language models are currently state-of-the-art for solving most tasks . however, many of them are in the low-data regime, making fine-tuning challenging . a new data augmentation strategy uses a teacher LLM to augment a small seed dataset . |
| Approach: | They propose a targeted and iterative data augmentation strategy that augments a teacher LLM to fine-tune a small seed dataset by adding additional data. |
| Outcome: | The proposed approach outperforms fine-tuning and other data augmentation strategies on a small seed dataset. |
Squeezed Attention: Accelerating Long Context Length LLM Inference (2025.acl-long)
Copied to clipboard
Coleman Richard Charles Hooper, Sehoon Kim, Hiva Mohammadzadeh, Monishwaran Maheswaran, Sebastian Zhao, June Paik, Michael W. Mahoney, Kurt Keutzer, Amir Gholami
| Challenge: | Emerging Large Language Models require long input context to perform complex tasks. |
| Approach: | They propose an algorithm to reduce the complexity of attention with respect to the fixed context length. |
| Outcome: | The proposed method reduces the complexity of attention from linear to logarithmic with respect to the fixed context length. |
TinyAgent: Function Calling at the Edge (2024.emnlp-demo)
Copied to clipboard
Lutfi Erdogan, Nicholas Lee, Siddharth Jha, Sehoon Kim, Ryan Tabrizi, Suhong Moon, Coleman Hooper, Gopala Anumanchipalli, Kurt Keutzer, Amir Gholami
| Challenge: | Recent large language models (LLMs) have enabled the development of advanced agentic systems that can integrate various tools and APIs to fulfill user queries. |
| Approach: | They propose an end-to-end framework for training and deploying task-specific small language model agents capable of function calling for driving agentic systems at the edge. |
| Outcome: | The proposed model outperforms existing models by reducing the input prompt length and quantizing the inference speed. |