Papers by Jungwon Park
Soft Head Selection for Injecting ICL-Derived Task Embeddings (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are commonly adapted to downstream tasks using parameter-efficient fine-tuning (PEFT) or in-context learning (ICL). |
| Approach: | They propose a gradient-based method that derives task-specific embeddings from activations using few-shot prompts and injects them during inference. |
| Outcome: | The proposed method outperforms existing methods on open-ended generation, reasoning, and natural language understanding tasks while using fewer trainable parameters. |