Papers by Tomer Natan
In-context Learning and Gradient Descent Revisited (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) has shown impressive results in few-shot learning tasks, yet its underlying mechanism remains elusive. |
| Approach: | They propose a simple gradient descent-based optimization procedure that respects layer causality and improves similarity scores significantly. |
| Outcome: | The proposed procedure improves similarity scores on untrained models despite not showing ICL. |