Papers by Muhan Gao
ICL CIPHERS: Quantifying ”Learning” in In-Context Learning via Substitution Ciphers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies suggest that In-Context Learning operates in dual modes . however, disentangling these modes remains a challenging goal . |
| Approach: | They propose a class of task reformulations based on substitution ciphers borrowed from classic cryptography. |
| Outcome: | The proposed model can solve tasks with a BIJECTIVE mapping, but it requires 'deciphering' the latent cipher. |
Insights into LLM Long-Context Failures: When Transformers Know but Don’t Tell (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit positional bias, struggling to utilize information from the middle or end of long contexts. |
| Approach: | They propose to examine LLMs' long-context generalizations by probing their hidden representations. |
| Outcome: | The proposed models excel at processing extended contexts while preserving their positional bias. |