Papers by Rabin Adhikari
Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers (2026.acl-srw)
Copied to clipboard
| Challenge: | Large language models are difficult to reverse engineer because of their internal operations. |
| Approach: | They train small attention-only transformers on a symbolic version of the Indirect Object Identification task. |
| Outcome: | The proposed model with only two attention heads achieves perfect IOI accuracy despite lacking MLPs and normalization layers . |