Papers by Daking Rai
An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Prior work has focused on ablating components in the CoT prompt, but the reason why these components are important to LLM reasoning is not explored. |
| Approach: | They investigate "neuron activation" as a lens to provide a unified explanation to previous work . they propose an approach to automatically identify neurons that imply arithmetic reasoning . |
| Outcome: | The proposed approach can explain the importance of components in a CoT prompt . it also automatically identifies neurons that imply arithmetic reasoning . |
A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sparse Autoencoders (SAEs) can disentangle complex features into more interpretable components. |
| Approach: | They propose to use Sparse Autoencoders to disentangle LLM features into more interpretable components. |
| Outcome: | The proposed method disentangles complex features into more interpretable components. |
All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokens (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) perform well on a multitude of computational tasks, yet their inner workings remain unclear. |
| Approach: | They propose two techniques to inhibit input-specific token computations in initial layers . they propose a transformer that allows for any token to immediately access all preceding tokens . |
| Outcome: | The proposed algorithms can perform on a variety of mental math tasks with high accuracy and transfer across models. |
Improving Generalization in Language Model-based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques (2023.acl-short)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs)2 have been adopted for semantic parsing due to their promising performance and straightforward architectures. |
| Approach: | They propose to use token preprocessing to preserve semantic boundaries of tokens produced by LM tokenizers and special tokens to mark the boundaries of aligned components. |
| Outcome: | The proposed techniques improve the performance of pre-trained language models on two text-to-SQL semantic parsing datasets. |