Papers by Shota Takashiro
Answer When Needed, Forget When Not: Language Models Pretend to Forget via In-Context Knowledge Unlearning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly required to selectively unlearn specific information. |
| Approach: | They propose a method which fine-tunes pre-trained LLMs to enable prompt unlearning of target knowledge within the context while preserving unrelated information. |
| Outcome: | The proposed method achieves up to 95% forget accuracy while retaining 80% of unrelated knowledge, significantly outperforming baselines in both in-domain and out-of-domain scenarios. |
Infinity-MoE: Generalizing Mixture of Experts to Infinite Experts (2026.eacl-short)
Copied to clipboard
| Challenge: | Existing methods to increase the number of experts are -MoE and . |
| Approach: | They propose a mixture of experts that selects a few feed-forward networks per token to increase the number of experts. |
| Outcome: | The proposed model improves on a GPT-2 Small model with 129M active and 186M total parameters by 2.5% over the current model. |