Papers with sparsity
Simpler neural networks prefer subregular languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Inductive biases of neural networks are still poorly understood, says dr. johansen . subregular languages are thought to form a bound on human phonological patterns . |
| Approach: | They apply a relaxation of L0 regularization which induces sparsity to study inductive biases of LSTMs. |
| Outcome: | The proposed method is based on a relaxation of L0 regularization, which induces sparsity, and a subregular language bias in LSTMs is related to the cognitive bias observed in human phonology. |
HiEdit: Lifelong Model Editing with Hierarchical Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to lifelong model editing apply parameter perturbations to static and dense layers for all instances. |
| Approach: | They propose a hierarchical reinforcement learning framework that identifies the most knowledge-relevant layers for each editing instance. |
| Outcome: | The proposed framework boosts the performance of the competitive RLEdit by 8.48% with perturbing only half of the layers per edit. |