Papers by Nur Lan
Biasless Language Models Learn Unnaturally: How LLMs Fail to Distinguish the Possible from the Impossible (2026.eacl-long)
Copied to clipboard
| Challenge: | linguists have discovered patterns which hold across virtually all known natural languages . lingulists are able to learn languages by comparing their learning curves to those of humans . |
| Approach: | They compare LLM learning curves on existing and "impossible" datasets . they find that GPT-2 learns each language and its impossible counterpart equally easily . |
| Outcome: | The proposed model learns each language and its impossible counterpart equally easily, the study shows . the study also shows that the proposed model does not provide any kind of separation between the possible and the impossible . |
Bridging the Empirical-Theoretical Gap in Neural Network Formal Language Learning Using Minimum Description Length (2024.acl-long)
Copied to clipboard
| Challenge: | Neural networks offer good approximation to many tasks but fail to reach perfect generalization. |
| Approach: | They propose to use a formal language to test whether a theoretically correct solution is not an optimum of commonly used objectives. |
| Outcome: | The proposed model fails to reach the theoretically correct solution even with regularization techniques. |
Minimum Description Length Recurrent Neural Networks (2022.tacl-1)
Copied to clipboard
| Challenge: | We show that neural networks that optimize a minimum description length score master memory challenges and perform addition with 100% accuracy. |
| Approach: | They train neural networks to optimize a Minimum Description Length score . they show that they master tasks involving memory challenges and perform addition . |
| Outcome: | The proposed models master languages and perform addition with 100% accuracy . they show that they can generalize from small training corpora and large training corpus . |