Papers by Luca Malagutti
On the Proper Treatment of Tokenization in Psycholinguistics (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language models are used in computational psycholinguistics to test theories that relate the surprisal of a region of interest to its cognitive cost experienced by readers. |
| Approach: | They propose to marginalize token-level language models into character-level ones before they are used in psycholinguistic studies. |
| Outcome: | The proposed model over token strings is better than character-level model, the authors show . the proposed model marginalizes token-level models into character-based models before they are used in psycholinguistic studies. |
The Role of n-gram Smoothing in the Age of Neural Networks (2024.naacl-long)
Copied to clipboard
| Challenge: | n-gram smoothing techniques were used to overcome overfitting problems in neural language models for decades. |
| Approach: | They propose to convert any n-gram smoothing technique into a regularizer compatible with neural language models. |
| Outcome: | The proposed regularizers outperform label smoothing on language modeling and machine translation. |
Generating Text from Language Models (2023.acl-tutorials)
Copied to clipboard
| Challenge: | a growing percentage of natural language processing tasks focus on the generation of text from probabilistic language models. |
| Approach: | They will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
| Outcome: | This tutorial will provide a centralized discussion of critical considerations when choosing how to generate from a language model. |
On the Efficacy of Sampling Adapters (2023.acl-long)
Copied to clipboard
| Challenge: | Using sampling adapters can improve the quality of the generated text. |
| Approach: | They propose a framework for understanding sampling adapters and propose 'sampling adapters' they argue that the shift enforced by them can be viewed as a trade-off between precision and recall . |
| Outcome: | The proposed framework can be used to improve the quality of language models by modifying their distributions to improve their precision and recall. |