Papers by Daniel Fadlon
How Much Pretraining Does Structured Data Need? (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models are increasingly adopted for handling structured data, despite pretraining on unstructured text. |
| Approach: | They propose to re-initialize subsets of layers with random weights before fine-tuning on structured datasets. |
| Outcome: | The proposed models are compared to unstructured datasets and show that they perform well over structured data. |