Papers by Patrycja Cieplicka
Domain Adaptation of Foundation LLMs for e-Commerce (2025.acl-industry)
Copied to clipboard
Christian Herold, Michael Kozielski, Tala Bazazo, Pavel Petrushkov, Yannick Versley, Seyyed Hadi Hashemi, Patrycja Cieplicka, Dominika Basaj, Shahram Khadivi
| Challenge: | Large Language Models (LLMs) have greatly improved the performance on most natural language tasks, and often show surprisingly good zero-shot generalization to new domains. |
| Approach: | They propose to continuously pretrain the Llama 3.1 base models on 1 trillion tokens of e-commerce data to introduce domain specific knowledge into the model while at the same time keeping the general capabilities intact. |
| Outcome: | The proposed model can be adapted to the new domain without sacrificing performance on general domain tasks. |