Papers by Petros Karypis
Fine-Tuning Language Models on Multiple Datasets for Citation Intention Classification (2024.findings-emnlp)
Copied to clipboard
Zeren Shui, Petros Karypis, Daniel Karls, Mingjian Wen, Saurav Manchanda, Ellad Tadmor, George Karypis
| Challenge: | Prior research has shown that pretrained language models (PLMs) can achieve state-of-the-art performance on CIC benchmarks. |
| Approach: | They propose a multi-task learning framework that fine-tunes pretrained language models on a dataset of primary interest together with multiple auxiliary CIC datasets to take advantage of additional supervision signals. |
| Outcome: | The proposed framework outperforms current state-of-the-art models on small datasets while aligning with the best-performing model on a large dataset. |
Extending Input Contexts of Language Models through Training on Segmented Sequences (2024.findings-naacl)
Copied to clipboard
| Challenge: | Effectively training languages models on long sequences poses many technical challenges. |
| Approach: | They propose a method for extending positional embeddings by sub-sampling segments from long inputs while maintaining their original position. |
| Outcome: | The proposed method extends the input con-text size of pretrained models without any changes in the model's memory and memory costs. |