Papers by Savas Ozkan
Multi-Task Pre-Finetuning of Lightweight Transformer Encoders for Text Classification and NER (2025.emnlp-industry)
Copied to clipboard
| Challenge: | nave multitask pre-finetuning introduces conflicting optimization signals that degrade overall performance. |
| Approach: | They propose a framework that enables a single shared encoder backbone with modular adapters. |
| Outcome: | The proposed framework achieves comparable performance to individual pre-finetuning while meeting practical deployment constraint. |
Decoding Text Spans for Efficient and Accurate Named-Entity Recognition (2026.acl-industry)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a key component in industrial information extraction pipelines, where systems must satisfy strict latency and throughput constraints in addition to strong accuracy. |
| Approach: | They propose a span-based NER framework that can be used to compute span representations at the final transformer stage, avoiding redundant computation in earlier layers. |
| Outcome: | The proposed framework matches competitive baselines while improving throughput and reducing computational cost. |
A Study of Parameter Efficient Fine-tuning by Learning to Efficiently Fine-Tune (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning (PEFT) are limited due to the need for increased computational resources. |
| Approach: | They propose a method to learn PEFT parameters from data by projecting high dimensional parameters onto low dimensional parameter manifolds or identifying PEFT parametrically. |
| Outcome: | The proposed method can be used to identify PEFT parameters on GLUE tasks. |