Papers by Ozan Irsoy
Collective Entity Disambiguation with Structured Gradient Tree Boosting (N18-1)
Copied to clipboard
| Challenge: | Existing work on structured gradient tree boosting for collective entity disambiguation is limited to regular classification or regression problems. |
| Approach: | They propose a structured learning model that uses gradient tree boosting to disambiguate named entities in a document. |
| Outcome: | The proposed model outperforms the previous state-of-the-art neural system by near 1% absolute accuracy on the popular AIDA-CoNLL dataset. |
Distillation of encoder-decoder transformers for sequence labelling (2023.findings-eacl)
Copied to clipboard
| Challenge: | despite the strong trend in NLP to explore the use of large language models, there is still limited work evaluating prompting and decoding mechanisms for SL tasks. |
| Approach: | They propose a hallucination-free framework for sequence tagging that is especially suited for distillation. |
| Outcome: | The proposed framework performs well across multiple sequence labelling datasets and in a few-shot learning scenario. |
Semantic Role Labeling as Syntactic Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using propBank-style semantic role labeling, we reduce the task to syntactic dependency parsing. |
| Approach: | They propose to convert SRL annotations into dependency tree representations through joint labels that permit highly accurate recovery back to the original format. |
| Outcome: | The proposed scheme reduces the task of (span-based) PropBank-style semantic role labeling to syntactic dependency parsing. |
Academics Can Contribute to Domain-Specialized Language Models (2024.emnlp-main)
Copied to clipboard
Mark Dredze, Genta Winata, Prabhanjan Kambadur, Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, David Rosenberg, Sebastian Gehrmann
| Challenge: | Commercially available models dominate academic leaderboards, focusing on creating and adapting general-purpose models . however, general- purpose models often underperform in specialized domains, and domain-specific models yield superior results. |
| Approach: | They advocate for a renewed focus on developing and evaluating domain- and task-specific models . they advocate for an adapted or adapted model that can be used to improve academic leaderboard standings . |
| Outcome: | The proposed model can do well on professional and linguistic examinations, college-level knowledge questions, and collections of reasoning tasks. |
MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies (2023.acl-long)
Copied to clipboard
| Challenge: | Autoregressive language models are trained by minimizing the cross-entropy of the model distribution Q relative to the data distribution P. However, these systems still struggle in many openended generation settings, where they are asked to produce a long text following a short prompt. |
| Approach: | They propose to combine forward and reverse cross-entropy to train autoregressive language models by minimizing the cross-Entropy of the model distribution Q relative to the data distribution P. |
| Outcome: | The proposed model overgeneralizes and produces non-human-like text without complex decoding strategies. |
Improving Instruct Models for Free: A Study on Partial Adaptation (2025.emnlp-main)
Copied to clipboard
Ozan Irsoy, Pengxiang Cheng, Jennifer L Chen, Daniel Preotiuc-Pietro, Shiyue Zhang, Duccio Pappadopulo
| Challenge: | Instruct models are deemed superior and more usable but can be eroded by instruction tuning . a recent study shows that instruct models are better at following instructions than base models . |
| Approach: | They scale down the strength of instruction tuning to improve model performance . they show that reducing instruction tuning results in material improvement . |
| Outcome: | The proposed model improves on a few-shot in-context learning benchmark . but it loses some degree of its in-training ability . |