Papers by Duccio Pappadopulo
Distillation of encoder-decoder transformers for sequence labelling (2023.findings-eacl)
Copied to clipboard
| Challenge: | despite the strong trend in NLP to explore the use of large language models, there is still limited work evaluating prompting and decoding mechanisms for SL tasks. |
| Approach: | They propose a hallucination-free framework for sequence tagging that is especially suited for distillation. |
| Outcome: | The proposed framework performs well across multiple sequence labelling datasets and in a few-shot learning scenario. |
Disentangling Online Chats with DAG-structured LSTMs (2021.starsem-1)
Copied to clipboard
| Challenge: | a number of messaging systems allow fast and synchronous textual communication but they often have a more complicated structure in which independent sub-conversations are interwoven with one another. |
| Approach: | They propose a model that can handle directed acyclic dependencies and integrates structured information into the conversation. |
| Outcome: | The proposed model achieves state-of-the-art status on the task of recovering reply-to relations and is competitive on other disentanglement metrics. |
Non-contrastive sentence representations via self-supervision (2024.findings-naacl)
Copied to clipboard
| Challenge: | Text embeddings are an important tool for a variety of NLP tasks. |
| Approach: | They compare sample contrastive methods with the standard baseline for contrastive sentence embeddings, SimCSE, and a class of self-supervised non-contrastive loss functions and methods. |
| Outcome: | The proposed methods outperform the standard baseline for contrastive sentence embeddings, SimCSE, on downstream tasks without auxiliary loss functions. |
Improving Instruct Models for Free: A Study on Partial Adaptation (2025.emnlp-main)
Copied to clipboard
Ozan Irsoy, Pengxiang Cheng, Jennifer L Chen, Daniel Preotiuc-Pietro, Shiyue Zhang, Duccio Pappadopulo
| Challenge: | Instruct models are deemed superior and more usable but can be eroded by instruction tuning . a recent study shows that instruct models are better at following instructions than base models . |
| Approach: | They scale down the strength of instruction tuning to improve model performance . they show that reducing instruction tuning results in material improvement . |
| Outcome: | The proposed model improves on a few-shot in-context learning benchmark . but it loses some degree of its in-training ability . |