Papers by Katerina Margatina
Attention-based Conditioning Methods for External Knowledge Integration (P19-1)
Copied to clipboard
| Challenge: | Existing approaches for incorporating external knowledge into deep neural networks (RNNs) lexicon features are used to concatenate external information into the input or hidden network layers. |
| Approach: | They propose a method for conditioning external knowledge into RNNs by concatenating a representation of the external information to the input or hidden network layers. |
| Outcome: | The proposed approach improves performance on six benchmark datasets. |
Active Learning Principles for In-Context Learning with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning has significantly enhanced predictive performance in few-shot learning settings. |
| Approach: | They propose to use pool-based Active Learning to identify the most informative demonstrations for few-shot learning over a single iteration to identify best demonstrations. |
| Outcome: | The proposed model outperforms all other methods, including random sampling, in the analysis of 24 classification and multi-choice tasks. |
CONFETTI: Conversational Function-Calling Evaluation Through Turn-Level Interactions (2025.acl-long)
Copied to clipboard
Tamer Alkhouli, Katerina Margatina, James Gung, Raphael Shu, Claudia Zaghi, Monica Sunkara, Yi Zhang
| Challenge: | Current benchmarks lack comprehensive assessment of large language models in complex conversational scenarios. |
| Approach: | They introduce a conversational benchmark to evaluate function-calling capabilities of large language models. |
| Outcome: | The proposed benchmark evaluates the function-calling capabilities and response quality of large language models. |
Understanding the Role of Input Token Characters in Language Models: How Does Information Loss Affect Performance? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models capture semantic and syntactic information, but no study has examined how information loss in input token characters affects their performance. |
| Approach: | They address this gap by pre-training language models using small subsets of token characters. |
| Outcome: | The proposed model retains 90% and 77% of the full-token model in standard NLU benchmarks and probing tasks even under extreme settings. |
On the Limitations of Simulating Active Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Active learning (AL) is a human-and-model-in-the-loop paradigm that iteratively selects informative unlabeled data for human annotation. |
| Approach: | They propose to simulate active learning by using an already labeled dataset as the pool of unlabeled data. |
| Outcome: | The proposed model-in-the-loop paradigm can be used to perform experiments with human annotations on-the fly. |
Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views (2023.eacl-main)
Copied to clipboard
| Challenge: | Temporal concept drift is a problem of data changing over time. |
| Approach: | They benchmark 11 pretrained masked language models on a series of tests to evaluate temporal concept drift. |
| Outcome: | The proposed framework evaluates 11 pretrained masked language models on a series of tests . it aims to reveal how robust an MLM is over time and provide a signal in case it has become outdated . |
Challenges and Strategies in Cross-Cultural NLP (2022.acl-long)
Copied to clipboard
Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, Anders Søgaard
| Challenge: | Various efforts have been made to accommodate linguistic diversity and serve speakers of many different languages. |
| Approach: | They propose a framework to examine cultural differences in NLP to better serve users . they argue that cultural knowledge, preferences and values can affect NLP practices . |
| Outcome: | The proposed framework examines how cultural knowledge, preferences and values can affect NLP practices. |
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning (2022.acl-short)
Copied to clipboard
| Challenge: | Recent active learning approaches in NLP use off-the-shelf pretrained language models (LMs) . a poor training strategy can be catastrophic for AL, authors argue . |
| Approach: | They propose to first adapt the pretrained LM to the target task and then use it for AL. |
| Outcome: | The proposed approach provides substantial data efficiency improvements compared to the standard fine-tuning approach. |
Active Learning by Acquiring Contrastive Examples (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using uncertainty and diversity sampling, active learning acquisition functions select difficult and diverse data points from a pool of unlabeled data. |
| Approach: | They propose an active learning acquisition function that selects contrastive examples from unlabeled data. |
| Outcome: | The proposed approach performs better or equal to the best performing baseline on all tasks, on both in-domain and out-of-domain data. |
Explicit Trait Inference for Multi-Agent Coordination (2026.acl-long)
Copied to clipboard
| Challenge: | Large language model (LLM) based multi-agent systems (MAS) show promise on complex tasks but remain prone to failures of coordination, such as goal drift, error cascades, and misaligned behaviors. |
| Approach: | They propose a psychologically grounded method for improving coordination using Explicit Trait Inference (ETI) ETI enables agents to infer and track partner characteristics along two established psychological dimensions—warmth (e.g., trust) and competence (eg. skill) |
| Outcome: | The proposed method reduces payoff loss in controlled and realistic multi-agent settings by 45–77% and improves performance by 3–29% depending on scenario and model. |
Investigating Multi-source Active Learning for Natural Language Inference (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies often assume that training and test data are drawn from the same distribution. |
| Approach: | They propose to apply active learning to unlabelled data pools to test for learning and generalisation. |
| Outcome: | The proposed strategies outperform random selection and outperformed hard-to-learn data on the task of natural language inference. |
Domain Adversarial Fine-Tuning as an Effective Regularizer (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing fine-tuning techniques can degrade general-domain representations . however, fine-timing can lead to catastrophic forgetting of knowledge . |
| Approach: | They propose a new regularization technique that complements the task-specific loss used during fine-tuning with an adversarial objective. |
| Outcome: | Empirical results show that AFTER improves performance on various natural language understanding tasks compared to standard fine-tuning. |
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) is widely used in natural language processing for self-supervised learning of text representations. |
| Approach: | They propose to use token-level classification tasks as main pretraining objectives instead of Masked language modeling (MLM) . Empirical results show that pretraining a model with 41% of the BERT-BASE’s parameters, BERT MEDIUM results in only a 1% drop in GLUE scores with their best objective. |
| Outcome: | Empirical results show that the proposed methods achieve comparable or better performance to MLM using a BERT-BASE architecture. |