Papers by George Chrysostomou
On the Impact of Temporal Concept Drift on Model Explanations (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Explanation faithfulness of model predictions is typically evaluated on held-out data from the same temporal distribution as the training data. |
| Approach: | They examine the impact of temporal variation on model explanations extracted by eight feature attribution methods and three select-then-predict models across six text classification tasks. |
| Outcome: | The proposed method shows the most robust faithfulness scores across datasets and in asynchronous settings. |
An Empirical Study on Explanations in Out-of-Domain Settings (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work in Natural Language Processing has focused on extracting faithful explanations . yet, little is known about how post-hoc explanations perform in out-of-domain settings . |
| Approach: | They propose to use a random baseline to evaluate out-of-domain post-hoc explanation faithfulness . they suggest select-then-predict models demonstrate comparable predictive performance in out- of-domain settings to full-text trained models. |
| Outcome: | The proposed models perform better in out-of-domain settings than full-text models. |
Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained transformer-based language models have demonstrated state-of-the-art predictive performance when adapted into a range of language understanding tasks. |
| Approach: | They propose to use salient information extracted a priori from training data to complement the task-specific information learned by the model during fine-tuning on a downstream task. |
| Outcome: | The proposed model can provide more faithful explanations across four different feature attribution methods compared to vanilla BERT. |
Compressing Language Models for Specialized Domains (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) excel at tasks across diverse domains, yet require substantial computational resources during inference. |
| Approach: | They propose a calibration method to improve the in-domain performance of compressed LMs in a post-training setting. |
| Outcome: | The proposed method outperforms existing methods on domain-specific tasks while preserving general performance. |
Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that attention cannot be considered as a faithful explanation across encoders and tasks. |
| Approach: | They propose a new family of Task-Scaling mechanisms that scale attention weights across tasks and two attention mechanisms. |
| Outcome: | The proposed models improve explanation faithfulness across two attention mechanisms, five encoders and five text classification datasets without sacrificing predictive performance. |
Self-calibration for Language Model Quantization and Pruning (2025.naacl-long)
Copied to clipboard
| Challenge: | Quantization and pruning are fundamental approaches for model compression, but they require large computational resources. |
| Approach: | They propose to use model calibration data to generate synthetic calibrations to improve model performance. |
| Outcome: | The proposed method outperforms other methods using real data in a post-training setting. |
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) is widely used in natural language processing for self-supervised learning of text representations. |
| Approach: | They propose to use token-level classification tasks as main pretraining objectives instead of Masked language modeling (MLM) . Empirical results show that pretraining a model with 41% of the BERT-BASE’s parameters, BERT MEDIUM results in only a 1% drop in GLUE scores with their best objective. |
| Outcome: | Empirical results show that the proposed methods achieve comparable or better performance to MLM using a BERT-BASE architecture. |