Papers by Giorgos Stamou
GreekMMLU: A Native-Sourced Multitask Benchmark for Evaluating Language Models in Greek (2026.findings-acl)
Copied to clipboard
Yang Zhang, Mersin Konomi, Christos Xypolopoulos, Konstantinos Divriotis, Konstantinos Skianis, Giannis Nikolentzos, Giorgos Stamou, Guokan Shang, Michalis Vazirgiannis
| Challenge: | Existing evaluation benchmarks for large language models are limited for Greek . Existing datasets are often machine-translated from English, failing to capture Greek linguistic and cultural characteristics. |
| Approach: | They propose a native-sourced benchmark for massive multitask language understanding in Greek . they publicize 16,857 samples and reserve 4,948 samples for a private leaderboard . |
| Outcome: | The proposed model is based on 21,805 multiple-choice questions across 45 subject areas . the model is publicly released and reserved for a private leaderboard . |
Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized product recommenders, but their susceptibility to adversarial manipulations is difficult to detect. |
| Approach: | They propose to use large language models to investigate cognitive biases as adversarial strategies in product research using LLMs. |
| Outcome: | The proposed approach is the first to tap into human psychological principles, making such manipulations hard to detect. |
Don’t Erase, Inform! Detecting and Contextualizing Harmful Language in Cultural Heritage Collections (2025.acl-long)
Copied to clipboard
| Challenge: | Cultural Heritage metadata can contain outdated or offensive terms that reflect historical cultural and societal norms. |
| Approach: | They propose an AI-powered tool that detects offensive terms in CH metadata . the tool has processed over 7.9 million records and provides contextual insights . |
| Outcome: | The proposed tool has processed over 7.9 million records and provides contextual insights . it pairs biased language with contextual information and suggestions for appropriate usage . |
Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors (2023.findings-acl)
Copied to clipboard
George Filandrianos, Edmund Dervakos, Orfeas Menis Mastromichalakis, Chrysoula Zerva, Giorgos Stamou
| Challenge: | Existing explanations for classifiers are counterfactual or contrastive . lack of universal ground truth for counterf actual edits hinders their evaluation . |
| Approach: | They propose a back translation-inspired evaluation methodology that utilises earlier outputs of the explainer as ground truth proxies to investigate the consistency of explainers. |
| Outcome: | The proposed method can provide valuable insights into the behaviour of predictor and explainer models and infer patterns that would otherwise be obscured. |
RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for prompting Large Language Models (LLMs) are lacking in advanced reasoning skills. |
| Approach: | They propose a method that generates and utilizes contextually reconstructed sentences to generate few-shot exemplars. |
| Outcome: | The proposed method significantly improves the performance of large language models in vertical and lateral thinking tasks, surpassing traditional exemplar selection strategies across a variety of few-shot settings. |
Towards Explainable Evaluation of Language Models on the Semantic Similarity of Visual Concepts (2022.coling-1)
Copied to clipboard
Maria Lymperaiou, George Manoliadis, Orfeas Menis Mastromichalakis, Edmund G. Dervakos, Giorgos Stamou
| Challenge: | Recent advances in NLP research have focused on robustness and explainability issues of their evaluation strategies. |
| Approach: | They propose to use pre-trained transformers to evaluate semantic similarity for visual vocabularies . they propose to provide explainable metrics for understanding the quality of retrieved instances . |
| Outcome: | The proposed metrics highlight inabilities of widely used evaluation methods and highlight weaknesses in learned linguistic representations. |
PAKTON: A Multi-Agent Framework for Question Answering in Long Legal Agreements (2025.emnlp-main)
Copied to clipboard
| Challenge: | Contract review is a complex and time-intensive task that typically requires legal expertise. |
| Approach: | a new open-source contract review framework is designed to handle complexities of contract analysis . PAKTON is a retrieval-augmented generation framework with plug-and-play capabilities . |
| Outcome: | The open-source framework outperforms models in predictive accuracy, retrieval performance, explainability, completeness, and grounded justifications. |
GreekBART: The First Pretrained Greek Sequence-to-Sequence Model (2024.lrec-main)
Copied to clipboard
Iakovos Evdaimon, Hadi Abdine, Christos Xypolopoulos, Stamatis Outsios, Michalis Vazirgiannis, Giorgos Stamou
| Challenge: | Transfer learning has revolutionized the fields of Computer Vision and Natural Language Processing. |
| Approach: | They introduce a new language model, GreekBART, that is based on a BART-base architecture. |
| Outcome: | The proposed model outperforms BERT, GPT and other transformer-based models on discriminative tasks. |
Large Language Models and Multimodal Retrieval for Visual Word Sense Disambiguation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Visual word sense disambiguation (VWSD) is a challenging task involving multiple candidates . context given for an ambiguous word is minimal, most often limited to a single word . |
| Approach: | They propose to use large language models to enhance given phrases and resolve ambiguity related to the target word. |
| Outcome: | The proposed frameworks improve the image representation of ambiguous words among candidates and achieve competitive ranking results. |
”I Never Said That”: A dataset, taxonomy and baselines on response clarity classification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Equivocation and ambiguity in public speech are well-studied discourse phenomena . a new taxonomy aims to detect and classify response clarity in political interviews . |
| Approach: | They propose a taxonomy that uses Large Language Models and human annotations to detect and classify response clarity in political interviews. |
| Outcome: | The proposed taxonomy combines ChatGPT and human annotations to identify clarity in political questions . it provides a fine-grained taxonomies for evasion techniques related to unclear, ambiguous responses . |
Pitfalls of Scale: Investigating the Inverse Task of Redefinition in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable results in several linguistic, reasoning and knowledge retrieval tasks. |
| Approach: | They propose to scale Large Language Models (LLMs) to scale up to reveal potential reasoning gaps as LLMs scale up. |
| Outcome: | The proposed redefinition task shows that model performance degrades with scale, and false confidence rises. |
Puzzle Solving using Reasoning of Large Language Models: A Survey (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated their logical reasoning abilities across various domains. |
| Approach: | They propose to divide puzzles into rule-based and rule-less categories and critically assess LLMs' performance through various methodologies. |
| Outcome: | The proposed models have demonstrated capabilities in deductive reasoning and inductive reasoning, but they face limitations in inductive thinking. |
Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Terms (2025.emnlp-main)
Copied to clipboard
| Challenge: | ailsntua researchers examine whether machine translation systems exhibit gender biases that reinforce societal stereotypes. |
| Approach: | They propose a probability-based metric to evaluate gender bias by analyzing aggregated model responses. |
| Outcome: | The proposed metric evaluates whether translations in Greek and French align with or diverge from societal stereotypes. |