Papers by Marek Suppa
RAGthoven: A Configurable Toolkit for RAG-enabled LLM Experimentation (2025.coling-demos)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have significantly altered the landscape of Natural Language Processing (NLP), but their use as a baseline method has not been extensive. |
| Approach: | They propose a tool for automatic evaluation of RAG-based pipelines that provides a simple yet powerful abstraction. |
| Outcome: | The proposed tool provides an automatic evaluation of RAG-based pipelines. |
Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark (2024.naacl-long)
Copied to clipboard
Stephen Mayhew, Terra Blevins, Shuheng Liu, Marek Suppa, Hila Gonen, Joseph Marvin Imperial, Börje Karlsson, Peiqin Lin, Nikola Ljubešić, Lester James Miranda, Barbara Plank, Arij Riabi, Yuval Pinter
| Challenge: | In named entity recognition, the majority of annotation efforts are centered on English, and cross-lingual transfer performance remains brittle. |
| Approach: | They propose to develop gold-standard named entity recognition benchmarks in many languages using a cross-lingual consistent schema. |
| Outcome: | The proposed benchmarks will be released to the public in 2022 . they will provide baselines on in-language and cross-lingual learning settings. |
SlovakSum: A Large Scale Slovak Summarization Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets with hundreds and thousands of documents are mainly in the English language, but the available data is small or non-existent. |
| Approach: | They propose to use a large Slovak news summarization dataset to evaluate its performance . the dataset contains headlines, short abstracts, and full source text . |
| Outcome: | The proposed dataset is compared with a standard ROUGE metric and a mT5 model to evaluate its performance. |
Trojsten Benchmark: Evaluating LLM Problem-Solving in Slovak STEM Competition Problems (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have been used for grading open-ended responses and providing feedback beyond traditional methods. |
| Approach: | They propose a Slovak-language dataset and a rubric-based LLM grading framework . they quantify multistep reasoning performance by difficulty and show consistency under difficult items . |
| Outcome: | The proposed model outperforms existing models on Slovak-language competition problems . the model shows consistent underperformance on harder items and language sensitivity . |
skLEP: A Slovak General Language Understanding Benchmark (2025.findings-acl)
Copied to clipboard
Marek Suppa, Andrej Ridzik, Daniel Hládek, Tomáš Javůrek, Viktória Ondrejová, Kristína Sásiková, Martin Tamajka, Marian Simko
| Challenge: | skLEP is the first comprehensive benchmark specifically designed for evaluating Slovak natural language understanding models. |
| Approach: | They introduce a benchmark specifically designed for evaluating Slovak natural language understanding models. |
| Outcome: | The proposed benchmark covers nine tasks that span token-level, sentence-pair, document-level tasks. |
A Summarization Dataset of Slovak News Articles (2020.lrec-1)
Copied to clipboard
| Challenge: | a number of studies on document summarization have focused on the English language . however, most of the work on this task is done on English datasets . |
| Approach: | They propose to use a news site's ROUGE metric to adapt it to Slovak texts . they propose to introduce a large-scale news-based summarization dataset . |
| Outcome: | The proposed approach is better suited for Slovak texts than the dominant ROUGE metric. |
SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation (2026.acl-long)
Copied to clipboard
| Challenge: | Slovak embeddings are core infrastructure for semantic search, retrieval-augmented generation (RAG), clustering, and classification. |
| Approach: | They propose a MTEB-style text embedding benchmark for Slovak, a low-resource West Slavic language . they use 31 datasets across 7 task types to evaluate the performance of the models . |
| Outcome: | The proposed model achieves competitive performance with proprietary APIs while remaining locally deployable for RAG . the model is based on 31 datasets across 7 task types and is 4 the depth of existing benchmark for Slovak . |
Cost-effective Deployment of BERT Models in Serverless Environment (2021.naacl-industry)
Copied to clipboard
| Challenge: | a large upfront infrastructure investment makes machine learning models difficult to deploy . however, serverless architectures have strict limits on the size of the deployment package . |
| Approach: | They propose to fine-tune BERT-style models on proprietary datasets for tasks . they use knowledge distillation to obtain models that are tuned for a specific domain . |
| Outcome: | The proposed model deployments report acceptable latency levels and cost-effectiveness without infrastructure overhead. |