Papers by Jakub Simko
MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection Benchmark (2023.emnlp-main)
Copied to clipboard
Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, Maria Bielikova
| Challenge: | MULTITuDE benchmarks lack authentic and machine-generated text in languages other than English . defining characteristic of new generation of LLMs is increased quality of text . |
| Approach: | They propose a benchmarking dataset for multilingual machine-generated text detection that compares detectors with authentic and machine-generated texts in 11 languages. |
| Outcome: | The proposed dataset compares detectors with zero-shot and fine-tuned detectors in 11 languages. |
A Rigorous Evaluation of LLM Data Generation Strategies for Low-Resource Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used to generate synthetic textual data for training smaller specialized models. |
| Approach: | They evaluate the performance of large language models and their generation strategies in 11 different languages using 3 NLP tasks and 4 open-source LLMs. |
| Outcome: | The proposed generation strategies and their combinations yield strong results across 11 languages, including several extremely low-resource ones. |
Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable multilingual capabilities, making them promising tools in both high- and low-resource languages. |
| Approach: | They use a multilingual LLM to generate synthetic datasets covering 11 languages and 4 classification tasks and use them to train smaller models. |
| Outcome: | The proposed model outperforms the large generator in low-resource languages and tasks. |
Multilingual Previously Fact-Checked Claim Retrieval (2023.emnlp-main)
Copied to clipboard
Matúš Pikuliak, Ivan Srba, Robert Moro, Timo Hromadka, Timotej Smoleň, Martin Melišek, Ivan Vykopal, Jakub Simko, Juraj Podroužek, Maria Bielikova
| Challenge: | Fact-checkers are often hampered by the sheer amount of online content that needs to be fact-checked. |
| Approach: | They propose a multilingual dataset for previously fact-checked claim retrieval using social media posts and 206k fact- checks in 39 languages written by professional fact- checkers. |
| Outcome: | The proposed method improves on the previously unsupervised method and shows that a multilingual dataset has its complexities and needs to be carefully interpreted. |
Fighting Randomness with Randomness: Mitigating Optimisation Instability of Fine-Tuning using Delayed Ensemble and Noisy Interpolation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Despite the success of fine-tuning, it still displays model performance instability, especially with limited data. |
| Approach: | They propose a new mitigation strategy that leverages the strengths of ensembling, noise regularisation and model interpolation while retaining computational efficiency. |
| Outcome: | The proposed mitigation strategy outperforms the best performing mitigation strategy (Ensemble) while using only a fraction of its cost. |
Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentation (2024.acl-long)
Copied to clipboard
| Challenge: | generative large language models (LLMs) have found their application in data augmentation tasks, where small numbers of text samples are paraphrased and then used to fine-tune downstream models. |
| Approach: | They propose to use taboo words, hints by previous outlier solutions, and chaining on previous outliest solutions to augment text datasets as part of instructions to LLMs augmenting text dataset. |
| Outcome: | The proposed methods increase diversity of generated texts, but performance is highest with hints. |
MultiCW: A Large-Scale Balanced Benchmark Dataset for Training Robust Check-Worthiness Detection Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are beginning to reshape how media professionals verify information, but support for detecting check-worthy claims remains limited. |
| Approach: | They propose a multilingual benchmark for check-worthy claim detection spanning 16 languages, six topical domains, and two writing styles. |
| Outcome: | The proposed model outperforms zero-shot LLMs on claim classification and strong generalization across languages, domains, and styles. |
Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | generative large language models are increasingly used for data augmentation tasks . text samples are mostly selected randomly and a comprehensive overview of other sample selection strategies is lacking. |
| Approach: | They compare random sample selection strategies and random sample sampling strategies to evaluate their effects in a low-resource setting. |
| Outcome: | The proposed model performance improvements are compared with other sample selection strategies. |
ChatGPT to Replace Crowdsourcing of Paraphrases for Intent Classification: Higher Diversity and Comparable Model Robustness (2023.emnlp-main)
Copied to clipboard
| Challenge: | generative large language models (LLMs) are replacing human workers for some tasks . crowdsourcing has several downsides: 1) the workforce is costly, 2) output quality is difficult to achieve, and 3) there are overheads related to the design and organization of the process. |
| Approach: | They investigate whether ChatGPT-created paraphrases are more diverse and robust . they use a crowdsourcing tool to collect training or validation examples . |
| Outcome: | The proposed models are more diverse and robust than the existing models. |
RoSE: Round-robin Synthetic Data Evaluation for Selecting LLM Generators without Human Test Sets (2026.eacl-long)
Copied to clipboard
| Challenge: | Current large language models (LLMs) are powerful generators of synthetic data, which are used for training smaller, more efficient models. |
| Approach: | They propose a proxy metric for selecting the best LLM generator without human annotations and a metric that measures the performance of a model. |
| Outcome: | The proposed proxy metric outperforms intrinsic heuristics and comes within 0.76 percentage points of the optimal generator baseline. |
LLMs vs Established Text Augmentation Techniques for Classification: When do the Benefits Outweight the Costs? (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have compared LLM-based augmentations with established methods, but the results are contradictory. |
| Approach: | They compare the performance of LLM-based augmentation methods with established ones . they found that LLMs are worthy of deployment only when very small number of seeds is used . |
| Outcome: | The proposed methods are worthy of deployment only when very small number of seeds is used. |
Authorship Obfuscation in Multilingual Machine-Generated Text Detection (2024.findings-emnlp)
Copied to clipboard
Dominik Macko, Robert Moro, Adaku Uchendu, Ivan Srba, Jason Lucas, Michiharu Yamashita, Nafis Irtiza Tripto, Dongwon Lee, Jakub Simko, Maria Bielikova
| Challenge: | Recent advances in Language Modeling have birthed Large Language Models (LLMs), which exhibit significant improvements, including the ability to generate texts easily misconstrued as humanwritten. |
| Approach: | They compare authorship obfuscation methods against machine-generated text (MGT) in 11 languages and analyze their performance against 37 well-known AO methods. |
| Outcome: | The proposed methods can cause evasion of detection in all languages, with homoglyph attacks particularly successful. |