Papers by Giovanni Puccetti
M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection (2024.acl-long)
Copied to clipboard
Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | Large Language Models (LLMs) have brought an unprecedented surge in machine-generated text (MGT) societal implications are posed by their potential misuse and lack of training data. |
| Approach: | They propose a benchmark to detect machine-generated text in multiple languages . they use multi-domain and multi-generator corpus to identify which model generated the text . |
| Outcome: | The proposed benchmark compares a multilingual, multi-domain and multi-generator corpus of MGTs with human-generated content. |
Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors (2025.findings-acl)
Copied to clipboard
Andrea Pedrotti, Michele Papucci, Cristiano Ciaccio, Alessio Miaschi, Giovanni Puccetti, Felice Dell’Orletta, Andrea Esuli
| Challenge: | Recent advances in Generative AI and Large Language Models (LLMs) have enabled the creation of highly realistic synthetic content, raising concerns about the potential for malicious use, such as misinformation and manipulation. |
| Approach: | They evaluate the resilience of state-of-the-art MGT detectors to linguistically informed adversarial attacks by using Direct Preference Optimization to shift the MGT style toward human-written text. |
| Outcome: | The proposed pipeline fine-tunes language models to shift the MGT style toward human-written text (HWT) it obtains generations more challenging to detect by current models, and shows that detectors can be easily fooled with relatively few examples, resulting in a significant drop in detecting performances. |
Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI (2026.acl-long)
Copied to clipboard
Yuxia Wang, Rui Xing, Jonibek Mansurov, Giovanni Puccetti, Zhuohan Xie, Minh Ngoc Ta, Jiahui Geng, Jinyan Su, Mervat Abassy, Saadeldine Eletter, Kareem Elozeiri, Nurkhan Laiyk, Maiya Goloburda, Tarek Mahmoud, Raj Vardhan Tomar, Alexander Aziz, Ryuto Koike, Masahiro Kaneko, Artem Shelmanov, Ekaterina Artemova, Vladislav Mikhailov, Akim Tsvigun, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | Prior studies have shown that distinguishing text generated by Large Language Models from human-written text is challenging for humans and often no better than random guessing. |
| Approach: | They conduct extensive case study to determine the upper bound of human detection accuracy. |
| Outcome: | The findings challenge previous conclusions on human detection accuracy across languages and domains. |
The Invalsi Benchmarks: measuring the Linguistic and Mathematical understanding of Large Language Models in Italian (2025.coling-main)
Copied to clipboard
| Challenge: | Invalsi MATE is a high-resource language, but there are few benchmarks to evaluate generative Large Language Models in this language. |
| Approach: | They propose three benchmarks to evaluate language models on mathematical understanding in italian . they use the Invalsi tests, which are administered to students aged 6 to 18 in the italian school system . |
| Outcome: | The proposed benchmarks are based on the Invalsi tests and the Italian highschool math Olympics. |
Outlier Dimensions that Disrupt Transformers are Driven by Frequency (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Disabling only 48 out of 110M parameters in BERT-base drops its performance by nearly 30% on MNLI. |
| Approach: | They find that disabling only 48 out of 110M parameters in BERT-base drops its performance by nearly 30% on MNLI. |
| Outcome: | The proposed model outlier phenomenon is associated with the frequency of encoded tokens in pre-training data. |
Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation (2025.findings-naacl)
Copied to clipboard
Luca Moroni, Giovanni Puccetti, Pere-Lluís Huguet Cabot, Andrei Stefan Bejgu, Alessio Miaschi, Edoardo Barba, Felice Dell’Orletta, Andrea Esuli, Roberto Navigli
| Challenge: | Pretrained Large Language Models (LLMs) are mainly designed for the English language, but are not optimized for non-English languages due to language contamination or multilingual pretraining data. |
| Approach: | They propose a method that leverages neural mapping for vocabulary substitution to optimize LLMs for the Italian language. |
| Outcome: | The proposed method reduces token fertility by 25% and improves grounded alignment strategies. |