Papers by Alessio Miaschi
Evaluating Large Language Models via Linguistic Profiling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) undergo extensive evaluation against various benchmarks collected in established leaderboards to assess their performance across multiple tasks. |
| Approach: | They propose a new evaluation methodology to test LLMs' sentence generation abilities under specific linguistic constraints. |
| Outcome: | The proposed evaluation methodology is based on the 'linguistic profiling' approach and is not intended to be a task-oriented evaluation. |
Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors (2025.findings-acl)
Copied to clipboard
Andrea Pedrotti, Michele Papucci, Cristiano Ciaccio, Alessio Miaschi, Giovanni Puccetti, Felice Dell’Orletta, Andrea Esuli
| Challenge: | Recent advances in Generative AI and Large Language Models (LLMs) have enabled the creation of highly realistic synthetic content, raising concerns about the potential for malicious use, such as misinformation and manipulation. |
| Approach: | They evaluate the resilience of state-of-the-art MGT detectors to linguistically informed adversarial attacks by using Direct Preference Optimization to shift the MGT style toward human-written text. |
| Outcome: | The proposed pipeline fine-tunes language models to shift the MGT style toward human-written text (HWT) it obtains generations more challenging to detect by current models, and shows that detectors can be easily fooled with relatively few examples, resulting in a significant drop in detecting performances. |
All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark (2025.findings-emnlp)
Copied to clipboard
Davide Testa, Giovanni Bonetta, Raffaella Bernardi, Alessandro Bondielli, Alessandro Lenci, Alessio Miaschi, Lucia Passaro, Bernardo Magnini
| Challenge: | MAIA evaluates visual language models on video-related tasks using reasoning categories that aim to disentangle language and vision relations. |
| Approach: | a native-italian benchmark is designed for fine-grained investigation of the reasoning abilities of visual language models on videos. |
| Outcome: | The benchmark evaluates visual language models on two aligned tasks and a visual question-answering task. |
Beyond the Spelling Miracle: Investigating Substring Awareness in Character-Blind Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Current Pre-trained Language Models are character-blind and struggle in spelling tasks . ability to identify characters and substrings within words is trivial but fundamental to robust language understanding. |
| Approach: | They propose to evaluate pre-trained language models with a binary substring identification task . they propose to examine where, when, and how a PLMs develop awareness of characters and substrings . |
| Outcome: | The proposed model identifies characters and substrings in a binary substring identification task. |
Evaluating Lexical Proficiency in Neural Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Natural Language Processing have been significantly shaped by the Deep Learning tsunami and the introduction of Transformer-based Language Models. |
| Approach: | They validated a framework to assess the lexical proficiency and linguistic creativity of Transformer-based Language Models (LMs) by analyzing performance of LMs of different sizes across tasks involving the generation, definition, and contextual usage of lexicals, neologisms, and nonce words. |
| Outcome: | The framework evaluates LMs in mono- and multilingual configuration across tasks involving the generation, definition, and contextual usage of lexicalized words, neologisms, and nonce words. |
Linguistic Knowledge Can Enhance Encoder-Decoder Models (If You Let It) (2024.lrec-main)
Copied to clipboard
| Challenge: | a recent study has shown that pre-trained NLMs can capture syntax- and semantic-sensitive phenomena. |
| Approach: | They investigate whether fine-tuning pre-trained models with linguistic knowledge improves their performance in a target task. |
| Outcome: | The proposed enhancements improve models' performance in a target task, the authors show . the study includes models in Italian and English, and multilingual models in English and Italian . |
Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation (2025.findings-naacl)
Copied to clipboard
Luca Moroni, Giovanni Puccetti, Pere-Lluís Huguet Cabot, Andrei Stefan Bejgu, Alessio Miaschi, Edoardo Barba, Felice Dell’Orletta, Andrea Esuli, Roberto Navigli
| Challenge: | Pretrained Large Language Models (LLMs) are mainly designed for the English language, but are not optimized for non-English languages due to language contamination or multilingual pretraining data. |
| Approach: | They propose a method that leverages neural mapping for vocabulary substitution to optimize LLMs for the Italian language. |
| Outcome: | The proposed method reduces token fertility by 25% and improves grounded alignment strategies. |
Linguistic Profiling of a Neural Language Model (2020.coling-main)
Copied to clipboard
| Challenge: | Neural Language Models (NLMs) have become a central component in NLP systems over the last few years, showing outstanding performance and improving the state-of-the-art on many tasks. |
| Approach: | They use a wide set of probing tasks to investigate how a Neural Language Model learns linguistic properties before and after a fine-tuning process. |
| Outcome: | The proposed model can encode a wide range of linguistic characteristics but loses this information when trained on specific downstream tasks. |