Papers by Paola Merlo
BLM-AgrF: A New French Benchmark to Investigate Generalization of Agreement in Neural Networks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing benchmarks for deep learning are based on massive amounts of data, which are effective in hiding some of the shallowness of the learned models. |
| Approach: | They propose to use a French dataset to learn the underlying rules of subject-verb agreement in sentences, inspired by visual IQ tests known as Raven’s Progressive Matrices. |
| Outcome: | The proposed method is based on Raven’s Progressive Matrices, a visual IQ test, and a dataset built using the BLM framework. |
Analogical Structure, Minimal Contextual Cues and Contrastive Distractors: Input Design for Sample-Efficient Linguistic Rule Induction (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent systems that use analogical reasoning require extensive knowledge engineering and even transformer-based models show inconsistent results across complexity levels. |
| Approach: | They propose to implement analogical structure, contrastive learning, and minimal contextual cue principles into large language models that train on English verb alternations. |
| Outcome: | The proposed models learn the alternation rules with high F1 on English verb alternations. |
Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings (D19-1)
Copied to clipboard
| Challenge: | Current unsupervised adversarial methods for mapping monolingual word embeddings require high quality parallel data, such as a dictionary or sentence-aligned corpus. |
| Approach: | They propose a weakly-supervised adversarial training method which improves performance of previous unsupervised adversaries for most languages. |
| Outcome: | The proposed method improves the performance of previous unsupervised adversarial methods for most languages and especially for typologically distant language pairs. |
Multi-Adversarial Learning for Cross-Lingual Word Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | Generative adversarial networks (GANs) have succeeded in inducing cross-lingual word embeddings without supervision, but their performance for distant languages is still not satisfactory. |
| Approach: | They propose a multi-adversarial method that induces the seed cross-lingual dictionary through multiple mappings, each induced to fit the mapping for one subspace. |
| Outcome: | The proposed method improves performance on bilingual lexicon induction and cross-lingual document classification on unsupervised bilingual linguistic induction. |
Blackbird language matrices (BLM), a new task for rule-like generalization in neural networks: Can Large Language Models pass the test? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to evaluate large language models for generalization lack generalization ability . current methods for evaluating LLMs are based on tests of human intelligence . |
| Approach: | They propose to use a language task to evaluate large language models' generalisation ability . they propose to ask LLMs to solve simple variants of the RAVEN IQ test . |
| Outcome: | The proposed task can be used to evaluate the generalisation ability of large language models . it shows that current generative models can handle the task in the sense that they understand instructions . |
BLM-s/lE: A structured dataset of English spray-load verb alternations for testing generalization in LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Current NLP models are achieving performance comparable to human capabilities on well-established benchmarks. |
| Approach: | They propose a BLM task to identify a missing element in a linguistic pattern from a list of candidate options based on a given matrix. |
| Outcome: | The proposed framework is based on the spray-load verb alternations in English as a case study. |