Papers by Alessandro Sordoni
Increasing Robustness to Spurious Correlations using Forgettable Examples (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural NLP models often exploit spurious correlations to perform tasks. minority examples have been shown to increase the out-of-distribution generalization of pre-trained language models. |
| Approach: | They propose to use example forgetting to find minority examples without prior knowledge of spurious correlations in the dataset. |
| Outcome: | The proposed approach improves out-of-distribution generalization on minorities . it shows that minority examples are more robust on challenging datasets . |
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle (2021.naacl-main)
Copied to clipboard
| Challenge: | Failing to capture the structure of input language could lead to generalization problems and over-parametrization. |
| Approach: | They propose a new syntax-aware language model that explicitly models the structure with an incremental parser and maintains the conditional probability setting of a standard language model. |
| Outcome: | The proposed model can achieve strong results in language modeling, parsing, and syntactic generalization tests while using fewer parameters than other models. |
Recursive Top-Down Production for Sentence Generation with Latent Trees (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Various studies have shown that incorporating syntactic structures into recursive encoders can be beneficial for various natural language tasks. |
| Approach: | They propose a dynamic programming algorithm that marginalises over latent binary tree structures with N leaves to train a recursive neural function. |
| Outcome: | The proposed model outperforms previous models on the LENGTH split and English question formation tasks on the Multi30k dataset. |
Combining Parameter-efficient Modules for Task-level Generalisation (2023.eacl-main)
Copied to clipboard
| Challenge: | A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. |
| Approach: | They propose a modular neural network where a subset of latent skills is associated with a parameter-efficient model adapter. |
| Outcome: | The proposed model improves sample efficiency and few-shot generalisation in supervised learning compared to baselines. |
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance (P18-1)
Copied to clipboard
| Challenge: | Compared to traditional shift-reduce parsing schemes, our approach is free from the potentially disastrous compounding error. |
| Approach: | They propose a model that predicts a scalar for each split position in a sentence and then determines the topology of grammar tree based on syntactic distances. |
| Outcome: | The proposed model achieves the state-of-the-art single model F1 score of 92.1 on PTB and 86.4 on CTB dataset, surpassing the previous single model results by a large margin. |
Unsupervised Dependency Graph Network (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work has identified properties of pretrained self-attention models that mirror those of dependency parse structures. |
| Approach: | They propose a model that encourages attention heads to model different dependency relations from raw corpora and a masked language modeling task. |
| Outcome: | The proposed model can induce dependency structures from raw corpora and the masked language modeling task without gold POS tags and any external information. |
Better Language Model with Hypernym Class Prediction (2022.acl-long)
Copied to clipboard
| Challenge: | Class-based language models (LMs) have been devised to address context sparsity in n-gram LMs for decades. |
| Approach: | They propose to use class-based prediction to improve generalization for rare words by annealing from predicting the class to token prediction during training. |
| Outcome: | The proposed model improves perplexity without sacrificing performance on rare words. |
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment (2025.acl-long)
Copied to clipboard
Jean-Philippe Corbeil, Amin Dada, Jean-Michel Attendu, Asma Ben Abacha, Alessandro Sordoni, Lucas Caccia, Francois Beaulieu, Thomas Lin, Jens Kleesiek, Paul Vozila
| Challenge: | Large language models such as GPT-4 have limited their deployment in clinical settings . a novel framework for adapting SLMs into high-performing clinical models is needed . |
| Approach: | They propose a framework for adapting large language models into high-performing clinical models . they pre-instruct experts on relevant medical and clinical corpora and model merging . |
| Outcome: | The proposed framework outperforms the existing model on the CLUE+ benchmark on medical entities and radiology reports. |
Self-training with Few-shot Rationalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work focused on training largescale and complex neural network models, but they are opaque in terms of their decision-making process. |
| Approach: | They propose a multi-task teacher-student framework for self-training pre-trained language models with limited task-specific labels and annotated rationales. |
| Outcome: | The proposed model improves performance in low-resource settings by making it aware of its rationalized predictions. |
Measuring Morphological Fusion Using Partial Information Decomposition (2022.coling-1)
Copied to clipboard
| Challenge: | agglutinative and fusional languages have a systematic relationship between meaning and form, but are less systematic when it comes to morphological relations. |
| Approach: | They propose a mathematically precise way of characterizing morphological systems using partial information decomposition. |
| Outcome: | The proposed framework decomposes mutual information into three components: unique, redundant, and synergistic information. |
Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evidence suggests that pre-trained Transformers encode commonsense knowledge . however, the extent to which this knowledge is acquired is unclear . |
| Approach: | They inject verbalized knowledge into pre-training minibatches and evaluate generalization . they find generalization does not improve over the course of pre- training from scratch . |
| Outcome: | The proposed model generalizes to supported inferences after pre-training on the injected knowledge. |
MedRiskEval: Medical Risk Evaluation Benchmark of Language Models, On the Importance of User Perspectives in Healthcare Settings (2026.eacl-industry)
Copied to clipboard
Jean-Philippe Corbeil, Minseon Kim, Maxime Griot, Sheela Agarwal, Alessandro Sordoni, Francois Beaulieu, Paul Vozila
| Challenge: | Existing risk evaluations focused on general safety benchmarks, resulting in role-dependent vulnerabilities in real-world medical and clinical deployments. |
| Approach: | They propose a patient-oriented dataset called PatientSafetyBench that evaluates a variety of open- and closed-source LLMs. |
| Outcome: | The proposed benchmark examines medical risks from 466 open- and closed-source LLMs across 5 risk categories. |
Understanding by Understanding Not: Modeling Negation in Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Negation is a core construction in natural language, but state-of-the-art pre-trained language models often handle it incorrectly. |
| Approach: | They propose to augment language modeling objective with unlikelihood objective based on negated generic sentences from a raw text corpus. |
| Outcome: | The proposed approach reduces the top 1 error rate to 4% on negated LAMA dataset and improves on negating NLI benchmarks. |
Linguistic Dependencies and Statistical Dependence (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that pairs of words that occur together are likely to stand in a linguistic dependency. |
| Approach: | They propose to use large pretrained language models to compute contextualized estimates of the pointwise mutual information between words (CPMI) |
| Outcome: | The proposed model maximizes CPMI and achieves an unlabelled undirected attachment score of 0.5 . |
Exploring and Predicting Transferability across NLP Tasks (2020.emnlp-main)
Copied to clipboard
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
| Challenge: | Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. |
| Approach: | They conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems. |
| Outcome: | The proposed model can improve performance even with low-data source tasks that differ substantially from the target task. |