Papers by Tom Sherborne
Compositional Generalisation for Explainable Hate Speech Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Hate speech detection models struggle to generalise beyond their training data . dataset biases and the use of sentence-level labels fail to teach the underlying structure of hate speech. |
| Approach: | They propose to use a dataset to train models with fine-grained, span-level annotations . they find that combinations of expressions that deviate from those seen during training are difficult to detect . |
| Outcome: | The proposed model can generalise to a dataset with equal frequency across all contexts while achieving state-of-the-art performance on the human-sourced PLEAD. |
Meta-Learning a Cross-lingual Manifold for Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Recent work has found success with machine translation or zero-shot methods . however, these approaches can struggle to model how native speakers ask questions . |
| Approach: | They propose a meta-learning algorithm to leverage minimal annotated examples in new languages for few-shot cross-lingual semantic parsing. |
| Outcome: | The proposed approach trains a parser with maximum sample efficiency in six languages on ATIS. |
Extrinsic Evaluation of Machine Translation Metrics (2023.acl-long)
Copied to clipboard
| Challenge: | MT metrics are widely used to distinguish the quality of machine translation systems across relatively large test sets. |
| Approach: | They evaluate the segment-level performance of the most widely used MT metrics by correlating them with how useful they are for downstream tasks. |
| Outcome: | The MT metrics are widely used to distinguish the quality of machine translation systems across relatively large test sets. |
Scalable Data Ablation Approximations for Language Models through Modular Training and Merging (2024.emnlp-main)
Copied to clipboard
Clara Na, Ian Magnusson, Ananya Harsh Jha, Tom Sherborne, Emma Strubell, Jesse Dodge, Pradeep Dasigi
| Challenge: | Training data compositions for Large Language Models (LLMs) can significantly affect their downstream performance. |
| Approach: | They propose a method which trains individual models on subsets of a training corpus and reuses them across evaluations of combinations of subset. |
| Outcome: | The proposed method improves training efficiency by scaling only linearly with respect to new data. |
Optimal Transport Posterior Alignment for Cross-lingual Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing work on cross-lingual semantic parsing has focused on English . a few-shot approach to parse from natural languages is comparatively unexplored . |
| Approach: | They propose a method that minimizes cross-lingual divergence between probabilistic latent variables by Optimal Transport. |
| Outcome: | The proposed method improves performance even without parallel input translations on two datasets. |
Zero-Shot Cross-lingual Semantic Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work in cross-lingual semantic parsing assumes access to high-quality machine translation systems and word alignment tools. |
| Approach: | They propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-logical form paired data and in-domain natural language corpora. |
| Outcome: | The proposed model performs significantly above translation-based baselines and competes with the supervised upper-bound. |
Bootstrapping a Crosslingual Semantic Parser (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in semantic parsing are limited to English but professional translation can be prohibitively expensive. |
| Approach: | They adapt a semantic parser trained on a single language to new languages and multiple domains with minimal annotation. |
| Outcome: | The proposed approach achieves parsing accuracy within 2% of translation using only 50% of training data. |
On Leakage of Code Generation Evaluation Datasets (2024.findings-emnlp)
Copied to clipboard
Alexandre Matton, Tom Sherborne, Dennis Aumiller, Elena Tommasone, Milad Alizadeh, Jingyi He, Raymond Ma, Maxime Voisin, Ellen Gilsenan-McMahon, Matthias Gallé
| Challenge: | In this paper, we discuss contamination by code generation test sets in large language models. |
| Approach: | They propose to use Python to test code generation test sets for contamination . they find that code generation is an important skill for large language models to master . |
| Outcome: | The proposed benchmarks are uncontaminated and provide a new insight into code generation. |