Papers by Michael Sullivan
Exploring Graph Representations of Logical Forms for Language Modeling (2025.findings-acl)
Copied to clipboard
| Challenge: | Graph-based formal-logical distributional semantics models are more data-efficient than textual counterparts. |
| Approach: | They propose a pretrained language model over graph representations of logical forms as a proof-of-concept. |
| Outcome: | The proposed model outperforms textual, transformer LMs on downstream tasks . the model is likely to scale with additional parameters and pretraining data . |
It is not True that Transformers are Inductive Learners: Probing NLI Models with External Negation (2024.eacl-long)
Copied to clipboard
| Challenge: | et al., 2017) show that NLI models learn to treat external negation as a distractor . e-learning models fail to inductively learn the role of negation for NLI tasks . |
| Approach: | They propose that models fine-tuned on NLI datasets learn to treat external negation as a distractor, effectively ignoring its presence in hypothesis sentences. |
| Outcome: | The proposed models learn to treat external negation as a distractor, the authors show . they also fail to inductively learn the law of the excluded middle for a single prefix . |
Evaluating Spatiotemporal Consistency in Automatically Generated Sewing Instructions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate instructions using large language models require spatiotemporal awareness of multiple objects and their surroundings. |
| Approach: | They propose a tree-based evaluation metric for LLM-generated step-by-step assembly instructions that more accurately reflects spatiotemporal aspects of construction than traditional metrics such as BLEU and BERT similarity scores. |
| Outcome: | The proposed metric better correlates with manually-annotated error counts, and is more robust against artificially-constructed counterfactual examples that are specifically constructed to confound metrics that rely on textual similarity. |
Procedural Environment Generation for Tool-Use Agents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to curation of tool-use training data are non-interactive and/or non-compositional. |
| Approach: | They propose a pipeline for the procedural generation of interactive tools and compositional tool-use data. |
| Outcome: | The proposed pipeline improves on a range of tool-use benchmarks and sets the new SoTA for two metrics on the NESTFUL dataset. |
Playpen: An Environment for Exploring Learning From Dialogue Game Feedback (2025.emnlp-main)
Copied to clipboard
Nicola Horst, Davide Mazzaccara, Antonia Schmidt, Michael Sullivan, Filippo Momentè, Luca Franceschetti, Philipp Sadler, Sherzod Hakimov, Alberto Testoni, Raffaella Bernardi, Raquel Fernández, Alexander Koller, Oliver Lemon, David Schlangen, Mario Giulianelli, Alessandro Suglia
| Challenge: | In this paper, we investigate whether Dialogue Games—goal-directed and rule-governed activities driven predominantly by verbal actions—can also serve as a source of feedback signals for learning. |
| Approach: | They introduce Playpen, an environment for off- and online learning through Dialogue Game self-play, and investigate a representative set of post-training methods: supervised fine-tuning, direct alignment and reinforcement learning with Group Relative Policy Optimization. |
| Outcome: | The proposed model improves performance on unseen instances, but negatively impacts other skills, while interactive learning shows balanced improvements without loss of skills. |