Papers by Marco Valentino
Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models (2024.lrec-main)
Copied to clipboard
| Challenge: | a number of studies have reported high accuracies in NLP tasks due to simple heuristics and dataset artifacts. |
| Approach: | They use a case where two words/terms occur in a shared context to construct a causal diagram . they also investigate the robustness to irrelevant changes and sensitivity to impactful changes of Transformers . |
| Outcome: | The proposed method bolsters the fact that similar benchmark accuracy scores may be observed for models that exhibit very different behaviour. |
Unification-based Reconstruction of Multi-hop Explanations for Science Questions (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches build explanations considering each question in isolation, but new approach leverages explanatory patterns emerging in scientific explanations. |
| Approach: | They propose a framework for reconstructing multi-hop explanations in science Question Answering . they integrate lexical relevance with the notion of unification power to rank atomic facts . |
| Outcome: | The proposed method achieves results competitive with Transformers, but is faster and scalable to large explanatory corpora. |
SylloBio-NLI: Evaluating Large Language Models on Biomedical Syllogistic Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing models are far from achieving the robustness and consistency required for safe biomedical NLI applications. |
| Approach: | They propose a framework that leverages external ontologies to instantiate diverse syllogistic arguments for biomedical NLI by identifying valid conclusions and extracting supporting evidence. |
| Outcome: | The proposed framework evaluates large language models on identifying valid conclusions and extracting supporting evidence across 28 syllogistic schemes instantiated with human genome pathways. |
Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on the application and evaluation of Large Language Models (LLMs) but LLMs are still prone to factual errors and inconsistencies in their explanations, offering limited control and interpretability for inference in complex domains. |
| Approach: | They propose an abductive-deductive framework that integrates Large Language Models with an external backward-chaining solver to refine step-wise natural language explanations. |
| Outcome: | The proposed framework improves explanations generated via in-context learning methods and Chain-of-Thought (CoT) on ethical NLI tasks while producing formal proofs describing and supporting models’ reasoning. |
A Framework for Evaluation of Machine Reading Comprehension Gold Standards (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing literature on machine reading comprehension (MRC) data is limited on the data design of gold standards. |
| Approach: | They propose a framework to investigate linguistic features, lexical cues and ambiguity in MRC gold standards. |
| Outcome: | The proposed framework investigates the present linguistic features, required reasoning and background knowledge and factual correctness on the one hand, and the presence of lexical cues as a lower bound for the requirement of understanding on the other. |
Inference to the Best Explanation in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have found success in real-world applications, but their underlying explanatory process is still poorly understood. |
| Approach: | They propose to use a framework inspired by philosophical accounts on Inference to the Best Explanation (IBE) to advance the interpretation and evaluation of LLMs’ explanations. |
| Outcome: | The proposed framework can identify the best explanation with up to 77% accuracy (27% above random) while being intrinsically more efficient and interpretable. |
Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies on syntactic injection in Variational AutoEncoders (VAEs) are limited to LSTM-based VAEs. |
| Approach: | They propose to use latent space separation techniques to inject syntactic information into Variational AutoEncoders (VAEs) using graph-based models. |
| Outcome: | The proposed end-to-end VAE architecture can improve the organisation of the latent space, alleviating the information loss occurring in standard VAE setups, and resulting in enhanced performances on language modelling and downstream generation tasks. |
Does My Representation Capture X? Probe-Ably (2021.acl-demo)
Copied to clipboard
| Challenge: | Probing (or diagnostic classification) has become a popular strategy for investigating whether a given set of intermediate features is present in the representations of neural models. |
| Approach: | They propose to use an extendable probing framework to automate the application of probing methods to the user’s inputs. |
| Outcome: | The proposed framework automates the application of probing methods to the user’s inputs. |
Eliciting Critical Reasoning in Retrieval-Augmented Generation via Contrastive Explanations (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies show that LLMs struggle to critically analyse RAG-based in-context information. |
| Approach: | They propose a framework that elicits critical arguments in RAG via contrastive explanations . they propose CRAG to retrieve relevant documents given a query and generate explanations that explicitly contrast relevance of passages to support the final answer. |
| Outcome: | The proposed framework improves state-of-the-art RAG models while requiring significantly fewer prompts and demonstrations and robust to perturbations in the retrieved documents. |
Introductory Tutorial: Reasoning with Natural Language Explanations (2024.emnlp-tutorials)
Copied to clipboard
| Challenge: | Existing paradigms for explanation-based NLIs lack a clear understanding of the nature of human reasoning. |
| Approach: | They propose to use natural language explanations to build models that address downstream tasks through explicit construction of a natural language. |
| Outcome: | In contrast to the existing paradigm based on Deep Learning, explanation-based NLI focuses on developing and evaluating models that address downstream tasks through the explicit construction of a natural language explanation. |
Diff-Explainer: Differentiable Convex Optimization for Explainable Multi-hop Inference (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing explainable multi-hop inference models are regarded as black-boxes due to their ability to transfer linguistic and semantic information to downstream tasks, posing concerns about interpretability and transparency of their predictions. |
| Approach: | They propose a hybrid framework that integrates explicit constraints with neural architectures through differentiable convex optimization to answer and explain multi-hop questions in natural language. |
| Outcome: | The proposed framework improves performance on scientific and commonsense QA tasks while still providing structured explanations in support of its predictions. |
Fundamental Reasoning Paradigms Induce Out-of-Domain Generalization in Language Models (2026.findings-acl)
Copied to clipboard
Mingzi Cao, Xingwei Tan, Mahmud Elahi Akhter, Marco Valentino, Maria Liakata, Xi Wang, Nikolaos Aletras
| Challenge: | Deduction, induction, and abduction are fundamental reasoning paradigms, core for human logical thinking. |
| Approach: | They propose to use a dataset of symbolic tasks to induce deductive skills into large language models (LLMs) they then use FT to fine-tune models to improve OOD generalization . |
| Outcome: | The proposed approach yields strong generalizability with substantial performance gains (up to 14.60) across realistic out-of-domain tasks. |
A Differentiable Integer Linear Programming Solver for Explanation-Based Natural Language Inference (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing ILP frameworks are non-differentiable and cannot be integrated as part of a broader deep learning architecture. |
| Approach: | They propose a neuro-symbolic architecture for explanation-based NLI based on DBCS. |
| Outcome: | The proposed approach achieves superior performance when compared to existing solvers and black-box solver. |
Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Proving (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for assessing the validity of explanations for NLI are time-consuming and prone to logical errors. |
| Approach: | They propose a framework that integrates Large Language Models and Theorem Provers to verify and refine natural language explanations through crowd-sourcing . they propose to use TPs to generate and formalise explanatory sentences and suggest potential inference strategies for NLI. |
| Outcome: | The proposed framework generates and formalises explanatory sentences and suggests potential inference strategies for NLI. |
Multi-Operational Mathematical Derivations in Latent Space (2024.naacl-long)
Copied to clipboard
| Challenge: | Using a symbolic engine, we investigate the possibility of approximating multiple mathematical operations in latent space for expression derivation. |
| Approach: | They propose to model mathematical operations as explicit geometric transformations by leveraging a symbolic engine and a large-scale dataset. |
| Outcome: | The proposed paradigms can be used to approximate multiple mathematical operations in latent space, while discriminating the conclusions for a single operation is achievable in the original expression encoder. |
Explainable Inference Over Grounding-Abstract Chains for Science Questions (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing inference models for science questions are black-box by nature, lacking explanations for their predictions. |
| Approach: | They propose an explainable inference approach for science questions by reasoning on grounding and abstract inference chains. |
| Outcome: | The proposed model generates plausible explanations for science questions using a weighted graph of relevant facts and a Bayesian Optimisation formalism. |
A Symbolic Framework for Evaluating Mathematical Reasoning and Generalisation with Transformers (2024.naacl-long)
Copied to clipboard
| Challenge: | evaluating the generalisability of Transformers to out-of-distribution mathematical reasoning problems is a challenge for many open-source models. |
| Approach: | They propose a method for generating and perturbing detailed derivations of equations at scale, aided by a symbolic engine, and compare their results to sequence classification tasks. |
| Outcome: | The proposed framework outperforms GPT-4, GPT-3.5 and a canon of fine-tuned BERT models in classification tasks . perturbations to input reasoning can reduce their performance by up to 80 F1 points . |
Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies on reasoning in language models have sparked a debate on whether they can learn systematic inferential principles or merely exploit superficial patterns in the training data. |
| Approach: | They propose a method for circuit discovery aimed at interpreting syllogistic inference . they uncover a circuit involving middle-term suppression that elucidates how LMs transfer information to derive valid conclusions from premises. |
| Outcome: | The proposed method elucidates how LMs transfer information to derive valid conclusions from premises. |
Faithful and Robust LLM-Driven Theorem Proving for NLI Explanations (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that the interaction of large language models (LLMs) with theorem provers (TPs) can help verify and improve the validity of NLI explanations. |
| Approach: | They propose to use logical expressions to guide LLMs in generating structured proof sketches and to use them to improve their accuracy. |
| Outcome: | The proposed strategies improve autoformalisation, syntactic errors and explanation refinement over the state-of-the-art model. |
Dissecting Clinical Reasoning in Natural Language Inference for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies on large language models (LLMs) have demonstrated the impact of prompting strategies and fine-tuning techniques on their reasoning capabilities. |
| Approach: | They examine four classes of prompting strategies to elicit reasoning in large language models . they then construct demonstrations using a frontier model to distil multi-step reasoning capabilities into smaller models based on Low-Rank Adaptation (LoRA). |
| Outcome: | The proposed model improves in 75% of the models on MedNLI and TREC Clinical Trials. |
Can Activation Steering Generalize Across Languages? A Study on Syllogistic Reasoning in Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Prior work has focused on activation steering for Large Language Models (LLMs) this technique can be used to improve reasoning accuracy and transferability across languages. |
| Approach: | They propose to use activation steering to steer models towards a cross-lingual reasoning space. |
| Outcome: | The proposed techniques generalise well to multilingual datasets while minimizing language modelling performance. |
To be or not to be an Integer? Encoding Variables for Mathematical Text (2022.findings-acl)
Copied to clipboard
| Challenge: | a number of natural language inference models are limited in interpreting mathematical knowledge written in Natural Language . a variable's meaning is determined exclusively by its defining type, i.e., its context . |
| Approach: | They propose a method that can create context-based representations for variables . they propose 'variable slot' approach which can be used to model variables based on their meaning . |
| Outcome: | The proposed model can be used to represent variables in natural language . it can be applied to a task of variable typing and create context-based representations for variables . |
Unravelling the Logic: Investigating the Generalisation of Transformers in Numerical Satisfiability Problems (2025.acl-long)
Copied to clipboard
| Challenge: | Transformer models exhibit minimal scale and noise invariance, along with limited vocabulary and number invariancy. |
| Approach: | They probe the generalisation prowess of Transformer models with respect to the hitherto unexplored domain of numerical satisfiability problems. |
| Outcome: | The proposed models exhibit minimal scale and noise invariance, along with limited vocabulary and number invariancy. |
Autoformalization in the Wild: Assessing LLMs on Real-World Mathematical Definitions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable potential in assisting with mathematical reasoning on different downstream tasks. |
| Approach: | They propose two new tools for autoformalizing real-world mathematical definitions from Wikipedia and arXiv papers. |
| Outcome: | The proposed methods improve definitions by up to 16% and undefined errors by 43%. |
Neuro-Symbolic Natural Language Processing (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have limitations in terms of safe and controlled reasoning, interpretability and adaptability . this tutorial aims to bridge the gap between the practical performance of LLMs and the principled modelling of language and inference of formal methods. |
| Approach: | This tutorial aims to bridge the gap between the practical performance of Large Language Models and the principled modelling of language and inference of formal methods. |
| Outcome: | This tutorial aims to bridge the gap between the performance of LLMs and the principled modelling of language and inference of formal methods. |
Enhancing Logical Reasoning in Language Models via Symbolically-Guided Monte Carlo Process Supervision (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown strong performance in many reasoning benchmarks, but lack robust planning or symbolic abstractions. |
| Approach: | They propose to synthesize high-quality symbolic reasoning trajectories with stepwise pseudo-labels at scale via Monte Carlo estimation. |
| Outcome: | The proposed method can be trained on high-quality symbolic reasoning trajectories with stepwise pseudo-labels at scale using Monte Carlo estimation. |
MASA: LLM-Driven Multi-Agent Systems for Autoformalization (2025.emnlp-demos)
Copied to clipboard
| Challenge: | This paper presents a framework for building multi-agent systems for autoformalization driven by Large Language Models. |
| Approach: | They propose a framework for building multi-agent systems for autoformalization driven by Large Language Models. |
| Outcome: | The proposed framework leverages collaborative agents to convert natural language statements into formal representations. |
Identifying Supporting Facts for Multi-hop Question Answering with Document Graph Networks (D19-53)
Copied to clipboard
| Challenge: | Recent advances in reading comprehension have resulted in models that surpass human performance when the answer is contained in a single, continuous passage of text. |
| Approach: | They propose a document-structured message passing architecture for the identification of supporting facts over a graph-structure based representation of text. |
| Outcome: | The proposed model outperforms a baseline reading comprehension test on raw text and shows that it is relevant for multi-hop reasoning. |
Interventional Probing in High Dimensions: An NLI Case Study (2023.findings-eacl)
Copied to clipboard
| Challenge: | Probing strategies have been shown to detect the presence of various linguistic features inlarge language models; in particular, semantic features intermediate to the “natural logic”fragment of the NLI. |
| Approach: | They propose to use amnesic probing and mnestic probing to investigate the effect of these semantic fea-tures on NLI classification by examining the effects of a mnemonic probing variation on the model. |
| Outcome: | The proposed methods have been shown to detect features intermediate to the “natural logic”fragment of the Natural Language Inferencetask (NLI). |
Adaptive LLM-Symbolic Reasoning via Dynamic Logical Solver Composition (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to NLP are static and require manual formalization. |
| Approach: | They propose an adaptive, multi-paradigm, neuro-symbolic inference framework that automatically identifies formal reasoning strategies from problems expressed in natural language and dynamically selects and applies specialized formal logical solvers. |
| Outcome: | The proposed framework outperforms baselines on individual and multi-paradigm reasoning tasks by 17% and 6%. |
Case-Based Abductive Natural Language Inference (2022.coling-1)
Copied to clipboard
| Challenge: | Recent approaches for multi-hop inference construct explanations considering each test case in isolation, but semantic drift causes wrong conclusions. |
| Approach: | They propose an abductive framework for multi-hop NLI exploring the retrieve-reuse-refine paradigm in Case-Based Reasoning. |
| Outcome: | The proposed model can be integrated with sparse and dense pre-trained encoders to improve multi-hop inference, or adopted as an evidence retriever for Transformers. |
Multi-Relational Hyperbolic Word Embeddings from Natural Language Definitions (2024.eacl-long)
Copied to clipboard
| Challenge: | a fundamental characteristic of natural language definitions is that they are widely abundant, pos-1. |
| Approach: | They propose a multi-relational model that explicitly leverages definitions' semantic structure to derive word embeddings. |
| Outcome: | The proposed model can preserve the semantic mapping required for interpretable traversal while imposing constraints on definitions while maintaining the recursive semantic structure. |
Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work suggests using logical formalisms coupled with external symbolic solvers to solve complex tasks. |
| Approach: | They propose a framework to disentangle content from logical reasoning without a complete formalisation. |
| Outcome: | The proposed methods improve CoT-based methods by up to 8% accuracy on challenging adversarial variations on both natural language and symbolic reasoning tasks. |
Improving Semantic Control in Discrete Latent Spaces with Transformer Quantized Variational Autoencoders (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent work has struggled to achieve consistent results due to the inevitable loss of semantic information in the variational bottleneck and limited control over the decoding mechanism. |
| Approach: | They propose a model that leverages the controllability of VQVAE to guide the self-attention mechanism in Transformer-based VAEs to improve semantic control and generation. |
| Outcome: | The proposed model outperforms existing state-of-the-art VAE models in terms of control and preservation of semantic information across different tasks. |
NLI4CT: Multi-Evidence Natural Language Inference for Clinical Trial Reports (2023.emnlp-main)
Copied to clipboard
| Challenge: | Clinical trial reports (CTRs) are indispensable for the development of personalized medicine. |
| Approach: | They propose a resource to help researchers interpret clinical trial reports . they use natural language inference to compute textual entailment . |
| Outcome: | The proposed resource is the first to cover interpretation of full clinical trial reports . it includes tasks to determine inference relation between natural language statements and CTRs . |
Compartmentalised Agentic Reasoning for Clinical NLI (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models produce fluent judgments for clinical natural language inference, yet fail when the decision requires the correct inferential schema rather than surface matching. |
| Approach: | They propose a compartmentalised agentic framework that routes each premise–statement pair to a reasoning family and applies a specialised solver with explicit verification and targeted refinement. |
| Outcome: | The proposed framework improves mean accuracy from 23% with direct prompting to 57%, with the largest gains on structurally demanding reasoning types. |
PEIRCE: Unifying Material and Formal Reasoning via LLM-Driven Neuro-Symbolic Refinement (2025.acl-demo)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are capable of material inference but lack formal rigour and verifiability. |
| Approach: | They propose a framework to unify material and formal inference through an iterative conjecture–criticism process. |
| Outcome: | The proposed framework unifies material and formal inference through an iterative conjecture–criticism process. |