Papers by Ehsan Shareghi

Logical Reasoning with Outcome Reward Models for Test-Time Scaling (2025.emnlp-main)

Challenge:	Large Language Models (LLMs) reasoning processes are complex and lack of organized visualization tools creates barriers to understanding, evaluation, and improvement.
Approach:	They propose a web-based platform for visualizing and analyzing LLM reasoning processes.
Outcome:	The proposed platform shows high parsing reliability, efficient processing, and excellent usability across various downstream applications.

Copied to clipboard

Ramya Keerthy Thatikonda, Wray Buntine, Ehsan Shareghi

All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning (2025.emnlp-main)

Challenge:	Logical reasoning is a critical benchmark for evaluating the capabilities of large language models (LLMs), but it is under-explored in deductive reasoning.
Approach:	They propose to use Chain-of-Thought to generate data using single and multiple samples to train ORMs.
Outcome:	The proposed model expands the type of errors covered in the training dataset, covering previously unexplored error types.

Copied to clipboard

Caiqi Zhang, Chang Shu, Ehsan Shareghi, Nigel Collier

Assessing the Sensitivity and Alignment of FOL Closeness Metrics (2025.findings-emnlp)

Challenge:	Existing methods for confidence estimation are primarily designed for factual QA tasks and fail to generalize to reasoning tasks.
Approach:	They propose a set of training-free, graph-based confidence estimation methods tailored to reasoning tasks that exploit graph properties such as centrality, path convergence, and path weighting.
Outcome:	The proposed methods improve confidence estimation and performance on two downstream tasks.

Copied to clipboard

Ramya Keerthy Thatikonda, Wray Buntine, Ehsan Shareghi

Robustness in NLP

Challenge:	a recent paradigm of logical reasoning with tool-augmented large language models has been successful, but the correctness of FOL statements often go unverified due to the lack of a reliable evaluation metric.
Approach:	They compare sensitivity of existing metrics and their alignment with LLM to evaluate FOLs.
Outcome:	The proposed metrics improve the sensitivity and robustness of the proposed models.

A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters (2021.acl-long)

Copied to clipboard

Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vulić, Roi Reichart, Anna Korhonen, Hinrich Schütze

It Is Not As Good As You Think! Evaluating Simultaneous Machine Translation on Interpretation Data (2021.emnlp-main)

Challenge:	Few-shot crosslingual transfer outperforms zero-shot with pretrained encoders like multilingual BERT.
Approach:	They conduct an experimental study on 40 sets of sampled few shots for six diverse NLP tasks across up to 40 languages.
Outcome:	The proposed model outperforms state-of-the-art approaches on lexical features and a full model finetuning approach outperformed several state- of-the art approaches.

Copied to clipboard

Jinming Zhao, Philip Arthur, Gholamreza Haffari, Trevor Cohn, Ehsan Shareghi

Text Generation

Challenge:	Existing siMT systems are trained and evaluated on offline translations . however, evaluation gap remains notable, calling for constructing large-scale interpretation corpora .
Approach:	They propose a translation-to-interpretation transfer method which converts offline translations into interpretation-style data.
Outcome:	The proposed interpretation test set shows that SiMT models improve on translation vs interpretation data.

Generating Synthetic Speech from SpokenVocab for Speech Translation (2023.findings-eacl)

Copied to clipboard

Jinming Zhao, Gholamreza Haffari, Ehsan Shareghi

Multilinguality

Challenge:	End-to-end speech-totext translation (ST) models require large amounts of data to train, but their size is considerably smaller than text-based MT data.
Approach:	They propose a method to convert MT data to ST data via text-to-speech systems.
Outcome:	The proposed method improves translation quality by an average of 1.83 BLEU score while performing equally well as TTS-generated speech in improving translation quality.

RedApt: An Adaptor for wav2vec 2 EncodingFaster and Smaller Speech Translation without Quality Compromise (2022.findings-emnlp)

Copied to clipboard

Jinming Zhao, Hao Yang, Gholamreza Haffari, Ehsan Shareghi

Speech & Audio in NLP

Challenge:	Pre-trained speech Transformers in speech translation systems have facilitated state-of-the-art (SotA) results, but their computational cost is high.
Approach:	They propose a Reducer Adaptor block that could be seamlessly integrated within any Transformer-based speech encoding architecture.
Outcome:	The proposed Reducer Adaptor block outperforms the existing SotA architecture by an average of 0.68 BLEU score on 8 language pairs from Must-C.

Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights (2024.emnlp-main)

Copied to clipboard

Hao Yang, Lizhen Qu, Ehsan Shareghi, Reza Haf

Multimodality

Challenge:	Large Multimodal Models have demonstrated a strong capability to understand multimodal information and interact with human users.
Approach:	They propose a speech-specific risk taxonomy to assess LMMs' ability to detect high-risk interactions in multimodal settings.
Outcome:	The proposed model is based on a speech-specific risk taxonomy covering 8 risk categories . it shows that the models are ineffective in detecting paralinguistic-specific risks in speech .

COMETA: A Corpus for Medical Entity Linking in the Social Media (2020.emnlp-main)

Copied to clipboard

Marco Basaldella, Fangyu Liu, Ehsan Shareghi, Nigel Collier

Knowledge Representation

Challenge:	Existing datasets for Entity Linking (EL) fail to address the complex nature of health terminology in layman’s language.
Approach:	They propose to use a corpus of 20k English biomedical entity mentions from Reddit expert-annotated with links to a widely-used medical knowledge graph to investigate the ability of these systems to perform complex inference on entities and concepts.
Outcome:	The proposed corpus satisfies a combination of desirable properties, from scale and coverage to diversity and quality, that to the best of our knowledge has not been met by existing resources in the field.

Discrete Minds in a Continuous World: Do Language Models Know Time Passes? (2025.findings-emnlp)

Copied to clipboard

Minghan Wang, Ye Bai, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation (D19-56)

Challenge:	Large Language Models excel at temporal reasoning tasks, but their ability to perceive the passage of time remains unexplored.
Approach:	They propose a Token-Time Hypothesis to test whether LLMs perceive the passage of time . they also propose an interactive navigation challenge to examine how LLM responds to time pressure .
Outcome:	The proposed model can map discrete token counts to wall-clock time and validate this through a dialogue duration judgment task.

Copied to clipboard

Victor Prokhorov, Ehsan Shareghi, Yingzhen Li, Mohammad Taher Pilehvar, Nigel Collier

Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models (2022.acl-long)

Challenge:	Variational Autoencoders suffer from learning uninformative latent representations due to issues such as approximated posterior collapse or entanglement of the latent space.
Approach:	They propose to impose an explicit constraint on the Kullback-Leibler divergence term inside the VAE objective function to understand the significance of the KL term in controlling the information transmitted through the VAe channel.
Outcome:	The proposed constraint avoids posterior collapse, but it also controls the information transmitted through the VAE channel.

Copied to clipboard

Zaiqiao Meng, Fangyu Liu, Ehsan Shareghi, Yixuan Su, Charlotte Collins, Nigel Collier

PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs (2024.findings-acl)

Challenge:	Despite the growing progress of probing knowledge for pre-trained language models, specialised areas such as the biomedical domain are vastly under-explored.
Approach:	They propose a biomedical knowledge probing benchmark, MedLAMA, constructed based on the Unified Medical Language System (UMLS) Metathesaurus.
Outcome:	The proposed approach pushes the acc@10 to 28%, but the performance gap remains notable.

Copied to clipboard

Jiuzhou Han, Nigel Collier, Wray Buntine, Ehsan Shareghi

Harnessing the Power of Large Language Models for Natural Language to First-Order Logic Translation (2024.acl-long)

Challenge:	Existing large language models are pre-trained on unstructured data, which leads to poor performance when dealing with structured data.
Approach:	They propose a framework to train large language models to act as verifier modules and to apply iterative corrections offline.
Outcome:	The proposed framework improves graph-based generative capability of large language models by iterating corrective instructions on three graph-derived datasets.

Copied to clipboard

Yuan Yang, Siheng Xiong, Ali Payani, Ehsan Shareghi, Faramarz Fekri

Fire Burns, Sword Cuts: Commonsense Inductive Bias for Exploration in Text-based Games (2022.acl-short)

Challenge:	Logic-based approaches to reasoning have lost popularity due to limited scalability and coverage.
Approach:	They present a dataset of 28K sentence-level NL-FOL pairs from GPT4 and a LogicLLaMA2-7B/13B fine-tuned on MALLS for NL translation.
Outcome:	The proposed model can be used standalone or to correct previously generated rules by GPT3.5.

Copied to clipboard

Dongwon Ryu, Ehsan Shareghi, Meng Fang, Yunqiu Xu, Shirui Pan, Reza Haf

On the Effect of Isotropy on VAE Representations of Text (2022.acl-short)

Challenge:	Existing RL agents are far away from solving text-based games due to their combinatorially large action spaces that hinders efficient exploration.
Approach:	They propose an exploration technique that injects external commonsense knowledge, via a pretrained language model, into the agent during training when the agent is the most uncertain about its next action.
Outcome:	The proposed method exhibits improvement on the collected game scores during the training in four out of nine games from Jericho.

Copied to clipboard

Lan Zhang, Wray Buntine, Ehsan Shareghi

Representation Learning

Challenge:	Injecting desired geometric properties into text representations has attracted a lot of attention due to its better utilisation of representation space.
Approach:	They propose to use an isotropic Gaussian posterior instead of the ellipsoidal Gausssian priori to inject isotropy into text representations.
Outcome:	The proposed method improves classification performance, robustness to input perturbation, and generative behavior compared to the ellipsoidal Gaussian posterior.

Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification (2021.eacl-main)

Copied to clipboard

Yi Zhu, Ehsan Shareghi, Yingzhen Li, Roi Reichart, Anna Korhonen

Information Extraction & Text Mining

Challenge:	Semi-supervised learning and multilingual pretraining have been shown to be effective for task-specific labelled data shortages.
Approach:	They propose to combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task.
Outcome:	The proposed method outperforms state-of-the-art models in low-resource settings across several languages and outperformed existing models in English.

Show Some Love to Your n-grams: A Bit of Progress and Stronger n-gram Language Modeling Baselines (N19-1)

Copied to clipboard

Ehsan Shareghi, Daniela Gerz, Ivan Vulić, Anna Korhonen

Reward Engineering for Generating Semi-structured Explanation (2024.findings-eacl)

Challenge:	Experimental results show that standalone n-gram models lend themselves as natural choices for resource-lean or morphologically rich languages.
Approach:	They run experiments on 50 languages covering all morphological language families to compare n-gram models with lstm models.
Outcome:	The proposed extension outperforms an lstm language model on 42 languages while its extension which explicitly injects linguistic knowledge outperformed the character-aware neural model on 8 languages.

Copied to clipboard

Jiuzhou Han, Wray Buntine, Ehsan Shareghi

Towards Uncertainty-Aware Language Agent (2024.findings-acl)

Challenge:	Unstructured natural language explanations lack a comprehensive explanation mechanism to verify a model's true reasoning capabilities.
Approach:	They propose a reward engineering method which uses semi-structured explanations to verify a model's true reasoning capabilities.
Outcome:	The proposed method achieves new state-of-the-art on two semi-structured explanation generation benchmarks (ExplaGraph and COPA-SSE) .

Copied to clipboard

Jiuzhou Han, Wray Buntine, Ehsan Shareghi

Challenge:	Existing Language Agents neglect the notion of uncertainty during interactions with external worlds.
Approach:	They propose a framework that orchestrates the interaction between the agent and the external world using uncertainty quantification.
Outcome:	The proposed framework improves performance on 3 representative tasks and lowers reliance on external world.

TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning (2022.findings-naacl)

Copied to clipboard

Yixuan Su, Fangyu Liu, Zaiqiao Meng, Tian Lan, Lei Shu, Ehsan Shareghi, Nigel Collier

Bayesian Learning for Neural Dependency Parsing (N19-1)

Challenge:	Existing pre-trained MLMs produce an anisotropic distribution of token representations . this is not ideal for tasks that require discriminative semantic meanings of distinct tokens - a problem that exists in pre-training models .
Approach:	They propose a continual pre-training approach that encourages BERT to learn an isotropic distribution of token representations.
Outcome:	The proposed approach improves on a wide range of English and Chinese benchmarks.

Copied to clipboard

Ehsan Shareghi, Yingzhen Li, Yi Zhu, Roi Reichart, Anna Korhonen

Syntactic Text Processing

Challenge:	Several approaches for dependency parsing in the small data regime have been proposed.
Approach:	They propose to use stochastic gradient Langevin dynamics to generate samples from the approximated posterior to overcome the computational and statistical costs of the approximate inference step.
Outcome:	The proposed model outperforms the biaffine model on 6 languages with less than 5k training instances and improves across five languages.

Koala: An Index for Quantifying Overlaps with Pre-training Corpora (2023.emnlp-demo)

Copied to clipboard

Thuy-Trang Vu, Xuanli He, Gholamreza Haffari, Ehsan Shareghi

Reshaping Representation Space to Balance the Safety and Over-rejection in Large Audio Language Models (2025.emnlp-main)

Challenge:	Recent studies have shown that large language models can be influenced by the frequency of overlap between pre-training corpora.
Approach:	They propose to search over large pre-training corpora using lossless compressed suffix arrays with highly efficient compression rate and search support.
Outcome:	Koala is a searchable index over large pre-training corpora using lossless compressed suffix arrays with highly efficient compression rate and search support.

Copied to clipboard

Hao Yang, Lizhen Qu, Ehsan Shareghi, Gholamreza Haffari

Self-supervised Graph Masking Pre-training for Graph-to-Text Generation (2022.emnlp-main)

Challenge:	Large Audio Language Models (LALMs) have demonstrated unprecedented capabilities in natural language understanding and generation, revolutionizing human-machine dialogue.
Approach:	They propose an unsupervised safety-fine-tuning strategy that reshapes LALMs representation space to enhance existing LALM safety-alignment while balancing the risk of over-rejection.
Outcome:	The proposed approach improves LALMs safety under three input conditions while increasing over-rejection rate by only 0.88% on average.

Copied to clipboard

Jiuzhou Han, Ehsan Shareghi

Text Generation

Challenge:	Large-scale pre-trained language models (PLMs) have advanced Graph-to-Text generation by processing the linearised version of a graph.
Approach:	They propose to mask pre-training tasks that neither require supervision signals nor adjust the architecture of the underlying pre-trained encoder-decoder model.
Outcome:	The proposed method achieves state-of-the-art results on WebNLG+2020 and EventNarrative datasets and is very efficient in the low-resource setting.

Can LLMs Reason in the Wild with Programs? (2024.findings-emnlp)

Copied to clipboard

Yuan Yang, Siheng Xiong, Ali Payani, Ehsan Shareghi, Faramarz Fekri

Self-Alignment Pretraining for Biomedical Entity Representations (2021.naacl-main)

Challenge:	Large language models have shown superior capability to solve reasoning problems with programs.
Approach:	They propose a task where an LLM is tasked to solve a reasoning problem of unknown type by identifying the sub-problems and their corresponding formalisms.
Outcome:	The proposed model can be fine tuned to achieve better performance on ambiguous and mixed scope problems.

Copied to clipboard

Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, Nigel Collier

Equipping Language Models with Tool Use Capability for Tabular Data Analysis in Finance (2024.eacl-short)

Challenge:	Existing approaches to self-supervised learning of biomedical entities are limited in the biomedic domain.
Approach:	They propose a pretraining scheme that self-aligns the representation space of biomedical entities.
Outcome:	The proposed framework achieves state-of-the-art on six MEL benchmarking datasets.

Copied to clipboard

Adrian Theuma, Ehsan Shareghi

Multimodality

Challenge:	Large language models (LLMs) have an array of reasoning capabilities but face limitations such as error propagation and hallucination.
Approach:	They propose to use a LLAMA-2 13B CHAT model to act as a task router and task solver to offload certain reasoning steps to external tools that are more suited for the task.
Outcome:	The proposed model improves by 35.2% and 5.06% over baseline models and strong GPT-3.5 results.

Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learning (2026.acl-long)

Copied to clipboard

Zheng Hui, Yijiang River Dong, Sanhanat Sivapiromrat, Ehsan Shareghi, Nigel Collier

Exploring the Potential of Multimodal LLM with Knowledge-Intensive Multimodal ASR (2024.findings-emnlp)

Challenge:	Prior approaches to rewriting large language models shatters linguistic coherence and removes privacy-sensitive information.
Approach:	They propose a framework that trains an agent to dynamically route text chunks . it implicitly distinguishes between replaceable Personally Identifiable Information (PII) and task-critical PII .
Outcome:	The proposed framework achieves state-of-the-art on the privacy-utility frontier . it trains an agent to dynamically route text chunks, learning a policy that balances privacy leakage and task performance.

Copied to clipboard

Minghan Wang, Yuxia Wang, Thuy-Trang Vu, Ehsan Shareghi, Reza Haf

Multimodality

Challenge:	Recent advances in multimodal large language models have made significant progress in integrating information across various modalities, yet real-world applications in educational and scientific domains remain challenging.
Approach:	They propose a task that focuses on transcribing scientific conference videos by leveraging visual information from slides to enhance the accuracy of technical terminologies.
Outcome:	The proposed framework improves transcript quality through post-editing and improves performance over speech-only baselines.

VerifiAgent: a Unified Verification Agent in Language Model Reasoning (2025.findings-emnlp)

Copied to clipboard

Jiuzhou Han, Wray Buntine, Ehsan Shareghi

Self-supervised Rewiring of Pre-trained Speech Encoders: Towards Faster Fine-tuning with Less Labels in Speech Processing (2022.findings-emnlp)

Challenge:	Existing verification methods are typically model-specific or domain-restricted . Existing methods require significant computational resources and lack scalability .
Approach:	a unified verification agent integrates two levels of verification: meta-verification and tool-based adaptive verification.
Outcome:	The proposed agent outperforms baseline verification methods among reasoning tasks.

Copied to clipboard

Hao Yang, Jinming Zhao, Gholamreza Haffari, Ehsan Shareghi

Audio Is the Achilles’ Heel: Red Teaming Audio Large Multimodal Models (2025.naacl-long)

Challenge:	Pre-trained speech encoders have facilitated great success across various speech processing tasks, but fine-tuning them for downstream tasks requires large training data to converge or to achieve state-of-the-art.
Approach:	They propose to rewire pre-trained speech encoders to improve their representation space without task-specific labels by neutrally synthesising audio inputs and frame masking.
Outcome:	The proposed model shows consistent improvement in isotropy in the representation space on 6 speech processing tasks.

Copied to clipboard

Hao Yang, Lizhen Qu, Ehsan Shareghi, Gholamreza Haffari

Towards Inference-time Scaling for Continuous Space Reasoning (2026.findings-acl)

Challenge:	Large Language Models (LMMs) have demonstrated ability to interact with humans through text . however, safety of audio LMMs remains under-explored .
Approach:	They red team the safety of five audio LMMs under three settings . they find that audio Lmms suffer an average attack success rate of 69.14% on harmful questions .
Outcome:	a new study shows that audio LMMs suffer an average success rate on harmful questions . the authors also show that the models exhibit safety vulnerabilities when distracted .

Copied to clipboard

Minghan Wang, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

POSQA: Probe the World Models of LLMs with Size Comparisons (2023.findings-emnlp)

Challenge:	Recent advances in reasoning large language models have expanded along training and inferencetime dimensions.
Approach:	They propose to use COCONUT (CITATION) continuous space reasoning LM as the backbone to generate diverse reasoning paths through dropout-based sampling.
Outcome:	The proposed method could enable performance gains similar to those observed in the discrete space, but only marginally improves in the continuous space.

Copied to clipboard

Chang Shu, Jiuzhou Han, Fangyu Liu, Ehsan Shareghi, Nigel Collier