Papers by Nigel Collier

Conformity in Large Language Models (2025.acl-long)

Challenge:	Infusing factual knowledge into pre-trained models is fundamental for many knowledge-intensive tasks.
Approach:	They propose an infusion approach that partitions a large knowledge graph into smaller sub-graphs and infuses their specific knowledge into various BERT models using lightweight adapters.
Outcome:	The proposed approach improves the underlying BERTs and achieves new SOTA performance on six downstream tasks.

Copied to clipboard

Xiaochen Zhu, Caiqi Zhang, Tom Stafford, Nigel Collier, Andreas Vlachos

Time to Revisit Exact Match (2025.findings-emnlp)

Challenge:	Conformity is a form of social influence that affects the way people respond to information.
Approach:	They adapt psychological experiments to examine the extent of conformity in large language models.
Outcome:	The proposed interventions mitigate conformity by reducing the naturalness of majority tones and reducing instruction-tuned models.

Copied to clipboard

Auss Abbood, Zaiqiao Meng, Nigel Collier

Question Answering

Challenge:	Temporal question answering is an established method for evaluating temporal reasoning in large language models.
Approach:	They propose a numerical estimation task where all questions require a numeric, temporal answer, allowing us to evaluate models beyond EM.
Outcome:	The proposed model responses are based on a numerical estimation task and are distilled from Test of Time and TempTabQA.

Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders (2023.eacl-main)

Copied to clipboard

Ivan Vulić, Goran Glavaš, Fangyu Liu, Nigel Collier, Edoardo Maria Ponti, Anna Korhonen

ReasonGraph: Visualization of Reasoning Methods and Extended Inference Paths (2025.acl-demo)

Challenge:	Pretrained multilingual language models (LMs) can be 'rewired' into effective multilingual sentence encoders (SEs) however, it remains unclear how to best leverage them to represent sub-sentence lexical items in cross-lingual lexicals.
Approach:	They propose a method for exposing cross-lingual lexical knowledge by additional fine-tuning through inexpensive contrastive learning that requires only a small amount of word translation pairs.
Outcome:	The proposed method exposes cross-lingual lexical knowledge by additional fine-tuning through inexpensive contrastive learning that requires only a small amount of word translation pairs.

Copied to clipboard

Zongqian Li, Ehsan Shareghi, Nigel Collier

All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning (2025.emnlp-main)

Challenge:	Large Language Models (LLMs) reasoning processes are complex and lack of organized visualization tools creates barriers to understanding, evaluation, and improvement.
Approach:	They propose a web-based platform for visualizing and analyzing LLM reasoning processes.
Outcome:	The proposed platform shows high parsing reliability, efficient processing, and excellent usability across various downstream applications.

Copied to clipboard

Caiqi Zhang, Chang Shu, Ehsan Shareghi, Nigel Collier

When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learning (2025.findings-emnlp)

Challenge:	Existing methods for confidence estimation are primarily designed for factual QA tasks and fail to generalize to reasoning tasks.
Approach:	They propose a set of training-free, graph-based confidence estimation methods tailored to reasoning tasks that exploit graph properties such as centrality, path convergence, and path weighting.
Outcome:	The proposed methods improve confidence estimation and performance on two downstream tasks.

Copied to clipboard

Yijiang River Dong, Tiancheng Hu, Yinhong Liu, Ahmet Üstün, Nigel Collier

Challenge:	Reinforcement Learning from Human Feedback assumes homogeneous preferences across users . personalization can introduce up to 20% safety misalignment .
Approach:	They propose a framework to assess personalized preference learning by tailoring preferences for users . they compare eight personalization methods across three preference datasets .
Outcome:	The proposed framework measures performance, fairness, unintended effects, adaptability across preferences . performance differences between personalization methods could reach 36% when users strongly disagree .

Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders (2021.emnlp-main)

Copied to clipboard

Fangyu Liu, Ivan Vulić, Anna Korhonen, Nigel Collier

LUQ: Long-text Uncertainty Quantification for LLMs (2024.emnlp-main)

Challenge:	Existing studies have shown that pretrained Masked Language Models are not effective as universal lexical and sentence encoders off-the-shelf, i.e., without further task-specific fine-tuning on NLI, sentence similarity, or paraphrasing tasks using annotated task data.
Approach:	They propose a contrastive learning technique which turns pretrained MLMs into effective universal lexical and sentence encoders without additional data.
Outcome:	The proposed technique can turn MLMs into effective universal lexical and sentence encoders even without additional data.

Copied to clipboard

Caiqi Zhang, Fangyu Liu, Marco Basaldella, Nigel Collier

Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking (2021.acl-short)

Challenge:	Existing research on Uncertainty Quantification (UQ) predominantly targets short text generation, however, real-world applications often necessitate much longer responses.
Approach:	They propose a method that ensembles responses from multiple models and selects the response with the lowest uncertainty.
Outcome:	The proposed method outperforms baseline methods in correlating with the model’s factuality scores (negative coefficient of -0.85 observed for Gemini Pro).

Copied to clipboard

Fangyu Liu, Ivan Vulić, Anna Korhonen, Nigel Collier

MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering (2023.acl-long)

Challenge:	Existing work on transferring domain-specific knowledge from a pretraining model to a resource-poor language is limited to English . a novel cross-lingual biomedical entity linking task is proposed to improve this capability.
Approach:	They propose a cross-lingual biomedical entity linking task and establish a new benchmark spanning 10 typologically diverse languages.
Outcome:	The proposed methods yield consistent gains across all target languages, sometimes up to 20 Precision@1 points, without any in-domain knowledge in the target language and without any parallel data.

Copied to clipboard

Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, Julian Eisenschlos

Biomedical Named Entity Recognition via Dictionary-based Synonym Generalization (2023.emnlp-main)

Challenge:	Visual language models that are pretraining on natural images or image-text pairs crawled from the web perform poorly on visual language tasks such as ChartQA and ChartQA.
Approach:	They propose to perform several pretraining tasks that cover plot deconstruction and numerical reasoning which are key capabilities in visual language modeling.
Outcome:	The proposed model outperforms state-of-the-art methods on benchmarks such as PlotQA and ChartQA by as much as 20%.

Copied to clipboard

Zihao Fu, Yixuan Su, Zaiqiao Meng, Nigel Collier

Information Extraction & Text Mining

Challenge:	Existing methods for biomedical named entity recognition require laborious human effort.
Approach:	They propose a Synonym Generalization framework that recognizes biomedical concepts using span-based predictions.
Outcome:	The proposed framework outperforms dictionary-based approaches on a wide range of benchmarks.

Demystifying Multi-Agent Debate: The Role of Confidence and Diversity (2026.findings-acl)

Copied to clipboard

Xiaochen Zhu, Caiqi Zhang, Yizhou Chi, Tom Stafford, Nigel Collier, Andreas Vlachos

Challenge:	Multi-agent debate (MAD) is widely used to improve large language models' (LLMs) reasoning and test-time scaling.
Approach:	They propose a diversity-aware initialisation that selects a more diverse pool of candidate answers, increasing the likelihood that a correct hypothesis is present at the start of debate.
Outcome:	The proposed protocol outperforms vanilla MAD and majority vote on six reasoning-oriented QA benchmarks.

iNews: A Multimodal Dataset for Modeling Personalized Affective Responses to News (2025.acl-long)

Copied to clipboard

Tiancheng Hu, Nigel Collier

Card-660: Cambridge Rare Word Dataset - a Reliable Benchmark for Infrequent Word Representation Models (D18-1)

Challenge:	Current approaches to modeling individual behavior ignore individual differences in how people interpret and react to identical stimuli.
Approach:	They propose a large-scale dataset specifically designed to facilitate the modeling of personalized affective responses to news content.
Outcome:	The proposed dataset includes annotations from 291 demographically diverse UK participants across 2,899 multimodal Facebook news posts from major UK outlets.

Copied to clipboard

Mohammad Taher Pilehvar, Dimitri Kartsaklis, Victor Prokhorov, Nigel Collier

Representation Learning

Challenge:	Existing benchmarks for rare word representation are lacking for evaluation and comparison . a task-based evaluation does not provide a solid basis for comparing different models .
Approach:	They propose to use an expert-annotated word similarity dataset to evaluate rare word representation techniques.
Outcome:	The proposed dataset provides a reliable benchmark for rare word representation techniques.

Quantifying the Persona Effect in LLM Simulations (2024.acl-long)

Copied to clipboard

Tiancheng Hu, Nigel Collier

Few-Shot Table-to-Text Generation with Prototype Memory (2021.findings-emnlp)

Challenge:	Large language models (LLMs) have shown remarkable promise in simulating human language and behavior.
Approach:	They investigate how integrating persona variables—demographic, social, and behavioral factors—impacts LLMs’ ability to simulate diverse perspectives.
Outcome:	The proposed model improves on a zero-shot model with persona prompting.

Copied to clipboard

Yixuan Su, Zaiqiao Meng, Simon Baker, Nigel Collier

Visually Grounded Reasoning across Languages and Cultures (2021.emnlp-main)

Challenge:	Neural table-to-text generation models are data-hungry and require large amounts of training data to learn the mapping between tables and texts.
Approach:	They propose a framework for table-to-text generation under the few-shot scenario that uses retrieved prototypes and a prototype selector to bridge the structural gap between tables and texts.
Outcome:	The proposed framework significantly improves the model performance on three benchmark datasets with state-of-the-art models.

Copied to clipboard

Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, Desmond Elliott

Multimodality

Challenge:	a new protocol allows for a multilingual hierarchy of concepts and images based on native speakers . the results suggest that the current models are not robust enough to handle multilingual data .
Approach:	They propose a protocol to construct an ImageNet-style hierarchy representative of more languages and cultures.
Outcome:	The proposed protocol lets the selection of concepts and images be entirely driven by native speakers, rather than scraping them automatically.

500xCompressor: Generalized Prompt Compression for Large Language Models (2025.acl-long)

Copied to clipboard

Zongqian Li, Yixuan Su, Nigel Collier

COMETA: A Corpus for Medical Entity Linking in the Social Media (2020.emnlp-main)

Challenge:	Prompt compression is important for large language models to increase inference speed, reduce computation cost, and improve user experience.
Approach:	They propose a method that compresses natural language contexts into a special token . they propose to reduce computations and memory costs by reducing the complexity .
Outcome:	The proposed method reduces computations and memory costs by 27-90% . it retains 70-74% and 77-84% of the LLM capabilities at high compression ratios .

Copied to clipboard

Marco Basaldella, Fangyu Liu, Ehsan Shareghi, Nigel Collier

Knowledge Representation

Challenge:	Existing datasets for Entity Linking (EL) fail to address the complex nature of health terminology in layman’s language.
Approach:	They propose to use a corpus of 20k English biomedical entity mentions from Reddit expert-annotated with links to a widely-used medical knowledge graph to investigate the ability of these systems to perform complex inference on entities and concepts.
Outcome:	The proposed corpus satisfies a combination of desirable properties, from scale and coverage to diversity and quality, that to the best of our knowledge has not been met by existing resources in the field.

On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation (D19-56)

Copied to clipboard

Victor Prokhorov, Ehsan Shareghi, Yingzhen Li, Mohammad Taher Pilehvar, Nigel Collier

DePlot: One-shot visual language reasoning by plot-to-table translation (2023.findings-acl)

Challenge:	Variational Autoencoders suffer from learning uninformative latent representations due to issues such as approximated posterior collapse or entanglement of the latent space.
Approach:	They propose to impose an explicit constraint on the Kullback-Leibler divergence term inside the VAE objective function to understand the significance of the KL term in controlling the information transmitted through the VAe channel.
Outcome:	The proposed constraint avoids posterior collapse, but it also controls the information transmitted through the VAE channel.

Copied to clipboard

Fangyu Liu, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, Yasemin Altun

Multimodality

Challenge:	Existing models for visual language reasoning require tens of thousands of training examples and their reasoning capabilities are limited.
Approach:	They propose a one-shot solution to visual language reasoning by combining plot-to-text translation and reasoning over the translated text into a modality conversion module.
Outcome:	The proposed method improves on human-written queries on plots and charts compared with a fine-tuned SOTA model on human queries.

Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models (2022.acl-long)

Copied to clipboard

Zaiqiao Meng, Fangyu Liu, Ehsan Shareghi, Yixuan Su, Charlotte Collins, Nigel Collier

Keep the Primary, Rewrite the Secondary: A Two-Stage Approach for Paraphrase Generation (2021.findings-acl)

Challenge:	Despite the growing progress of probing knowledge for pre-trained language models, specialised areas such as the biomedical domain are vastly under-explored.
Approach:	They propose a biomedical knowledge probing benchmark, MedLAMA, constructed based on the Unified Medical Language System (UMLS) Metathesaurus.
Outcome:	The proposed approach pushes the acc@10 to 28%, but the performance gap remains notable.

Copied to clipboard

Yixuan Su, David Vandyke, Simon Baker, Yan Wang, Nigel Collier

PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs (2024.findings-acl)

Challenge:	Existing approaches to generate paraphrases are decomposable, but some use a sequence-to-sequence model to generate each word in a uniform way.
Approach:	They propose a framework for identification then aggregation of input tokens and a custom decoder to generate paraphrases.
Outcome:	The proposed framework outperforms previous studies on two benchmark datasets and generates paraphrases in interpretable and controllable way.

Copied to clipboard

Jiuzhou Han, Nigel Collier, Wray Buntine, Ehsan Shareghi

Prix-LM: Pretraining for Multilingual Knowledge Base Construction (2022.acl-long)

Challenge:	Existing large language models are pre-trained on unstructured data, which leads to poor performance when dealing with structured data.
Approach:	They propose a framework to train large language models to act as verifier modules and to apply iterative corrections offline.
Outcome:	The proposed framework improves graph-based generative capability of large language models by iterating corrective instructions on three graph-derived datasets.

Copied to clipboard

Wenxuan Zhou, Fangyu Liu, Ivan Vulić, Nigel Collier, Muhao Chen

BioReddit: Word Embeddings for User-Generated Biomedical NLP (D19-62)

Challenge:	Existing methods to build and enrich multilingual knowledge bases have not been successful . knowledge expressed in different languages may be complementary and unequally distributed .
Approach:	They propose a model that integrates useful multilingual and KB-based factual knowledge into a single model.
Outcome:	The proposed model can provide richer combined knowledge than monolingual KBs.

Copied to clipboard

Marco Basaldella, Nigel Collier

Representation Learning

Challenge:	a corpus of medical-themed posts was scrapped from Reddit to train word embeddings on downstream tasks.
Approach:	They propose to train word embeddings from a corpus of medical forums from reddit scrapping posts from medical-themed subreddits.
Outcome:	The proposed system outperforms embeddings trained on general purpose data or on scientific papers when applied on user-generated content.

Do ever larger octopi still amplify reporting biases? Evidence from judgments of typical colour (2022.aacl-short)

Copied to clipboard

Fangyu Liu, Julian Eisenschlos, Jeremy Cole, Nigel Collier

A Richer-but-Smarter Shortest Dependency Path with Attentive Augmentation for Relation Extraction (N19-1)

Challenge:	Language models trained on text-only corpora have no direct access to the physical world and thus suffer from reporting bias.
Approach:	They investigate reporting bias from the perspective of colour in larger language models such as PaLM and GPT-3.
Outcome:	The proposed models outperform smaller models on the basis of colour and more closely track human judgements than smaller models.

Copied to clipboard

Duy-Cat Can, Hoang-Quynh Le, Quang-Thuy Ha, Nigel Collier

Information Extraction & Text Mining

Challenge:	Existing approaches to extract relationship between entities in sentences suffer from missing or redundant information.
Approach:	They propose a deep neural model that combines the advantages of the two approaches to extract the relationship between two entities in a sentence.
Outcome:	The proposed model outperforms baseline models on the SemEval-2010 dataset.

Navigating the Alignment-Calibration Trade-off: A Pareto-Superior Frontier via Model Merging (2026.findings-acl)

Copied to clipboard

Tiancheng Hu, Benjamin Minixhofer, Nigel Collier

Challenge:	We show that the "alignment tax" of post-training is framed as a drop in task accuracy.
Approach:	They propose a more holistic view of the alignment tax by framing it as a drop in accuracy and a degradation of model calibration.
Outcome:	The proposed method improves accuracy beyond both parents while recovering calibration lost during alignment.

TOAD: Task-Oriented Automatic Dialogs with Diverse Response Styles (2024.findings-acl)

Copied to clipboard

Yinhong Liu, Yimai Fang, David Vandyke, Nigel Collier

Dialogue Systems & Conversational Agents

Challenge:	Existing datasets for Task-Oriented Dialogs (TOD) lack consideration for adaptive response styles and neglect to simulate interactions with app contexts like calendars or alarms.
Approach:	They propose to generate an annotated task-oriented dialog dataset and an automatic pipeline to generate it.
Outcome:	The proposed dataset provides a variety of system response styles and provides verbose or non-verbal responses.

How to tackle an emerging topic? Combining strong and weak labels for Covid news NER (2022.aacl-short)

Copied to clipboard

Aleksander Ficek, Fangyu Liu, Nigel Collier

Information Extraction & Text Mining

Challenge:	Named Entity Recognition (NER) models can be trained for emerging topics such as medical domain where new topics are constantly evolving out of the scope of existing models and datasets.
Approach:	They propose a recipe to combine weak and strong labels to improve Named Entity Recognition (NER) models for emerging topics.
Outcome:	The proposed model outperforms methods trained on weak data while combining out-of-domain and in-domain weak label training.

TopViewRS: Vision-Language Models as Top-View Spatial Reasoners (2024.emnlp-main)

Copied to clipboard

Chengzu Li, Caiqi Zhang, Han Zhou, Nigel Collier, Anna Korhonen, Ivan Vulić

Large-scale Exploration of Neural Relation Classification Architectures (D18-1)

Challenge:	Top-view perspective is a typical way in which humans read and reason over different types of maps, but spatial reasoning capabilities of modern VLMs in this setup remain unattested and underexplored.
Approach:	They introduce a top-view spatial reasoning dataset and use it to evaluate VLMs across 4 perception and reasoning tasks with different levels of complexity.
Outcome:	The proposed model can understand and reason over spatial relations from the top view and can be controlled at different granularities of spatial reasoning.

Copied to clipboard

Hoang-Quynh Le, Duy-Cat Can, Sinh T. Vu, Thanh Hai Dang, Mohammad Taher Pilehvar, Nigel Collier

Text Classification

Challenge:	Existing studies on relation classification have been limited to a very narrow range of datasets, making comparisons between approaches difficult.
Approach:	They propose a multi-channel LSTM model combined with a CNN that takes advantage of all currently popular linguistic and architectural features.
Outcome:	The proposed model achieves state-of-the-art on two datasets and provides direct insights into the challenges faced by language models on relation classification.

Mapping Text to Knowledge Graph Entities using Multi-Sense LSTMs (D18-1)

Copied to clipboard

Dimitri Kartsaklis, Mohammad Taher Pilehvar, Nigel Collier

UNCLE: Benchmarking Uncertainty Expressions in Long-Form Generation (2025.emnlp-main)

Challenge:	a paper addresses the problem of mapping natural language text to knowledge base entities.
Approach:	They propose a model for mapping natural language text to knowledge base entities using a multi-dimensional entity space obtained from a knowledge graph.
Outcome:	The proposed model is applied to large-scale text-to-entity mapping and entity classification tasks with state-of-the-art results.

Copied to clipboard

Ruihan Yang, Caiqi Zhang, Zhisong Zhang, Xinting Huang, Dong Yu, Nigel Collier, Deqing Yang

LoGU: Long-form Generation with Uncertainty Expressions (2025.acl-long)

Challenge:	Existing work lacks direct and fair evaluation of Large Language Models’ ability to express uncertainty effectively in long-form generation.
Approach:	They propose a benchmark to evaluate uncertainty expression in both long- and short-form question answering (QA) they propose prompt-based and training-based methods to improve models’ performance.
Outcome:	The proposed method mitigates this issue but a misalignment persists in uncertainty expression between long- and short-form generation.

Copied to clipboard

Ruihan Yang, Caiqi Zhang, Zhisong Zhang, Xinting Huang, Sen Yang, Nigel Collier, Dong Yu, Deqing Yang

Failure Modes in Multi-Hop QA: The Weakest Link Effect and the Recognition Bottleneck (2026.acl-long)

Challenge:	Large Language Models (LLMs) generate factually incorrect content, i.e., hallucinations, despite impressive performance.
Approach:	They propose a framework to enable models to express uncertainty when unsure . they propose atomic claims to refine uncertainty and refine it using supervised fine-tuning and direct preference optimization to enhance uncertainty expression.
Outcome:	The proposed framework significantly improves accuracy, reduces hallucinations, and maintains comprehensiveness of responses.

Copied to clipboard

Meiru Zhang, Zaiqiao Meng, Nigel Collier

An Individualized News Affective Response Dataset (2024.acl-srw)

Challenge:	Existing studies have identified a position bias in Large Language Models that causes them to overlook information at certain positions.
Approach:	They propose a semantic probe to disentangle position bias in Large Language Models . they propose MFAI to steer attention towards selected positions .
Outcome:	The proposed model can locate and integrate information at certain positions even in noisy, long-context settings.

Copied to clipboard

Tiancheng Hu, Nigel Collier

Will-They-Won’t-They: A Very Large Dataset for Stance Detection on Twitter (2020.acl-main)

Challenge:	a new dataset captures subjective affective responses to news headlines . current methods to assess emotion detection ignore subjective differences in groups and individuals .
Approach:	They propose a large-scale dataset capturing subjective affective responses to news headlines . the dataset includes Facebook post screenshots from popular UK media outlets .
Outcome:	The proposed dataset captures subjective affective responses to headlines from popular media outlets.

Copied to clipboard

Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, Nigel Collier

TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning (2022.findings-naacl)

Challenge:	stance detection is a key component of fake news detection, fact-checking and rumor verification.
Approach:	They propose to use a large dataset of English tweets for stance detection for a rumor verification task.
Outcome:	The proposed dataset contains 51,284 tweets in English, making it the largest available dataset of the type.

Copied to clipboard

Yixuan Su, Fangyu Liu, Zaiqiao Meng, Tian Lan, Lei Shu, Ehsan Shareghi, Nigel Collier

Prompt Compression for Large Language Models: A Survey (2025.naacl-long)

Challenge:	Existing pre-trained MLMs produce an anisotropic distribution of token representations . this is not ideal for tasks that require discriminative semantic meanings of distinct tokens - a problem that exists in pre-training models .
Approach:	They propose a continual pre-training approach that encourages BERT to learn an isotropic distribution of token representations.
Outcome:	The proposed approach improves on a wide range of English and Chinese benchmarks.

Copied to clipboard

Zongqian Li, Yinhong Liu, Yixuan Su, Nigel Collier

LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generation (2026.acl-long)

Challenge:	Current methods for improving LLM efficiency focus on optimizing the model itself, while prompt-centric methods focus on lowering the complexity of input.
Approach:	They propose to use prompt compression to optimize the compression encoder and combine hard and soft prompt methods to improve the efficiency of LLMs.
Outcome:	The proposed methods are categorized into hard prompt methods and soft prompt methods.

Copied to clipboard

Caiqi Zhang, Xiaochen Zhu, Chengzu Li, Nigel Collier, Andreas Vlachos

Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments (2024.emnlp-main)

Challenge:	Existing methods for hallucination detection are limited to short-form question answering tasks and do not generalize well to open-ended generation.
Approach:	They propose a method that trains LLMs to append a numerical confidence score to each generated statement during long-form generation.
Outcome:	The proposed method is 20 faster than traditional self-consistency methods while achieving better calibration.

Copied to clipboard

Han Zhou, Xingchen Wan, Yinhong Liu, Nigel Collier, Ivan Vulić, Anna Korhonen

Which Melbourne? Augmenting Geocoding with Maps (P18-1)

Challenge:	Large language models (LLMs) have shown promising abilities as cost-effective and reference-free evaluators for assessing language quality.
Approach:	They propose an automatic Zero-shot Evaluation-oriented Prompt Optimization framework which produces fairer preference decisions and improves human alignment.
Outcome:	The proposed framework produces fairer preference decisions and better aligns LLMs with humans.

Copied to clipboard

Milan Gritta, Mohammad Taher Pilehvar, Nigel Collier

Representation Learning

Challenge:	Existing methods to associate geographic information in text with coordinates are limited by lexical features and cartesian coordinates.
Approach:	They propose a geocoder that exploits implicit lexical clues to associate coordinates with text . they propose encoding of geographic metadata to generate two distinct views of the same text.
Outcome:	The proposed method improves state-of-the-art results on three datasets and an open-source dataset for disease outbreaks and epidemics.

Self-Alignment Pretraining for Biomedical Entity Representations (2021.naacl-main)

Copied to clipboard

Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, Nigel Collier

Plan-then-Generate: Controlled Data-to-Text Generation via Planning (2021.findings-emnlp)

Challenge:	Existing approaches to self-supervised learning of biomedical entities are limited in the biomedic domain.
Approach:	They propose a pretraining scheme that self-aligns the representation space of biomedical entities.
Outcome:	The proposed framework achieves state-of-the-art on six MEL benchmarking datasets.

Copied to clipboard

Yixuan Su, David Vandyke, Sihui Wang, Yimai Fang, Nigel Collier

Confidence Estimation for LLMs in Multi-turn Interactions (2026.findings-acl)

Challenge:	Existing studies focus on producing results that are close to the references, i.e. what to generate and in what order (the output structure) cannot be explicitly controlled by the users.
Approach:	They propose a Plan-then-Generate framework to improve the controllability of neural data-to-text models.
Outcome:	The proposed model can control both the intra-sentence and inter-sentent structure of the generated output.

Copied to clipboard

Caiqi Zhang, Ruihan Yang, Xiaochen Zhu, Chengzu Li, Tiancheng Hu, Yijiang River Dong, Deqing Yang, Nigel Collier

Dialogue Systems & Conversational Agents

Challenge:	Despite recent progress, most prior work studies confidence in single-turn question answering.
Approach:	They propose a logit-based probe that measures confidence in multi-turn dialogues . they propose 'infoECE' and a "hinter-guesser" paradigm for generating controlled evaluations based on data .
Outcome:	The proposed framework is grounded in calibration and monotonicity of confidence as more information becomes available.

Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learning (2026.acl-long)

Copied to clipboard

Zheng Hui, Yijiang River Dong, Sanhanat Sivapiromrat, Ehsan Shareghi, Nigel Collier

Can LLM be a Personalized Judge? (2024.findings-emnlp)

Challenge:	Prior approaches to rewriting large language models shatters linguistic coherence and removes privacy-sensitive information.
Approach:	They propose a framework that trains an agent to dynamically route text chunks . it implicitly distinguishes between replaceable Personally Identifiable Information (PII) and task-critical PII .
Outcome:	The proposed framework achieves state-of-the-art on the privacy-utility frontier . it trains an agent to dynamically route text chunks, learning a policy that balances privacy leakage and task performance.

Copied to clipboard

Yijiang Dong, Tiancheng Hu, Nigel Collier

Improving Word Translation via Two-Stage Contrastive Learning (2022.acl-long)

Challenge:	a new study examines the reliability of large language models (LLMs) for personalization and role-playing evaluation without examining its validity.
Approach:	They investigate the reliability of LLM-as-a-Personalized-Judge for personalization . they find that personas provided to LLMs have limited predictive power .
Outcome:	The proposed model is less reliable than previously thought, the authors show . human annotation reveals that third-person crowd worker evaluations of personalized preferences are even worse than LLM predictions.

Copied to clipboard

Yaoyiran Li, Fangyu Liu, Nigel Collier, Anna Korhonen, Ivan Vulić

Dialogue Response Selection with Hierarchical Curriculum Learning (2021.acl-long)

Challenge:	Existing approaches to bilingual lexicon induction (BLI) are limited to two stages, but we propose a robust and effective two-stage contrastive learning framework for the task.
Approach:	They propose a two-stage contrastive learning framework for the task . they propose to refine cross-lingual linear maps between static word embeddings via a contrastive objective and integrate it into the self-learning procedure for even more refined cross-linguistic maps.
Outcome:	The proposed framework improves cross-lingual maps and word translation capability by integrating it into the self-learning procedure.

Copied to clipboard

Yixuan Su, Deng Cai, Qingyu Zhou, Zibo Lin, Simon Baker, Yunbo Cao, Shuming Shi, Nigel Collier, Yan Wang

Dialogue Systems & Conversational Agents

Challenge:	Empirical studies on three benchmark datasets with three state-of-the-art matching models demonstrate that the proposed learning framework significantly improves the model performance across various evaluation metrics.
Approach:	They propose a hierarchical curriculum learning framework that trains matching models in an “easy-to-difficult” scheme.
Outcome:	The proposed framework significantly improves the model performance across evaluation metrics on three benchmark datasets with three state-of-the-art matching models.

STANDER: An Expert-Annotated Dataset for News Stance Detection and Evidence Retrieval (2020.findings-emnlp)

Copied to clipboard

Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, Nigel Collier

Can We Instruct LLMs to Compensate for Position Bias? (2024.findings-emnlp)

Challenge:	a new news dataset targets both stance detection (SD) and fine-grained evidence retrieval (ER) . stance Detection (SD), which is a form of multitask learning, has gained increasing interest in recent work .
Approach:	They propose a news dataset that targets both stance detection (SD) and fine-grained evidence retrieval (ER) their dataset is an expert-annotated news dataset with 3,291 articles.
Outcome:	The proposed dataset is a high-quality benchmark for future research in stance detection and evidence retrieval.

Copied to clipboard

Meiru Zhang, Zaiqiao Meng, Nigel Collier

Generating Knowledge Graph Paths from Textual Definitions using Sequence-to-Sequence Models (N19-1)

Challenge:	Recent studies reveal that position bias in large language models (LLMs) leads to difficulty in accessing information retrieved from the retriever.
Approach:	They propose to direct LLMs to allocate more attention towards a selected segment of the context through prompting.
Outcome:	The proposed approach improves the performance of large language models by promoting instruction with an exact document index.

Copied to clipboard

Victor Prokhorov, Mohammad Taher Pilehvar, Nigel Collier

Incorporating Stock Market Signals for Twitter Stance Detection (2022.acl-long)

Challenge:	a novel method for mapping unrestricted text to knowledge graph entities is proposed . a proof-of-concept experiment has encouraging results comparable to those of state-of the-art systems.
Approach:	They propose a method for mapping unrestricted text to knowledge graph entities by framing the task as a sequence-to-sequence problem.
Outcome:	The proposed method produces highly interpretable predictions comparable to state-of-the-art systems.

Copied to clipboard

Costanza Conforti, Jakob Berndt, Mohammad Taher Pilehvar, Chryssi Giannitsarou, Flavio Toxvaerd, Nigel Collier

Value of Information: A Framework for Human–Agent Communication (2026.acl-long)

Challenge:	stance detection is the task of automatically classifying the writer's opinion expressed in a text towards a particular target.
Approach:	They propose a robust multi-task neural architecture that combines textual input with high-frequency intra-day time series from stock market prices.
Outcome:	The proposed system achieves state-of-the-art on the wt–wt dataset.

Copied to clipboard

Yijiang River Dong, Tiancheng Hu, Zheng Hui, Caiqi Zhang, Ivan Vulić, Andreea Bobu, Nigel Collier

Challenge:	Existing approaches to large language model (LLM) agents fail to account for stakes of different decisions.
Approach:	They propose a framework that balances task risk, query ambiguity, user effort . they use a value-of-information framework to dynamically weigh the expected utility gain .
Outcome:	The proposed model matches or exceeds the best manually-tuned baselines in four domains . it explicitly balances task risk, query ambiguity, and user effort .

POSQA: Probe the World Models of LLMs with Size Comparisons (2023.findings-emnlp)

Copied to clipboard

Chang Shu, Jiuzhou Han, Fangyu Liu, Ehsan Shareghi, Nigel Collier