Papers by Haonan Li

Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models (2025.findings-emnlp)

Challenge:	Pre-trained language models have improved performance for many NLP tasks in finance and healthcare.
Approach:	They propose a large-scale commercial universal language generation model which is pre-trained on a corpus drawn from 10 markets across 7 languages.
Outcome:	The proposed model outperforms other models on commercial generation tasks and on other markets, languages, and tasks.

Copied to clipboard

Haonan He, Yuchen Ren, Yining Tang, Ziyang Xu, Junxian Li, Minghao Yang, Di Zhang, Yuan Dong, Tao Chen, Shufei Zhang, Yuqiang Li, Nanqing Dong, Wanli Ouyang, Dongzhan Zhou, Peng Ye

A Chinese Dataset for Evaluating the Safeguards in Large Language Models (2024.findings-acl)

Challenge:	Biology-Instructions is the first large-scale instruction-tuning dataset for multi-omics biological sequences.
Approach:	They propose a large-scale instruction-tuning dataset for multi-omics biological sequences . they propose 'chatMultiOmics' to overcome limitations of current LLMs on multi-ome tasks .
Outcome:	The proposed dataset bridges LLMs and complex biological sequence-related tasks while maintaining conversational fluency.

Copied to clipboard

Yuxia Wang, Zenan Zhai, Haonan Li, Xudong Han, Shom Lin, Zhenxuan Zhang, Angela Zhao, Preslav Nakov, Timothy Baldwin

Responsible & Trustworthy NLP

Challenge:	a recent study has shown that large language models can produce harmful responses, exposing users to unexpected risks.
Approach:	They propose a dataset for the safety evaluation of Chinese LLMs in Mandarin Chinese . they extend the dataset to better identify false negative and false positive examples .
Outcome:	The proposed dataset is for the safety evaluation of Chinese LLMs, and is based on a Chinese dataset.

CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval (2026.acl-long)

Copied to clipboard

Jiahui Geng, Fengyu Cai, Shaobo Cui, Qing Li, Liangwei Chen, Chenyang Lyu, Haonan Li, Derui Zhu, Alexander Pretschner, Heinz Koeppl, Fakhri Karray

Information Retrieval

Challenge:	Existing benchmarks focus on functional relevance while neglecting code quality.
Approach:	They propose a multilingual benchmark to evaluate quality-aware code retrieval . they include fine-grained quality annotations over 42,725 queries and 134,907 code snippets .
Outcome:	The proposed benchmarks show that state-of-the-art models fail to separate buggy or insecure code from robust counterparts.

Prompt Space Optimizing Few-shot Reasoning Success with Large Language Models (2024.findings-naacl)

Copied to clipboard

Fobo Shi, Peijun Qing, Dong Yang, Nan Wang, Youbo Lei, Haonan Lu, Xiaodong Lin, Duantengchuan Li

MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct (2025.findings-acl)

Challenge:	Prompt engineering is an essential technique for enhancing the abilities of large language models (LLMs) by providing explicit and specific instructions.
Approach:	They propose a new approach that uses text embeddings to obtain basis vectors by matrix decomposition and constructs a space for representing all prompts.
Outcome:	The proposed approach significantly outperforms state-of-the-art prompt paradigms on ten public reasoning benchmarks.

Copied to clipboard

Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang, Yongbin Li, Minzheng Wang, Pengpeng Zeng, Lianli Gao, Heng Tao Shen, Yunshui Li, Hamid Alinejad-Rokny, Xiaobo Xia, Jingkuan Song, Fei Huang

Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis (2022.emnlp-main)

Challenge:	a new framework for image-text instruction data evolution improves MLLM performance . lack of high-quality instruction data remains a major bottleneck in ML modeling .
Approach:	They propose a multimodal instruction data evolution framework that iteratively enhances data quality through fine-grained perception, cognitive reasoning, and interaction evolution.
Outcome:	The proposed approach improves MLLM performance in nine vision-language tasks while using significantly less data.

Copied to clipboard

Shuai Fan, Chen Lin, Haonan Li, Zhenghao Lin, Jinsong Su, Hang Zhang, Yeyun Gong, JIan Guo, Nan Duan

Sentiment Analysis

Challenge:	Existing pre-trained language representation models (PLMs) capture sentiment information from word-level while under-considering sentence-level information.
Approach:	They propose a Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks that enhance the PLM’s knowledge about sentiment words.
Outcome:	The proposed model achieves state-of-the-art on various sentence-level and aspect-level sentiment classification benchmarks.

FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning (2026.acl-long)

Copied to clipboard

Zhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan, Hachem Madmoun, Fan Zhang, Debopriyo Banerjee, Georgi Nenkov Georgiev, Xueqing Peng, Lingfei Qian, Jimin Huang, Jinyan Su, Aaryamonvikram Singh, Rui Xing, Rania Elbadry, Chen Xu, Haonan Li, Fajri Koto, Ivan Koychev, Tanmoy Chakraborty, Yuxia Wang, Salem Lahlou, Veselin Stoyanov, Sophia Ananiadou, Preslav Nakov

Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity (2026.acl-long)

Challenge:	Existing benchmarks emphasize final numerical answers while neglecting intermediate reasoning steps.
Approach:	They propose a symbolic benchmark for verifiable Chain-of-Thought evaluation in finance . FINCHAIN spans 58 topics across 12 financial domains and three difficulty levels .
Outcome:	The proposed benchmark aims to bridge symbolic reasoning and factual verification.

Copied to clipboard

Feiteng Fang, Dingwei Chen, Xiang Huang, Ting-En Lin, Yuchuan Wu, Xiong Liu, Jing Ye, Ziqiang Liu, Haonan Zhang, Liang Zhu, Hamid Alinejad-Rokny, Min Yang, Yongbin Li

Challenge:	Existing approaches to reward modeling in reinforcement learning tasks are limited when dealing with ambiguous preferences.
Approach:	They propose to use AAM to dynamically calibrate preference margins using the Bradley-Terry model's internal parameter knowledge to improve reward modeling in subjective tasks.
Outcome:	The proposed approach improves reward modeling by dynamically calibrating preference margins using the model’s internal parameter knowledge.

Loki: An Open-Source Tool for Fact Verification (2025.coling-demos)

Copied to clipboard

Haonan Li, Xudong Han, Hao Wang, Yuxia Wang, Minghan Wang, Rui Xing, Yilin Geng, Zenan Zhai, Preslav Nakov, Timothy Baldwin

Responsible & Trustworthy NLP

Challenge:	Loki is an open-source fact-checking tool designed to address the growing problem of misinformation.
Approach:	They propose a tool that breaks down the fact-checking task into five steps . they propose LOKI, which offers a semiautomated, human-in-the-loop approach .
Outcome:	a new open-source tool is designed to address the growing problem of misinformation . the tool breaks down the fact-checking task into five steps to assist human judgment .

CMMLU: Measuring massive multitask language understanding in Chinese (2024.findings-acl)

Copied to clipboard

Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, Timothy Baldwin

SCALAR: Scientific Citation-based Live Assessment of Long-context Academic Reasoning (2026.eacl-long)

Challenge:	Existing large language models struggle to achieve an accuracy of even 60%, which is the pass mark for Chinese exams.
Approach:	They propose to use CMMLU to evaluate Chinese multilingual and Chinese LLMs in a comprehensive benchmark that covers various subjects and settings.
Outcome:	The proposed benchmark covers natural sciences, social sciences, engineering, and the humanities and aims to improve on existing models.

Copied to clipboard

Renxi Wang, Honglin Mu, Liqun Ma, Lizhi Lin, Yunlong Feng, Timothy Baldwin, Xudong Han, Haonan Li

Do-Not-Answer: Evaluating Safeguards in LLMs (2024.findings-eacl)

Challenge:	Long-context understanding is a critical capability for large language models . evaluating this capability requires extensive human annotation, which is time-consuming and costly.
Approach:	They propose a benchmark to assess citation-grounded long-context reasoning in academic writing.
Outcome:	The proposed benchmark compares state-of-the-art models with human experts on two tasks . human experts achieve 90% accuracy, but most models struggle with the cloze-style task .

Copied to clipboard

Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, Timothy Baldwin

ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic (2024.findings-acl)

Challenge:	a dataset evaluating harmful capabilities in large language models is available at https://github.com/Libr-AI/do-not-answer.
Approach:	They collect an open-source dataset to evaluate the safeguards in large language models . they find that simple BERT-style classifiers can achieve results comparable to GPT-4 .
Outcome:	The proposed dataset compares the safety of six popular LLMs to GPT-4 on automatic safety evaluation.

Copied to clipboard

Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, Timothy Baldwin

Challenge:	evaluating language models in Arabic remains challenging due to limited datasets . focus has shift to reasoning and knowledge-intensive tasks due to lack of relevant datasets.
Approach:	They propose to use ArabicMMLU to evaluate models' understanding of Arabic . they use 40 tasks and 14,575 multiple-choice questions from school exams in different countries .
Outcome:	The ArabicMMLU is the first multi-task language understanding benchmark for the Arabic language . it is based on 40 tasks and 14,575 multiple-choice questions in modern standard Arabic . the models are based in different countries across North Africa, the Levant, and the Gulf regions .

PAMN: Multi-phase Correlation Modeling for Contrast-Enhanced 3D Medical Image Retrieval (2025.findings-emnlp)

Copied to clipboard

Haonan Tong, Ke Liu, Chuang Zhang, Xinglin Zhang, Tao Chen, Jenq-Neng Hwang, Lei Li

Multimodality

Challenge:	Current 3D medical imaging models focus on spatial features, neglecting phase-specific progression detailed in clinical reports.
Approach:	They propose a framework that fuses imaging phases with clinical text to enhance 3D medical image retrieval.
Outcome:	The proposed framework outperforms state-of-the-art models on a phase-series dataset of 12,230 hospital CT scans.

NAT: Enhancing Agent Tuning with Negative Samples (2025.naacl-long)

Copied to clipboard

Renxi Wang, Xudong Han, Yixuan Zhang, Timothy Baldwin, Haonan Li

Challenge:	Existing methods for fine-tuning and reinforcement learning use only positive examples, limiting their efficiency in low-resource scenarios.
Approach:	They propose a method that leverages both successful and failed trajectories for fine-tuning, maximizing the utility of limited resources.
Outcome:	The proposed method surpasses existing methods, including SFT, DPO, and PPO, across various tasks.

Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification (2024.findings-acl)

Copied to clipboard

Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov

Target Word Masking for Location Metonymy Resolution (2020.coling-main)

Challenge:	Large language models are notorious for producing erroneous claims in their output.
Approach:	They propose a fact-checking and hallucination detection pipeline based on token-level uncertainty quantification that removes the impact of uncertainty about what claim to generate on the current step and what surface form to use.
Outcome:	The proposed method can fact-check the atomic claims in the output of large language models.

Copied to clipboard

Haonan Li, Maria Vasardani, Martin Tomko, Timothy Baldwin

Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource Languages (2026.acl-long)

Challenge:	Existing word sense disambiguation and named entity recognition systems have no explicit metonymy detection.
Approach:	They propose an end-to-end word-level classification approach based only on BERT . they show that their approach generalises well to unseen data .
Outcome:	The proposed approach surpasses conventional models and benchmarks on 5 datasets and generalises well to unseen data.

Copied to clipboard

Saeed Almheiri, Bilal Elbouardi, Salsabila Zahirah Pranida, Irina Nikishina, Ashwath Rao B, Parameswari Krishnamurthy, Muhammad Cendekia Airlangga, Rifo Ahmad Genadi, Nguyen Phan Gia Bao, Amir Hossein Yari, Hawau Olamide Toyin, Nurdaulet Mukhituly, Mena Attia, Besher Hassan, Ahmad Fathan Hidayatullah, Tatsuki Kuribayashi, Haonan Li, Suma Bhat, Fajri Koto

Natural Language Interfaces

Challenge:	idioms are a major challenge for multilingual NLP because their meanings shift between figurative and literal usage, often requiring context for accurate interpretation.
Approach:	They propose a multilingual idiom dataset that provides idiomatic expressions in both sentence-level and conversational contexts.
Outcome:	The proposed model performs well with low-resource idioms, but lacks contextual inference.

KFCNet: Knowledge Filtering and Contrastive Learning for Generative Commonsense Reasoning (2021.findings-emnlp)

Copied to clipboard

Haonan Li, Yeyun Gong, Jian Jiao, Ruofei Zhang, Timothy Baldwin, Nan Duan

InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions (2024.naacl-long)

Challenge:	Pre-trained language models have led to substantial gains over a broad range of NLP tasks, but have limitations for high-quality tasks such as commonsense generation and ad keyword generation.
Approach:	They propose a Knowledge Filtering and Contrastive learning Network which references external knowledge and achieves better generation performance.
Outcome:	The proposed model outperforms the current state of the art on the CommonGen benchmark by a large margin.

Copied to clipboard

Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, Yujiu Yang

Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU (2023.emnlp-main)

Challenge:	In order to perform downstream tasks, Large Language Models (LLMs) need continual adaptation without catastrophic forgetting.
Approach:	They propose a new paradigm that allows for continual adaptation without catastrophic forgetting . they propose to replay previous data based on task similarity with instructions .
Outcome:	The proposed method improves performance over 16 tasks with different training orders.

Copied to clipboard

Fajri Koto, Nurul Aisyah, Haonan Li, Timothy Baldwin

EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models (2024.acl-long)

Challenge:	Existing studies on large language models based on English datasets do not provide adequate data for evaluating their capabilities beyond English.
Approach:	They propose a multi-task language understanding benchmark for Indonesian culture and languages . it measures language proficiency, reasoning abilities and real-world knowledge .
Outcome:	The proposed model passes the primary school level in Indonesia, while other models perform at lower levels.

Copied to clipboard

Rocktim Das, Simeon Hristov, Haonan Li, Dimitar Dimitrov, Ivan Koychev, Preslav Nakov

Multimodality

Challenge:	Existing benchmarks for vision language models are outdated and unable to accurately assess their performance.
Approach:	They propose a multi-discipline multimodal multilingual exam benchmark for vision language models . they collect multiple-choice questions across 20 disciplines across 11 languages from 7 language families .
Outcome:	The EXAMS-V exam includes 20,932 multiple-choice questions across 20 disciplines . the questions come in 11 languages from 7 language families and require advanced reasoning skills .

Getting More Juice Out of Your Data: Hard Pair Refinement Enhances Visual-Language Models Without Extra Data (2025.naacl-long)

Copied to clipboard

Haonan Wang, Minbin Huang, Runhui Huang, Lanqing Hong, Hang Xu, Tianyang Hu, Xiaodan Liang, Zhenguo Li, Hong Cheng, Kenji Kawaguchi

Nanda Family: Open-Weights Generative Large Language Models for Hindi (2026.eacl-long)

Challenge:	Contrastive Language-Image Pre-training (CLIP) is a standard for cross-modal image-text representation learning.
Approach:	They propose a framework that enhances pre-trained CLIP models by exploiting challenging text-image pairs within existing datasets.
Outcome:	The proposed framework improves CLIP models by exploiting text-image pairs in training.

Copied to clipboard

Aaryamonvikram Singh, Debopriyo Banerjee, Dhruv Sahnan, Monojit Choudhury, Shivam Chauhan, Rocktim Jyoti Das, Xudong Han, Haonan Li, Alok Anil Jadhav, Utkarsh Agarwal, Mukund Choudhary, Fajri Koto, Junaid Hamid Bhat, Awantika Shukla, Samujjwal Ghosh, Samta Kamboj, Onkar Pandit, Lalit Pradhan, Rahul Pal, Sunil Kumar Sahu, Parvez Mullah, Ali El Filali, Zainul Abedien Ahmed Quraishi, Neha Sengupta, Gokulakrishnan Ramakrishnan, Rituraj Joshi, Gurpreet Gosal, Avraham Sheinin, Natalia Vassilieva, Preslav Nakov

MultiSpanQA: A Dataset for Multi-Span Question Answering (2022.naacl-main)

Challenge:	Large language models remain predominantly English-centric, which limits their utility for underrepresented languages.
Approach:	They propose to extend Llama’s vocabulary with 20% Hindi-specific tokens, thus halving Hindi tokenization fertility while preserving English efficiency.
Outcome:	The proposed models outperform open-weight models of comparable size on a 65B-token corpus and bilingual instruction and safety alignment on . a culturally grounded dataset.

Copied to clipboard

Haonan Li, Martin Tomko, Maria Vasardani, Timothy Baldwin

Natural Language Interfaces

Challenge:	Existing reading comprehension datasets focus on single-span answers, but multi-spread questions are less studied.
Approach:	They propose a new reading comprehension dataset that focuses on multi-span questions . they introduce new metrics for the purposes of multi--spontaneous question answering evaluation .
Outcome:	The proposed model beats baselines and achieves state-of-the-art on the existing dataset.

Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation (2026.acl-long)

Copied to clipboard

Jiaying Wu, Zihang Fu, Haonan Wang, Fanxiao Li, Jiafeng Guo, Preslav Nakov, Min-Yen Kan

GammaE: Gamma Embeddings for Logical Queries on Knowledge Graphs (2022.emnlp-main)

Challenge:	X (formerly Twitter) users can flag misleading posts, attach contextual notes, and rate the notes’ helpfulness, but there is a significant latency in Community Notes, which is unable to provide accurate notes.
Approach:	They propose a framework that augments Community Notes for faster and more reliable health misinformation governance.
Outcome:	The proposed framework outperforms human contributors in correctness, helpfulness, and evidence utility in health misinformation surges.

Copied to clipboard

Dong Yang, Peijun Qing, Yang Li, Haonan Lu, Xiaodong Lin

Demystifying Instruction Mixing for Fine-tuning Large Language Models (2024.acl-srw)

Challenge:	Existing methods for embedding knowledge graphs are difficult due to complicated query structures and incomplete graph data.
Approach:	They propose a probabilistic embedding model for encoding entities and queries to answer different types of FOL queries on KGs.
Outcome:	The proposed model outperforms state-of-the-art models on public benchmarks on three large logical query datasets.

Copied to clipboard

Renxi Wang, Haonan Li, Minghao Wu, Yuxia Wang, Xudong Han, Chiyu Zhang, Timothy Baldwin