Papers by Xudong Han
FairLib: A Unified Framework for Assessing and Improving Fairness (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Existing approaches to assess and improve model fairness have been inconsistent and inconsistent. |
| Approach: | They propose an open-source python library for assessing and improving model fairness. |
| Outcome: | The proposed framework can be used for natural language, images, and audio. |
Evaluating Debiasing Techniques for Intersectional Biases (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for debiasing protected attributes have been limited to binary attributes in isolation, however many corpora involve multiple such attributes, possibly with higher cardinality. |
| Approach: | They propose to evaluate a bias-constrained model which is new to NLP and an extension of the iterative nullspace projection technique which can handle multiple identities. |
| Outcome: | The proposed model is based on a new iterative nullspace projection technique which can handle multiple identities. |
A Chinese Dataset for Evaluating the Safeguards in Large Language Models (2024.findings-acl)
Copied to clipboard
Yuxia Wang, Zenan Zhai, Haonan Li, Xudong Han, Shom Lin, Zhenxuan Zhang, Angela Zhao, Preslav Nakov, Timothy Baldwin
| Challenge: | a recent study has shown that large language models can produce harmful responses, exposing users to unexpected risks. |
| Approach: | They propose a dataset for the safety evaluation of Chinese LLMs in Mandarin Chinese . they extend the dataset to better identify false negative and false positive examples . |
| Outcome: | The proposed dataset is for the safety evaluation of Chinese LLMs, and is based on a Chinese dataset. |
Optimising Equal Opportunity Fairness in Model Training (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to reduce bias have been shown to be effective over real-world datasets. |
| Approach: | They propose two new training objectives which directly optimise for the widely-used criterion of equal opportunity. |
| Outcome: | The proposed training objectives directly optimise for the widely-used criterion of equal opportunity while maintaining high performance over two classification tasks. |
Weakly-Supervised Temporal Article Grounding (2022.emnlp-main)
Copied to clipboard
Long Chen, Yulei Niu, Brian Chen, Xudong Lin, Guangxing Han, Christopher Thomas, Hammad Ayyubi, Heng Ji, Shih-Fu Chang
| Challenge: | Existing VG models make unrealistic assumptions about how to ground video segments . a recent study has shown that video grounding can be useful for downstream applications . |
| Approach: | They propose a new task: Weakly-Supervised temporal Article Grounding (WSAG) given an article and a relevant video, WSAG aims to localize all "groundable" sentences to the video. |
| Outcome: | The proposed method is simple but effective, and it can be used in real-world applications. |
Loki: An Open-Source Tool for Fact Verification (2025.coling-demos)
Copied to clipboard
Haonan Li, Xudong Han, Hao Wang, Yuxia Wang, Minghan Wang, Rui Xing, Yilin Geng, Zenan Zhai, Preslav Nakov, Timothy Baldwin
| Challenge: | Loki is an open-source fact-checking tool designed to address the growing problem of misinformation. |
| Approach: | They propose a tool that breaks down the fact-checking task into five steps . they propose LOKI, which offers a semiautomated, human-in-the-loop approach . |
| Outcome: | a new open-source tool is designed to address the growing problem of misinformation . the tool breaks down the fact-checking task into five steps to assist human judgment . |
Balancing out Bias: Achieving Fairness Through Balanced Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to reducing group bias do not account for correlations between author demographics and linguistic variables, limiting their effectiveness. |
| Approach: | They extend a method for countering group bias using balanced training by balancing each demographic group in training and using protected attributes as input. |
| Outcome: | The proposed model outperforms all other methods when combined with balanced training. |
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)
Copied to clipboard
Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji
| Challenge: | We present a new information extraction system that can construct temporal event graphs from news documents. |
| Approach: | They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction . |
| Outcome: | The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities. |
Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring (2025.naacl-long)
Copied to clipboard
Honglin Mu, Han He, Yuxin Zhou, Yunlong Feng, Yang Xu, Libo Qin, Xiaoming Shi, Zeming Liu, Xudong Han, Qi Shi, Qingfu Zhu, Wanxiang Che
| Challenge: | Existing black-box jailbreak methods often rely on model feedback . existing methods may be intercepted by content moderators during the search process . |
| Approach: | They propose a method that guides malicious prompt construction by local training a mirror model of the target black-box model through benign data distillation. |
| Outcome: | The proposed method achieves a 92% attack success rate and 80% stealth rate on a subset of AdvBench. |
RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)
Copied to clipboard
Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji
| Challenge: | Existing methods for event prediction are incomplete and noisy. |
| Approach: | They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework . |
| Outcome: | The proposed framework can be applied to a wide variety of newsworthy scenarios. |
SCALAR: Scientific Citation-based Live Assessment of Long-context Academic Reasoning (2026.eacl-long)
Copied to clipboard
| Challenge: | Long-context understanding is a critical capability for large language models . evaluating this capability requires extensive human annotation, which is time-consuming and costly. |
| Approach: | They propose a benchmark to assess citation-grounded long-context reasoning in academic writing. |
| Outcome: | The proposed benchmark compares state-of-the-art models with human experts on two tasks . human experts achieve 90% accuracy, but most models struggle with the cloze-style task . |
Systematic Evaluation of Predictive Fairness (2022.aacl-main)
Copied to clipboard
| Challenge: | Several methods have been proposed to mitigate bias in training on biased datasets. |
| Approach: | They propose to examine the effect of target class imbalance and stereotyping on model performance by analyzing binary classification, profession prediction and regression tasks. |
| Outcome: | The proposed methods show that data conditions have a strong influence on relative model performance. |
Grounding learning of modifier dynamics: An application to color naming (D19-1)
Copied to clipboard
| Challenge: | Existing models for grounding are unable to understand modified color expressions, such as “light blue”. |
| Approach: | They propose a model that learns more complex transformations in RGB space and a hard ensemble model that selects a color space depending on the modifier-color pair. |
| Outcome: | The proposed model performs better in the HSV color space than the state-of-the-art model. |
Decoupling Adversarial Training for Fair NLP (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing work assumes main task labels and protected attributes are available in the dataset, but protected labels are often unavailable or only available in limited numbers. |
| Approach: | They propose a method which uses only a small volume of protected labels to train adversarial models using a dataset with a discriminator. |
| Outcome: | The proposed method can be used to transfer private-labelled instances from one dataset to another without requiring large amounts of protected labels. |
Do-Not-Answer: Evaluating Safeguards in LLMs (2024.findings-eacl)
Copied to clipboard
| Challenge: | a dataset evaluating harmful capabilities in large language models is available at https://github.com/Libr-AI/do-not-answer. |
| Approach: | They collect an open-source dataset to evaluate the safeguards in large language models . they find that simple BERT-style classifiers can achieve results comparable to GPT-4 . |
| Outcome: | The proposed dataset compares the safety of six popular LLMs to GPT-4 on automatic safety evaluation. |
NAT: Enhancing Agent Tuning with Negative Samples (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning and reinforcement learning use only positive examples, limiting their efficiency in low-resource scenarios. |
| Approach: | They propose a method that leverages both successful and failed trajectories for fine-tuning, maximizing the utility of limited resources. |
| Outcome: | The proposed method surpasses existing methods, including SFT, DPO, and PPO, across various tasks. |
VIEWS: Entity-Aware News Video Captioning (2024.emnlp-main)
Copied to clipboard
Hammad Ayyubi, Tianqi Liu, Arsha Nagrani, Xudong Lin, Mingda Zhang, Anurag Arnab, Feng Han, Yukun Zhu, Xuande Feng, Kevin Zhang, Jialu Liu, Shih-Fu Chang
| Challenge: | Existing video captioning benchmarks and models produce generic captions for videos that lack specific identification of individuals, locations, or organizations. |
| Approach: | They propose a task of directly summarizing news videos into captions that are entity-aware . they validate the effectiveness of their approach across three video captioning models . |
| Outcome: | The proposed approach is effective across three video captioning models. |
Diverse Adversaries for Mitigating Bias in Training (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing adversarial methods only partially mitigate the problem of model bias, added to which their training procedures are unstable. |
| Approach: | They propose a method where discriminators are encouraged to learn orthogonal hidden representations from one another to reduce model bias. |
| Outcome: | The proposed method significantly reduces bias and stability of training over standard methods. |
Nanda Family: Open-Weights Generative Large Language Models for Hindi (2026.eacl-long)
Copied to clipboard
Aaryamonvikram Singh, Debopriyo Banerjee, Dhruv Sahnan, Monojit Choudhury, Shivam Chauhan, Rocktim Jyoti Das, Xudong Han, Haonan Li, Alok Anil Jadhav, Utkarsh Agarwal, Mukund Choudhary, Fajri Koto, Junaid Hamid Bhat, Awantika Shukla, Samujjwal Ghosh, Samta Kamboj, Onkar Pandit, Lalit Pradhan, Rahul Pal, Sunil Kumar Sahu, Parvez Mullah, Ali El Filali, Zainul Abedien Ahmed Quraishi, Neha Sengupta, Gokulakrishnan Ramakrishnan, Rituraj Joshi, Gurpreet Gosal, Avraham Sheinin, Natalia Vassilieva, Preslav Nakov
| Challenge: | Large language models remain predominantly English-centric, which limits their utility for underrepresented languages. |
| Approach: | They propose to extend Llama’s vocabulary with 20% Hindi-specific tokens, thus halving Hindi tokenization fertility while preserving English efficiency. |
| Outcome: | The proposed models outperform open-weight models of comparable size on a 65B-token corpus and bilingual instruction and safety alignment on . a culturally grounded dataset. |
Does Representational Fairness Imply Empirical Fairness? (2022.findings-aacl)
Copied to clipboard
| Challenge: | Neural methods have been trained on datasets which embody cultural and societal stereotypes, captured in spurious correlations between target labels and protected attributes. |
| Approach: | They propose a debiasing method that encourages a latent space that separates instances based on target label, while mixing instances that share protected attributes. |
| Outcome: | The proposed method shows that representational fairness does not imply empirical fairness across methods. |
Demystifying Instruction Mixing for Fine-tuning Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Instruction tuning is effective for aligning large language models with human instructions, but the procedure to optimizing the mixing of instruction datasets is still unclear. |
| Approach: | They categorize instructions into three primary types: NLP downstream tasks, coding, and general chat. |
| Outcome: | The proposed method improves performance of large language models (LLMs) but it is difficult to combine different instruction datasets to optimize overall performance. |
Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP (2023.eacl-main)
Copied to clipboard
| Challenge: | Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. |
| Approach: | They propose to clarify the current situation and plot a course for meaningful progress in fair learning by making clear inter-relations among the current gamut of methods and their relation to fairness theory. |
| Outcome: | The proposed approach addresses the practical problem of model selection, which involves a trade-off between fairness and accuracy and has led to systemic issues in fairness research. |