Papers by Xudong Shen
FairLib: A Unified Framework for Assessing and Improving Fairness (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Existing approaches to assess and improve model fairness have been inconsistent and inconsistent. |
| Approach: | They propose an open-source python library for assessing and improving model fairness. |
| Outcome: | The proposed framework can be used for natural language, images, and audio. |
Optimising Equal Opportunity Fairness in Model Training (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to reduce bias have been shown to be effective over real-world datasets. |
| Approach: | They propose two new training objectives which directly optimise for the widely-used criterion of equal opportunity. |
| Outcome: | The proposed training objectives directly optimise for the widely-used criterion of equal opportunity while maintaining high performance over two classification tasks. |
Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models (2026.findings-acl)
Copied to clipboard
Xudong Wang, Chaoning Zhang, Chenghao Li, Shuxu Chen, Qigan Sun, Jiaquan Zhang, Fachrina Dewi Puspitasari, Tae-Ho Kim, Jiwei Wei, Malu Zhang, Guoqing Wang, Yang Yang, Heng Tao Shen
| Challenge: | Existing automatic prompt optimization methods fail to optimize prompts and decoding hyperparameters within a unified framework to achieve stable global improvements. |
| Approach: | They propose a dynamic prompt optimization framework for complex reasoning that unifies prompt templates and decodes hyperparameters as inheritable agent configurations. |
| Outcome: | Experiments on multiple mathematical and hybrid reasoning benchmarks show that Agent-GWO improves accuracy and stability over existing prompt optimization methods. |
CODEMENV: Benchmarking Large Language Models on Code Migration (2025.findings-acl)
Copied to clipboard
Keyuan Cheng, Xudong Shen, Yihao Yang, TengyueWang TengyueWang, Yang Cao, Muhammad Asif Ali, Hanbin Wang, Lijie Hu, Di Wang
| Challenge: | Large language models (LLMs) have demonstrated remarkable proficiency in handling a wide range of tasks within the software engineering domain, but their ability to perform code migration—adapting code to different environments—remains underexplored. |
| Approach: | They propose a benchmark to evaluate large language models’ performance in handling code migration tasks. |
| Outcome: | The proposed benchmark comprises 922 data points across 19 Python and Java packages and offers three tasks to systematically evaluate code migration: identifying version-incompatible functions, determining function changes, and adapting code to target environments. |
Systematic Evaluation of Predictive Fairness (2022.aacl-main)
Copied to clipboard
| Challenge: | Several methods have been proposed to mitigate bias in training on biased datasets. |
| Approach: | They propose to examine the effect of target class imbalance and stereotyping on model performance by analyzing binary classification, profession prediction and regression tasks. |
| Outcome: | The proposed methods show that data conditions have a strong influence on relative model performance. |
Truth or Sophistry? LoFa: A Benchmark for LLM Robustness Against Logical Fallacies (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has focused on the ability of Large Language Models to **identify** or **classify** fallacies, but their robustness against these fallacias in persuasive contexts remains largely unexplored. |
| Approach: | They propose a new metric to assess LLM robustness against fallacies by pairing factual questions with fallacious arguments and developing a multi-round debate framework to assess model resilience. |
| Outcome: | The proposed metric disentangles robustness from a model’s knowledge limitations and demonstrates unique vulnerability profiles across models. |
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks (2022.emnlp-main)
Copied to clipboard
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, Xudong Shen
| Challenge: | a benchmark of 1,616 diverse NLP tasks and their expert-written instructions is used to test generalization of models to unseen tasks . a recent study shows that instruction-following models outperform instruction-based models by over 9% . |
| Approach: | They build a benchmark of 1,616 diverse NLP tasks and their expert-written instructions. |
| Outcome: | The proposed model outperforms existing instruction-following models by over 9% on the benchmark despite being smaller. |
Does Representational Fairness Imply Empirical Fairness? (2022.findings-aacl)
Copied to clipboard
| Challenge: | Neural methods have been trained on datasets which embody cultural and societal stereotypes, captured in spurious correlations between target labels and protected attributes. |
| Approach: | They propose a debiasing method that encourages a latent space that separates instances based on target label, while mixing instances that share protected attributes. |
| Outcome: | The proposed method shows that representational fairness does not imply empirical fairness across methods. |
CodeV: Issue Resolving with Visual Data (2025.findings-acl)
Copied to clipboard
Linhao Zhang, Daoguang Zan, Quanshun Yang, Zhirong Huang, Dong Chen, Bo Shen, Tianyu Liu, Yongshun Gong, Huang Pengjie, Xudong Lu, Guangtai Liang, Lizhen Cui, Qianxiang Wang
| Challenge: | Large Language Models (LLMs) have expanded to more complex repository-level tasks. |
| Approach: | They propose a first approach to leveraging visual data to enhance the issue-resolving capabilities of Large Language Models (LLMs) they demonstrate the effectiveness of CodeV and provide valuable insights into leveraging visualization to resolve GitHub issues. |
| Outcome: | The proposed approach improves the issue-resolving capabilities of Large Language Models (LLMs) by using visual data. |
SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models (2025.naacl-long)
Copied to clipboard
Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Nikita Nangia, Anaelia Ovalle, Giada Pistilli, Dragomir Radev, Beatrice Savoldi, Vipul Raheja, Jeremy Qin, Esther Ploeger, Arjun Subramonian, Kaustubh Dhole, Kaiser Sun, Amirbek Djanibekov, Jonibek Mansurov, Kayo Yin, Emilio Villa Cueva, Sagnik Mukherjee, Jerry Huang, Xudong Shen, Jay Gala, Hamdan Al-Ali, null Tair Djanibekov, Nurdaulet Mukhituly, Shangrui Nie, Shanya Sharma, Karolina Stanczak, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Marcelo Viridiano, Oskar Van Der Wal, Adina Yakefu, Aurélie Névéol, Mike Zhang, Sydney Zink, Zeerak Talat
| Challenge: | Large Language Models reproduce and exacerbate social biases present in training data, and resources to quantify this issue are limited. |
| Approach: | They propose a multilingual parallel dataset to examine culturally-specific stereotypes that may be learned by LLMs. |
| Outcome: | The proposed dataset includes stereotypes from 20 regions around the world and 16 languages, spanning multiple identity categories subject to discrimination worldwide. |