Papers by Xitong Zhang
Discourse Heuristics For Paradoxically Moral Self-Correction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | moral self-correction is a promising approach for aligning output of Large Language Models with human moral values . authors show that moral self correction relies on discourse constructions that reflect heuristic shortcuts . |
| Approach: | a new method is proposed to strengthen moral self-correction using heuristics extracted from curated datasets. |
| Outcome: | a new method to strengthen moral self-correction is proposed . the proposed method is based on heuristics extracted from curated datasets. |
Diagnosing Moral Reasoning Acquisition in Language Models: Pragmatics and Generalization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior research has shown that LLMs fail to perform satisfactorily on moral cognizance tasks . |
| Approach: | They propose to use curated datasets to improve LLMs' moral cognizance . they find pragmatic dilemma constrains generalization ability of current learning paradigms . |
| Outcome: | The proposed learning paradigms fail to perform on moral cognizance tasks, the authors show . they show that the pragmatic dilemma is the primary bottleneck for moral reasoning acquisition . |
Not All Voices Are Rewarded Equally: Probing and Repairing Reward Models across Human Diversity (2025.findings-emnlp)
Copied to clipboard
Zihao Li, Feihao Fang, Xitong Zhang, Jiaru Zou, Zhining Liu, Wei Xiong, Ziwei Wu, Baoyu Jing, Jingrui He
| Challenge: | Using real-world datasets, we conduct the most comprehensive study to date, auditing various state-of-the-art reward models across nine sensitive attributes, including age, gender, ethnicity, etc. |
| Approach: | They propose a method to mitigate group disparities in reward modeling by using real-world data. |
| Outcome: | The proposed method is based on a population-based dataset with nine demographic attributes, including gender, ethnicity, age, gender, and ethnicity. |
Towards Understanding Task-agnostic Debiasing Through the Lenses of Intrinsic Bias and Forgetfulness (2024.findings-acl)
Copied to clipboard
Guangliang Liu, Milad Afshari, Xitong Zhang, Zhiyu Xue, Avrajit Ghosh, Bidhan Bashyal, Rongrong Wang, Kristen Johnson
| Challenge: | Debiasing Pretrained Language Models (PLMs) are task-agnostic and can be generalizable, but its impact on language modeling ability and the risk of relearning social biases remain as the two most significant challenges. |
| Approach: | They propose a framework which can Propagate Socially-fair Debiasing to Downstream Fine-tuning to alleviate the forgetting issue of PLMs by regularizing debiased attention heads based on the PLM’s bias levels from stages of pretraining and debiase. |
| Outcome: | The proposed framework can Propagate Socially-fair Debiasing to Downstream Fine-tuning, indicating that the ineffectiveness of debiase can be alleviated by overcoming the forgetting issue through regularizing successfully debiased attention heads based on the PLMs’ bias levels from stages of pretraining and debiases. |
Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models require expensive training on consumer-grade hardwares. |
| Approach: | They propose a sparsity-based approach that introduces trainable sparse adaptations to the weight matrices in the model and offers greater flexibility in selecting fine-tuned parameters. |
| Outcome: | The proposed method outperforms other methods for a simple yet effective baseline for nLP tasks while sacrificing performance. |
OAgents: An Empirical Study of Building Effective Agents (2025.findings-emnlp)
Copied to clipboard
He Zhu, Tianrui Qin, King Zhu, Heyuan Huang, Yeyi Guan, Jinxiang Xia, Hanhao Li, Yi Yao, Ningning Wang, Pai Liu, Tianhao Peng, Xin Gui, Li Xiaowan, Yuhui Liu, Xiangru Tang, Jian Yang, Ge Zhang, Xitong Gao, Yuchen Eleanor Jiang, Changwang Zhang, Jun Wang, Jiaheng Liu, Wangchunshu Zhou
| Challenge: | a recent study shows that agent research practices are far from standard, rigorous . lack of a standard evaluation protocol makes previous works not reproducible, authors say . |
| Approach: | They conduct an empirical study on the GAIA benchmark to investigate agent design choices . they find that lack of a standard evaluation protocol makes previous works not reproducible . |
| Outcome: | The proposed framework achieves state-of-the-art performance among open-source projects. |
PAC-tuning: Fine-tuning Pre-trained Language Models with PAC-driven Perturbed Gradient Descent (2023.emnlp-main)
Copied to clipboard
| Challenge: | PAC-tuning is a two-stage fine-tune method for pretrained language models . PAC training minimizes the PACBayes generalization bound to learn proper parameter distribution . |
| Approach: | They propose a two-stage fine-tuning method to minimize the PAC-Bayes generalization bound . they use PAC to inject noise with variance learned in the first stage into the model parameters . |
| Outcome: | The proposed method outperforms baseline methods on 5 GLUE benchmark tasks. |