Papers by Zhongbin Xie
An Empirical Analysis of Parameter-Efficient Methods for Debiasing Pre-Trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models inherit more human-like biases from the training corpora, causing computationally expensive problems. |
| Approach: | They propose parameter-efficient methods in combination with counterfactual data augmentation for bias mitigation. |
| Outcome: | The proposed methods are effective in mitigating gender bias, prompt tuning is more suitable for GPT-2 than BERT, and less effective when it comes to racial and religious bias. |
Counter-GAP: Counterfactual Bias Evaluation through Gendered Ambiguous Pronouns (2023.eacl-main)
Copied to clipboard
| Challenge: | a number of studies have focused on gender bias in language models, but these methods fail to detect it. |
| Approach: | They propose to use gender bias in coreference resolution to evaluate gender bias . they propose to construct an annotated quadruple-level dataset with 4008 instances . |
| Outcome: | The proposed method is able to detect gender bias in a quadruple dataset . previous methods failed to detect bias or cancel it, the authors argue . |
Investigating Multi-layer Representations for Dense Passage Retrieval (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Dense retrieval models adopt vectors from the last hidden layer of the document encoder to represent a document, which is in contrast to the fact that representations in different layers of a pre-trained language model contain different kinds of linguistic knowledge and behave differently during fine-tuning. |
| Approach: | They propose to utilize representations from multiple encoder layers to make up the representation of a document, which they denote Multi-layer Representations (MLR). |
| Outcome: | The proposed model outperforms dual encoder, ME-BERT and ColBERT in the single-vector retrieval setting and with other advanced training techniques. |