Papers by Aili Shen
FairLib: A Unified Framework for Assessing and Improving Fairness (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Existing approaches to assess and improve model fairness have been inconsistent and inconsistent. |
| Approach: | They propose an open-source python library for assessing and improving model fairness. |
| Outcome: | The proposed framework can be used for natural language, images, and audio. |
Optimising Equal Opportunity Fairness in Model Training (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to reduce bias have been shown to be effective over real-world datasets. |
| Approach: | They propose two new training objectives which directly optimise for the widely-used criterion of equal opportunity. |
| Outcome: | The proposed training objectives directly optimise for the widely-used criterion of equal opportunity while maintaining high performance over two classification tasks. |
Systematic Evaluation of Predictive Fairness (2022.aacl-main)
Copied to clipboard
| Challenge: | Several methods have been proposed to mitigate bias in training on biased datasets. |
| Approach: | They propose to examine the effect of target class imbalance and stereotyping on model performance by analyzing binary classification, profession prediction and regression tasks. |
| Outcome: | The proposed methods show that data conditions have a strong influence on relative model performance. |
On the (In)Effectiveness of Images for Text Classification (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on text classification, but have shown that images do not improve NLP tasks. |
| Approach: | They focus on text classification, where images complement the text and the Wikipedia page can be in one of a number of different languages. |
| Outcome: | The proposed model trains without external pre-training, but when combined with BERT models pre-trained on large-scale external data, images contribute nothing. |
Evaluating Document Coherence Modeling (2021.tacl-1)
Copied to clipboard
| Challenge: | a new study examines pretrained language models' ability to model discourse and pragmatic phenomena. |
| Approach: | They propose a sentence intrusion detection task using a dictionary dataset . they show that pretrained LMs perform impressively in in-domain evaluation . |
| Outcome: | The proposed dataset shows that pretrained LMs perform impressively in in-domain evaluation, but experience a substantial drop in the cross-domain setting, indicating limited generalization capacity. |
Modelling Uncertainty in Collaborative Document Quality Assessment (D19-55)
Copied to clipboard
| Challenge: | Existing work on document quality assessment relies on predicting the quality of a document relative to a putative gold standard, without paying attention to the subjectivity of this task. |
| Approach: | They propose to use Gaussian processes and random forests to measure the uncertainty in document quality predictions. |
| Outcome: | The proposed methods can predict the quality of Wikipedia articles while providing an estimate of uncertainty when there is inconsistency in the quality labels from the contributors. |
Does Representational Fairness Imply Empirical Fairness? (2022.findings-aacl)
Copied to clipboard
| Challenge: | Neural methods have been trained on datasets which embody cultural and societal stereotypes, captured in spurious correlations between target labels and protected attributes. |
| Approach: | They propose a debiasing method that encourages a latent space that separates instances based on target label, while mixing instances that share protected attributes. |
| Outcome: | The proposed method shows that representational fairness does not imply empirical fairness across methods. |