Papers by Stanislas Lauly
MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation (2022.emnlp-main)
Copied to clipboard
Anna Currey, Maria Nadejde, Raghavendra Reddy Pappagari, Mia Mayer, Stanislas Lauly, Xing Niu, Benjamin Hsu, Georgiana Dinu
| Challenge: | Existing benchmarks have limited diversity in terms of gender phenomena, sentence structure, or language coverage. |
| Approach: | They propose a benchmark to evaluate gender accuracy in translation from English into eight widely-spoken languages. |
| Outcome: | The proposed benchmark provides realistic, gender-balanced, counterfactual data in eight language pairs where the gender of individuals is unambiguous in the input segment. |