| Challenge: | a study on hate speech against minorities in Italian tweets found that 1 women are the most targeted group. |
| Approach: | They propose to train monolingual transformers and multilingual transformer models with monolingual data in English, Italian, and Spanish to detect misogyny in tweets. |
| Outcome: | The proposed model achieves state-of-the-art on English, Italian, and Spanish. |
Similar Papers
Emerging Cross-lingual Structure in Pretrained Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work has shown that multilingual pretraining works, but is unable to measure these effects. |
| Approach: | They propose to use multilingual masked language modeling to train a model on concatenated text from multiple languages to find universal latent symmetries in embedding spaces. |
| Outcome: | The proposed models can be trained on concatenated text from multiple languages without shared vocabulary or domain similarity. |
XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond (2022.lrec-1)
Copied to clipboard
| Challenge: | Language models are ubiquitous in NLP, but current analyses focus on (multilingual variants of) standard benchmarks and task-specific corpora as multilingual signals. |
| Approach: | They propose a model to train and evaluate multilingual language models in Twitter using a set of Twitter datasets in eight different languages and a XLM-T model. |
| Outcome: | The proposed model trains and evaluates multilingual models on Twitter. |
Assessing the Syntactic Capabilities of Transformer-based Multilingual Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Multilingual Transformer-based language models have been shown to be excellent learners in crosslingual transfer tasks. |
| Approach: | They evaluate the syntactic generalization capabilities of BERT and RoBERTa models on English and Spanish tests. |
| Outcome: | The proposed models perform well on English and Spanish tests, and the proposed tests are compared against models on the same language and models on two different languages. |
Can Monolingual Pretrained Models Help Cross-Lingual Classification? (2020.aacl-main)
Copied to clipboard
| Challenge: | Multilingual pretrained language models have shown impressive results for cross-lingual transfer, but due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors. |
| Approach: | They propose to transfer the knowledge from monolingual pretrained models to multilingual ones to improve zero-shot cross-lingual classification by using machine translation systems. |
| Outcome: | The proposed methods outperform vanilla multilingual fine-tuning on two cross-lingual classification benchmarks. |
How to Solve Few-Shot Abusive Content Detection Using the Data We Actually Have (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for abusive language detection are expensive and lack of knowledge about the target is a challenge. |
| Approach: | They propose to build models cheaply for a new target label set and/or language, using only a few training examples of the target domain. |
| Outcome: | The proposed model improves monolingually and across languages using existing datasets and only a few-shots of the target domain. |
Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models (2022.findings-aacl)
Copied to clipboard
| Challenge: | Zero-shot cross-lingual transfer learning has been shown to be challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between languages, such as hate speech detection. |
| Approach: | They propose to train on multilingual auxiliary tasks to improve zero-shot transfer of hate speech detection models across languages by bringing a cross-lingual knowledge proxy to the task. |
| Outcome: | The proposed methods improve zero-shot transfer of hate speech detection models across languages and domains using multilingual auxiliary tasks fine-tuned. |
Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on multilingual models have focused on their cross-lingual transfer behavior . a recent study examined multilingual model learning from the multilingual pretraining signal . |
| Approach: | They analyze checkpoints during multilingual pretraining to identify when models acquire in-language and cross-lingual abilities. |
| Outcome: | The proposed model achieves high in-language performance early on, with lower-level linguistic skills acquired before more complex ones. |
Cross-domain and Cross-lingual Abusive Language Detection: A Hybrid Approach with Deep Learning and a Multilingual Lexicon (P19-2)
Copied to clipboard
| Challenge: | Detecting online abusive language in social media messages is gaining increasing attention from scholars and stakeholders. |
| Approach: | They propose a hybrid approach with deep learning and a multilingual lexicon to cross-domain and cross-lingual detection of abusive content. |
| Outcome: | The proposed system can detect abusive content across domains and languages using a multilingual lexicon and a domain-independent lexical. |
Comparing Biases and the Impact of Multilingual Training across Multiple Languages (2023.emnlp-main)
Copied to clipboard
Sharon Levy, Neha John, Ling Liu, Yogarshi Vyas, Jie Ma, Yoshinari Fujinuma, Miguel Ballesteros, Vittorio Castelli, Dan Roth
| Challenge: | Currently, studies on bias and fairness in natural language processing focus on a single language and/or across few attributes (e.g. gender, race). However, biases can manifest differently across languages for individual attributes. |
| Approach: | They adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for race, religion, nationality, and gender. |
| Outcome: | The proposed model favors groups that are dominant in each language's culture, indicating bias amplification, after multilingual finetuning. |
MiTTenS: A Dataset for Evaluating Gender Mistranslation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on gender mistranslation in translation systems have highlighted the problem . a dataset of 26 languages is presented to measure the extent of such errors . |
| Approach: | They propose a dataset that measures the extent of gender mistranslation in translation systems . they use handcrafted passages that target known failure patterns and synthetically generated passages . |
| Outcome: | The proposed dataset covers 26 languages from a variety of language families and scripts, including several traditionally under-represented in digital resources. |