Papers by Tommi Jauhiainen
Language Variety Identification with True Labels (2024.lrec-main)
Copied to clipboard
Marcos Zampieri, Kai North, Tommi Jauhiainen, Mariano Felice, Neha Kumari, Nishant Nair, Yash Mahesh Bangera
| Challenge: | Language identification datasets are compiled with the assumption that the gold label of each instance is determined by where texts are retrieved from. |
| Approach: | They present a human-annotated multilingual dataset for language variety identification . they use a model to train multiple models to discriminate between different languages . |
| Outcome: | The proposed dataset provides a reliable benchmark toward robust and fairer language variety identification systems. |
HeLI-OTS, Off-the-shelf Language Identifier for Text (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing off-the-shelf language identification tools favor widely used languages, but Heli-OTS can be used to identify a large group of languages. |
| Approach: | They introduce an off-the-shelf text language identification tool using the HeLI method . they compare the He LI-OTS language identifier with fastText on two different data sets . |
| Outcome: | The proposed language identification tool is compared with fastText on two different data sets. |