| Challenge: | et al., a leading database of humanitarian documents, compiled with ReliefWeb reports . documents selected with language identification and noise reduction techniques . authors present corpora of English, French and Spanish humanitarian documents . |
| Approach: | They present three corpora of English, French and Spanish humanitarian documents compiled with ReliefWeb reports . documents were tokenized, lemmatized, tagged by part of speech, and enriched with metadata . authors propose a project to develop a humanitarian dictionary with a focus on conceptual variation . |
| Outcome: | The corpora were compiled to satisfy the research needs of the Humanitarian Encyclopedia project with a focus on conceptual variation. |
Similar Papers
The DReaM Corpus: A Multilingual Annotated Corpus of Grammars for the World’s Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Until recently, language descriptions were available in paper form only, with indexes as the only search aid. |
| Approach: | They propose to digitize a multilingual corpus of language descriptions and annotate it with various meta, word, and text attributes to make searching and analysis easier and more useful. |
| Outcome: | The proposed corpus is searchable through a couple of well-established corpus infrastructures. |
An Empirical Evaluation of Annotation Practices in Corpora from Language Documentation (2020.lrec-1)
Copied to clipboard
| Challenge: | Language documentation projects have produced substantial amounts of primary data from a wide variety of endangered languages. |
| Approach: | They propose to use common annotation conventions in existing corpora to facilitate their future processing. |
| Outcome: | The proposed formats are based on the common ELAN and Toolbox formats and are used to facilitate their future processing. |
A Multilingual Parallel Corpora Collection Effort for Indian Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Currently, neural network based approaches for machine translation are data hungry and sentence-level aligned parallel pairs are the currency. |
| Approach: | They propose to build sentence aligned parallel corpora across 10 Indian languages using online sources which have content shared across languages. |
| Outcome: | The proposed corpora significantly extends existing resources that are either not large enough or are restricted to a specific domain (such as health). |
A review of Spanish corpora annotated with negation (C18-1)
Copied to clipboard
| Challenge: | Existing corpora annotated with negation information are small and not always compatible . negation is a linguistic phenomenon that is not addressed in English . |
| Approach: | They review existing corpora annotated with negation in Spanish and analyze compatibility . they propose to develop a supervised negation processing system for Spanish . |
| Outcome: | The proposed system will not be able to merge the small corpora in Spanish due to lack of compatibility in annotations. |
Elote, Choclo and Mazorca: on the Varieties of Spanish (2024.naacl-long)
Copied to clipboard
| Challenge: | Spanish is the official language in 20 countries and the second most-spoken native language . available corpora treat it as one monolithic language, damping prediction power . |
| Approach: | They compile and curate datasets in different varieties of Spanish around the world at an unprecedented scale and create the CEREAL corpus. |
| Outcome: | The results show that Spanish is a multilingual language with a wide range of cultural and cultural influences. |
BasqueParl: A Bilingual Corpus of Basque Parliamentary Transcriptions (2022.lrec-1)
Copied to clipboard
Nayla Escribano, Jon Ander Gonzalez, Julen Orbegozo-Terradillos, Ainara Larrondo-Ureta, Simón Peña-Fernández, Olatz Perez-de-Viñaspre, Rodrigo Agerri
| Challenge: | a new corpus of Basque parliamentary transcripts is released to study political discourse in contrasting languages . a corpus containing political discourses from public institutions can be used for computational social science research . |
| Approach: | They present a corpus from Basque parliamentary transcripts and enrich it with metadata related to relevant attributes of speakers and speeches. |
| Outcome: | The proposed corpus is characterized by heavy Basque-Spanish code-switching . it provides interesting insights about language use of political representatives across time, parties and gender . |
The ParCoLab Parallel Corpus and Its Extension to Four Regional Languages of France (2024.lrec-main)
Copied to clipboard
Dejan Stosic, Saša Marjanović, Delphine Bernhard, Myriam Bras, Laurent Kevers, Stella Retali-Medori, Marianne Vergez-Couret, Carole Werner
| Challenge: | Parallel corpora are scarce for most of the world's language pairs. |
| Approach: | They propose to extend ParCoLab with a parallel corpus for Alsatian, Corsican, Occitan and Poitevin-Saintongeais. |
| Outcome: | The proposed corpus contains more than 20k tokens per regional language. |
The PALMA Corpora of African Varieties of Portuguese (2022.lrec-1)
Copied to clipboard
Tjerk Hagemeijer, Amália Mendes, Rita Gonçalves, Catarina Cornejo, Raquel Madureira, Michel Généreux
| Challenge: | a corpus of urban varieties of Portuguese is being studied in Angola, Mozambique and So Tomé and Prncipe . the corpora are transcribed spoken data, complemented by metadata describing the setting of the audio recordings and sociolinguistic information about the speakers. |
| Approach: | They present three new corpora of urban varieties of Portuguese spoken in Angola, Mozambique and So Tomé and Prncipe . they provide new, contemporary data for the study of each variety and for comparative research on African, Brazilian and European varieties . |
| Outcome: | The corpora are transcribed spoken data and annotated with POS and lemma information . they are already being used for comparative research on possession and location . |
No Data to Crawl? Monolingual Corpus Creation from PDF Files of Truly low-Resource Languages in Peru (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for extracting text from PDF files are expensive and limited by the absence of web content of endangered languages. |
| Approach: | They propose a method for creating monolingual corpora for four endangered languages . they use a PDF file format with multilingual sentences and noisy pages . |
| Outcome: | The proposed method allows the creation of clean corpora for the four languages, a key resource for natural language processing tasks nowadays. |
Parallel Corpora for the Biomedical Domain (L18-1)
Copied to clipboard
| Challenge: | Existing corpora of parallel corporata are being used in the biomedical domain . MT is known to support readers' access to textual documents in a language other than their native language . |
| Approach: | They propose to leverage parallel corpora to implement cross-lingual information retrieval or machine translation tools. |
| Outcome: | The proposed corpus is being used in the biomedical task at the conference on machine translation (WMT'16 and WMT'17) it can be leveraged to provide access to health information in languages other than English. |