| Challenge: | Various corpora of dialects have been collected using a well-equipped recording environment due to geographical and expense issues. |
| Approach: | They construct a crowdsourced parallel speech corpus of Japanese dialects using crowdsourcing platforms. |
| Outcome: | The proposed corpus includes parallel text and speech data of 21 Japanese dialects. |
Similar Papers
Creating dialect sub-corpora by clustering: a case in Japanese for an adaptive method (L18-1)
Copied to clipboard
| Challenge: | a mixed corpus composed of different dialects is sufficiently resourced to cluster them into dialects. |
| Approach: | They propose a pipeline to derive clusters of dialects from a mixed corpus when their standard counterpart is sufficiently resourced. |
| Outcome: | The proposed pipeline can identify dialectal content when its standard counterpart is sufficiently resourced and can then cluster it into four dialects. |
Parallel Corpus for Japanese Spoken-to-Written Style Conversion (2020.lrec-1)
Copied to clipboard
| Challenge: | spoken-to-written style conversion is becoming an important technology to increase the readability of ASR transcriptions. |
| Approach: | They propose to build a Japanese parallel corpus of spoken-to-written style conversions . they use crowdsourcing to convert spoken-style text into written-style texts . |
| Outcome: | The proposed corpus can handle general and specific spoken-to-written style conversion problems in Japanese. |
Construction of the Corpus of Everyday Japanese Conversation: An Interim Report (L18-1)
Copied to clipboard
Hanae Koiso, Yasuharu Den, Yuriko Iseki, Wakako Kashino, Yoshiko Kawabata, Ken’ya Nishikawa, Yayoi Tanaka, Yasuyuki Usuda
| Challenge: | a new corpus of everyday conversations is being developed in the field of everyday conversation . the corpus is based on 94 hours of recordings of everyday Japanese conversations . |
| Approach: | They propose to build a large-scale corpus of everyday Japanese conversation in a balanced manner. |
| Outcome: | The proposed corpus will be published in 2022 and consist of more than 200 hours of recordings. |
Crowdsourced Corpus of Sentence Simplification with Core Vocabulary (L18-1)
Copied to clipboard
| Challenge: | a crowdsourced corpus of simplified sentences is used to generate complex sentences from more complex ones. |
| Approach: | They propose to use crowdsourced data set of simplified sentences from Japanese textbooks and reference books to generate simplified sentences. |
| Outcome: | The proposed set of simplified sentences is a good quality data set for machine learning. |
Construction of a Japanese Word Similarity Dataset (L18-1)
Copied to clipboard
| Challenge: | evaluating distributed word representations in languages that do not have such resources is difficult . et al., 2015: distributed word represent a sparse vector indicating the word itself or the context of the word. |
| Approach: | They constructed a Japanese word similarity dataset to evaluate distributed representations in Japanese. |
| Outcome: | a Japanese word similarity dataset is the first resource that can be used to evaluate distributed representations in Japanese . the dataset contains various parts of speech and includes rare words in addition to common words . |
The DReaM Corpus: A Multilingual Annotated Corpus of Grammars for the World’s Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Until recently, language descriptions were available in paper form only, with indexes as the only search aid. |
| Approach: | They propose to digitize a multilingual corpus of language descriptions and annotate it with various meta, word, and text attributes to make searching and analysis easier and more useful. |
| Outcome: | The proposed corpus is searchable through a couple of well-established corpus infrastructures. |
Improving Crowdsourcing-Based Annotation of Japanese Discourse Relations (L18-1)
Copied to clipboard
| Challenge: | Discourse parsing is an important task in natural language processing, but few languages have corpora annotated with discourse relations . crowdsourcing-based annotations are of poor quality and require expensive and time-consuming . et al. (2009) evaluated the quality of annotations using expert annotations. |
| Approach: | They construct a Japanese corpus with discourse annotations through crowdsourcing . they propose improvement techniques based on language tests . |
| Outcome: | The proposed methods improve the quality of the annotations, and will make them publicly available. |
JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent machine translation algorithms rely on parallel corpora, but only some resource-rich language pairs can benefit from them. |
| Approach: | They construct a parallel corpus for English-Japanese, which has 8.7 million sentence pairs . they use a web crawler to automatically align parallel sentences in the corpus . |
| Outcome: | The proposed corpus includes a broader range of domains and can be trained with a pre-trained model. |
A Multilingual Parallel Corpora Collection Effort for Indian Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Currently, neural network based approaches for machine translation are data hungry and sentence-level aligned parallel pairs are the currency. |
| Approach: | They propose to build sentence aligned parallel corpora across 10 Indian languages using online sources which have content shared across languages. |
| Outcome: | The proposed corpora significantly extends existing resources that are either not large enough or are restricted to a specific domain (such as health). |
JParaCrawl v3.0: A Large-scale English-Japanese Parallel Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing parallel corpora for English-Japanese are limited, limiting the accuracy of machine translation models. |
| Approach: | They propose a web-based English-Japanese parallel corpus with 21 million unique sentence pairs . this is more than twice as many as the previous corpus JParaCrawl v2.0 . |
| Outcome: | The proposed corpus boosts the accuracy of machine translation models on various domains. |