Building an English Vocabulary Knowledge Dataset of Japanese English-as-a-Second-Language Learners Using Crowdsourcing (L18-1)
Copied to clipboard
| Challenge: | a dataset for analyzing the English vocabulary of English-as-a-second language learners is available . a vocabulary size test was performed by 100 test takers hired via crowdsourcing . |
| Approach: | They propose a dataset for analyzing the English vocabulary of English-as-a-second language learners. |
| Outcome: | a dataset for analyzing the English vocabulary of English-as-a-second language learners is available online . the results show that the test is reliable and can be predicted with high accuracy . |
Similar Papers
Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resources (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have examined the quality of labeled data in non-English languages. |
| Approach: | They annotate how datasets are created, input text and label sources, tools used to build them and what they study. |
| Outcome: | The results show that language-proficient NLP researchers' estimated availability correlates with dataset availability. |
TallVocabL2Fi: A Tall Dataset of 15 Finnish L2 Learners’ Vocabulary (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing work on second language knowledge has focused on the knowledge of small numbers of words, often geared towards measuring vocabulary size. |
| Approach: | They propose a “tall” word knowledge response dataset containing information about a few learners’ knowledge of many words. |
| Outcome: | The proposed dataset is based on a self-rating test and translation test and is compared with previous comparable datasets. |
Korean L2 Vocabulary Prediction: Can a Large Annotated Corpus be Used to Train Better Models for Predicting Unknown Words? (L18-1)
Copied to clipboard
| Challenge: | a recent study suggests that a classifier trained on unknown words may yield better results for L2 learners. |
| Approach: | They propose to use a supervised learning classifier to predict word complexity in Korean . they propose to train models on annotated corpus of unknown words with 71 % precision . |
| Outcome: | The proposed model recalls 80 % of unknown words with 71 % precision. |
Crowdsourcing in the Development of a Multilingual FrameNet: A Case Study of Korean FrameNet (2020.lrec-1)
Copied to clipboard
| Challenge: | Using current methods, the construction of multilingual FrameNets is expensive and complex. |
| Approach: | They evaluated whether crowdsourcing approaches captured cross-cultural and cross-linguistic meanings . they found that crowd workers made intuitive choices comparable to trained FrameNet experts . |
| Outcome: | The results are now available in Korean FrameNet 1.1. |
Crowdsourced Corpus of Sentence Simplification with Core Vocabulary (L18-1)
Copied to clipboard
| Challenge: | a crowdsourced corpus of simplified sentences is used to generate complex sentences from more complex ones. |
| Approach: | They propose to use crowdsourced data set of simplified sentences from Japanese textbooks and reference books to generate simplified sentences. |
| Outcome: | The proposed set of simplified sentences is a good quality data set for machine learning. |
A Method for Building a Commonsense Inference Dataset based on Basic Events (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to acquire commonsense are limited by the general-purpose language models. |
| Approach: | They propose a method for building a commonsense inference dataset using crowdsourcing and automatic extraction from a corpus. |
| Outcome: | The proposed method can solve 104k commonsense inference problems in a Japanese corpus with high accuracy, but low bias. |
Crowdsourcing Natural Language Data at Scale: A Hands-On Tutorial (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | a tutorial on crowdsourcing for efficient data annotation will introduce crowdsourcing and provide an overview of the technology. |
| Approach: | This tutorial will introduce users to efficient data annotation via crowdsourcing marketplaces. |
| Outcome: | This tutorial will introduce users to the use of crowdsourcing for data annotation. |
Towards an Automatic Assessment of Crowdsourced Data for NLU (L18-1)
Copied to clipboard
| Challenge: | Recent development of spoken dialog systems aims at allowing a natural input style. |
| Approach: | They investigate how crowdsourced data can be assessed with respect to its naturalness and usefulness by using a word based language model to identify valid data. |
| Outcome: | The proposed methods show that valid data can be identified with the help of a word based language model. |
Rethinking Annotation: Can Language Learners Contribute? (2023.acl-long)
Copied to clipboard
| Challenge: | Researchers have traditionally recruited native speakers to provide annotations for benchmark datasets, but there are languages for which recruiting native speakers is difficult. |
| Approach: | They recruit 36 language learners and provide two types of additional resources and perform mini-tests to measure their language proficiency. |
| Outcome: | The proposed method improves learners' language proficiency in terms of vocabulary and grammar. |
Using Crowdsourced Exercises for Vocabulary Training to Expand ConceptNet (2020.lrec-1)
Copied to clipboard
Christos Rodosthenous, Verena Lyding, Federico Sangati, Alexander König, Umair ul Hassan, Lionel Nicolas, Jolita Horbacauskiene, Anisia Katinskaia, Lavinia Aparaschivei
| Challenge: | Language resources (LRs) are expensive to create and maintain, and this makes it difficult to create or extend LRs. |
| Approach: | They propose to use a Telegram chatbot interface to gather knowledge on word relations suitable for expanding ConceptNet with new words. |
| Outcome: | The proposed model allows to gather 12,000 answers from learners on different question types over 16 days and shows that it is a potential tool for crowdsourcing and fostering vocabulary skills. |