Exploring the Influence of Spelling Errors on Lexical Variation Measures (C18-1)
Copied to clipboard
| Challenge: | Lexical richness measures such as Type-Token Ratio and Yule's K are often used for learner English analysis and assessment but are unstable because of spelling errors. |
| Approach: | They propose to use a dictionary to calculate the difference between TTR and Yule’s K caused by spelling errors and to deepen the understanding of the influence of spelling errors on them. |
| Outcome: | The proposed measures are based on English learner corpora of three groups and estimate their values before and after spelling errors are manually corrected. |
Similar Papers
An In-Depth Comparison of 14 Spelling Correction Tools on a Common Benchmark (2020.lrec-1)
Copied to clipboard
| Challenge: | False positives and false negatives are common spelling and grammar errors. |
| Approach: | They evaluate 14 spelling correction tools on a common benchmark . they compare sentences from the English Wikipedia distorted using a realistic error model . |
| Outcome: | The evaluation provides a detailed comparison with respect to 12 error categories. |
Spelling convention sensitivity in neural language models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Various long-distance dependencies have been investigated using neural language models. |
| Approach: | They examine whether large neural language models learn the long-distance dependency of British versus American spelling conventions . a large T5 language model does internalize consistency, but only with respect to observed lexical items . |
| Outcome: | The proposed model internalizes consistency with the training corpora, but only with respect to observed lexical items. |
LeSpell - A Multi-Lingual Benchmark Corpus of Spelling Errors to Develop Spellchecking Methods for Learner Language (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing spellcheckers do not work well with learner data. |
| Approach: | They propose a multi-lingual evaluation data set of spelling mistakes in context that is highly customizable for the DKPro architecture. |
| Outcome: | The proposed spellchecker improves performance in many settings and can be customized to meet learners' needs. |
Grammatical Error Correction Using Pseudo Learner Corpus Considering Learner’s Error Tendency (2020.acl-srw)
Copied to clipboard
| Challenge: | Recent studies have focused on improving the performance of grammatical error correction (GEC) tasks using pseudo data. |
| Approach: | They propose to extract sentences similar to those written by language learners and generate pseudo errors by considering error types that learners often make. |
| Outcome: | The proposed model significantly improves the performance of the Russian GEC task compared with other models using pseudo data. |
Grammatical Error Correction: Are We There Yet? (2022.coling-1)
Copied to clipboard
| Challenge: | grammatical error correction (GEC) systems outperform humans on the CoNLL-2014 test set, but there are still classes of errors that they fail to correct. |
| Approach: | They found that state-of-the-art GEC systems outperform humans by a wide margin on the CoNLL-2014 test set . however, they found that there are still classes of errors that they fail to correct . |
| Outcome: | The F0.5 evaluation metric outperforms the CoNLL-2014 test set, but there are still classes of errors that they fail to correct. |
Word Complexity is in the Eye of the Beholder (2021.naacl-main)
Copied to clipboard
| Challenge: | Lexical complexity is a subjective notion, yet it is often neglected in lexical simplification and readability systems which use a ”one-size-fits-all” approach. |
| Approach: | They propose to use a dataset of complex words annotated by readers with different backgrounds to investigate which aspects contribute to the notion of lexical complexity. |
| Outcome: | The proposed approach can be replicated in a dataset of complex words annotated by readers with different backgrounds. |
On Generalization across Measurement Systems: LLMs Entail More Test-Time Compute for Underrepresented Cultures (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) should be able to provide accurate information irrespective of the measurement system at hand . |
| Approach: | They use newly compiled datasets to test if this is true for seven open-source LLMs. |
| Outcome: | The proposed model can provide accurate information regardless of the measurement system at hand. |
How Good (really) are Grammatical Error Correction Systems? (2021.eacl-main)
Copied to clipboard
| Challenge: | Standard evaluations of Grammatical Error Correction systems use a fixed reference text generated relative to the original text. |
| Approach: | They propose to use a gold reference text to evaluate Grammatical Error Correction systems that is generated relative to the original text and is independent of the system output. |
| Outcome: | The proposed evaluations show that the system performs 20-40 points better than standard evaluations. |
CUTE: Measuring LLMs’ Understanding of Their Tokens (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) perform well on a wide variety of tasks, authors say . they lack direct access to characters, which can be difficult to generalize to new languages . |
| Approach: | They propose a benchmark to test the orthographic knowledge of Large Language Models . they find that most LLMs seem to know the spelling of their tokens - yet fail to manipulate text . |
| Outcome: | The proposed benchmark tests the orthographic knowledge of large language models . it finds that most LLMs seem to know the spelling of their tokens, but fail to manipulate text . |
Building Knowledge-Guided Lexica to Model Cultural Variation (2024.naacl-long)
Copied to clipboard
| Challenge: | Cultural variation exists between nations, but also within regions . Historically, it has been difficult to computationally model cultural variation due to a lack of training data and scalability constraints. |
| Approach: | They propose a method to measure cultural variation using a knowledge-guided lexical model using geolocated tweets. |
| Outcome: | The proposed method could help us better understand the way people communicate and build more culturally-aware NLP systems. |