Phonetic Normalization for Machine Translation of User Generated Content (D19-55)
Copied to clipboard
| Challenge: | a method to correct noisy User Generated Content (UGC) in French is proposed . it leverages on the existence of UGC specific noise due to the misuse of words with similar pronunciations. |
| Approach: | They propose a phonetizer-based method to correct noisy User Generated Content (UGC) they use phonetic similarity to generate IPA pronunciations of words . |
| Outcome: | The proposed method improves translation quality of noisy User Generated Content (UGC) in french. |
Similar Papers
Benefits of Data Augmentation for NMT-based Text Normalization of User-Generated Content (D19-55)
Copied to clipboard
| Challenge: | Social media texts are considered important language resources for several NLP tasks, but their use of non-standard words makes it difficult to process and analyze UGC. |
| Approach: | They propose to use a Neural Machine Translation approach to normalize lexical variants to their canonical forms to overcome performance drop in UGC. |
| Outcome: | The proposed approach overcomes a data bottleneck in Dutch, a low-resource language. |
Enhancing BERT for Lexical Normalization (D19-55)
Copied to clipboard
| Challenge: | Pre-trained contextual language models have improved performance of many NLP tasks. |
| Approach: | They propose to use a pre-trained language model to perform lexical normalisation without UGC resources. |
| Outcome: | The proposed model can perform lexical normalisation without the need for training sentences and 3,000 tokens. |
A Taxonomy for In-depth Evaluation of Normalization for User Generated Content (L18-1)
Copied to clipboard
| Challenge: | Existing taxonomies for lexical normalization are not suitable for the task of normalization since the categories are substantially different. |
| Approach: | They propose a taxonomy of error categories for lexical normalization . they annotate a recent normalization dataset and read a near-perfect agreement . |
| Outcome: | The proposed taxonomy is based on a recent normalization dataset and it performs well. |
Towards User-Driven Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | a good translation should implicitly mirror user traits rather than translate the original content semantically. |
| Approach: | They propose a framework that captures user traits from historical inputs . they propose 'user-driven' NMT to model user behavior under a zero-shot learning fashion . |
| Outcome: | The proposed framework can capture user traits from historical inputs under zero-shot learning fashion. |
Handling Normalization Issues for Part-of-Speech Tagging of Online Conversational Text (L18-1)
Copied to clipboard
Géraldine Damnati, Jeremy Auguste, Alexis Nasr, Delphine Charlet, Johannes Heinecke, Frédéric Béchet
| Challenge: | a new approach to POS tagging noisy user generated text is proposed . word embeddings are trained on a noisy corpus to address both normalization and POS. |
| Approach: | They propose to use word embeddings to normalize text before tagging it, while a gated neural network based tagger handles the remaining errors. |
| Outcome: | The proposed approach normalizes some errors before tagging, while a gated neural network handles the remaining errors. |
Dialect-to-Standard Normalization: A Large-Scale Multilingual Evaluation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Text normalization is a range of tasks that consist in replacing non-standard spellings with their standard equivalents. |
| Approach: | They introduce dialect-to-standard normalization as a sentence-level character transduction task and provide a large-scale analysis of these methods. |
| Outcome: | The proposed model performs best for Finnish, Swiss German and Slovene while the pre-trained model using full sentences performs the best for Norwegian. |
Synthetic Data for English Lexical Normalization: How Close Can We Get to Manually Annotated Data? (2020.lrec-1)
Copied to clipboard
| Challenge: | Social media data is a valuable data resource for natural language processing tasks. |
| Approach: | They propose to adapt input text to a more standard form, a task also referred to as normalization. |
| Outcome: | The proposed system scores 94.29 accuracy on the test data compared to 95.22 when trained on human-annotated data. |
Normalising Non-standardised Orthography in Algerian Code-switched User-generated Data (D19-55)
Copied to clipboard
| Challenge: | a new corpus of unstructured data from social media is presenting challenges to NLP research . standardisation is neither natural nor universal, it is rather a human invention. |
| Approach: | They compile a parallel corpus of Arabic textual data matched with human annotations . they use a deep neural model designed to deal with context-dependent spelling correction . |
| Outcome: | The proposed model performs best with two CNN sub-network encoders and an LSTM decoder . pre-processing data token-by-token with edit-distance aligner significantly improves performance . |
Normalizing Non-canonical Turkish Texts Using Machine Translation Approaches (P19-2)
Copied to clipboard
| Challenge: | a study using non-canonical text normalization shows that it can surpass the current best performing system by a large margin. |
| Approach: | They propose a fully automated, context-aware machine translation approach with fewer stages of processing. |
| Outcome: | The proposed approach surpasses the current best-performing system by a large margin . the proposed method is more data-hungry and more data sensitive than other methods . |
PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies suggest that Neural Machine Translation still struggles with certain kinds of input with considerable noise, such as User-Generated Contents (UGC) on the Internet. |
| Approach: | They propose to evaluate the robustness of Neural Machine Translation models against specific linguistic phenomena in Japanese-English translation. |
| Outcome: | The proposed model can handle user-generated content (UGC) on the Internet, but it is difficult to translate clean inputs. |