| Challenge: | We examine different types of noise generated by human errors and how these noisy inputs affect the performance of cognate generation models. |
| Approach: | They evaluate two popular neural cognate generation models’ robustness to human-plausible noise. |
| Outcome: | The proposed models are robust to deletion, duplication, swapping, keyboard errors, and a new type of error, phonological errors. |
Similar Papers
How Reliable are Model Diagnostics? (2021.findings-acl)
Copied to clipboard
| Challenge: | Contemporary statistical models trade off interpretability and simplicity for powerful parameterizations and inductive biases, enabling impressive performance. |
| Approach: | They examine three recent models and find they are not yet reliable . they also formulate recommendations for practitioners and researchers . |
| Outcome: | The proposed models are not as reliable as previously assumed, the authors argue . their findings suggest that they are needed for improving models and training setups . |
Did Translation Models Get More Robust Without Anyone Even Noticing? (2025.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation models are highly sensitive to “noisy” inputs, such as spelling errors, abbreviations, and formatting issues. |
| Approach: | They revisit this insight in light of recent multilingual MT models and large language models applied to machine translation. |
| Outcome: | The proposed models perform better on clean data than previous models, but none of the open models use robustness techniques. |
Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation (D19-55)
Copied to clipboard
| Challenge: | Recent machine translation methods are highly sensitive to orthographical variations such as spelling errors. |
| Approach: | They propose to train machine translation models with random synthetic noise at training time . they focus on translation performance on natural typos, and show robustness to such noise . |
| Outcome: | The proposed method significantly improves translation models on natural typos without accessing natural noise data or distribution. |
Visual Cues and Error Correction for Translation Robustness (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing robustness techniques fail when faced with unseen types of noise and their performance degrades on clean texts. |
| Approach: | They propose visual context to improve translation robustness for noisy texts . they also propose an error correction training regime that can be used as an auxiliary task . |
| Outcome: | The proposed training regime improves translation robustness on noisy texts while maintaining translation quality on clean texts. |
Whispers of Doubt Amidst Echoes of Triumph in NLP Robustness (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to measure robustness are problematic, and out-of-domain evaluations are no longer relevant. |
| Approach: | They examine models of different sizes spanning different architectural choices and pretraining objectives. |
| Outcome: | The results show that not all out-of-domain tests provide insight into robustness . merely scaling models does not make them adequately robust . |
Robust to Noise Models in Natural Language Processing Tasks (P19-2)
Copied to clipboard
| Challenge: | Existing spelling correction systems are far from perfect for noise-sensitive texts . a new way to handle noise is to make models robust to noise. |
| Approach: | They propose a robust to noise word embeddings model which outperforms existing models in different tasks. |
| Outcome: | The proposed model outperforms existing models in three downstream tasks and shows improvements in noise robustness over existing models. |
Improving Robustness of Machine Translation with Synthetic Noise (N19-1)
Copied to clipboard
| Challenge: | Recent work on MT robustness has demonstrated the need to build or adapt systems that are resilient to such noise. |
| Approach: | They propose to synthesize natural noise in social media data to enhance robustness of MT systems by leveraging natural noise. |
| Outcome: | The proposed method can make a vanilla MT system more resilient to noise, partially mitigating loss in accuracy resulting therefrom. |
Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on this topic focus on the robustness of specific detectors or particular attack methods. |
| Approach: | They stress test the detectors’ robustness to malicious attacks under realistic scenarios using LLMs and metric-based detectors. |
| Outcome: | The proposed methods are based on a set of LLM-based models and their performance is compared under different budget levels. |
Proceedings of the 3rd Workshop on Neural Generation and Translation (D19-56)
Copied to clipboard
| Challenge: | The third workshop on neural generation and translation is held in london . the workshop received 68 submissions from leading minds in the field . |
| Approach: | the third workshop on neural generation and translation is held in london . the workshop will feature four invited talks from leading minds in the field . |
| Outcome: | the third workshop on neural generation and translation is held in london . the conference received 68 submissions from which 36 accepted . |
When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are sensitive to subtle, non-semantic variations in prompt phrasing and formatting. |
| Approach: | They propose to evaluate 4 methods for improving prompt robustness within a unified experimental framework. |
| Outcome: | The proposed methods are compared to 8 models from Llama, Qwen and Gemma families and are generalized against multiple types of distribution shifts. |