| Challenge: | Numerous aspects of written language have been studied for native language identification (NLI) but its impact has not been studied. |
| Approach: | They propose to use punctuation marks as indicators of native language . they propose to apply them to native language identification . |
| Outcome: | The proposed methods support the hypothesis that punctuation marks are persistent and robust indicators of the native language of the author, even when a high proficiency level in a non-native language is achieved. |
Similar Papers
Pulling Out All The Full Stops: Punctuation Sensitivity in Neural Machine Translation and Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Several studies have documented the sensitivity of machine translation models to noise, including spelling errors and dialects. |
| Approach: | They propose to test machine translation systems for punctuation sensitivity by performing sentence-final perturbation tests with full stops, exclamation and questions marks. |
| Outcome: | The proposed tests show that commercial translation systems are vulnerable to single punctuation insertion or deletion, resulting in unreliable translations. |
Using Punctuation as an Adversarial Attack on Deep Learning-Based NLP Systems: An Empirical Study (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies show that insertions of a few symbols are a general attack mechanism, but grammar checks do not mitigate them. |
| Approach: | They propose to use punctuation insertions as adversarial attacks on NLP systems to create a toolbox of methods to attack models while also pointing out flaws. |
| Outcome: | The results show that punctuation insertions outperform word-level attacks in settings with a limited number of word synonyms and queries to the victim’s model. |
Punctuations and Predicates in Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent work has shown that LLMs perform tasks in ways that diverge significantly from human reasoning. |
| Approach: | They examine the computational importance of punctuation tokens in large language models . they use zeroing and layer-swapping techniques to examine their necessity and sufficiency . |
| Outcome: | The proposed model differs in GPT-2, DeepSeek, and Gemma in that punctuation is necessary and sufficient in multiple layers . the findings offer new insight into the internal mechanisms of punctuations in LLMs and have implications for interpretability and model analysis. |
Pun Unintended: LLMs and the Illusion of Humor Understanding (2025.emnlp-main)
Copied to clipboard
Alessandro Zangari, Matteo Marcuzzo, Andrea Albarelli, Mohammad Taher Pilehvar, Jose Camacho-Collados
| Challenge: | Existing models for pun detection lack nuanced grasp typical of human interpretation. |
| Approach: | They analyze existing pun detection benchmarks and human evaluation across recent LLMs to find subtle changes in puns that mislead LLM. |
| Outcome: | The proposed models lack the nuance typical of human interpretation and lack the depth of their analysis to detect puns. |
Comprehensive Punctuation Restoration for English and Polish (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Punctuation restoration is a fundamental requirement for the readability of text derived from Automatic Speech Recognition systems. |
| Approach: | They evaluate several methods in the comprehensive punctuation reconstruction task by comparing two languages with a model to determine the quality of the punctuated word. |
| Outcome: | The proposed model improves on two languages with relatively simple and complex morphologies. |
A Survey of Pun Generation: Datasets, Evaluations and Methodologies (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Pun generation aims to modify linguistic elements in text to produce humour or evoke double meanings. |
| Approach: | They propose to review pun generation datasets and methods across different stages . pun generation aims to produce humour or evoke double meanings . |
| Outcome: | This paper summarises both automated and human evaluation metrics used to assess the quality of pun generation. |
Overestimation of Syntactic Representation in Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Several testing methodologies have been developed to probe models’ syntactic representations. |
| Approach: | They propose a method to determine syntactic structure by training a model on strings generated according to a template and testing its ability to distinguish between similar ones with different syntax. |
| Outcome: | The proposed method reproduces positive results with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs. |
When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are sensitive to subtle, non-semantic variations in prompt phrasing and formatting. |
| Approach: | They propose to evaluate 4 methods for improving prompt robustness within a unified experimental framework. |
| Outcome: | The proposed methods are compared to 8 models from Llama, Qwen and Gemma families and are generalized against multiple types of distribution shifts. |
Exploring Linguistic Probes for Morphological Inflection (2023.emnlp-main)
Copied to clipboard
| Challenge: | morphological inflection models typically employ language-independent data splitting algorithms. |
| Approach: | They propose language-specific probes to test aspects of morphological generalization . they use three morphology-distinct languages to test their generalization abilities . |
| Outcome: | The proposed language-specific probes are used to test morphological generalization abilities on three distinct languages. |
A unified approach to sentence segmentation of punctuated text in many languages (2021.acl-long)
Copied to clipboard
| Challenge: | Existing tools for segmenting punctuated text in many languages are limited in their language coverage and evaluation is ad hoc. |
| Approach: | They propose a new context-based modeling approach that can be trained on noisily-annotated data. |
| Outcome: | The proposed model exceeds baselines set by existing methods on English corpora and performs well on average on new multilingual evaluation set. |