Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach (2021.emnlp-main)
Copied to clipboard
| Challenge: | cuneiform clay tablets were written in 2500 BCE - 100 CE and are a target of extensive transcription and transliteration efforts due to their deterioration. |
| Approach: | They propose to use a masked language modelling task to complete missing text given cuneiform clay tablets written on cuniform signswedges (2500 BCE - 100 CE) they develop models which automatically complete these missing signs based on contextual cues and greedy decoding schemes. |
| Outcome: | The proposed models perform well on missing token prediction (89% hit@5) despite data scarcity (1M tokens), and human evaluations show that they are able to transcribe texts in extinct languages. |
Similar Papers
At the Crossroad of Cuneiform and NLP: Challenges for Fine-grained Part-of-speech Tagging (2024.lrec-main)
Copied to clipboard
| Challenge: | cuneiform texts are dominated by multiple languages and language families . the most dominant language written in cuniform is the Semitic Akkadian . existing cnl models are not suitable for digital editions of Akkadi . |
| Approach: | They focus on letters written in the Semitic Akkadian, a cuneiform language dominated by cuniform texts . they propose to use pre-trained embeddings, sentence segmentation and cnl to fine-tune language models . |
| Outcome: | The dominant language written in cuneiform is the Semitic Akkadian . the paper examines the input material and tries to initiate a discussion about best-practices . |
Embible: Reconstruction of Ancient Hebrew and Aramaic Texts Using Transformers (2024.findings-eacl)
Copied to clipboard
| Challenge: | inscriptions are damaged over time due to earthquakes, fires, political conflicts, and other natural and human-related causes. |
| Approach: | They evaluate masked language modeling approach to reconstruct illegible inscriptions . they use transformer-based models to fine-tune models on Biblical texts . |
| Outcome: | The proposed model achieves the highest text completion accuracy on masked parts of the Biblical texts. |
Automated Phonological Transcription of Akkadian Cuneiform Text (2020.lrec-1)
Copied to clipboard
| Challenge: | Akkadian was an east-semitic language spoken in ancient Mesopotamia . cuneiform text does not mark the inflection for logograms, so the inflected form needs to be inferred from the sentence context. |
| Approach: | They propose to automate phonological transcription of the transliterated Akkadian corpora . transcription is normalized according to the grammatical description of a given dialect . they find that cuneiform text does not mark the inflection for logograms . |
| Outcome: | The proposed transcriptions show the Akkadian renderings for Sumerian logograms, while the logogram transcription is more challenging. |
BabyFST - Towards a Finite-State Based Computational Model of Ancient Babylonian (2020.lrec-1)
Copied to clipboard
| Challenge: | morphological analyzer for Akkadian is not yet available for the extinct language . we present a general finite-state based model for Babylonian that can achieve a coverage of 97.3% and a recall of 93.7% on token level. |
| Approach: | They propose a general finite-state based morphological model for Babylonian that can achieve a coverage of 97.3% and recall up to 93.7% on lemmatization and POS-tagging tasks. |
| Outcome: | The proposed model can achieve coverage and recall of 97.3% on lemmatization and POS-tagging tasks on token level from a transcribed input. |
PHD: Pixel-Based Language Modeling of Historical Documents (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have seen a boom in efforts to digitise historical documents in numerous languages and sources, leading to a transformation in the way historians work. |
| Approach: | They propose a method for generating synthetic scans to resemble real historical documents by pre-training a model to reconstruct masked patches instead of predicting token distributions. |
| Outcome: | The proposed model can reconstruct masked patches and understand language well. |
Validator-Guided Hard Negative Mining for Masked Language Modeling in Low-Resource Ancient Languages (2026.acl-srw)
Copied to clipboard
| Challenge: | Pre-trained multilingual models lack exposure to low-resource ancient languages such as Sumerian. |
| Approach: | They build a hierarchical validator capturing subword, word, and part-of-speech patterns from 4,545 annotated sequences and use it to filter candidates and identify hard negatives for fine-tuning. |
| Outcome: | The proposed model achieves 18.0% hit@10 accuracy, while the validator alone improves this to 72.8%, while hard negative fine-tuning reaches 78.3%. |
Restoring ancient text using deep learning: a case study on Greek epigraphy (D19-1)
Copied to clipboard
| Challenge: | illegible parts of ancient texts must be restored by specialists, known as epigraphists, using deep neural networks to recover missing characters from text input. |
| Approach: | They propose a model that recovers missing characters from a damaged text input using deep neural networks. |
| Outcome: | The proposed model achieves a 30.1% character error rate, compared to the 57.3% of human epigraphists. |
Reconstruction of Cuneiform Literary Texts as Text Matching (2024.lrec-main)
Copied to clipboard
| Challenge: | cuneiform fragment identification is a slow and unsystematic process for reconstructing ancient texts . fragments of cuniform script are often found in fragments written on clay tablets . cnl is able to identify fragments and match them with existing text collections . |
| Approach: | They propose a character-level n-gram-based similarity matching approach to identify fragments . they compare different approaches to identify overlaps between fragments and texts . |
| Outcome: | The proposed approach speeds up the process and reduces the time it takes to complete the work. |
Exploring Large Language Models for Classical Philology (2023.acl-long)
Copied to clipboard
| Challenge: | Recent advances in NLP have led to the creation of powerful language models for many languages including Ancient Greek and Latin. |
| Approach: | They propose to use encoder-only and encoder decoder architectures to create four models for Ancient Greek that vary along two dimensions for tasks of interest for Classical languages. |
| Outcome: | The proposed models improve on existing models of Ancient Greek and Latin and provide a large pre-training corpus for Ancient Greek to support the creation of a larger, comparable model zoo for Classical Philology. |
Enabling Language Models to Fill in the Blanks (2020.acl-main)
Copied to clipboard
| Challenge: | Infilling is the task of predicting missing spans of text at any position in a document. |
| Approach: | They propose a framework which can be used to infill entire sentences . they train off-the-shelf LMs on sequences containing concatenation of masked text . |
| Outcome: | The proposed approach can infill entire sentences on short stories, scientific abstracts, and lyrics. |