Andy Luecking, Giuseppe Abrami, Leon Hammerla, Marc Rahn, Daniel Baumartz, Steffen Eger, Alexander Mehler
| Challenge: | Using the examples of English and German, we examine how parsers trained on modern variants of these languages can be transferred to older language levels without loss. |
| Approach: | They develop a treebank of diachronic corpora enriched with dependency annotations using 3 parsers, 6 pre-trained language models, 5 newly trained models for German, and two tag sets. |
| Outcome: | The proposed treebank covers the time period from 1800 until today and is based on the DependencyAnnotator annotation tool. |
Similar Papers
BERTrade: Using Contextual Embeddings to Parse Old French (2022.lrec-1)
Copied to clipboard
| Challenge: | a growing interest in digital humanities for automatic processing and annotation of historical texts is generating new models for historical languages. |
| Approach: | They use POS-tagging and dependency parsing to evaluate contextual word embedding models . Old French is one of the historical languages for which they have the largest amount of syntactically annotated data . |
| Outcome: | The proposed model can be used to improve performance in Old French, the authors show . they use POS-tagging and dependency parsing to evaluate the model's quality . |
Universal Dependencies: Extensions for Modern and Historical German (2024.lrec-main)
Copied to clipboard
| Challenge: | a new UD treebank is being developed for Middle High German annotations . the annotation scheme is inconsistent with other treebanks for this period . |
| Approach: | They propose to extend the UD scheme for modern and historical German by a range of tokens . they propose to use a treebank that is the first UD treebank for Middle High German . |
| Outcome: | The proposed extensions relate in part to differences between arguments and modifiers . the proposed treebank is the first UD treebank for Middle High German . |
Detecting Syntactic Change with Pre-trained Transformer Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a fine-tuned BERT model can distinguish between text from the early 1800s and late 1900s . we use it to identify specific instances of syntactic change and specific words for which a new part of speech was introduced. |
| Approach: | They propose to use a BERT-based model to find syntactic differences between English of the early 1800s and that of the late 1900s. |
| Outcome: | The proposed model can distinguish between English of the early 1800s and that of the late 1900s using only syntactic information. |
Cheating a Parser to Death: Data-driven Cross-Treebank Annotation Transfer (L18-1)
Copied to clipboard
| Challenge: | Using annotated corpus for linguistic purposes is no longer justified . hand-crafted syntactic resources such as grammars and lexicons can be used as sources of features to guide data driven systems. |
| Approach: | They propose an efficient method for transferring annotations between two different treebanks of the same language. |
| Outcome: | The proposed method is based on the Universal Dependency annotation scheme and was evaluated on the gold standard (94.75% of LAS, 99.40% UAS on the test set). |
Possessors Change Over Time: A Case Study with Artworks (D18-1)
Copied to clipboard
| Challenge: | Existing methods to extract possession relations from Wikipedia articles can be used to extract possessors over time. |
| Approach: | They propose to extract possession relations from Wikipedia articles and temporal information indicating when these relations are true. |
| Outcome: | The proposed annotation scheme yields many possessors over time for a given artwork, and an LSTM ensemble can automate the task. |
How Bad are PoS Tagger in Cross-Corpora Settings? Evaluating Annotation Divergence in the UD Project. (N19-1)
Copied to clipboard
| Challenge: | Using annotation variation principles, Part-of-Speech tagging performance degrades when applied to test sentences that depart from training data. |
| Approach: | They propose to use the annotation variation principle to identify inconsistencies between annotations . they also evaluate their impact on prediction performance . |
| Outcome: | The proposed method can detect errors in gold standard annotations and improve prediction performance. |
Universal Dependencies and Quantitative Typological Trends. A Case Study on Word Order (L18-1)
Copied to clipboard
| Challenge: | a new method is proposed to acquire typological evidence from "gold" treebanks for different languages. |
| Approach: | They propose a method for acquiring typological evidence from "gold" treebanks for different languages. |
| Outcome: | The proposed method can shed light on key issues of the linguistic typological literature. |
Penn-Helsinki Parsed Corpus of Early Modern English: First Parsing Results and Analysis (2022.findings-naacl)
Copied to clipboard
| Challenge: | PPCEME has a large set of function tags and is difficult to parse . authors present results for PPceME using a modified version of the Berkeley Neural Parser . |
| Approach: | They propose to use a modified version of the Berkeley Neural Parser to parse PPCEME using function tags. |
| Outcome: | The proposed parser will be used to parse Early English Books Online, a 1.5 billion word corpus. |
Tracing Syntactic Change in the Scientific Genre: Two Universal Dependency-parsed Diachronic Corpora of Scientific English and German (2022.lrec-1)
Copied to clipboard
| Challenge: | a recent study has focused on the syntactic development of scientific discourse in English and German. |
| Approach: | They present two comparable diachronic corpora of scientific English and German from the Late Modern Period (17th c.–19th d.) annotated with Universal Dependencies. |
| Outcome: | The presented corpora are comparable to existing studies on grammatical change in English and German . the results show that the pre-processing steps significantly improve parsing accuracy . |
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining (2025.acl-long)
Copied to clipboard
Jeffrey Li, Mohammadreza Armandpour, Seyed Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemulapalli, Samy Bengio, Oncel Tuzel, Mehrdad Farajtabar, Hadi Pouransari, Fartash Faghri
| Challenge: | Large language models (LLMs) trained on historical web data inevitably become outdated. |
| Approach: | They propose a web-scale dataset for time-continual pretraining of LLMs derived from 114 dumps of Common Crawl (CC) they also design time-stratified evaluations to assess how well various continual learning methods adapt to new data while retaining past knowledge. |
| Outcome: | The proposed benchmarks show that autoregressive meta-schedules combined with a fixed-ratio replay of older data can achieve comparable held-out loss to re-training from scratch, while requiring significantly less computation (2.6x). |