| Challenge: | Historical text normalization often relies on small training datasets. |
| Approach: | They evaluate 63 multi-task learning configurations for sequence-to-sequence-based historical text normalization across ten datasets from eight languages. |
| Outcome: | The proposed learning architecture outperforms the simple, but strong identity baseline. |
Similar Papers
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models (2022.acl-long)
Copied to clipboard
| Challenge: | Massively Multilingual Transformer based Language Models have been shown to be effective on zero-shot transfer across languages, though performance varies from language to language depending on pivot language(s) used for fine-tuning. |
| Approach: | They propose to combine multi-task learning problems with multi-lingual Transformers to model zero-shot transfer across languages. |
| Outcome: | The proposed model can predict zero-shot transfer across languages with a multi-task learning problem with pretraining data in very few languages. |
Zero- and Few-Shot NLP with Pretrained Language Models (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce NLP researchers to the latest techniques for learning from little-to-no data . aims at bringing interested researchers up to speed about the latest and ongoing techniques . |
| Approach: | They aim to introduce techniques for learning from little-to-no data using pretrained language models. |
| Outcome: | This tutorial aims to bring interested NLP researchers up to speed about recent techniques . it will cover methods from manual engineering, better inference algorithms to better tuning methods . |
Zero-Shot Dependency Parsing with Worst-Case Aware Automated Curriculum Learning (2022.acl-short)
Copied to clipboard
| Challenge: | Large multilingual pretrained language models such as mBERT and XLM-RoBERTa have been found to be effective for cross-lingual transfer of syntactic parsing models but only between related languages. |
| Approach: | They propose to use multi-task learning to dynamically optimize for parsing performance on outlier languages by using a multi-level learning approach. |
| Outcome: | The proposed method significantly outperforms uniform and size-proportional sampling in the zero-shot setting. |
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning (2023.acl-long)
Copied to clipboard
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Yang Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
| Challenge: | Existing methods for data-to-text generation focus on specific types of structured data. |
| Approach: | They propose a method that provides a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. |
| Outcome: | The proposed method improves zero-shot and few-shot scenarios and can adapt to new structured data. |
A Large-Scale Comparison of Historical Text Normalization Systems (N19-1)
Copied to clipboard
| Challenge: | a large study of historical text normalization is done on eight languages . there is no consensus on the state-of-the-art approach to normalization . |
| Approach: | They present a large study of historical text normalization done on eight languages . they evaluate four different systems based on supervised learning on datasets from eight different languages based in the literature . |
| Outcome: | The proposed methods are based on supervised learning and are available online. |
Semi-supervised Contextual Historical Text Normalization (2020.acl-main)
Copied to clipboard
| Challenge: | Historical text normalization is the task of mapping historical word forms to their modern counterparts. |
| Approach: | They propose to use a generative normalization model to obtain contextualization from the target-side language model. |
| Outcome: | et al., 2018) show that the most effective approach reduces manual normalization time and manual training costs. |
Evaluating Historical Text Normalization Systems: How Well Do They Generalize? (N18-2)
Copied to clipboard
| Challenge: | Historical text normalization systems aim to convert historical wordforms to their modern equivalents . many of these systems have been developed and tested on a single language . |
| Approach: | They propose to use a nave baseline system to evaluate historical text normalization systems . they show that the models generalize well to unseen words in tests on five languages . |
| Outcome: | The proposed models generalize well to unseen words on five languages, but provide no clear benefit over the nave baseline. |
Few-shot Learning with Multilingual Generative Language Models (2022.emnlp-main)
Copied to clipboard
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li
| Challenge: | Large-scale generative language models such as GPT-3 are competitive few-shot learners. |
| Approach: | They train multilingual generative language models on a corpus covering a diverse set of languages and study their few- and zero-shot learning capabilities. |
| Outcome: | The proposed model outperforms GPT-3 on 171 out of 182 directions with 32 training examples and surpasses the official supervised baseline in 45 directions. |
X-Shot: A Unified System to Handle Frequent, Few-shot and Zero-shot Learning Simultaneously in Classification (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on few-shot and zero-shot learning, but label occurrences vary widely . authors propose a new classification challenge that can be used to manage labels across the full frequency spectrum . |
| Approach: | They propose a new classification challenge that allows for label co-occurrences without predefined limits. |
| Outcome: | The proposed system can handle freq-shot, few-shot and zero-shot labels without limits. |
Zero-Shot Text Classification with Self-Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models have increased attention to zero-shot text classification. |
| Approach: | They propose a plug-and-play method to bridge this gap by requiring only class names along with an unlabeled dataset. |
| Outcome: | The proposed model can be trained on a natural language inference dataset and performs on dozens of unseen tasks without the need for domain expertise or trial and error. |