Argument Mining in Data Scarce Settings: Cross-lingual Transfer and Few-shot Techniques (2024.acl-long)
Copied to clipboard
| Challenge: | Recent work on sequence labelling has explored different strategies to mitigate the lack of manually annotated data for the large majority of the world languages. |
| Approach: | They propose to use the mask objective to exploit the few-shot capabilities of pre-trained language models to improve their performance. |
| Outcome: | The proposed model-transfer outperforms data-transference and fine-tuning outperformed few-shot methods for Argument Mining task. |
Similar Papers
Multilingual Argument Mining: Datasets and Analysis (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Argument mining tasks in non-English languages are dominated by English . we use a pre-trained language model that supports 104 languages to train models . |
| Approach: | They propose a multilingual BERT model to address argument mining tasks in non-English languages . they use English datasets and machine translation to facilitate transfer learning . |
| Outcome: | The proposed model is well suited for classifying the stance of arguments and detecting evidence, but less so for assessing the quality of arguments. |
Few-shot Learning with Multilingual Generative Language Models (2022.emnlp-main)
Copied to clipboard
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li
| Challenge: | Large-scale generative language models such as GPT-3 are competitive few-shot learners. |
| Approach: | They train multilingual generative language models on a corpus covering a diverse set of languages and study their few- and zero-shot learning capabilities. |
| Outcome: | The proposed model outperforms GPT-3 on 171 out of 182 directions with 32 training examples and surpasses the official supervised baseline in 45 directions. |
Make the Best of Cross-lingual Transfer: Evidence from POS Tagging with over 100 Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual generalisability of large pre-trained models use English training data and test data in unseen languages. |
| Approach: | They propose to use multilingual pre-trained models to model cross-lingual transfer in a selection of target languages. |
| Outcome: | The proposed model can be used to improve cross-lingual transfer performance in low-resource languages with no labeled training data. |
Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need! (C18-1)
Copied to clipboard
| Challenge: | Argumentation mining (AM) requires the identification of complex discourse structures . existing resources are not adequate for assessing cross-lingual AM due to their heterogeneity or lack of complexity. |
| Approach: | They propose to use a dataset to translate persuasive student essays into German, French, Spanish, and Chinese to compare arguments mining and annotation projection. |
| Outcome: | The proposed methods perform better when using expensive human or cheap machine translations and almost eliminate loss from cross-lingual transfer. |
Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to train models to provide natural language explanations (NLEs) require acquisition of task-specific NLEs, which is time- and resource-consuming. |
| Approach: | They propose a few-shot out-of-domain transfer of NLEs from a parent task to a child task . they propose four methods that cover possible fine-tuning combinations of NLESs and labels . |
| Outcome: | The proposed methods cover the possible fine-tuning combinations of labels and NLEs for the parent and child tasks. |
How to Translate Your Samples and Choose Your Shots? Analyzing Translate-train & Few-shot Cross-lingual Transfer (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have focused on zero-shot cross-lingual transfer of pretrained languages. |
| Approach: | They propose to use few-shot cross-lingual transfer to improve zero-shot performance of multilingual pretrained language models. |
| Outcome: | The proposed model can be scaled to high-quality samples and improves on zero-shot performance. |
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer (2024.naacl-long)
Copied to clipboard
Akari Asai, Sneha Kudugunta, Xinyan Yu, Terra Blevins, Hila Gonen, Machel Reid, Yulia Tsvetkov, Sebastian Ruder, Hannaneh Hajishirzi
| Challenge: | Recent advances in few-shot generalization in natural language processing focus on English. |
| Approach: | They propose a benchmark that unifies 15 diverse tasks across 54 languages in a sequence-to-sequence format and provides a fixed set of few-shot examples and instructions. |
| Outcome: | The proposed framework unifies 15 diverse tasks across 54 languages in a sequence-to-sequence format and provides a fixed set of few-shot examples and instructions. |
Massively Multilingual Transfer for NER (P19-1)
Copied to clipboard
| Challenge: | Existing approaches for cross-lingual transfer use a single source language, but there are exceptions. |
| Approach: | They propose two techniques for modulating the transfer, suitable for zero-shot or few-shot learning, respectively. |
| Outcome: | The proposed methods are much more effective than baseline models and rival oracle selection of the single best individual model. |
”Diversity and Uncertainty in Moderation” are the Key to Data Selection for Multilingual Few-shot Transfer (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches for few-shot transfer show significant gain over zero-shot transfers . language resource distribution is skewed across the world's languages . proposed methods use multiple measures such as data entropy and gradient embedding . |
| Approach: | They propose a loss embedding method for sequence labeling tasks that induces diversity and uncertainty sampling similar to gradient embeddment. |
| Outcome: | The proposed methods outperform baseline methods for POS tagging, NER, and NLI tasks for up to 20 languages. |
Don’t Prompt, Search! Mining-based Zero-Shot Learning with Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has obtained strong zero-shot results by prompting language models. |
| Approach: | They propose a mining-based approach that uses regular expressions to mine labeled examples from unlabeled corpora and fine tune a pretrained model. |
| Outcome: | The proposed method outperforms prompting on a wide range of tasks when using comparable templates. |