DistaLs: a Comprehensive Collection of Language Distance Measures (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing work on how to measure distances between languages has focused on intuition and typological distance. |
| Approach: | They propose a toolkit that provides users with easy access to language distance measures. |
| Outcome: | The proposed toolkit provides easy access to a wide variety of language distance measures. |
Similar Papers
Systematic Inequalities in Language Technology Performance across the World’s Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have revealed that NLP is limited to a subset of the world’s 6,500 languages. |
| Approach: | They propose a framework for estimating the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. |
| Outcome: | The proposed framework estimates the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. |
A Tutorial on Evaluation Metrics used in Natural Language Generation (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial presents the evolution of automatic evaluation metrics to their current state along with emerging trends in this field. |
| Approach: | This tutorial presents the evolution of automatic evaluation metrics to their current state . it aims to assess the extent of scientific progress made and identify areas/components that need improvement . |
| Outcome: | This tutorial presents the evolution of automatic evaluation metrics to their current state along with emerging trends in this field. |
Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resources (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have examined the quality of labeled data in non-English languages. |
| Approach: | They annotate how datasets are created, input text and label sources, tools used to build them and what they study. |
| Outcome: | The results show that language-proficient NLP researchers' estimated availability correlates with dataset availability. |
On Measures of Biases and Harms in NLP (2022.findings-aacl)
Copied to clipboard
Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang
| Challenge: | Recent studies show that natural language processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality. |
| Approach: | They propose a framework for harms and questions to help practitioners understand biases . they propose measurable measures to detect and mitigate biased groups . |
| Outcome: | The proposed framework provides a framework for harms and questions for practitioners to answer to guide the development of bias measures. |
Quantifying the Dialect Gap and its Correlates Across Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Historically, studies investigating minority variants of languages have been limited to a select few languages. |
| Approach: | They evaluate state-of-the-art large language models for regional dialects of several high- and low-resource languages and analyze how regional dialect gap is correlated with economic, social, and linguistic factors. |
| Outcome: | The proposed model is compared with two high-use applications and shows that it can solve the regional dialect gap. |
Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World (2022.aacl-main)
Copied to clipboard
| Challenge: | Linguistic disparity in the NLP world is widely acknowledged, but the reasons behind it are rarely discussed within the field. |
| Approach: | They propose to categorise languages based on speaker population and vitality . they also analyse the distribution of language data resources and amount of NLP/CL research . |
| Outcome: | The proposed model identifies the reasons for the disparity and suggests ways to overcome it. |
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations (2021.acl-demo)
Copied to clipboard
| Challenge: | ACL-IJCNLP 2021 will be an online conference . submissions range from early prototypes to mature production-ready systems . |
| Approach: | the ACL-IJCNLP 2021 will be an online conference . the system demonstration track invites submissions ranging from early prototypes to mature production-ready systems. |
| Outcome: | the ACL-IJCNLP 2021 system demonstration track received 133 submissions . the submission rate was 32.3% . |
NLG-Metricverse: An End-to-End Library for Evaluating Natural Language Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Natural language generation models are a key component of deep learning, says aaron eliott . he says it is crucial to develop and apply better metrics for NLG evaluation . |
| Approach: | a new open-source library for NLG evaluation is created to facilitate researchers to judge the effectiveness of their models. the framework provides a living collection of NLG metrics in a unified and easy-to-use environment. |
| Outcome: | a new open-source library for NLG evaluation aims to improve performance of models . the framework provides tools to apply, analyze, compare, and visualize the metrics . |
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Tutorial Abstracts (2021.acl-tutorials)
Copied to clipboard
| Challenge: | . - (EN) |
| Approach: | . - (EN) |
| Outcome: | . - (EN) |
Analyzing the Surprising Variability in Word Embedding Stability Across Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are powerful representations that form the foundation of many natural language processing architectures. |
| Approach: | They explore word embedding stability in a wide range of languages to gain insight into their stability. |
| Outcome: | The proposed results provide insights into word embedding stability in English and other languages. |