A Major Obstacle for NLP Research: Let’s Talk about Time Allocation! (2022.emnlp-main)
Copied to clipboard
| Challenge: | Subpar time allocation has been a major obstacle for natural language processing research in recent years, argues a new position paper . |
| Approach: | They propose to identify the biggest traps the NLP community falls into and suggest solutions to solve them. |
| Outcome: | The authors outline multiple concrete problems together with their negative consequences and suggest remedies to improve the status quo. |
Similar Papers
A Systematic Review of Reproducibility Research in Natural Language Processing (2021.eacl-main)
Copied to clipboard
| Challenge: | Despite the recent progress in reproducibility, the field is far from reaching a consensus on how reproducibility should be defined, measured and addressed. |
| Approach: | They propose to provide a wide-angle snapshot of current work on reproducibility in NLP. |
| Outcome: | The proposed work will provide a wide-angle snapshot of current work on reproducibility in NLP. |
Efficient Methods for Natural Language Processing: A Survey (2023.tacl-1)
Copied to clipboard
Marcos Treviso, Ji-Ung Lee, Tianchu Ji, Betty van Aken, Qingqing Cao, Manuel R. Ciosici, Michael Hassid, Kenneth Heafield, Sara Hooker, Colin Raffel, Pedro H. Martins, André F. T. Martins, Jessica Zosa Forde, Peter Milder, Edwin Simpson, Noam Slonim, Jesse Dodge, Emma Strubell, Niranjan Balasubramanian, Leon Derczynski, Iryna Gurevych, Roy Schwartz
| Challenge: | Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data, but using only scale to improve performance means resource consumption also grows. |
| Approach: | They propose to use data, time, storage, or energy to improve model performance. |
| Outcome: | The proposed methods and findings provide guidance for conducting NLP under limited resources and point towards promising research directions for developing more efficient methods. |
On the Gap between Adoption and Understanding in NLP (2021.findings-acl)
Copied to clipboard
| Challenge: | a recent paper argues that current publications foster a gap between adoption and understanding of models . it also makes it easier to meet publication demands with method papers, argues the paper . |
| Approach: | They argue that current NLP publication models foster a gap between adoption and understanding of models . they argue that everlarger models make it harder to explain how our methods work . |
| Outcome: | The authors argue that current publications foster a gap between adoption and understanding of models . they argue that the rise of everlarger models makes it harder to explain how our methods work . |
NLP Needs Diversity outside of ‘Diversity’ (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new position paper argues that diversity in NLP is concentrated on a small number of areas surrounding fairness . |
| Approach: | a new position paper argues that diversity in NLP is disproportionately concentrated on fairness areas. |
| Outcome: | a new position paper argues that diversity in NLP is disproportionately concentrated on fairness areas. |
Is NLP Ready for Standardization? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a number of scientific fields, including telecommunications, networks and multimedia, lack standards in the field of NLP. |
| Approach: | They propose to examine how NLP lacks standards and how that can impact society, industry and regulations. |
| Outcome: | The proposed standards examine the needs of NLP researchers and industry . they argue that the lack of standards can impact the field, society and industry. |
Should We Ban English NLP for a Year? (2022.emnlp-main)
Copied to clipboard
| Challenge: | aaron carroll: two thirds of NLP research is devoted to developing technology for speakers of English . carroll says this bias feeds into consumer technologies to widen existing inequality gaps . he says we need to consider more concrete measures to mitigate climate change . |
| Approach: | a new paper argues that NLP is contributing to global inequalities through a digital language divide . a carbon tax, cap-and-trade and car-free Sundays are examples of measures to mitigate climate change . |
| Outcome: | a new paper argues that NLP is contributing to global inequalities through a digital language divide . a carbon tax, cap-and-trade and car-free Sundays are examples of measures to mitigate climate change . |
Defining a New NLP Playground (2023.findings-emnlp)
Copied to clipboard
Sha Li, Chi Han, Pengfei Yu, Carl Edwards, Manling Li, Xingyao Wang, Yi Fung, Charles Yu, Joel Tetreault, Eduard Hovy, Heng Ji
| Challenge: | Recent explosion of performance of large language models (LLMs) has changed the field more abruptly and seismically than any other shift in the field’s 80 year history. |
| Approach: | They propose 20+ PhD-dissertation-worthy research directions to define a new NLP playground by combining theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications. |
| Outcome: | The proposed research will cover theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications. |
The glass ceiling in NLP (D18-1)
Copied to clipboard
| Challenge: | a glass ceiling exists within the field of NLP, but no study has examined this issue . female representation in Computer Science is lower than the average STEM field . |
| Approach: | They propose to use a mathematical model to show that a glass ceiling exists in NLP . they find that there is a growing mentor gender gap and a disparity between mentors . |
| Outcome: | The proposed model shows that a glass ceiling exists within the field of NLP since the mid 2000s. |
NLP Reproducibility For All: Understanding Experiences of Beginners (2023.acl-long)
Copied to clipboard
| Challenge: | a study with 93 students in an introductory NLP class shows that beginners' programming skill and comprehension of research papers have a limited impact on their effort spent completing the exercise. |
| Approach: | a study conducted with 93 students in an introductory NLP course questioned them on their programming background and programming background. |
| Outcome: | The results show that beginners' programming skill and comprehension of research papers have a limited impact on their effort spent on the exercise. |
Systematic Inequalities in Language Technology Performance across the World’s Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have revealed that NLP is limited to a subset of the world’s 6,500 languages. |
| Approach: | They propose a framework for estimating the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. |
| Outcome: | The proposed framework estimates the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. |