| Challenge: | The authors of this paper analyze the publications in the field of speech and language processing. |
| Approach: | They propose to analyze publications in the field of speech and language processing to measure innovation . they use the corpus of papers published over 50 years and enlarge it to the SNLP corpus . |
| Outcome: | The proposed method enlarges the corpus of 65,003 documents published over 50 years to the Speech and Language Processing (SNLP) corpus. |
Similar Papers
Systematic Inequalities in Language Technology Performance across the World’s Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have revealed that NLP is limited to a subset of the world’s 6,500 languages. |
| Approach: | They propose a framework for estimating the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. |
| Outcome: | The proposed framework estimates the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. |
A Systematic Review of Reproducibility Research in Natural Language Processing (2021.eacl-main)
Copied to clipboard
| Challenge: | Despite the recent progress in reproducibility, the field is far from reaching a consensus on how reproducibility should be defined, measured and addressed. |
| Approach: | They propose to provide a wide-angle snapshot of current work on reproducibility in NLP. |
| Outcome: | The proposed work will provide a wide-angle snapshot of current work on reproducibility in NLP. |
NLP Scholar: A Dataset for Examining the State of NLP Research (2020.lrec-1)
Copied to clipboard
| Challenge: | Google Scholar is the largest web search engine for academic literature and provides access to rich metadata associated with the papers. |
| Approach: | They extracted citation information from the ACL Anthology (AA) for about 44 thousand NLP papers and identified authors who published at least three papers there. |
| Outcome: | The ACL Anthology (AA) is the largest repository of articles on Natural Language Processing (NLP). |
A Tutorial on Evaluation Metrics used in Natural Language Generation (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial presents the evolution of automatic evaluation metrics to their current state along with emerging trends in this field. |
| Approach: | This tutorial presents the evolution of automatic evaluation metrics to their current state . it aims to assess the extent of scientific progress made and identify areas/components that need improvement . |
| Outcome: | This tutorial presents the evolution of automatic evaluation metrics to their current state along with emerging trends in this field. |
Examining Citations of Natural Language Processing Literature (2020.acl-main)
Copied to clipboard
| Challenge: | citations of NLP papers have decreased in recent years, but long papers get three times as many citation as short papers . citation data from the ACL Anthology and Google Scholar can be used to understand the field and quantify the impact of different types of papers. |
| Approach: | They extract data from the ACL Anthology and Google Scholar to examine trends in citations of NLP papers. |
| Outcome: | The results show that only about 56% of the papers in AA are cited ten or more times . CL Journal has the most cited papers, but its citation dominance has lessened . |
The Nature of NLP: Analyzing Contributions in NLP Papers (2025.acl-long)
Copied to clipboard
| Challenge: | despite this, what constitutes NLP research remains debated . |
| Approach: | They propose a taxonomy of research contributions and introduce a task of automatically identifying contribution statements and classifying their types from NLP research papers. |
| Outcome: | The proposed model analyzes 29k NLP research papers to understand their contributions . |
Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges (2025.acl-long)
Copied to clipboard
Bolei Ma, Yuting Li, Wei Zhou, Ziwei Gong, Yang Janet Liu, Katja Jasinskaja, Annemarie Friedrich, Julia Hirschberg, Frauke Kreuter, Barbara Plank
| Challenge: | linguistics studies how context influences meaning of language and how people use it to convey implied meanings, emotions, and intentions. |
| Approach: | They analyze task designs, data collection methods, evaluation approaches and their relevance to real-world applications. |
| Outcome: | The findings highlight emerging trends, challenges, and gaps in existing benchmarks . the findings will contribute to more nuanced and context-aware NLP models . |
A Survey on Patent Analysis: From NLP to Multimodal AI (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in pretrained language models and large language models have demonstrated transformative capabilities across diverse domains. |
| Approach: | They propose a taxonomy for categorization based on tasks in the patent life cycle . they introduce a novel taxonomies for categorizing based upon tasks in patent life cycles . |
| Outcome: | The proposed method is based on tasks in the patent life cycle and provides a taxonomy for categorization based upon tasks in patent life cycles. |
The Road to Success: Assessing the Fate of Linguistic Innovations in Online Communities (C18-1)
Copied to clipboard
| Challenge: | a longitudinal study of online social networks investigates the birth and spread of lexical innovations. |
| Approach: | They investigate the birth and diffusion of lexical innovations in online communities . they build on sociolinguistic theories and focus on the relationship between the spread of a new term and the social role of the individuals who use it . |
| Outcome: | The proposed method predicts whether an innovation will succeed in a community. |
Efficient Methods for Natural Language Processing: A Survey (2023.tacl-1)
Copied to clipboard
Marcos Treviso, Ji-Ung Lee, Tianchu Ji, Betty van Aken, Qingqing Cao, Manuel R. Ciosici, Michael Hassid, Kenneth Heafield, Sara Hooker, Colin Raffel, Pedro H. Martins, André F. T. Martins, Jessica Zosa Forde, Peter Milder, Edwin Simpson, Noam Slonim, Jesse Dodge, Emma Strubell, Niranjan Balasubramanian, Leon Derczynski, Iryna Gurevych, Roy Schwartz
| Challenge: | Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data, but using only scale to improve performance means resource consumption also grows. |
| Approach: | They propose to use data, time, storage, or energy to improve model performance. |
| Outcome: | The proposed methods and findings provide guidance for conducting NLP under limited resources and point towards promising research directions for developing more efficient methods. |