NLP Workbench: Efficient and Extensible Integration of State-of-the-art Text Mining Tools (2023.eacl-demo)
Copied to clipboard
| Challenge: | NLP Workbench is a web-based text mining platform that allows non-expert users to obtain semantic understanding of large-scale corpora using state-of-the-art text mining models. |
| Approach: | They propose to use a microservice architecture to replace existing models or integrate a new one. |
| Outcome: | The proposed model is extensible and can be easily replaced or integrated with existing models. |
Similar Papers
Dynabench: Rethinking Benchmarking in NLP (2021.naacl-main)
Copied to clipboard
Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, Adina Williams
| Challenge: | Dynabench is an open-source platform for dynamic dataset creation and model benchmarking. |
| Approach: | They propose an open-source platform for dynamic dataset creation and model benchmarking. |
| Outcome: | The proposed platform can be used to create models that fail on simple challenges and falter in real-world scenarios. |
NLP-ADBench: NLP Anomaly Detection Benchmark (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Anomaly detection (AD) is an important machine learning task, but its effectiveness in detecting harmful content, phishing attempts, and spam reviews is limited. |
| Approach: | They introduce NLP-ADBench, the most comprehensive NLP anomaly detection benchmark to date . it includes eight curated datasets and 19 state-of-the-art algorithms . |
| Outcome: | The NLP-ADBench benchmark includes 19 state-of-the-art methods and 8 curated datasets . no single model dominates across all datasets, indicating need for automated model selection . |
Mining Biomedical Publications With The LAPPS Grid (L18-1)
Copied to clipboard
| Challenge: | Natural language processing (NLP) text mining can increase productivity and innovation in the sciences by orders of magnitude. |
| Approach: | The Language Applications Grid is an infrastructure for rapid development of natural language processing applications (NLP) it provides an intuitive and easy-to-use platform for users to exploit NLP tools and resources . the Grid integrates the services and resources provided by PubAnnotation to greatly enhance the user's ability to annotate scientific publications . |
| Outcome: | The Language Applications (LAPPS) Grid is an infrastructure for rapid development of natural language processing applications (NLP) it integrates services and resources provided by PubAnnotation to greatly enhance user's ability to annotate scientific publications and share the results. |
SUMMARY WORKBENCH: Unifying Application and Evaluation of Text Summarization Models (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Summary Workbench is a tool for developing and evaluating text summarization models. |
| Approach: | They propose a tool for developing and evaluating text summarization models that integrates with Docker plugins and provides visual analysis of models’ strengths and weaknesses. |
| Outcome: | The proposed model and evaluation measures can be easily integrated as Docker-based plugins and provide insights into the models’ strengths and weaknesses. |
CogCompNLP: Your Swiss Army Knife for NLP (L18-1)
Copied to clipboard
Daniel Khashabi, Mark Sammons, Ben Zhou, Tom Redman, Christos Christodoulopoulos, Vivek Srikumar, Nicholas Rizzolo, Lev Ratinov, Guanheng Luo, Quang Do, Chen-Tse Tsai, Subhro Roy, Stephen Mayhew, Zhili Feng, John Wieting, Xiaodong Yu, Yangqiu Song, Shashank Gupta, Shyam Upadhyay, Naveen Arivazhagan, Qiang Ning, Shaoshi Ling, Dan Roth
| Challenge: | a corpus-reader module supports popular corpora, feature extraction and annotation modules for semantic and syntactic tasks. |
| Approach: | They propose a library that provides modules to address different challenges . they provide a corpus-reader module that supports popular corpora in the NLP community . |
| Outcome: | The proposed library simplifies the process of design and development of NLP applications by providing modules to address different challenges. |
GlobalBench: A Benchmark for Global Progress in Natural Language Processing (2023.emnlp-main)
Copied to clipboard
Yueqi Song, Simran Khanuja, Pengfei Liu, Fahim Faisal, Alissa Ostapenko, Genta Winata, Alham Aji, Samuel Cahyawijaya, Yulia Tsvetkov, Antonios Anastasopoulos, Graham Neubig
| Challenge: | despite advances in NLP, significant disparities in performance across languages still exist . prior benchmarks focused on a limited number of tasks and languages, but now GlobalBench tracks progress on all languages. |
| Approach: | They propose to use global benchmarks to track progress on all NLP datasets in all languages. |
| Outcome: | a new tool tracks progress on all NLP datasets in all languages and tracks per-speaker utility and equity . globalbench is designed to identify the most under-served languages and reward research efforts . a globalbech is available at https://github.com/neulab/globalbench. |
LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarking (2024.eacl-demo)
Copied to clipboard
Fahim Dalvi, Maram Hasanain, Sabri Boughorbel, Basel Mousi, Samir Abdaljalil, Nizi Nazar, Ahmed Abdelali, Shammur Absar Chowdhury, Hamdy Mubarak, Ahmed Ali
| Challenge: | Recent development and success of Large Language Models necessitate evaluation of their performance across diverse NLP tasks in different languages. |
| Approach: | They propose a framework that can be customized to evaluate LLMs for any NLP task, regardless of language. |
| Outcome: | The LLMeBench framework can be customized to evaluate LLMs for any NLP task, regardless of language. |
A Data-Centric Framework for Composable NLP Workflows (2020.emnlp-demos)
Copied to clipboard
Zhengzhong Liu, Guanxiong Ding, Avinash Bukkittu, Mansi Gupta, Pengzhi Gao, Atif Ahmed, Shikun Zhang, Xin Gao, Swapnil Singhavi, Linwei Li, Wei Wei, Zecong Hu, Haoran Shi, Xiaodan Liang, Teruko Mitamura, Eric Xing, Zhiting Hu
| Challenge: | Empirical natural language processing (NLP) systems involve interoperation among multiple components . a wealth of NLP toolkits exist ( 4), such as spaCy, DKPro, CoreNLP. |
| Approach: | They propose a unified open-source framework that supports fast development of NLP workflows . framework includes processors for NLP tasks, visualization, and annotation . |
| Outcome: | The framework offers processors for NLP tasks, visualization, and annotation, and is extensible . it is delivered through two modularized yet integratable open-source projects, Forte and Stave . |
DeepNLPF: A Framework for Integrating Third Party NLP Tools (2020.lrec-1)
Copied to clipboard
Francisco Rodrigues, Rinaldo Lima, William Domingues, Robson Fidalgo, Adrian Chifu, Bernard Espinasse, Sébastien Fournier
| Challenge: | Many third-party NLP tools perform distinct NLP subtasks, but integration is difficult . authors present a framework that enables easy integration of third-parties into a pipeline . |
| Approach: | They propose a framework that enables easy integration of third-party NLP tools . it provides an API for complete pipeline customization including definition of input/output formats . |
| Outcome: | The proposed framework reduces runtime processing time compared to executing the same pipeline in a sequential manner. |
GLiNER2: Schema-Driven Multi-Task Learning for Structured Information Extraction (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing solutions for information extraction (IE) require specialized models for different tasks or require expensive large language models. |
| Approach: | They propose a framework that enhances the original GLiNER architecture to support named entity recognition, text classification, and hierarchical structured data extraction within a single efficient model. |
| Outcome: | The proposed framework improves performance across diverse IE tasks and accessibility compared to LLM-based alternatives. |