Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

EasyNLP: A Comprehensive and Easy-to-use Toolkit for Natural Language Processing (2022.emnlp-demos)

Challenge:	Transformer-based language models (LMs) are opaque and unexplained, causing problems for endusers and developers who wish to debug or fix their behaviour.
Approach:	They propose an interactive debugger tool for transformer-based LMs that provides a fine-grained interpretation of the model's internal prediction process and a powerful framework for intervening in LM behavior.
Outcome:	The proposed tool provides a fine-grained interpretation of the model's internal prediction construction process, and a powerful framework for intervening in LM behavior.

Copied to clipboard

Chengyu Wang, Minghui Qiu, Taolin Zhang, Tingting Liu, Lei Li, Jianing Wang, Ming Wang, Jun Huang, Wei Lin

An Explainable Toolbox for Evaluating Pre-trained Vision-Language Models (2022.emnlp-demos)

Challenge:	Pre-Trained Models (PTMs) have reshaped the development of natural language processing (NLP) but it is not easy to obtain high-performing PTMs without a large amount of labeled training data and deploy them online with fast inference speed.
Approach:	They propose to make it easy to build NLP applications with knowledge-enhanced pre-training and knowledge distillation.
Outcome:	EasyNLP supports a comprehensive suite of NLP algorithms and features knowledge-enhanced pre-training, knowledge distillation and few-shot learning functionalities.

Copied to clipboard

Tiancheng Zhao, Tianqi Zhang, Mingwei Zhu, Haozhan Shen, Kyusong Lee, Xiaopeng Lu, Jianwei Yin

TweetNLP: Cutting-Edge Natural Language Processing for Social Media (2022.emnlp-demos)

Challenge:	Existing studies evaluate VLP models by comparing the fine-tuned downstream task performance with the average downstream task accuracy.
Approach:	They propose a toolbox for evaluating Vision-Language Pretraining (VLP) models.
Outcome:	The proposed toolbox provides the preliminary datasets that deepen the image-texting ability of a VLP model.

Copied to clipboard

Jose Camacho-collados, Kiamehr Rezaee, Talayeh Riahi, Asahi Ushio, Daniel Loureiro, Dimosthenis Antypas, Joanne Boisson, Luis Espinosa Anke, Fangyu Liu, Eugenio Martínez Cámara

Challenge:	TweetNLP is an integrated platform for natural language processing in social media.
Approach:	They propose a Python-based platform for natural language processing in social media that supports a variety of NLP tasks.
Outcome:	The proposed platform supports generic focus areas such as sentiment analysis and named entity recognition, as well as social media-specific tasks such as emoji prediction and offensive language identification.

JoeyS2T: Minimalistic Speech-to-Text Modeling with JoeyNMT (2022.emnlp-demos)

Copied to clipboard

Mayumi Ohta, Julia Kreutzer, Stefan Riezler

FairLib: A Unified Framework for Assessing and Improving Fairness (2022.emnlp-demos)

Challenge:	JoeyS2T is a simple, lightweight NMT extension for speech-to-text tasks such as automatic speech recognition and end-to end speech translation.
Approach:	JoeyS2T is a JoeyNMT extension for automatic speech recognition and end-to-end speech translation.
Outcome:	JoeyS2T performs competitively on English speech recognition and English-to-German speech translation benchmarks.

Copied to clipboard

Xudong Han, Aili Shen, Yitong Li, Lea Frermann, Timothy Baldwin, Trevor Cohn

Responsible & Trustworthy NLP

Challenge:	Existing approaches to assess and improve model fairness have been inconsistent and inconsistent.
Approach:	They propose an open-source python library for assessing and improving model fairness.
Outcome:	The proposed framework can be used for natural language, images, and audio.

ELEVANT: A Fully Automatic Fine-Grained Entity Linking Evaluation and Analysis Tool (2022.emnlp-demos)

Copied to clipboard

Hannah Bast, Matthias Hertel, Natalie Prange

Knowledge Representation

Challenge:	a tool for fine-grained evaluation of entity linkers on benchmarks is presented . a typical evaluation shows that overall precision and recall are poor . particular benchmarks often require very particular skills from an entity linker .
Approach:	They propose a tool for fine-grained evaluation of entity linkers on benchmarks . they use a graph-based tool to analyze performance of a set of entity links .
Outcome:	The proposed tool provides an automatic breakdown of the performance by error categories and by entity type.

A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering (2022.emnlp-demos)

Copied to clipboard

Matt Maufe, James Ravenscroft, Rob Procter, Maria Liakata

DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population (2022.emnlp-demos)

Challenge:	Question Answering (QA) is a growing area of research . state-of-the-art QA models struggle on out-of domain documents without fine-tuning .
Approach:	They propose a pipeline for validating and training QA data and an interface for human annotation.
Outcome:	The proposed pipeline improves QA performance on domain-specific datasets while preserving the accuracy of the model.

Copied to clipboard

Ningyu Zhang, Xin Xu, Liankuan Tao, Haiyang Yu, Hongbin Ye, Shuofei Qiao, Xin Xie, Xiang Chen, Zhoubo Li, Lei Li

Information Extraction & Text Mining

Challenge:	Existing knowledge extraction tools are not complete due to emerging entities and relations in real-world applications.
Approach:	They propose an open-source knowledge extraction toolkit DeepKE that supports low-resource, document-level and multimodal scenarios in the knowledge base population.
Outcome:	The proposed toolkit supports low-resource, document-level and multimodal scenarios in the knowledge base population.

AnEMIC: A Framework for Benchmarking ICD Coding Models (2022.emnlp-demos)

Copied to clipboard

Juyong Kim, Abheesht Sharma, Suhas Shanbhogue, Jeremy Weiss, Pradeep Ravikumar

Challenge:	Diagnostic coding is the task of assigning diagnosis codes defined by the ICD (International Classification of Diseases) standard to patient visits based on clinical notes.
Approach:	They propose to use an ICD coding framework to train and benchmark models . they correct errors in preprocessing and provide an interactive demo to analyze the models based on custom inputs.
Outcome:	The framework corrects errors in preprocessing and provides key models and weights trained on correctly preprocessed datasets.

SPEAR : Semi-supervised Data Programming in Python (2022.emnlp-demos)

Copied to clipboard

Guttu Abhishek, Harshad Ingole, Parth Laturia, Vineeth Dorna, Ayush Maheshwari, Ganesh Ramakrishnan, Rishabh Iyer

Multimodality

Challenge:	a package for data programming with semi supervision implements several recent approaches to label and train machine learning models.
Approach:	They propose an open-source python library for data programming with semi supervision . the library implements several recent data programming approaches including heuristics and association of noisy labels to training datasets.
Outcome:	The proposed package implements several recent approaches for data programming with semi supervision.

Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements (2022.emnlp-demos)

Copied to clipboard

Leandro Von Werra, Lewis Tunstall, Abhishek Thakur, Sasha Luccioni, Tristan Thrush, Aleksandra Piktus, Felix Marty, Nazneen Rajani, Victor Mustar, Helen Ngo

Challenge:	Evaluation is a key part of machine learning, yet there is neo-tooling to support it . auxiliary techniques such as testing for significance, measuring statistical power, and auxiliary methods are not available in ML.
Approach:	They propose a set of tools to facilitate the evaluation of models and datasets in machine learning . they propose 'evaluation on the Hub' platform that enables large-scale evaluation of over 75,000 models .
Outcome:	The proposed tools can be used to evaluate models and datasets on the Hugging Face Hub.

KeywordScape: Visual Document Exploration using Contextualized Keyword Embeddings (2022.emnlp-demos)

Copied to clipboard

Henrik Voigt, Monique Meuschke, Sina Zarrieß, Kai Lawonn

Representation Learning

Challenge:	Existing tools for document visualization assume that keywords have static meanings, but contextualized word embeddings are unrealistic.
Approach:	They propose a visual exploration tool that visualizes contextualized word embeddings in documents based on keywords.
Outcome:	The proposed tool visualizes keywords in terms of their contextualized embeddings in a semantic landscape that keeps keywords with similar context close to each other, allowing for a more precise search and comparison of documents.

MedConQA: Medical Conversational Question Answering System based on Knowledge Graphs (2022.emnlp-demos)

Copied to clipboard

Fei Xia, Bin Li, Yixuan Weng, Shizhu He, Kang Liu, Bin Sun, Shutao Li, Jun Zhao

Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours (2022.emnlp-demos)

Challenge:	Existing medical dialogue systems have the problems of weak scalability, insufficient knowledge, and poor controllability.
Approach:	They propose a medical conversational question-answering system based on the knowledge graph to improve scalability and controllability.
Outcome:	The proposed system can conduct knowledge-grounded dialogues with users, using a Chinese medical knowledge graph and a large-scale dataset.

Copied to clipboard

Eyal Shnarch, Alon Halfon, Ariel Gera, Marina Danilevsky, Yannis Katsis, Leshem Choshen, Martin Santillan Cooper, Dina Epelboim, Zheng Zhang, Dakuo Wang

Text Classification

Challenge:	Label Sleuth is an open source system for labeling and creating text classifiers which does not require coding skills nor machine learning knowledge.
Approach:	Label Sleuth is an open source system for labeling and creating text classifiers which does not require coding skills nor machine learning knowledge.
Outcome:	Label Sleuth is an open source system for labeling and creating text classifiers.

AGReE: A system for generating Automated Grammar Reading Exercises (2022.emnlp-demos)

Copied to clipboard

Sophia Chan, Swapna Somasundaran, Debanjan Ghosh, Mengxuan Zhao

Challenge:	AGReE is a system that generates multiple-choice grammar practice items . common core standards for K-12 English literacy include grammar as a learning outcome .
Approach:	They propose a system that generates multiple-choice grammar practice exercises that can be completed while reading.
Outcome:	The proposed grammar-reading exercise system can be completed while reading . it offers immediate feedback, similar to a more formal incentive system .

BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systems (2022.emnlp-demos)

Copied to clipboard

Guangsen Wang, Samson Tan, Shafiq Joty, Gang Wu, Jimmy Au, Steven C.h. Hoi

DeepGen: Diverse Search Ad Generation and Real-Time Customization (2022.emnlp-demos)

Challenge:	evaluating and troubleshooting production TOD systems is still a largely manual process requiring large amount of human conversations with the systems.
Approach:	They propose a data-efficient end-to-end Bot SIMulation framework for commercial task-oriented dialog systems that can generate user queries and generate semantic-level dialog acts and entities from bot definitions.
Outcome:	The proposed framework is able to infer semantic-level dialog acts and entities from bot definitions and generate user queries via model-based paraphrasing.

Copied to clipboard

Konstantin Golobokov, Junyi Chai, Victor Ye Dong, Mandy Gu, Bingyu Chi, Jie Cao, Yulan Yan, Yi Liu

Information Retrieval

Challenge:	Existing systems that generate ads manually are not effective in generating ad copy and generating millions of ads for large businesses.
Approach:	They propose a system that generates fluent ads from advertiser’s web pages in an abstractive fashion and solves practical issues such as factuality and inference speed.
Outcome:	The proposed system generates fluent ads from advertiser’s web pages in an abstractive fashion and solves practical issues such as factuality and inference speed.

ACCoRD: A Multi-Document Approach to Generating Diverse Descriptions of Scientific Concepts (2022.emnlp-demos)

Copied to clipboard

Sonia Murthy, Kyle Lo, Daniel King, Chandra Bhagavatula, Bailey Kuehl, Sophie Johnson, Jonathan Borchardt, Daniel Weld, Tom Hope, Doug Downey

Challenge:	Current systems that automatically define unfamiliar terms only surface a single "best" description for all users, which may not be accessible for all readers, given varying background knowledge.
Approach:	They propose an end-to-end system that generates sets of descriptions of scientific concepts . ACCoRD corpus includes 1,275 labeled contexts and 1,787 expert-authored concept descriptions .
Outcome:	The proposed system produces diverse descriptions of concepts in terms of reference concepts.

Automatic Comment Generation for Chinese Student Narrative Essays (2022.emnlp-demos)

Copied to clipboard

Zhexin Zhang, Jian Guan, Guowei Xu, Yixiang Tian, Minlie Huang

MIC: A Multi-task Interactive Curation Tool (2022.emnlp-demos)

Challenge:	Existing studies focus on giving discrete scores for holistic quality or distinct traits, but real-world teachers usually provide detailed comments in natural language, which are more informative than single scores.
Approach:	They propose a model which generates comments for specified segments from given student narrative essays using a human-written Chinese dataset.
Outcome:	The proposed model outperforms baselines and has 91% success rate.

Copied to clipboard

Shi Yu, Mingfeng Yang, Jerrod Parker, Stephen Brock

Semantic Text Processing

Challenge:	MIC is a human-machine collaborative curation tool for multiple NLP tasks.
Approach:	They propose to use a multi-task interactive curation tool for multiple NLP tasks to help solve pain points in real NLP.
Outcome:	The proposed tool supports multiple projects with multiple users and allows for collaborative annotations.

SUMMARY WORKBENCH: Unifying Application and Evaluation of Text Summarization Models (2022.emnlp-demos)

Copied to clipboard

Shahbaz Syed, Dominik Schwabe, Martin Potthast

Arabic Word-level Readability Visualization for Assisted Text Simplification (2022.emnlp-demos)

Challenge:	Summary Workbench is a tool for developing and evaluating text summarization models.
Approach:	They propose a tool for developing and evaluating text summarization models that integrates with Docker plugins and provides visual analysis of models’ strengths and weaknesses.
Outcome:	The proposed model and evaluation measures can be easily integrated as Docker-based plugins and provide insights into the models’ strengths and weaknesses.

Copied to clipboard

Reem Hazim, Hind Saddiki, Bashar Alhafni, Muhamed Al Khalil, Nizar Habash

Paraphrasing

Challenge:	a Google Docs add-on for automatic Arabic word-level readability visualization is available for free.
Approach:	They propose a Google Docs add-on for automatic Arabic word-level readability visualization.
Outcome:	The proposed add-on can be used to assess the reading difficulty of a text and identify difficult words as part of manual text simplification.

LogiTorch: A PyTorch-based library for logical reasoning on natural language (2022.emnlp-demos)

Copied to clipboard

Chadi Helwe, Chloé Clavel, Fabian Suchanek

Reasoning

Challenge:	Logical reasoning on natural language is one of the most challenging tasks for deep learning models.
Approach:	They propose a library that includes logical reasoning benchmarks and models . they use a dataset to run the library or fine tune the models with different hyperparameters .
Outcome:	This library includes logical reasoning benchmarks, models, utility functions and co-reference resolution.

stopes - Modular Machine Translation Pipelines (2022.emnlp-demos)

Copied to clipboard

Pierre Andrews, Guillaume Wenzek, Kevin Heffernan, Onur Çelebi, Anna Sun, Ammar Kamran, Yingzhe Guo, Alexandre Mourachko, Holger Schwenk, Angela Fan

Multilinguality

Challenge:	Neural machine translation is a natural language deep learning application that needs data to be trained.
Approach:	They describe a framework that empowers scalability and versatility for research use cases.
Outcome:	The proposed framework empowers scalability and versatility for research use cases.

GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)

Copied to clipboard

Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou

KGI: An Integrated Framework for Knowledge Intensive Language Tasks (2022.emnlp-demos)

Challenge:	Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work.
Approach:	They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations.
Outcome:	The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work.

Copied to clipboard

Md Faisal Mahbub Chowdhury, Michael Glass, Gaetano Rossiello, Alfio Gliozzo, Nandana Mihindukulasooriya

Twitter-Demographer: A Flow-based Tool to Enrich Twitter Data (2022.emnlp-demos)

Challenge:	Existing state-of-the-art retrieval augmented generation models are not available for knowledge-intensive language tasks.
Approach:	They propose a retrieval augmented generation system that showcases the latest state-of-the-art retrieval models on knowledge-intensive language tasks.
Outcome:	The proposed system is based on the core of the KGI system.

Copied to clipboard

Federico Bianchi, Vincenzo Cutrona, Dirk Hovy

Responsible & Trustworthy NLP

Challenge:	199 million people communicate on twitter daily, making it essential to study policy and decision-making.
Approach:	They propose a flow-based tool to augment Twitter data with additional information about tweets and users.
Outcome:	The proposed tool is designed to enhance Twitter data with additional information about tweets and users.

Azimuth: Systematic Error Analysis for Text Classification (2022.emnlp-demos)

Copied to clipboard

Gabrielle Gauthier-melancon, Orlando Marquez Ayala, Lindsay Brin, Chris Tyler, Frederic Branchaud-charron, Joseph Marinier, Karine Grande, Di Le

Information Extraction & Text Mining

Challenge:	Azimuth is an open-source tool to perform error analysis for text classification.
Approach:	They propose an open-source tool to perform error analysis for text classification . they propose to combine a range of ML techniques to facilitate the error analysis stage .
Outcome:	The proposed approach helps AI practitioners discover and address areas where the model does not generalize by leveraging and integrating a range of ML techniques.

SynKB: Semantic Search for Synthetic Procedures (2022.emnlp-demos)

Copied to clipboard

Fan Bai, Alan Ritter, Peter Madrid, Dayne Freitag, John Niekrasz

Information Retrieval

Challenge:	SynKB is an open-source, automatically extracted knowledge base of chemical synthesis protocols.
Approach:	They propose to make SynKB available as an open-source tool for chemists . synKB supports more flexible queries about reaction conditions .
Outcome:	The proposed open-source tool has higher recall and high precision than proprietary chemistry databases.

Camelira: An Arabic Multi-Dialect Morphological Disambiguator (2022.emnlp-demos)

Copied to clipboard

Ossama Obeid, Go Inoue, Nizar Habash

Syntactic Text Processing

Challenge:	Camelira is a web-based Arabic multi-dialect morphological disambiguation tool that covers modern standard Arabic, Egyptian, Gulf, and Levantine.
Approach:	They propose a web-based Arabic multi-dialect morphological disambiguation tool that covers modern standard Arabic, Egyptian, Gulf, and Levantine.
Outcome:	The proposed tool covers modern standard Arabic, Egyptian, Gulf, and Levantine . it also provides an option to automatically choose an appropriate disambiguator based on the prediction of a dialect identification component.

POTATO: The Portable Text Annotation Tool (2022.emnlp-demos)

Copied to clipboard

Jiaxin Pei, Aparna Ananthasubramaniam, Xingyao Wang, Naitian Zhou, Apostolos Dedeloudis, Jackson Sargent, David Jurgens

Challenge:	POTATO is a free, fully open-sourced annotation system that supports labeling many types of text and multimodal data.
Approach:	They propose to use POTATO to design and deploy complex annotation tasks.
Outcome:	The proposed annotation system improves labeling speed and productivity over two tasks.

KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models (2022.emnlp-demos)

Copied to clipboard

Haris Widjaja, Kiril Gashteovski, Wiem Ben Rim, Pengfei Liu, Christopher Malon, Daniel Ruffinelli, Carolin Lawrence, Graham Neubig

Knowledge Representation

Challenge:	Knowledge Graphs (KGs) store information in the form of (head, predicate, tail)-triples.
Approach:	They propose a framework for performing fine-grained evaluation on meaningful subsets of data.
Outcome:	The proposed framework tests models on meaningful subsets of the data, which would have been impossible to detect with standard averaged single-score metrics.

FALTE: A Toolkit for Fine-grained Annotation for Long Text Evaluation (2022.emnlp-demos)

Copied to clipboard

Tanya Goyal, Junyi Jessy Li, Greg Durrett

SEAL: Interactive Tool for Systematic Error Analysis and Labeling (2022.emnlp-demos)

Challenge:	Existing tools to evaluate long text outputs are lacking in the field of NLP . human rating and error analysis remains a crucial component for any evaluation of long text generation.
Approach:	They propose a web-based toolkit to collect fine-grained error annotations for long texts . they use a taxonomy to identify errors and assign them to text spans .
Outcome:	The proposed tool can be used to evaluate the coherence of long generated summaries.

Copied to clipboard

Nazneen Rajani, Weixin Liang, Lingjiao Chen, Margaret Mitchell, James Zou

Visual Data in NLP

Challenge:	Existing models that fail on tail data or rare groups are difficult to identify due to lack of explicit labels.
Approach:	They propose a systematic error analysis and labeling tool that uses a two-step approach to identify high-error slices of data and then give human-understandable semantics to those underperforming slices.
Outcome:	The proposed tool identifies high-error slices of data and gives human-understandable semantics to those underperforming slices.

Hands-On Interactive Neuro-Symbolic NLP with DRaiL (2022.emnlp-demos)

Copied to clipboard

Maria Leonor Pacheco, Shamik Roy, Dan Goldwasser

Knowledge Representation

Challenge:	Existing methods to enhance and correct NLP models require feedback from users.
Approach:	They propose to enhance DRaiL with an easy to use Python interface that allows users to define, modify and augment models, as well as debug and visualize the predictions.
Outcome:	The proposed framework supports predicting sentence and entity level moral sentiment in political tweets.

Paraphrastic Representations at Scale (2022.emnlp-demos)

Copied to clipboard

John Wieting, Kevin Gimpel, Graham Neubig, Taylor Berg-kirkpatrick

Semantic Text Processing

Challenge:	a new system allows users to train their own state-of-the-art paraphrastic sentence representations in a variety of languages.
Approach:	They propose a system that allows users to train their own paraphrastic sentence representations in a variety of languages.
Outcome:	The proposed models outperform previous models on monolingual and cross-lingual tasks and can be used on CPUs with little difference in inference speed.

Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance (2022.emnlp-demos)

Copied to clipboard

Yasaman Razeghi, Raja Sekhar Reddy Mekala, Robert L Logan Iv, Matt Gardner, Sameer Singh

BMCook: A Task-agnostic Compression Toolkit for Big Models (2022.emnlp-demos)

Challenge:	Snoopy allows researchers to analyze the impact of the overlap between pretraining corpus and test data on model performance statistics.
Approach:	They propose to align terms in test instances with their frequency in the Pile to explore correlations between model accuracy and model size and number.
Outcome:	Snoopy allows researchers to analyze term frequency statistics in large language models on NLP benchmarks.

Copied to clipboard

Zhengyan Zhang, Baitao Gong, Yingfa Chen, Xu Han, Guoyang Zeng, Weilin Zhao, Yanxu Chen, Zhiyuan Liu, Maosong Sun

ALToolbox: A Set of Tools for Active Learning Annotation of Natural Language Texts (2022.emnlp-demos)

Challenge:	Existing efforts to compress medium-sized models for specific tasks have limited results.
Approach:	They propose a task-agnostic compression toolkit for big models that implements quantization, pruning, distillation and MoEfication methods.
Outcome:	The proposed tool improves performance on a model with 3 billion parameters by 12x . it also outperforms the original model on three typical NLP benchmarks.

Copied to clipboard

Akim Tsvigun, Leonid Sanochkin, Daniil Larionov, Gleb Kuzmin, Artem Vazhentsev, Ivan Lazichny, Nikita Khromov, Danil Kireev, Aleksandr Rubashevskii, Olga Shahmatova, Dmitry V. Dylov, Igor Galitskiy, Artem Shelmanov

Low-Resource NLP

Challenge:	Currently, the framework supports text classification, sequence tagging, and seq2seq tasks.
Approach:	They propose an open-source framework for active learning annotation in natural language processing that provides an easy-to-deploy GUI annotation tool directly in the Jupyter IDE.
Outcome:	The proposed framework reduces computational overhead and duration of AL iterations and increases annotated data reusability.

TextBox 2.0: A Text Generation Library with Pre-trained Language Models (2022.emnlp-demos)

Copied to clipboard

Tianyi Tang, Junyi Li, Zhipeng Chen, Yiwen Hu, Zhuohao Yu, Wenxun Dai, Wayne Xin Zhao, Jian-yun Nie, Ji-rong Wen