Papers with PyTorch
TextBox: A Unified, Modularized, and Extensible Framework for Text Generation (2021.acl-demo)
Copied to clipboard
Junyi Li, Tianyi Tang, Gaole He, Jinhao Jiang, Xiaoxuan Hu, Puzhao Xie, Zhipeng Chen, Zhuohao Yu, Wayne Xin Zhao, Ji-Rong Wen
| Challenge: | TextBox is an open-source text generation framework that is modularized and extensible. |
| Approach: | They propose to provide a unified, modularized, and extensible text generation framework that implements 21 text generation models on 9 benchmark datasets. |
| Outcome: | The proposed framework implements 21 models on 9 benchmark datasets and is available under the Apache License 2.0 license. |
JoeyS2T: Minimalistic Speech-to-Text Modeling with JoeyNMT (2022.emnlp-demos)
Copied to clipboard
| Challenge: | JoeyS2T is a simple, lightweight NMT extension for speech-to-text tasks such as automatic speech recognition and end-to end speech translation. |
| Approach: | JoeyS2T is a JoeyNMT extension for automatic speech recognition and end-to-end speech translation. |
| Outcome: | JoeyS2T performs competitively on English speech recognition and English-to-German speech translation benchmarks. |
OpenSLU: A Unified, Modularized, and Extensible Toolkit for Spoken Language Understanding (2023.acl-demo)
Copied to clipboard
| Challenge: | Spoken Language Understanding (SLU) is a task-oriented dialogue system . open-source toolkit provides a unified, modularized, and extensible toolkit for SLU . |
| Approach: | They introduce an open-source toolkit to provide a unified toolkit for spoken language understanding. |
| Outcome: | The proposed toolkit unifies 10 models for both single-intent and multi-intention scenarios. |
fairseq: A Fast, Extensible Toolkit for Sequence Modeling (N19-4)
Copied to clipboard
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli
| Challenge: | OpenNMT is a community-built toolkit written in multiple languages with an emphasis on extensibility. |
| Approach: | They propose to use PyTorch to train custom sequence models for translation, summarization, language modeling, and other tasks. |
| Outcome: | The proposed toolkit is fast, extensible, and useful for both research and production. |
SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for quantization of models are too complicated and can cause performance damage. |
| Approach: | They propose a self-adaptive mixed-precision (SAMP) toolkit to automatically control quantization rate by a mixed-presence architecture to balance model accuracy and efficiency. |
| Outcome: | The proposed toolkit has a higher speedup than PyTorch and FasterTransformer while ensuring the required accuracy. |
LightSeq: A High Performance Inference Library for Transformers (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing inference frameworks for natural language processing are not the best choice for online service of sequence processing problems. |
| Approach: | They propose a highly efficient inference library for Transformer models that includes GPU optimization techniques to streamline computation and reduce memory footprint. |
| Outcome: | The proposed library achieves 14x speedup compared with TensorFlow and 1.4x speed up compared to a concurrent CUDA implementation. |
NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit (P19-3)
Copied to clipboard
| Challenge: | NeuralClassifier is a toolkit for hierarchical multi-label text classification. |
| Approach: | They propose a toolkit for neural hierarchical multi-label text classification . they use a variety of text encoders to implement the model . |
| Outcome: | The proposed model achieves comparable performance with reported results in the literature. |
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding (2020.acl-demos)
Copied to clipboard
Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao
| Challenge: | MT-DNN is an open-source natural language understanding toolkit . it allows researchers and developers to train customized deep learning models . |
| Approach: | They present MT-DNN, an open-source natural language understanding toolkit . it is designed to facilitate rapid customization for a broad spectrum of NLU tasks . MT supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop. |
| Outcome: | The proposed model can significantly compress a large model without significant performance drop. |
Joey NMT: A Minimalist NMT Toolkit for Novices (D19-3)
Copied to clipboard
| Challenge: | a recent study shows that novices perform better than experts in a code quiz. |
| Approach: | They present a minimalist neural machine translation toolkit based on PyTorch . they evaluate the accessibility of the toolkit in a user study . |
| Outcome: | The proposed toolkit performs comparable to more complex toolkits on standard benchmarks. |
Box Embeddings: An open-source library for representation learning using geometric structures (2021.emnlp-demo)
Copied to clipboard
Tejas Chheda, Purujit Goyal, Trang Tran, Dhruvesh Patel, Michael Boratko, Shib Sankar Dasgupta, Andrew McCallum
| Challenge: | Recent studies have explored alternative vector representations with different inductive biases or capabilities. |
| Approach: | They propose a Python library that extends probabilistic box embeddings to geometric shapes and regions. |
| Outcome: | The proposed library is fully open source and compatible with PyTorch and TensorFlow. |
T2NER: Transformers based Transfer Learning Framework for Named Entity Recognition (2021.eacl-demos)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is an important task in information extraction due to large variations in entity names and flexibility in how entities are mentioned. |
| Approach: | They propose a Transformers based Transfer Learning framework for Named Entity Recognition (T2NER) that integrates transformer models with the state-of-the-art in NLP and provides a unified platform for transfer learning. |
| Outcome: | The proposed framework bridges the gap between the state-of-the-art in transformer models and the state of the art in NER with deep transformer models. |
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation (P19-3)
Copied to clipboard
Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan, Eric Xing
| Challenge: | Texar is an open-source text generation toolkit that supports a broad set of text generation tasks. |
| Approach: | They introduce Texar, an open-source text generation toolkit that supports text generation tasks. |
| Outcome: | Texar supports machine translation, summarization, dialog, content manipulation, and more. |
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)
Copied to clipboard
| Challenge: | OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch . |
| Approach: | OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment. |
| Outcome: | OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment . |
DeepPavlov 1.0: Your Gateway to Advanced NLP Models Backed by Transformers and Transfer Learning (2024.emnlp-demo)
Copied to clipboard
Maksim Savkin, Anastasia Voznyuk, Fedor Ignatov, Anna Korzanova, Dmitry Karpov, Alexander Popov, Vasily Konovalov
| Challenge: | Open-source framework for using NLP models is released for non-experts . complexity of building, fine-tuning and deploying state-of-the-art models remains a barrier . |
| Approach: | They present DeepPavlov 1.0, an open-source framework for using NLP models . the framework is based on PyTorch and supports HuggingFace transformers . |
| Outcome: | The DeepPavlov 1.0 framework is designed for practitioners with limited knowledge of NLP/ML. |
Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Named entity recognition models use a conditional random field as the final layer . current work eschews prior knowledge of how the span encoding scheme works . |
| Approach: | They propose to constrain the output to suppress illegal transitions to train a tagger with a cross-entropy loss twice as fast as a CRF. |
| Outcome: | The proposed model trains twice as fast as a CRF with statistically insignificant differences in F1 . the proposed model is open source and can be used in PyTorch and TensorFlow. |
The Spoken Language Understanding MEDIA Benchmark Dataset in the Era of Deep Learning: data updates, training and evaluation tools (2022.lrec-1)
Copied to clipboard
Gaëlle Laperrière, Valentin Pelloin, Antoine Caubrière, Salima Mdhaffar, Nathalie Camelin, Sahar Ghannay, Bassam Jabaian, Yannick Estève
| Challenge: | a growing number of studies address the spoken language understanding domain through a simple task like speech intent detection. |
| Approach: | They focus on the french MEDIA SLU dataset, which is distributed since 2005 . they propose a recipe for its use, including data preparation, training and evaluation scripts . |
| Outcome: | The MEDIA SLU dataset is used as a benchmark dataset for a large number of research projects. |
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context (P19-1)
Copied to clipboard
| Challenge: | Term memory networks (RNNs) are difficult to optimize due to gradient vanishing and explosion. |
| Approach: | They propose a neural architecture Transformer-XL that enables learning dependency beyond a fixed length without disrupting temporal coherence. |
| Outcome: | The proposed method improves state-of-the-art performance on short and long sequences and generates coherent, novel text articles with thousands of tokens. |
LibEvolutionEval: A Benchmark and Study for Version-Specific Code Generation (2025.naacl-long)
Copied to clipboard
Sachit Kuhar, Wasi Uddin Ahmad, Zijian Wang, Nihal Jain, Haifeng Qian, Baishakhi Ray, Murali Krishna Ramanathan, Xiaofei Ma, Anoop Deoras
| Challenge: | Recent code completion models focus on local file contexts, but do not fully capture the complexities of real-world software development. |
| Approach: | They propose a version-specific code-completion task across eight libraries as they evolve over the years and an in-depth analysis of two widely used public libraries: PyTorch and Matplotlib. |
| Outcome: | The proposed model improves performance with public libraries, compared with existing models. |
TARIC-SLU: A Tunisian Benchmark Dataset for Spoken Language Understanding (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing SLU resources are limited in high-resource languages such as English, Mandarin and French. |
| Approach: | They propose to use a Tunisian dialect dataset to build a semantic model of the system that is continuously annotated with dialogue acts and slots. |
| Outcome: | The proposed dataset is based on train-based and ASR-based models of train-driven conversations in Tunisian dialect. |