Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop
Sampling and Filtering of Neural Machine Translation Distillation Data (2021.naacl-srw)
Copied to clipboard
| Challenge: | In most of neural machine translation distillation or stealing scenarios, the highest-scoring hypothesis of the target model is used to train a new model. |
| Approach: | They propose to use the highest-scoring hypothesis of the target model (teacher) to train a new model (student). |
| Outcome: | The proposed method improves the performance of MT models in English to Czech and with reference translations. |
IceSum: An Icelandic Text Summarization Corpus (2021.naacl-srw)
Copied to clipboard
| Challenge: | Automatic Text Summarization (ATS) is the task of generating concise and fluent summaries from one or more documents. |
| Approach: | They present a corpus of 1,000 Icelandic news articles and extractive summaries . they train several neural network-based models on the corpus and evaluate them . |
| Outcome: | The proposed model outperforms baseline methods in the evaluation of the dataset and the models with an open license. |
Negation typology and general representation models for cross-lingual zero-shot negation scope resolution in Russian, French, and Spanish. (2021.naacl-srw)
Copied to clipboard
| Challenge: | Negation resolution remains an acute and continuously researched question in Natural Language Processing. |
| Approach: | They propose to use multilingual pre-trained general representation models to detect negation scope in languages without annotated data. |
| Outcome: | The proposed model achieves token-level F1 score between English, Spanish, French, and Russian. |
Representations of Meaning in Neural Networks for NLP: a Thesis Proposal (2021.naacl-srw)
Copied to clipboard
| Challenge: | Neural networks are the state-of-the-art method of machine learning for many problems in NLP. |
| Approach: | They propose to examine the distribution of meaning in the vector space representation of words in neural networks trained for NLP tasks. |
| Outcome: | The proposed method would be compatible with distributional hypothesis, structuralism, and semantic holism. |
Towards Layered Events and Schema Representations in Long Documents (2021.naacl-srw)
Copied to clipboard
| Challenge: | a thesis aims to explore the use of event extraction in literary texts . event extraction is a challenging domain based on its variety of genres . |
| Approach: | They propose to use event extraction to extract semantic information from literary texts . they propose to build on sequences of event embeddings to form schema embeddables . |
| Outcome: | The proposed approach will allow comparisons between sections of documents and entire literary works. |
Parallel Text Alignment and Monolingual Parallel Corpus Creation from Philosophical Texts for Text Simplification (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for text simplification require a lot of annotated data, however there are few suitable tools for this task. |
| Approach: | They propose an unsupervised method for aligning text based on Doc2Vec embeddings and an alignment algorithm capable of aligning texts at different levels. |
| Outcome: | The proposed method can be used to create a monolingual parallel corpus composed of the works of early modern philosophers and their corresponding simplified versions. |
Syntax-Based Attention Masking for Neural Machine Translation (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to extend transformers to source-side trees are linearized into sequences, but they are limited by positional encodings. |
| Approach: | They propose a method for extending transformers to source-side trees by using masks based on tree positions . they define a number of masks that limit self-attention based upon relationships among tree nodes . |
| Outcome: | The proposed method improves on translations from English to germany and English to english and germany by +2.1 BLEU. |
Multi-Modal Image Captioning for the Visually Impaired (2021.naacl-srw)
Copied to clipboard
| Challenge: | Current captioning models for blind people do not use textual data present in the image when generating captions. |
| Approach: | They propose to use text detected in the image as an input feature in captions . they also use a pointer-generator network to copy detected text to the caption . |
| Outcome: | The proposed system outperforms existing models on the VizWiz dataset, showing a 35% and 16.2% performance improvement. |
Open-Domain Question Answering with Pre-Constructed Question Spaces (2021.naacl-srw)
Copied to clipboard
| Challenge: | Open-domain question answering aims at locating answers to user-generated questions in massive collections of documents. |
| Approach: | They propose an algorithm with a novel reader-retriever design that differs from both families of algorithms. |
| Outcome: | The proposed algorithm outperforms retrieval-based methods with two large-scale datasets and is state-of-the-art. |
A Sliding-Window Approach to Automatic Creation of Meeting Minutes (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to extract utterances and keyphrases from transcripts are lacking in meeting minutes. |
| Approach: | They propose a sliding-window approach to automatic generation of meeting minutes . they use a neural abstractive abstractive to navigate through the raw transcript . |
| Outcome: | The proposed approach is evaluated on natural transcripts and two versions of automatic transcripts. |
Exploration and Discovery of the COVID-19 Literature through Semantic Visualization (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing semantic visualization methods are limited in finding connections between corpora targeting a specific topic. |
| Approach: | They propose to use semantic visualization to explore large datasets of complex networks by exploiting the semantics of the relations in them. |
| Outcome: | The proposed method can enable exploration and discovery over large datasets of complex networks by exploiting the semantics of the relations in them. |
Shuffled-token Detection for Refining Pre-trained RoBERTa (2021.naacl-srw)
Copied to clipboard
| Challenge: | State-of-the-art transformer models have achieved robust performance on a variety of NLP tasks. |
| Approach: | They propose to refine a pre-trained NLP model by detecting shuffled tokens . they use a sequential approach to train a model using random shuffling . |
| Outcome: | The proposed model achieves better performance on 4 out of 7 GLUE tasks. |
Morphology-Aware Meta-Embeddings for Tamil (2021.naacl-srw)
Copied to clipboard
| Challenge: | In this work, we focus on producing morphologically enhanced word embeddings for Tamil, a highly agglutinative South Indian language with rich morphology that remains low-resource with regards to NLP tasks. |
| Approach: | They present a first-ever word analogy dataset for Tamil using a rules-based segmenter and meta-embedding techniques. |
| Outcome: | The proposed embeddings outperform baselines on the word analogy task by 16% and appear to mitigate a trade-off between semantic and morphological accuracy. |
Seed Word Selection for Weakly-Supervised Text Classification with Unsupervised Error Estimation (2021.naacl-srw)
Copied to clipboard
| Challenge: | Weakly-supervised text classification aims to induce text classifiers from only a handful of user-provided seed words. |
| Approach: | They propose to use user-provided seed words to induce text classifiers using only a handful of carefully chosen seed words. |
| Outcome: | The proposed method outperforms baseline model using only category name seed words and achieves comparable performance as a counterpart using expert-annotated seed words. |
Multi-Task Learning of Generation and Classification for Emotion-Aware Dialogue Response Generation (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing models for human-like interaction with humans are not expected to improve the accuracy of emotion recognition, but instead focus on generating emotion-aware responses. |
| Approach: | They propose a neural response generation model with multi-task learning of generation and classification, focusing on emotion. |
| Outcome: | The proposed model makes generated responses more emotionally aware. |
Comparison of Grammatical Error Correction Using Back-Translation Models (2021.naacl-srw)
Copied to clipboard
| Challenge: | Currently, a mainstream approach to generate pseudo data is back-translation (BT). |
| Approach: | They propose to use back-translation to generate pseudo data that contains grammatical and ungrammatically produced sentences. |
| Outcome: | The proposed methods improve or interpolate the performance of each error type compared with a single BT model with different seeds. |
Parallel sentences mining with transfer learning in an unsupervised setting (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to mine parallel sentences in low-resource environments are not suitable for many low-level language pairs. |
| Approach: | They propose an approach based on transfer learning to mine parallel sentences in an unsupervised setting using bilingual corpora of low-resource language pairs. |
| Outcome: | The proposed model improves the performance of mined parallel sentences at two real-world low-resource language pairs compared with previous methods. |
Sentence Concatenation Approach to Data Augmentation for Neural Machine Translation (2021.naacl-srw)
Copied to clipboard
| Challenge: | Neural machine translation is known to show poor performance at long sentence translations . however, when the sentence length exceeds a certain value, the quality of NMT becomes inferior to that of statistical machine translation. |
| Approach: | They propose a method that uses given parallel corpora as train data to generate long sentences by concatenating two sentences at random. |
| Outcome: | The proposed method improves translation quality more when combined with back-translation. |
Emotion Classification in a Resource Constrained Language Using Transformer-based Approach (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods to classify Bengali text into six basic emotions are infancy for resource-constrained languages like English, Arabic, Chinese and French. |
| Approach: | They propose a transformer-based technique to classify Bengali text into one of the six basic emotions: anger, fear, disgust, sadness, joy, and surprise. |
| Outcome: | The proposed technique outperforms all other techniques by achieving highest weighted f_1-score on the test data. |
Hie-BART: Document Summarization with Hierarchical BART (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing document summarization models do not capture hierarchical structures of documents . proposed model incorporates multi-granularity self-attention (MG-SA) |
| Approach: | They propose a new abstractive document summarization model, hierarchical BART . the proposed model captures hierarchically structured sentences in the BART model . |
| Outcome: | The proposed model outperforms baseline models and improves performance on CNN/Daily Mail dataset. |
Towards Multi-Modal Text-Image Retrieval to improve Human Reading (2021.naacl-srw)
Copied to clipboard
| Challenge: | In primary school, children's books, as well as in modern language learning apps, multi-modal learning strategies like illustrations of terms and phrases are used to support reading comprehension. |
| Approach: | They propose to use multi-modal transformers to train multi-dimensional models on text-image retrieval to support a user's reading comprehension of arbitrary text. |
| Outcome: | The proposed model performs poorly because of the short and relatively simple textual data that the current models are trained with. |