Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Knowledge Router: Learning Disentangled Representations for Knowledge Graphs (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to learning from relational patterns and structural information ignore the intrinsic complexity of KGs. |
| Approach: | They propose to learn latent properties of KG entities by using a neighborhood mechanism to disentangle the inner properties of each entity. |
| Outcome: | The proposed method significantly improves performance on key metrics on several benchmark datasets. |
Distantly Supervised Relation Extraction with Sentence Reconstruction and Knowledge Base Priors (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to facilitate distantly supervised relation extraction are noisy instances, long-tail relations and unbalanced bag sizes. |
| Approach: | They propose a multi-task approach to facilitate distantly supervised relation extraction by bringing closer the representations of sentences that contain the same Knowledge Base pairs. |
| Outcome: | The proposed approach improves performance on two datasets created via distant supervision. |
Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on information extraction (IE) has solved the four main tasks separately, thus failing to benefit from inter-dependencies between tasks. |
| Approach: | They propose a model to solve four IE tasks in a single model that captures inter-dependencies between tasks. |
| Outcome: | The proposed model achieves state-of-the-art performance on monolingual and multilingual learning settings with three different languages. |
Abstract Meaning Representation Guided Graph Encoding and Decoding for Joint Information Extraction (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (IE) and Information Extraction (IE), both focus on extracting the main information from natural language texts. |
| Approach: | They propose an AMR-guided framework for joint information extraction using a pre-trained AMR parser. |
| Outcome: | The proposed framework achieves state-of-the-art on all IE subtasks. |
A Frustratingly Easy Approach for Entity and Relation Extraction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on end-to-end relation extraction models combine two tasks: named entity recognition and relation extraction. |
| Approach: | They propose a pipelined approach for entity and relation extraction that uses two independent encoders to construct the relation model. |
| Outcome: | The proposed approach achieves an 8.16 speedup with a slight reduction in accuracy on standard benchmarks. |
Event Time Extraction and Propagation via Graph Attention Networks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on grounding events into a precise timeline has been limited due to the inherent ambiguity of language and the requirement for information propagation over inter-related events. |
| Approach: | They propose a 4-tuple temporal representation for entity slot filling to ground events into a timeline using a graph attention network approach. |
| Outcome: | The proposed approach yields 7.0% match rate over contextualized embedding approaches and 16.3% higher match rate compared to sentence-level manual event time argument annotation. |
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has attracted wide attention in recent years. |
| Approach: | They propose a probing-based approach to measure word translation accuracy using transformer layers. |
| Outcome: | The proposed model outperforms previous probing-based translation models. |
Mediators in Determining what Processing BERT Performs First (2021.naacl-main)
Copied to clipboard
| Challenge: | Probing neural models for the ability to perform downstream tasks using their activation patterns is often used to localize what parts of the network specialize in performing which tasks. |
| Approach: | They propose to consider the prediction’s context length as a potential mediating factor and consider the length of the span whose processing is minimally required to perform the prediction. |
| Outcome: | The proposed model can get 196 different rankings when probing with seven tasks, the authors show . |
Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that supervised models exploit data artifacts to achieve good test scores while their performance severely degrades on samples outside their training distribution. |
| Approach: | They propose a method which automatically generates contrast sets for the visual question answering task by using a semantic input representation. |
| Outcome: | The proposed method computes the answer of perturbed questions, thus reducing annotation cost and enabling thorough evaluation of models’ performance on various semantic aspects. |
Multilingual Language Models Predict Human Reading Behavior (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that cognitively motivated "attention" mechanism in neural models is not a good indicator for relative importance. |
| Approach: | They compare the performance of language-specific and multilingual pretrained transformer models to predict reading time measures reflecting natural human sentence processing. |
| Outcome: | The proposed models predict reading time measures on Dutch, English, German, and Russian texts. |
Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing (2021.naacl-main)
Copied to clipboard
| Challenge: | a study of neural language models shows that syntactic probes do not properly isolate syntax. |
| Approach: | They show that syntactic probes do not properly isolate syntax . they train two probes trained on normal data and find they perform worse . |
| Outcome: | The proposed method outperforms the baseline models on the most popular models, but their lead is reduced by 53%. |
A Non-Linear Structural Probe (2021.naacl-main)
Copied to clipboard
| Challenge: | Probing is a method of investigating the encoding of knowledge in contextual representations. |
| Approach: | They propose to kernelize a metric and develop a non-linear variant with an identical number of parameters by using a kernel-based probe. |
| Outcome: | The proposed probe learns only linear transformations and achieves statistically significant performance improvement over baseline in all languages. |
Concealed Data Poisoning Attacks on NLP Models (2021.naacl-main)
Copied to clipboard
| Challenge: | In contrast, adversarial attacks can cause model errors by modifying inputs, such as the universal triggers attack. |
| Approach: | They propose a data poisoning attack that allows an adversary to control model predictions whenever a desired trigger phrase is present in the input. |
| Outcome: | The proposed attack can cause model errors by modifying inputs, but it can also cause extra human annotation. |
Backtranslation Feedback Improves User Confidence in MT, Not Quality (2021.naacl-main)
Copied to clipboard
Vilém Zouhar, Michal Novák, Matúš Žilinec, Ondřej Bojar, Mateo Obregón, Robin L. Hill, Frédéric Blain, Marina Fomicheva, Lucia Specia, Lisa Yankovskaya
| Challenge: | Inbound translation is a modern need for which the user experience has significant room for improvement, beyond the basic machine translation facility. |
| Approach: | They propose to provide cues that indicate the quality of MT output as well as suggest possible rephrasing of the source language. |
| Outcome: | The proposed feedback module increases user confidence in the produced translation, but not the objective quality. |
Data Filtering using Cross-Lingual Word Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | varying task definitions and data conditions make it difficult to draw a meaningful comparison. |
| Approach: | They propose to use language identification to perform data filtering on MT data based on cross-lingual word embeddings to identify weaknesses in language identification tool. |
| Outcome: | The proposed methods perform well on three real-life, high resource MT tasks while performing weakly within more realistic task conditions. |
Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised neural machine translation (UNMT) use cross-lingual pretraining to align the lexical- and high-level representations of two languages. |
| Approach: | They propose to use type-level cross-lingual subword embeddings to enhance the bilingual masked language model pretraining with lexical-level information to align the two languages. |
| Outcome: | Empirical results show that the method improves on UNMT (up to 4.5 BLEU) and bilingual lexicon induction compared to baseline models. |
Neural Machine Translation without Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models operate over subword tokens, but byte-based models employ a different approach . a one-hot representation of each byte does not hurt performance, but it improves BLEU scores . |
| Approach: | They propose to represent every computerized text as a sequence of bytes via UTF-8 . this eliminates the need for an embedding layer and improves performance . |
| Outcome: | The proposed model improves BLEU scores on byte-to-byte translation models compared to character-level models . the proposed model does not require an embedding layer and does not drop out of the decoder . |
Counterfactual Data Augmentation for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models often rely on large-scale parallel corpora for training, exhibiting degraded performance on low-resource languages. |
| Approach: | They propose a method that interprets language models and phrasal alignment causally and generates augmented parallel translation corpora by sampling new source phrases from a masked language model. |
| Outcome: | The proposed method improves translation, backtranslation and translation robustness on IWSLT’15 English Vietnamese, WMT’17 English - German, and WMT'18 English – Turkish. |
Cultural and Geographical Influences on Image Translatability of Words across Languages (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models produce poor translations when there are few/no parallel sentences to train the models. |
| Approach: | They define image translatability as the translability of words as images associated with words in different languages that have a high degree of visual similarity. |
| Outcome: | The proposed model improves upon text-only models only marginally. |
Multilingual BERT Post-Pretraining Alignment (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work improves on the success of monolingual pretrained language models by adding cross-lingual tasks that always involve English. |
| Approach: | They propose a method to align multilingual contextual embeddings as a post-pretraining step for improved cross-lingual transferability of pretrained language models. |
| Outcome: | The proposed model outperforms XLM-R_Base on translation-train tasks while using less parallel data and fewer parameters. |
A Million Tweets Are Worth a Few Points: Tuning Transformers for Customer Service Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | In domain-specific customer service applications, many companies struggle to deploy advanced NLP models due to the limited availability of and noise in their datasets. |
| Approach: | They analyze customer service conversations on a multilingual social media corpus and compare different approaches to pretraining and finetuning on different end tasks. |
| Outcome: | The proposed model improves performance on multilingual social media data, especially in non-English settings. |
Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases (2021.naacl-main)
Copied to clipboard
Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, Prodromos Malakasiotis
| Challenge: | Interpretability or explainability is an emerging field of research in NLP . experimental results indicate that the newly introduced task is very challenging . |
| Approach: | They propose to extract rationales as paragraphs in multi-paragraph structured court cases . they also propose a constraint that allows models to be more specific . |
| Outcome: | The proposed task is very challenging and there is a large scope for further research. |
Answering Product-Questions by Utilizing Questions from Other Contextually Similar Products (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to predict product-related questions fail for new or unpopular products . product-specific question answering is a popular service provided by many e-commerce websites . |
| Approach: | They propose a framework for predicting the answer to product-related questions based on the answers of similar products. |
| Outcome: | The proposed model outperforms baselines on some segments of product-related questions. |
EnSidNet: Enhanced Hybrid Siamese-Deep Network for grouping clinical trials into drug-development pathways (2021.naacl-main)
Copied to clipboard
| Challenge: | Siamese Neural Networks have been widely used to perform similarity classification in multi-class settings. |
| Approach: | They propose an Enhanced hybrid Siamese-Deep Neural Network (EnSidNet) that can be used to group clinical trials belonging to the same drug-development pathway along the several clinical trial phases. |
| Outcome: | The proposed model shows significant improvement above baselines in a 1-shot evaluation setting and in . a classical similarity setting. |
DATE: Detecting Anomalies in Text via Self-Supervision of Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent deep learning methods for anomalies in images learn better features of normality in an end-to-end self-supervised setting. |
| Approach: | They propose to use a novel pretext task to learn a deep learning model for Anomaly Detection in text to train a model to discriminate between different transformations applied to visual data. |
| Outcome: | The proposed method outperforms state-of-the-art methods on 20Newsgroups and AG News datasets in the semi-supervised setting and in the unsupervised setting. |
A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to handle out-of-vocabulary identifiers are not suitable for source code processing. |
| Approach: | They propose a method to handle out-of-vocabulary identifiers by identifies anonymization . they show that the method significantly improves the performance of the Transformer . |
| Outcome: | The proposed method significantly improves the performance of the Transformer in two code processing tasks. |
Fast and Scalable Dialogue State Tracking with Explicit Modular Decomposition (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches for dialogue state tracking are mainly based on classification-based and extraction-based methods. |
| Approach: | They propose a model which incorporates both classification-based and extraction-based methods and integrates four modules to jointly extract dialogue states. |
| Outcome: | The proposed model outperforms the state-of-the-art models in multi-domain dialogues with many turns of utterances. |
Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | Cross-encoders perform full-attention over the input pair, while bi-encoding requires substantial training data and fine-tuning over the target task to achieve competitive performance. |
| Approach: | They propose a data augmentation strategy that uses cross-encoders to label larger set of input pairs to augment training data for bi-encoding. |
| Outcome: | The proposed approach improves on multiple tasks and domain adaptation tasks by up to 37 points compared to the original bi-encoder performance. |
SmBoP: Semi-autoregressive Bottom-up Semantic Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing semantic parsers decode syntax using a top-down depth-first traversal. |
| Approach: | They propose a semi-autoregressive bottom-up parser that constructs at decoding step t the top-K sub-trees of height t. |
| Outcome: | The proposed method achieves 2.2x speed-up in decoding time and 5x speed up in training time on a zero-shot semantic parsing benchmark. |
SGL: Speaking the Graph Languages of Semantic Parsing via Multilingual Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Graph-based semantic parsing is one of the most promising general-purpose meaning representations . owing to this heterogeneity, most research focused on solutions specific to a given formalism . |
| Approach: | They propose a multilingual neural machine translation framework for Graph-based semantic parsing . they propose Graph2seq architecture that trains with an MNMT objective . |
| Outcome: | The proposed framework outperforms all competitors on cross-lingual parsing tasks. |
Unifying Cross-Lingual Semantic Role Labeling with Heterogeneous Linguistic Resources (2021.naacl-main)
Copied to clipboard
| Challenge: | Using cross-lingual techniques to perform Semantic Role Labeling (SRL) has been limited by the fact that each language adopts its own linguistic formalism . |
| Approach: | They propose a unified model to perform cross-lingual SRL over heterogeneous linguistic resources. |
| Outcome: | The proposed model is able to annotate a sentence in a single forward pass with all the inventories it was trained with, providing a tool for the analysis and comparison of linguistic theories across different languages. |
Fool Me Twice: Entailment from Wikipedia Gamification (2021.naacl-main)
Copied to clipboard
| Challenge: | a new dataset of entailment pairs is released to challenge the goal of examining arbitrary statements. |
| Approach: | They propose a multi-player game that challenges players to solve entailment pairs . the game is open source and encourages adversarial examples . |
| Outcome: | The proposed game lowers the number of examples that can be solved using "shortcuts" the game is open source and the code is available for free. |
Meta-Learning for Domain Generalization in Semantic Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to parsing use standard supervised learning, but little attention has been given to domain generalization. |
| Approach: | They propose a meta-learning framework which targets zero-shot domain generalization for semantic parsing. |
| Outcome: | The proposed framework significantly boosts parser performance on English and Chinese spider datasets. |
Aspect-Controlled Neural Argument Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Current argument generation models produce lengthy texts and allow the user little control over the aspect the argument should address. |
| Approach: | They propose a language model that can be controlled to generate sentence-level arguments for a given topic, stance, and aspect. |
| Outcome: | The proposed model generates high-quality arguments for argumentation and counter-arguments. |
Text Generation from Discourse Representation Structures (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models to generate text from formal meaning representations based on Discourse Representation Structures (DRSs) . |
| Approach: | They propose neural models to generate text from formal meaning representations based on Discourse Representation Structures (DRSs). |
| Outcome: | The proposed model achieves competitive performance on the GMB benchmark against several strong baselines. |
APo-VAE: Text Generation in Hyperbolic Space (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models that learn embeddings only in Euclidean vector space do not account for such structural property of language. |
| Approach: | They propose a Poincare Variational Autoencoder to capture latent hierarchies in hyperbolic space . they propose enabling adversarial learning procedures to empower robust model training . |
| Outcome: | The proposed model outperforms existing models in a hyperbolic latent space . it captures latent language hierarchies in hyperbolical space and is robust to training . |
DART: Open-Domain Structured Data Record to Text Generation (2021.naacl-main)
Copied to clipboard
Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Mutethia Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victoria Lin, Caiming Xiong, Richard Socher, Nazneen Fatema Rajani
| Challenge: | Data-to-text annotations can be costly when dealing with tables with nontrivial structures. |
| Approach: | They propose a procedure for extracting semantic triples from tables that encodes their structures by exploiting table headers and table title. |
| Outcome: | The proposed method exploits the semantic dependencies between table headers and title to extract semantic triples from tables. |
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Language models are a new standard to build state-of-the-art NLP systems. |
| Approach: | They compare multilingual and monolingual models on unseen languages . they show that some languages benefit from transfer learning whereas others don't . |
| Outcome: | The proposed model behaves in multiple ways on unseen languages, while others fail to transfer . the results provide a promising direction towards making multilingual models useful for a new set of unseense languages. |
Multi-Adversarial Learning for Cross-Lingual Word Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | Generative adversarial networks (GANs) have succeeded in inducing cross-lingual word embeddings without supervision, but their performance for distant languages is still not satisfactory. |
| Approach: | They propose a multi-adversarial method that induces the seed cross-lingual dictionary through multiple mappings, each induced to fit the mapping for one subspace. |
| Outcome: | The proposed method improves performance on bilingual lexicon induction and cross-lingual document classification on unsupervised bilingual linguistic induction. |
Multi-view Subword Regularization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing subword regularization methods for multilingual pretrained representations are suboptimal for multi-lingual transfer. |
| Approach: | They propose a method that enforces consistency between standard and probabilistic segmentations. |
| Outcome: | The proposed method improves the effectiveness of cross-lingual transfer by 2.5 points over standard methods. |
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer (2021.naacl-main)
Copied to clipboard
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel
| Challenge: | Current natural language processing pipelines often use transfer learning, where a model is pre-trained on a data-rich task before being fine-tuned on . this significantly limits their use given that roughly 80% of the world population does not speak English. |
| Approach: | They introduce a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. |
| Outcome: | The proposed model achieves state-of-the-art on multilingual benchmarks and a simple technique to prevent accidental translation in the zero-shot setting. |
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning (2021.naacl-main)
Copied to clipboard
Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Hassan Awadallah
| Challenge: | Recent work shows that multilingual representations are disjointed across languages, bringing additional challenges for transfer onto extremely low-resource languages. |
| Approach: | They propose a meta-learning based framework that learns to transform representations judiciously from auxiliary languages to a target one and brings their representation spaces closer for effective transfer. |
| Outcome: | The proposed framework learns to transform representations from auxiliary languages to a target language and brings their representation spaces closer for effective transfer. |
Open Domain Question Answering over Tables via Dense Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in open-domain QA focus on retrieving textual passages . a retriever designed to handle tabular context can improve retrieval quality . |
| Approach: | They propose a tabular-based retrieval model that improves retrieval quality over a BERT-based retriever. |
| Outcome: | The proposed retriever improves retrieval quality with mined hard negatives over a BERT-based retriever. |
Open-Domain Question Answering Goes Conversational via Question Rewriting (2021.naacl-main)
Copied to clipboard
Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, Srinivas Chappidi
| Challenge: | Existing large-scale benchmarks for conversational QA limit the topic of conversation to the content of a single document. |
| Approach: | They propose a dataset for Question Rewriting in Conversational Context (QReCC) the dataset contains 14K conversations with 80K question-answer pairs. |
| Outcome: | The proposed approach shows that the first baseline for the QReCC dataset is 19.10, compared to the human upper bound of 75.45, indicating the difficulty of the setup and a large room for improvement. |
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing question answering systems lack the ability to access relevant knowledge and reason over it. |
| Approach: | They propose a model that uses KGs to identify relevant knowledge in QA contexts and perform joint reasoning over them. |
| Outcome: | The proposed model improves on the CommonsenseQA and OpenBookQA datasets and performs interpretable and structured reasoning. |
XOR QA: Cross-lingual Open-Retrieval Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | a dataset of 40k information-seeking questions across seven languages is used to answer multilingual question answering tasks. |
| Approach: | They propose a task framework that allows questions from one language to be answered via answer content from another language. |
| Outcome: | The proposed framework can be used to answer questions from one language to another . the dataset was built on 40K questions across 7 languages, but could not find same-language answers . |
SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | SPARTA is a novel neural retrieval method for open-domain question answering . it learns a sparse representation that can be efficiently implemented as an Inverted Index . |
| Approach: | They propose a method that learns a sparse representation that can be implemented as an Inverted Index. |
| Outcome: | The proposed method achieves state-of-the-art results on 4 open-domain question answering tasks and 11 retrieval question answering (ReQA) tasks. |
Implicitly Abusive Language – What does it actually look like and why are we not getting there? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets make learning implicit abuse difficult, argues a new position paper . a lack of work on implicit abuse has limited the effectiveness of automatic detection . |
| Approach: | They argue that existing datasets make learning implicit abuse difficult . they propose a divide-and-conquer strategy to detect implicit abuse . |
| Outcome: | The proposed model could be improved to detect implicit abuse in a dataset with a standardized model. |
The Importance of Modeling Social Factors of Language: Theory and Practice (2021.naacl-main)
Copied to clipboard
| Challenge: | Current NLP models focus on information content while ignoring language’s social factors. |
| Approach: | They propose that NLP systems focus on information content while ignoring language’s social factors to improve performance. |
| Outcome: | The proposed approach improves the performance of existing systems, open up new applications, and increase fairness and usability for all users. |
On learning and representing social meaning in NLP: a sociolinguistic perspective (2021.naacl-main)
Copied to clipboard
| Challenge: | linguistic variation allows for the expression of social meaning, information about the social background and identity of the language user. |
| Approach: | They introduce the concept of social meaning to NLP and discuss how sociolinguistics can inform work on representation learning in NLP. |
| Outcome: | The proposed model can be used to learn social meaning in NLP and identify key challenges. |
Preregistering NLP research (2021.naacl-main)
Copied to clipboard
| Challenge: | Preregistration refers to specifying what you are going to do, and what you expect to find in your study, before carrying out the study. |
| Approach: | They propose to use preregistration to specify what you are going to do and what you expect to find in your study, and propose several preregistrations questions for different kinds of studies. |
| Outcome: | The proposed preregistration form could provide firmer grounds for slow science in NLP research. |
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence (2021.naacl-main)
Copied to clipboard
| Challenge: | Typical fact verification models use retrieved written evidence to verify claims . evidence sources change over time as more information is gathered and revised . a new benchmark for fact verification is VitaminC, which is contrastive in nature . |
| Approach: | They propose a benchmark that uses Wikipedia revisions to train models to discern and adjust to slight factual changes. |
| Outcome: | The proposed model improves accuracy by 10% on adversarial fact verification and 6% on adversary natural language inference. |
Representing Numbers in NLP: a Survey and a Vision (2021.naacl-main)
Copied to clipboard
| Challenge: | Numeracy is an essential skill for language understanding since numbers are often interspersed in text. |
| Approach: | They propose a comprehensive taxonomy of tasks and methods to represent numbers in text . they synthesize best practices for representing numbers in texts and articulate a vision for holistic numeracy . |
| Outcome: | The proposed model synthesizes best practices for representing numbers in text . it argues that the model is more effective than other approaches . |
Extending Multi-Document Summarization Evaluation to the Interactive Setting (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to interactive summarization are incomparable and divergent . a key gap in the development and adoption of interactive summaries is the lack of evaluation methodologies and benchmarks for meaningful comparison of systems. |
| Approach: | They propose an end-to-end evaluation framework for interactive summarization based on expansion-based interaction . framework includes procedure of collecting real user sessions, evaluation measures relying on summarizing standards, but adapted to reflect interaction. |
| Outcome: | The proposed evaluation framework is based on evaluations of baseline implementations and is available publicly as a benchmark. |
Identifying Helpful Sentences in Product Reviews (2021.naacl-main)
Copied to clipboard
| Challenge: | a key advantage of online shopping is the ability to read what other customers are saying about products of interest. |
| Approach: | They propose a task to extract a representative helpful sentence from reviews . they collect a dataset in english and use crowd-sourcing to test their model . |
| Outcome: | The proposed model outperforms baselines in a crowd-sourced model of representative helpful sentences from product reviews. |
Noisy Self-Knowledge Distillation for Text Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to summarize text using a single reference and noisy datasets are ill-suited to summarising on single reference datasets. |
| Approach: | They propose to use self-knowledge distillation to improve text summarization by generating smoothed labels for students and teachers to reduce model uncertainty. |
| Outcome: | The proposed framework improves on pretrained and non-pretrained models on three benchmarks. |
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation (2021.naacl-main)
Copied to clipboard
Alexander Fabbri, Simeng Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, Yashar Mehdad
| Challenge: | Recent advances in summarization are driven by the availability of large datasets such as the CNN-DailyMail corpus and the New York Times corpus. |
| Approach: | They propose a method for fine-tuning pretrained models for summarization in unsupervised manner . they use Wikipedia data to produce pseudo-summaries which contain characteristics of target dataset . |
| Outcome: | The proposed method achieves state-of-the-art, zero-shot abstractive summarization performance on CNN-DailyMail dataset and compares with other methods on other datasets. |
Enhancing Factual Consistency of Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often distort or fabricate facts in articles . factual inconsistency is a common problem with abstractive summaries . |
| Approach: | They propose a fact-aware summarization model FASum to extract factual relations into the summary generation process via graph attention. |
| Outcome: | The proposed model can produce abstractive summaries with higher factual consistency compared with existing systems and corrects factual errors via modifying only a few keywords. |
Few-shot Intent Classification and Slot Filling with Retrieved Examples (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for few-shot learning are based on labeled examples, but they are non-trivial . few-sshot learning is challenging due to the imbalance in the amount of data between the source and target domains. |
| Approach: | They propose retrieval-based methods for intent classification and slot filling tasks . they use a batch-softmax objective to learn similar contextualized representations for spans . |
| Outcome: | The proposed method outperforms previous systems on the CLINC and SNIPS benchmarks. |
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses (2021.naacl-main)
Copied to clipboard
| Challenge: | Ad hominem attacks target a person's character instead of the position the person is maintaining. |
| Approach: | They propose to use salient n-gram similarity as a soft constraint to reduce the amount of ad hominems generated in Twitter conversations. |
| Outcome: | The proposed method reduces the amount of ad hominems generated in human and dialogue system responses to English Twitter posts by using salient n-gram similarity as a soft constraint. |
Human-like informative conversations: Better acknowledgements using conditional mutual information (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing chatbots generate responses that are non-specific w.r.t. one of the contexts, typically the conversational history. |
| Approach: | They propose to build a dialogue agent that can weave new factual content into conversations as naturally as humans. |
| Outcome: | The proposed method trades off pmi for pcmi_h and is preferred by humans for overall quality over the Max-PMI baseline 60% of the time. |
A Comparative Study on Schema-Guided Dialogue State Tracking (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work proposes using natural language descriptions to define domain ontologies for dialog state tracking. |
| Approach: | They propose to use natural language descriptions to define domain ontologies instead of tag names for each intent or slot . they introduce a set of newly designed bench-marking descriptions and show model robustness . |
| Outcome: | The proposed model is robust on homogeneous and heterogeneously described descriptions in training and evaluation. |
Spoken Language Understanding for Task-oriented Dialogue Systems with Augmented Memory Networks (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent research shows promising results by jointly learning of slot filling and intent detection tasks. |
| Approach: | They propose a way to combine slot filling and slot filler learning to achieve state-of-the-art results. |
| Outcome: | The proposed model outperforms existing methods on benchmark datasets and ATIS datasets. |
How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds (2021.naacl-main)
Copied to clipboard
| Challenge: | a recent improvement in the quality of natural language processing and generation (NLG) is needed for goal-oriented ML driven agents. |
| Approach: | They propose a reinforcement learning system that integrates large-scale language modeling and commonsense reasoning-based pre-training to imbue the agent with relevant priors. |
| Outcome: | The proposed system is able to act and talk naturally with respect to their motivations. |
Linking Entities to Unseen Knowledge Bases with Arbitrary Schemas (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on entity linking relies on a knowledge base that is not known at training time. |
| Approach: | They propose a method to flexibly convert entities with several attribute-value pairs from arbitrary KBs into flat strings and use it to generalize the model. |
| Outcome: | The proposed model is 12% more accurate than baseline models on English datasets. |
Self-Training with Weak Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art deep neural networks require large amounts of labeled training data that is expensive to obtain or not available for many tasks. |
| Approach: | They propose a weak supervision framework that leverages all available data for a given task . they leverage task-specific unlabeled data through self-training with a model that predicts pseudo-labels for instances that may not be covered by weak rules . |
| Outcome: | The proposed framework improves on state-of-the-art datasets on six benchmark tasks. |
Neural Language Modeling for Contextualized Temporal Graph Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for temporal reasoning have been used for a number of applications, but their potential for tempor reasoning over event graphs has not been explored. |
| Approach: | They propose to use large-scale pre-trained language models to generate an event-level temporal graph from a document using existing IE/NLP tools. |
| Outcome: | The proposed method outperforms the closest existing method on several metrics on a hand-labeled, out-of-domain corpus. |
Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to generalize knowledge bases model triple-level uncertainty . Existing models only model triple level uncertainty, and reasoning results lack global consistency. |
| Approach: | They propose a method to embed knowledge graphs with calibrated probabilistic semantics . they model each entity as a box and relations between two entities as affine transforms based on affinity transforms. |
| Outcome: | Experiments show that the proposed method outperforms baseline methods on confidence prediction and fact ranking. |
Document-Level Event Argument Extraction by Conditional Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing event extraction models have been limited to the sentence level . this formulation signifies a misalignment between the information seeking behavior and the informative seeking behavior. |
| Approach: | They propose a document-level neural event argument extraction model by formulating the task as conditional generation following event templates. |
| Outcome: | The proposed model achieves 7.6% F1 and 5.7% F1 over the best baseline on the document-level event extraction dataset WikiEvents and 9.3% F1 on the informative argument extraction task. |
Template Filling with Generative Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Template filling tasks are usually tackled by a pipeline of two separate systems, one for role-filler extraction and another for template/event recognition. |
| Approach: | They propose a framework that naturally models the dependence between entities within a single event and across multiple events described in a document. |
| Outcome: | The proposed framework outperforms pipeline-based approaches and other neural baselines that do not model between-event dependencies on documents containing multiple events. |
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models (2021.naacl-main)
Copied to clipboard
Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, Xia Hu
| Challenge: | Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. |
| Approach: | They propose a shortcut mitigation framework to suppress NLU models from making overconfident predictions for samples with large shortcut degree. |
| Outcome: | The proposed framework suppresses the model from making overconfident predictions for samples with large shortcut degree. |
On Attention Redundancy: A Comprehensive Study (2021.naacl-main)
Copied to clipboard
| Challenge: | Attention redundancy has been observed among attention heads but has not been deeply studied in the literature. |
| Approach: | They propose a multi-layer multi-head self-attention mechanism which is widely applied in modern neural language models. |
| Outcome: | The proposed model is useful for interpretation and model compression. |
Does BERT Pretrained on Clinical Notes Reveal Sensitive Data? (2021.naacl-main)
Copied to clipboard
| Challenge: | Pretraining large (masked) language models over EHR data has yielded consistent performance gains across tasks. |
| Approach: | They propose to use large Transformers to release pretraining models over EHRs . they propose to recover patient names and conditions associated with them . |
| Outcome: | The proposed models recover patient names and conditions associated with patients . the proposed models share the model parameters for use by other researchers . |
Low-Complexity Probing via Finding Subnetworks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to probing neural networks for linguistic properties are to train a shallow multi-layer perceptron (MLP) on top of the model's internal representations. |
| Approach: | They propose a subtractive pruning-based probe where they find an existing subnetwork that performs the linguistic task of interest. |
| Outcome: | The proposed probe achieves higher accuracy on pre-trained models and lower accuracy on random models, and better learning on its own. |
An Empirical Comparison of Instance Attribution Methods for NLP (2021.naacl-main)
Copied to clipboard
| Challenge: | Influence functions provide machinery for identifying training instances that may have led to a specific prediction, but are computationally expensive and prohibitive in many cases. |
| Approach: | They evaluate the degree to which different potential instance attribution agrees with respect to the importance of training samples. |
| Outcome: | The proposed methods exhibit desirable characteristics similar to more complex methods, but are computationally expensive. |
Generalization in Instruction Following Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Understanding and executing natural language instructions in a grounded domain is one of the hallmarks of artificial intelligence. |
| Approach: | They propose a learning strategy that involves data augmentation to improve the model's performance. |
| Outcome: | The proposed learning strategy outperforms state-of-the-art models in the blocks world domain while satisfying our expectations much better. |
LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pre-trained models suffer from slow inference speed due to cross-modal attention in transformer architecture. |
| Approach: | They propose a multimodal approach that accelerates the inference time of ITR by thousands of times . they extract pre-cached feature indexes offline and employ instant dot-product matching online . |
| Outcome: | The proposed approach outperforms existing models that consume 1000 times magnitude of computational hours using the same features. |
Measuring Social Biases in Grounded Vision and Language Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to measure social biases in word embeddings are limited to visually grounded word embeds . a new study generalizes word embedment associations to visually ground word embeddas . |
| Approach: | They generalize word embeddings' biases to visually grounded word embeds . they propose two generalizations that answer questions about how biase, language, and vision interact . |
| Outcome: | The proposed measures are applied to a new dataset that includes 10,228 images from COCO, Conceptual Captions, and Google Images. |
MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences (2021.naacl-main)
Copied to clipboard
Jianing Yang, Yongxin Wang, Ruitao Yi, Yuying Zhu, Azaan Rehman, Amir Zadeh, Soujanya Poria, Louis-Philippe Morency
| Challenge: | a novel graph-based neural model for multimodal sequential data is proposed . fusion is the process of blending information from multiple modalities, usually preceded by alignment . |
| Approach: | They propose a graph-based neural model that converts unaligned data into a modal-temporal graph . they use a dynamic pruning and read-out technique to efficiently process the graph fusion operation . |
| Outcome: | The proposed model performs state-of-the-art on multimodal sentiment analysis and emotion recognition benchmarks while utilizing significantly fewer model parameters. |
Grounding Open-Domain Instructions to Automate Web Support Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | RUSS is a task and dataset to ground natural language instructions on the web to perform previously unseen tasks. |
| Approach: | They build a task and dataset to ground AI agents from open-domain, step-by-step instructions on the web. |
| Outcome: | The proposed model outperforms existing models that map instructions to actions without WebLang. |
Modular Networks for Compositional Instruction Following (2021.naacl-main)
Copied to clipboard
| Challenge: | Standard instruction following models struggle on novel compositions of subgoals observed during training. |
| Approach: | They propose a modular architecture that follows natural language instructions that describe sequences of diverse subgoals. |
| Outcome: | The proposed architecture improves generalization to novel subgoals and environments unseen in training. |
Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Information (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing vision language navigation tasks require soft attention over words to locate instructions . a new approach uses syntax information to ground instructions with visual information . |
| Approach: | They propose a vision language navigation agent that utilizes syntax information to enhance alignment between the instruction and the current visual scenes. |
| Outcome: | The proposed agent outperforms the baseline model that does not use syntax information on the Room-to-Room dataset, especially in the unseen environment. |
Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing zero-shot learning methods for multi-label text classification mostly learn a matching model between the feature space of text and the label space. |
| Approach: | They propose to use a graph encoder to incorporate label hierarchies to learn effective label representations on the zero-shot multi-label text classification problem. |
| Outcome: | The proposed approach outperforms previous non-pretrained methods on the zero-shot multi-label text classification task. |
Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach (2021.naacl-main)
Copied to clipboard
| Challenge: | Fine-tuned pre-trained language models (LMs) have enormous success in many natural language processing tasks, but they still require excessive labeled data in the fine-tuning stage. |
| Approach: | They propose a framework to enable fine-tuning pre-trained language models with weak supervision without any labeled data. |
| Outcome: | The proposed framework outperforms the strongest baseline and achieves competitive performance with fully-supervised fine-tuning methods. |
Posterior Differential Regularization with f-divergence for Improving Model Robustness (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained models suffer catastrophic degradation in out-of-domain generalization to datasets with domain shift or adversarial scenarios. |
| Approach: | They propose to regularize the posterior difference between clean and noisy inputs by using a Jacobian regularization framework and a virtual adversarial training framework. |
| Outcome: | The proposed framework can improve model robustness in fully supervised and semi-supervised settings. |
Understanding Hard Negatives in Noise Contrastive Estimation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing theoretical results in contrastive learning focus on unconditional negative distributions. |
| Approach: | They propose a method where highest-scoring incorrect labels are chosen as negatives . they also propose 'hard negative mining' where negatives are used as negative examples . |
| Outcome: | The proposed approach achieves strong results on the task of zero-shot entity linking. |
Certified Robustness to Word Substitution Attack with Differential Privacy (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can be easily fooled by DNNs, making the robustness and security of NLP models significantly important. |
| Approach: | They propose a differential privacy-based algorithm to achieve certified robustness against word substitution at- tacks in text classification via differential privacy. |
| Outcome: | The proposed model achieves higher accuracy and more than 30X efficiency improvement over existing defense algorithms. |
DReCa: A General Task Augmentation Strategy for Few-Shot Natural Language Inference (2021.naacl-main)
Copied to clipboard
| Challenge: | Meta-learning has not yet succeeded in NLP due to the lack of a well-defined task distribution . meta-learners tend to overfit their adaptation mechanism and datasets are heterogeneous . |
| Approach: | They propose a method for decomposing datasets into Reasoning Categories to form additional high quality tasks. |
| Outcome: | The proposed method improves the accuracy of meta-learners by 1.5-4% across four few-shot NLI problems. |
Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages (2021.naacl-main)
Copied to clipboard
| Challenge: | Unsupervised translation systems have impressive performance on resource-rich language pairs . however, in more realistic settings, unsupervised systems perform poorly . |
| Approach: | They propose a model for 5 low-resource languages that leverages monolingual and auxiliary parallel data from other high-resourced languages. |
| Outcome: | The proposed model outperforms state-of-the-art models on low-resource languages . it also matches the current state- of-the art model for Nepali-English . |
Macro-Average: Rare Types Are Important Too (2021.naacl-main)
Copied to clipboard
| Challenge: | MT metrics trained on segment-level human judgments are inherently non-transparent and reflect undesirable biases. |
| Approach: | They propose to use a type-based classifier metric to evaluate machine translation and compare it with a supervised and unsupervised one. |
| Outcome: | The proposed model outperforms other models in indicating cross-lingual information retrieval task performance and shows that it can be used to compare supervised and unsupervised neural machine translation. |
Assessing Reference-Free Peer Evaluation for Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate machine translation output are based on comparing MT output to one or more reference translations. |
| Approach: | They propose to use probabilities given by a large, multilingual model as a reference-free metric. |
| Outcome: | The proposed model is robust and likely to offer reasonable performance across a broad spectrum of domains and different system qualities. |
The Curious Case of Hallucinations in Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) suffers from well known pathologies such as coverage, mistranslation of named entities, etc. |
| Approach: | They propose a theory that explains hallucinations under source perturbation and a method that generates hallucines under corpus-level noise without any source perturbations. |
| Outcome: | The proposed hypothesis is validated by a corpus-level noise analysis and is validateable in other datasets. |
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to multilingual machine translation rely on training models on monolingual data for all languages in a multitask setup. |
| Approach: | They propose a vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models by combining monolingual data with a dictionary. |
| Outcome: | The proposed model improves on existing models by preserving the original model and allowing for competitive performance even with only monolingual data. |
Towards Modeling the Style of Translators in Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | a key ingredient of neural machine translation is the use of large datasets with different but consistent translation styles . however, the models do not capture the variety of translators' styles from the data . a recent study shows that style-augmented models can capture the style variations of translator . |
| Approach: | They propose to augment a neural machine translation model with translator information . they use TED talk datasets to model and control translator-related stylistic variations . |
| Outcome: | The proposed models capture the style variations of translators and generate translations with different styles on new data. |
Self-Supervised Test-Time Learning for Reading Comprehension (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised question answering shows that models can be trained with procedurally generated question-answer pairs and achieve performance competitive with supervised methods. |
| Approach: | They propose a method that performs "test-time learning" on a given context . they use self-supervision to train models on synthetically generated question-answer pairs . |
| Outcome: | The proposed method outperforms current unsupervised methods and outperformed supervised methods. |
Capturing Row and Column Semantics in Transformer Based Question Answering over Tables (2021.naacl-main)
Copied to clipboard
Michael Glass, Mustafa Canim, Alfio Gliozzo, Saneem Chemmengath, Vishwajeet Kumar, Rishav Chakravarti, Avi Sil, Feifei Pan, Samarth Bharadwaj, Nicolas Rodolfo Fauceglia
| Challenge: | Existing transformer based approaches have been used to answer questions over tables. |
| Approach: | They propose a transformer based architecture that independently classifies rows and columns to identify relevant cells and a model that incorporates existing tables to improve efficiency. |
| Outcome: | The proposed model outperforms the state-of-the-art transformer based approaches on WikiSQL lookup questions and achieves 3.4% and 18.86% additional precision improvement on the standard WikisQL benchmark. |
Explainable Multi-hop Verbal Reasoning Through Internal Monologue (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing state-of-the-art language models do not make intermediate reasoning steps explicit . large pretrained language models such as BERT and RoBERTa have been successfully used in multi-hop reasoning problems . |
| Approach: | They propose to decompose multi-hop reasoning problems into several simple ones and use natural language to guide intermediate reasoning hops. |
| Outcome: | The proposed model can generate subgoals and perform inference in natural language at each reasoning step. |
Robust Question Answering Through Sub-part Alignment (2021.naacl-main)
Copied to clipboard
| Challenge: | Current textual question answering models fail to generalize to out-of-domain settings. |
| Approach: | They propose to decompose question and context into smaller units and align them to find the answer. |
| Outcome: | The proposed model is more robust than the standard BERT QA model on adversarial and out-of-domain datasets. |
Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to decompose complex tasks into simpler ones do not require annotated decompositions. |
| Approach: | They propose a framework for building interpretable systems that learn to solve complex tasks by decomposing existing models into simpler ones solvable by existing models. |
| Outcome: | The proposed framework is more versatile than existing explainable systems for DROP and HotpotQA datasets, is more robust than state-of-the-art blackbox (uninterpretable) systems, and generates more understandable and trustworthy explanations compared to prior work. |
RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) achieve high recall amongst top few predictions, but low overall accuracy, motivating the need for answer re-ranking. |
| Approach: | They propose a method to make answer re-ranking successful for span-extraction tasks even beyond large pre-training. |
| Outcome: | The proposed approach achieves 45.5% Exact Match accuracy on Natural Questions and 61.7% on TriviaQA. |
On the Transferability of Minimal Prediction Preserving Inputs in Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work establishes the presence of short, uninterpretable input fragments that yield high confidence and accuracy in neural models. |
| Approach: | They investigate competing hypotheses for the existence of MPPIs in question answering . they discover a perplexing invariance of MPIs to random training seed, model architecture, pretraining, and training domain. |
| Outcome: | The proposed model performance is higher than comparable short queries. |
Understanding by Understanding Not: Modeling Negation in Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Negation is a core construction in natural language, but state-of-the-art pre-trained language models often handle it incorrectly. |
| Approach: | They propose to augment language modeling objective with unlikelihood objective based on negated generic sentences from a raw text corpus. |
| Outcome: | The proposed approach reduces the top 1 error rate to 4% on negated LAMA dataset and improves on negating NLI benchmarks. |
DuoRAT: Towards Simpler Text-to-SQL Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent text-to-SQL models can translate natural language questions to corresponding SQL queries on unseen databases. |
| Approach: | They propose a re-implementation of the RAT-SQL model that uses only relation-aware or vanilla transformers as the building blocks. |
| Outcome: | The proposed model is based on the spider dataset and shows it can be used on large databases without human intervention. |
Looking Beyond Sentence-Level Natural Language Inference for Question Answering and Text Summarization (2021.naacl-main)
Copied to clipboard
Anshuman Mishra, Dhruvesh Patel, Aparna Vijayakumar, Xiang Lorraine Li, Pavan Kapanipathi, Kartik Talamadupula
| Challenge: | Large-scale, open Natural Language Inference datasets have catalyzed the development of NLI models that exhibit close to human-level performance, but the use of these models for other downstream NLP tasks has met with limited success. |
| Approach: | They use multiple-choice reading comprehension and checking factual correctness of textual summarization tasks to investigate potential reasons for this . authors leverage abundance of data from reading comprehension datasets into longer-premise NLI datasets to improve their models . |
| Outcome: | The proposed models outperform models trained on converted datasets due to the difference in premise lengths. |
Structure-Grounded Pretraining for Text-to-SQL (2021.naacl-main)
Copied to clipboard
Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, Matthew Richardson
| Challenge: | STRUG is a weakly supervised structure-based pretraining framework for text-to-SQL . it can be used to learn to capture text-table alignment in a given database schema . |
| Approach: | They propose a weakly supervised structure-grounded pretraining framework for text-to-SQL that can effectively learn to capture text-table alignment based on a parallel text-tab corpus. |
| Outcome: | The proposed framework outperforms BERTLARGE and BERTLAGE on all text-to-SQL alignment settings. |
Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System (2021.naacl-main)
Copied to clipboard
| Challenge: | Text classification is usually studied by labeling texts with relevant categories from a predefined set. |
| Approach: | They propose a task where a system incrementally handles multiple rounds of new classes . they propose two entailment approaches, ENTAILMENT and HYBRID, which show promise . |
| Outcome: | The proposed task is based on a few-shot text classification task in the NLP domain. |
Temporal Reasoning on Implicit Events from Distant Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | a novel temporal reasoning dataset evaluates the degree to which systems understand implicit events . state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events - a recent paper . |
| Approach: | They propose a temporal reasoning dataset that evaluates the degree to which systems understand implicit events. |
| Outcome: | The proposed model outperforms baseline systems on TRACIE by 5% and 11% on MATRES, an explicit event benchmark. |
Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been successful on a wide range of NLP tasks . however, contextual representations from pre-trated models contain entangled semantic and syntactic information. |
| Approach: | They propose a semantic sentence embedding model that disentangles semantics and syntax from pre-trained models. |
| Outcome: | The proposed model outperforms state-of-the-art models on unsupervised semantic similarity tasks. |
Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive conversation summarization has received much attention, but it suffers from insufficient, redundant, or incorrect content due to the unstructured and complex characteristics of human-human interactions. |
| Approach: | They propose to model rich structures in conversations for more precise and accurate conversation summarization by incorporating discourse relations between utterances and action triples in utterrances and designing a multi-granularity decoder to generate summaries by combining all levels of information. |
| Outcome: | The proposed models outperform state-of-the-art methods and generalize well in other domains in terms of automatic evaluations and human judgments. |
A New Approach to Overgenerating and Scoring Abstractive Summaries (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization is a learning objective to produce system outputs that resemble reference summaries on a word-to-word basis. |
| Approach: | They propose a two-staged strategy to generate multiple variants of the target summary and score and select admissible ones according to users’ needs. |
| Outcome: | The proposed approach can achieve state-of-the-art on benchmark summarization datasets. |
D2S: Document-to-Slide Generation Via Query-Based Text Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing research efforts to automate the document-to-slide generation process face a critical challenge: no publicly available dataset for training and benchmarking. |
| Approach: | They propose a dataset SciDuet that gathers papers and their corresponding slides from recent years’ NLP and ML conferences. |
| Outcome: | The proposed system outperforms state-of-the-art summarization baselines on both automated ROUGE metrics and qualitative human evaluation. |
Efficient Attentions for Long Document Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models that use full attentions have quadratic computational and memory complexities, and are too costly for long documents. |
| Approach: | They propose an efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source. |
| Outcome: | The proposed model can process ten times more tokens than current models that use full attentions. |
RefSum: Refactoring Neural Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for text summarization are limited by limitations of reranking or stacking. |
| Approach: | They propose a framework that provides a unified view of text summarization and summaries combination. |
| Outcome: | The proposed method can be used by researchers as an off-the-shelf tool to achieve further performance improvements. |
Annotating and Modeling Fine-grained Factuality in Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent abstractive summarization systems produce factual errors that are not faithful to the input . current methods are lacking in identifying what errors are most important to target . |
| Approach: | They use synthetic and human-labeled data to identify factual errors in summarization and train models on the factuality detection task. |
| Outcome: | The proposed model detects factual errors on word, dependency, and sentence levels. |
Larger-Context Tagging: When and Why Does It Work? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing tagging systems that use sentence-level data are not well understood. |
| Approach: | They propose a larger-context approach to tagging tasks that incorporates contextual information into existing tapping systems. |
| Outcome: | The proposed aggregators improve on four tagging tasks and 13 datasets. |
Neural Sequence Segmentation as Determining the Leftmost Segments (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to segment sentences are mostly at token level, limiting their full potential to capture long-term dependencies. |
| Approach: | They propose a framework that incrementally segments natural language sentences at segment level. |
| Outcome: | The proposed framework outperforms baseline methods on syntactic chunking and Chinese part-of-speech tagging datasets. |
PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work shows that probabilistic context-free grammars with neural parameterization can be effective in unsupervised constituency parsing. |
| Approach: | They propose a parameterization form of PCFGs based on tensor decomposition which has at most quadratic computational complexity in the symbol number. |
| Outcome: | The proposed model improves unsupervised constituency parsing performance across ten languages. |
GEMNET: Effective Gated Gazetteer Representations for Recognizing Complex Entities in Low-context Input (2021.naacl-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is difficult in real-world settings due to short texts, emerging entities, and complex entities. |
| Approach: | They propose a flexible Gazetteer Representation encoder and a Mixture-of-Experts gating network for gazetteer knowledge integration. |
| Outcome: | The proposed approach shows large gains (up to +49% F1) in recognizing difficult entities compared to baselines. |
Video-aided Unsupervised Grammar Induction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods of multi-modal grammar induction focus on grammar inducing from text-image pairs, but videos provide even richer information, such as static objects and actions. |
| Approach: | They propose a video-aided grammar induction model which learns a constituency parser from unlabeled text and its corresponding video. |
| Outcome: | The proposed model outperforms existing systems on three benchmarks. |
Generating Negative Samples by Manipulating Golden Responses for Unsupervised Learning of a Response Evaluation Model (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing metrics that rely on comparisons to a set of known correct responses do not account for the variety of responses and therefore correlate poorly with human judgment. |
| Approach: | They propose a method of manipulating a golden response to create a new negative response that is designed to be inappropriate within the context while maintaining high similarity with the original golden response. |
| Outcome: | The proposed model can be made using unsupervised learning for the next-utterance prediction task on English datasets and shows that using the negative samples alongside random negative samples can increase the model’s correlation with human evaluations. |
How Robust are Fact Checking Systems on Colloquial Claims? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing fact checking systems that perform well on colloquial claims significantly degenerate on collotic claims with the same semantics. |
| Approach: | They propose to transfer the styles of claims from FEVER into colloquialism to investigate fact checking systems on colloqual claims. |
| Outcome: | The proposed system significantly degenerates on colloquial claims with the same semantics. |
Fine-grained Post-training for Improving Retrieval-based Dialogue Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to select the correct response for a dialogue system are generation-based and retrieval-based. |
| Approach: | They propose a fine-grained post-training method that reflects the characteristics of the multi-turn dialogue. |
| Outcome: | The proposed model achieves state-of-the-art with significant margins on three benchmark datasets. |
Put Chatbot into Its Interlocutor’s Shoes: New Framework to Learn Chatbot Responding with Intention (2021.naacl-main)
Copied to clipboard
| Challenge: | Currently, most work on improving the fluency and coherence of chatbots is focused on making them more human-like. |
| Approach: | They propose a framework to train chatbots to possess human-like intentions by making them learn from interactive conversation. |
| Outcome: | The proposed framework includes a guiding chatbot and an interlocutor model that plays the role of humans. |
Adding Chit-Chat to Enhance Task-Oriented Dialogues (2021.naacl-main)
Copied to clipboard
Kai Sun, Seungwhan Moon, Paul Crook, Stephen Roller, Becka Silvert, Bing Liu, Zhiguang Wang, Honglei Liu, Eunjoon Cho, Claire Cardie
| Challenge: | Existing dialogue systems focus on functional goals, open-domain chatbots on socially engaging conversations. |
| Approach: | They propose to add chit-chat to ENhance Task-ORiented dialogues by a human-assisted data collection approach to augment task-oriented dialogues with minimal annotation effort. |
| Outcome: | The proposed models can code-switch between task and chit-chat to be more engaging, interesting, knowledgeable, and humanlike while maintaining competitive task performance. |
Incorporating Syntax and Semantics in Coreference Resolution with Heterogeneous Graph Attention Network (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing neural coreference resolution models lack syntactic and semantic information . however, such information has been shown to benefit other tasks. |
| Approach: | They propose a graph-based model that incorporates syntactic and semantic structures of sentences. |
| Outcome: | The proposed model incorporates syntactic and semantic structures of sentences. |
Context Tracking Network: Graph-based Context Modeling for Implicit Discourse Relation Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models fail to fully utilize contextual information which plays an important role in interpreting sentences. |
| Approach: | They propose a graph-based Context Tracking Network to model the discourse context for IDRR. |
| Outcome: | The proposed model can integrate sentence-level and token-level contextual semantics better than existing models. |
Improving Neural RST Parsing Model with Silver Agreement Subtrees (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for Rhetorical Structure Theory (RST) parsing use supervised learning, but the RST-DT is small due to the costly annotation of RST trees. |
| Approach: | They propose to use silver data to improve RST parsing models by using annotated silver data. |
| Outcome: | The proposed method achieves the best micro-F1 scores for Nuclearity and Relation at 75.0 and 63.2 . it also achieves a remarkable gain in relation score against the previous state-of-the-art parser. |
RST Parsing from Scratch (2021.naacl-main)
Copied to clipboard
| Challenge: | Fig. 1 shows a document level discourse parser that performs top-down end-to-end parsing without requiring segmentation . |
| Approach: | They propose a top-down end-to-end formulation of document level discourse parsing in the Rhetorical Structure Theory framework. |
| Outcome: | The proposed model outperforms existing methods in end-to-end parsing and parse with gold segmentation without handcrafted features. |
Did they answer? Subjective acts and intents in conversational discourse (2021.naacl-main)
Copied to clipboard
| Challenge: | Discourse signals are often implicit, leaving it up to the interpreter to draw inferences . current discourse data and frameworks ignore the social aspect, expecting only a single ground truth . elisa f. and her team present a dataset with multiple and subjective interpretations of English conversation . |
| Approach: | They present a first discourse dataset with multiple and subjective interpretations of English conversation . they show disagreements are nuanced and require a deeper understanding of contextual factors . |
| Outcome: | The proposed dataset shows disagreements are nuanced and require deeper understanding of contextual factors. |
Evaluating the Impact of a Hierarchical Discourse Representation on Entity Coreference Resolution Performance (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on entity coreference resolution (CR) follows current trends in Deep Learning . traditional approaches do not make use of hierarchical representations of discourse structure . |
| Approach: | They propose to leverage automatically constructed discourse parse trees within a neural approach to generate anaphoric mentions. |
| Outcome: | The proposed model improves on two benchmark entity coreference-resolution datasets. |
Bridging Resolution: Making Sense of the State of the Art (2021.naacl-main)
Copied to clipboard
| Challenge: | bridging resolution is a task that involves identifying and resolving bridling/associative anaphors, which are anamorphic references to non-identical associated antecedents. |
| Approach: | They propose a hybrid rule-based and MTL approach that would enable a better understanding of their comparative strengths and weaknesses. |
| Outcome: | The proposed model would be able to better understand their strengths and weaknesses and perform a manual analysis of the errors made by the model. |
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle (2021.naacl-main)
Copied to clipboard
| Challenge: | Failing to capture the structure of input language could lead to generalization problems and over-parametrization. |
| Approach: | They propose a new syntax-aware language model that explicitly models the structure with an incremental parser and maintains the conditional probability setting of a standard language model. |
| Outcome: | The proposed model can achieve strong results in language modeling, parsing, and syntactic generalization tests while using fewer parameters than other models. |
Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | In neural sequence-to-sequence learning, Reinforcement Learning (RL) has gained popularity due to the suitability of Policy Gradient (PG) methods for the end-to end training paradigm. |
| Approach: | They propose to let the model explore the output space beyond the reference output that is used for standard cross-entropy minimization by reinforcing model outputs according to their quality, effectively increasing the likelihood of higher-quality samples. |
| Outcome: | The proposed model explores the output space beyond the reference output that is used for cross-entropy minimization, increasing the likelihood of higher-quality samples. |
Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to encode natural languages without orders are lacking. |
| Approach: | They conduct a comprehensive analysis of the ability of neural models to organize sentences from a bag of words under three typical scenarios. |
| Outcome: | The proposed models can reorder or reconstruct sentences from a bag of words under three typical scenarios. |
Mask Attention Networks: Rethinking and Strengthen Transformer (2021.naacl-main)
Copied to clipboard
Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang
| Challenge: | Existing research explores to enhance the two sublayers separately to improve the capability of Transformer for text representation. |
| Approach: | They propose to combine SAN and Feed-Forward Networks to create a dynamic mask attention network with a learnable mask matrix which can model localness adaptively. |
| Outcome: | The proposed model outperforms the original Transformer on translation and text summarization tasks. |
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to model coarse-grained linguistic information do not integrate coarse-gram information into pre-training. |
| Approach: | They propose an explicitly n-gram masking method to enhance integration of coarse-grained linguistic information into pre-training. |
| Outcome: | The proposed method outperforms existing models on English and Chinese text corpora and fine-tunes on 19 downstream tasks. |
Lattice-BERT: Leveraging Multi-Granularity Representations in Chinese Pre-trained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models process text as a sequence of characters, ignoring more coarse granularity, e.g., words. |
| Approach: | They propose a new pre-training paradigm for Chinese that incorporates word representations along with characters and can model a sentence in a multi-granular manner. |
| Outcome: | The proposed model can bring an average increase of 1.5% under the 12-layer setting, which achieves new state-of-the-art among base-size models on the CLUE benchmarks. |
Modeling Event Plausibility with Consistent Conceptual Abstraction (2021.naacl-main)
Copied to clipboard
| Challenge: | Understanding natural language requires common sense, one aspect of which is the ability to discern the plausibility of events. |
| Approach: | They propose a method of forcing model consistency that improves correlation with human plausibility judgements. |
| Outcome: | The proposed method improves correlation with human plausibility judgements. |
UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus (2021.naacl-main)
Copied to clipboard
| Challenge: | Contextual word embedding models do not take into account structured expert domain knowledge from a knowledge base. |
| Approach: | They propose a contextual embedding model that integrates domain knowledge during the pre-training process via a novel knowledge augmentation strategy. |
| Outcome: | The proposed model outperforms existing domain-specific models on common named-entity recognition (NER) and clinical natural language inference tasks. |
Field Embedding: A Unified Grain-Based Framework for Word Representation (2021.naacl-main)
Copied to clipboard
| Challenge: | Current methods focus on learning word embeddings while linguistic information is discarded after the learning. |
| Approach: | They propose a framework field embedding to jointly learn word and grain embedds by incorporating morphological, phonetic, and syntactical linguistic fields. |
| Outcome: | The proposed framework integrates morphological, phonetic, and syntactical linguistic fields to learn word embeddings and grain embedds. |
MelBERT: Metaphor Detection via Contextualized Late Interaction using Metaphorical Identification Theories (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have developed computational models to recognize metaphorical words in sentences. |
| Approach: | They propose a model that leverages contextualized word representation and linguistic metaphor identification theories to detect whether the target word is metaphorical. |
| Outcome: | The proposed model outperforms baseline models on four benchmark datasets . it leverages contextualized word representation and linguistic metaphor identification theories to detect whether the target word is metaphorical. |
Non-Parametric Few-Shot Learning for Word Sense Disambiguation (2021.naacl-main)
Copied to clipboard
| Challenge: | Word sense disambiguation (WSD) is a problem in natural language processing . 84% of annotated words have less than 10 examples in the long-tail distribution . |
| Approach: | They propose a non-parametric few-shot learning approach to mitigate word sense disambiguation . they use a metric space to compute distances among the senses of a given word . |
| Outcome: | The proposed method achieves a 75.1 F1 score on the unified evaluation benchmark. |
Why Do Document-Level Polarity Classifiers Fail? (2021.naacl-main)
Copied to clipboard
| Challenge: | a new method to characterize, quantify and measure the impact of hard instances is proposed . a method to label hard instances can shed light on why and when classifiers fail, authors say . |
| Approach: | They propose a method to characterize, quantify and measure the impact of hard instances in polarity classification of movie reviews. |
| Outcome: | The proposed method can quantify the impact of hard instances in polarity classification . it can shed light on why and when classifiers fail, the authors say . |
A Unified Span-Based Approach for Opinion Mining with Syntactic Constituents (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for fine-grained opinion mining (OM) are based on span-based annotations, but they are not effective. |
| Approach: | They propose a unified span-based approach for the end-to-end OM setting using syntactic constituents and multi-task learning to integrate them into the proposed model. |
| Outcome: | The proposed approach achieves significant improvements over previous work on the MPQA 2.0 dataset and reduces the number of wrongly-predicted opinion expressions and roles. |
Target-specified Sequence Labeling with Multi-head Self-attention for Target-oriented Opinion Words Extraction (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies on ABSA focus on Target-oriented Opinion Words (or Terms) Extraction . Experimental results indicate that TSMSA outperforms the benchmark methods on TOWE significantly . |
| Approach: | They propose to use a pre-trained language model with multi-head self-attention to integrate TOWE with AOPE to extract aspects and opinion terms in pairs. |
| Outcome: | The proposed structure outperforms the benchmark methods on TOWE significantly . the proposed structure is similar or even better than state-of-the-art AOPE models . |
Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa (2021.naacl-main)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis (ABSA) is a fine-grained task in sentiment analysis. |
| Approach: | They compare a model with a dependency parser and a tree from a fine-tuned RoBERTa model to find the polarities for aspects in a sentence. |
| Outcome: | The proposed model outperforms the parser-provided tree on six datasets across four languages. |
Domain Divergences: A Survey and Empirical Analysis (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing literature on divergence measures is lacking in predicting performance of models in new domains. |
| Approach: | They propose a taxonomy of divergence measures consisting of three classes — Information-theoretic, Geometric, and Higher-order measures and identify the relationships between them. |
| Outcome: | The proposed measures are based on three novel use-cases and identify that they are prevalent in three domains and higher-order measures are more common in two. |
Target-Aware Data Augmentation for Stance Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for stance detection are not diversified or inconsistent with the given target and label information. |
| Approach: | They propose to augment a text with a conditional masked word prediction task . they propose to replace a target mention with 'target-aware' sentences by replacing a reference word with . |
| Outcome: | The proposed method outperforms existing methods on 11 targets. |
End-to-end ASR to jointly predict transcriptions and linguistic annotations (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models generate audio transcripts by sequentially producing likely graphemes, or multi-graphemic units, from which lexical items of a language can be recovered. |
| Approach: | They propose a Transformer-based sequence-to-sequence model for automatic speech recognition that can produce high-quality transcriptions and linguistic annotations. |
| Outcome: | The proposed model can produce high-quality transcriptions and linguistic annotations on Japanese and English audio datasets. |
Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | End-to-end speech translation models can be trained to leverage source text . however, since the input modalities are different, it is difficult to leverage the source text successfully. |
| Approach: | They propose to leverage source transcriptions via pre-training and joint training with ASR and NMT tasks. |
| Outcome: | The proposed model predicts paraphrased transcriptions as an auxiliary task with a single decoder. |
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | ESPnet framework exploits compositionality to learn searchable hidden representations at intermediate stages of a sequence model using decomposed sub-tasks. |
| Approach: | They propose a framework that exploits compositionality to learn searchable hidden representations at intermediate stages of a sequence model using decomposed sub-tasks. |
| Outcome: | The proposed framework outperforms the state-of-the-art on speech translation tasks by +6 and +3 BLEU on the two test sets of Fisher-CallHome and +4 BLUE on the English-German and English-French test sets. |
SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Experimental results show that SPLAT improves the previous state-of-the-art performance on the Spoken SQuAD dataset by more than 10%. |
| Approach: | They propose a semi-supervised learning framework to jointly pre-train the speech and language modules using unpaired speech and text. |
| Outcome: | The proposed framework improves the previous state-of-the-art performance on the Spoken SQuAD dataset by more than 10%. |
Worldly Wise (WoW) - Cross-Lingual Knowledge Fusion for Fact-based Visual Spoken-Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Question-Answering has long been of interest, but its accessibility to users through a speech interface and its support to multiple languages have not been addressed in prior studies. |
| Approach: | They propose a task and a synthetically-generated dataset to do Fact-based Visual Spoken-Question Answering (FVSQA) the task requires a system to retrieve an entity from Knowledge Graphs (KGs) the question is spoken rather than typed. |
| Outcome: | The proposed task performs at same levels of accuracy across 3 languages, including English, Hindi, and Turkish. |
Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignment (2021.naacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive encoder-decoder models improve decoding speed, but generation quality suffers . editing at the level of output sequences limits model flexibility. |
| Approach: | They propose *iterative realignment* which iteratively realigns connectionist temporal alignments. |
| Outcome: | The proposed model matches an autoregressive baseline with a 14x speedup on the WSJ dataset; on LibriSpeech, it achieves an LM-free test-other WER of 9.0% (19% relative improvement on comparable work). |
Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies focus on analyzing structured data, while mining causal relationship among factors from unstructured data is of great importance. |
| Approach: | They propose a graph-based causal inference framework which builds causal graphs from fact descriptions without much human involvement. |
| Outcome: | The proposed framework can capture nuance from fact descriptions among confusing charges and provide explainable discrimination in few-shot settings. |
Counterfactual Supporting Facts Extraction for Explainable Medical Record Based Diagnosis with Graph Network (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods provide explanations based on a precise medical knowledge base, which is disease-specific and difficult to obtain for experts in reality. |
| Approach: | They propose a method to extract supporting facts from irregular EMR without external knowledge bases by constructing a hierarchical graph network and using it to obtain causal relationship between multi-granularity features and diagnosis results. |
| Outcome: | The proposed method diagnoses four types of EMR correctly and provides accurate supporting facts for the results. |
Personalized Response Generation via Generative Split Memory Network (2021.naacl-main)
Copied to clipboard
| Challenge: | Despite the success of text generation and dialogue systems, how to endow a text generation system with personality traits remains under-investigated. |
| Approach: | They propose a model to generate personalized responses on reddit using user profiles and posting histories. |
| Outcome: | The proposed model improves over the state-of-the-art response generation models. |
Towards Few-shot Fact-Checking via Perplexity (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models can perform few-shot learning for various downstream tasks, such as question answering and machine translation. |
| Approach: | They propose a method to leverage the powerful transfer learning ability of a language model via a perplexity score to learn few-shot for the fact-checking task. |
| Outcome: | The proposed method outperforms the Major Class baseline by 10% on the F1-Macro metric across multiple datasets. |
Active2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to deep learning for NLP require large amounts of labeled data. |
| Approach: | They propose an approach that iteratively selects a small number of examples for expert annotation based on their estimated utility in training the model. |
| Outcome: | The proposed approach reduces the data requirements of state-of-the-art AL strategies by 3-25% on multiple NLP tasks while achieving the same performance with virtually no additional computation overhead. |
Generating An Optimal Interview Question Plan Using A Knowledge Graph And Integer Linear Programming (2021.naacl-main)
Copied to clipboard
| Challenge: | a large multi-national IT company added roughly 70,000 employees in FY2018-19. |
| Approach: | They propose an interview assistant system to automatically select an optimal set of technical questions personalized for a candidate. |
| Outcome: | The proposed system can help human interviewers plan for an upcoming interview of that candidate. |
Model Extraction and Adversarial Transferability, Your BERT is Vulnerable! (2021.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models are used for natural language processing (NLP) but when they are deployed as a service, they can suffer from different attacks . |
| Approach: | They propose two defence strategies to protect the target model from adversarial attacks . they show that model extraction and adversarially transferable attacks can be effective . |
| Outcome: | The extracted model can lead to highly transferable adversarial attacks against the target model. |
A Global Past-Future Early Exit Method for Accelerating Inference of Pre-trained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to accelerate inference speed of pre-trained language models are limited to local representations of exit layer . current models are associated with large memory requirement and high computational cost, which slow down inference and further encumber the application of PLMs. |
| Approach: | They propose a method to exit early without passing through all inference layers . they take into consideration all the linguistic information embedded in the past layers a global perspective . |
| Outcome: | The proposed method outperforms existing methods by a large margin . it uses linguistic information embedded in the past layers and future features . the proposed method is scalable and cost-effective . |
Masked Conditional Random Fields for Sequence Labeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Conditional Random Fields (CRF) based neural models are among the most performant for sequence labeling problems, but they can sometimes generate illegal sequences of tags. |
| Approach: | They propose a conditional random field-based model that imposes restrictions on candidate paths during both training and decoding phases. |
| Outcome: | The proposed method improves on existing CRF models with near zero additional cost. |
Heterogeneous Graph Neural Networks for Concept Prerequisite Relation Learning in Educational Data (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to learn prerequisite relations among concepts are lacking . concepts are crucial for learning, organizing, applying and generating knowledge . |
| Approach: | They propose a concept prerequisite relation learning approach which combines concept representation and concept pairwise features to make it more practical. |
| Outcome: | The proposed method achieves state-of-the-art results on four datasets. |
Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies reveal a security threat to natural language processing models, called the Backdoor Attack. |
| Approach: | They propose to hack a model by modifying one single word embedding vector without sacrificing accuracy on clean samples. |
| Outcome: | The proposed method is more efficient and stealthier on sentiment analysis and sentence-pair classification tasks. |
DA-Transformer: Distance-aware Transformer (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models that capture token distances are not optimal for modeling the orders and relations of contexts. |
| Approach: | They propose a distance-aware Transformer that can exploit the real distances between tokens to re-scale the raw self-attention weights. |
| Outcome: | The proposed model outperforms the existing Transformer and its variants on five benchmark datasets and can improve the performance of many tasks. |
ASAP: A Chinese Review Dataset Towards Aspect Category Sentiment Analysis and Rating Prediction (2021.naacl-main)
Copied to clipboard
| Challenge: | Sentiment analysis is a key task in e-commerce to detect fine-to-coarse sentiment polarities. |
| Approach: | They propose to use a large-scale Chinese restaurant review dataset ASAP to investigate the sentiment polarities underlying user reviews. |
| Outcome: | The proposed model outperforms state-of-the-art models on both tasks. |
Are NLP Models really able to Solve Simple Math Word Problems? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing solvers for math word problems often achieve high performance on benchmark datasets . existing models rely on shallow heuristics to achieve high accuracy . |
| Approach: | They restrict their attention to English MWPs taught in grades four and lower . they propose a challenge dataset to test the accuracy of MWp solvers . |
| Outcome: | The proposed model can solve a large fraction of MWPs even with shallow heuristics . the proposed model is much lower on the challenge dataset SVAMP . |
WRIME: A New Dataset for Emotional Intensity Estimation with Subjective and Objective Annotations (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on emotion analysis use subjective emotional intensity labels by the writers and objective ones by the readers. |
| Approach: | They annotate 17,000 SNS posts with both the writer's subjective emotional intensity and the reader's objective emotional intensity to construct a Japanese emotion analysis dataset. |
| Outcome: | The results show that the reader cannot fully detect the emotions of the writer, especially anger and trust. |
KPQA: A Metric for Generative Question Answering Using Keyphrase Weights (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing n-gram similarity metrics fail to discriminate the incorrect answers due to the free-form of the answer. |
| Approach: | They propose a new metric that assigns different weights to each token via keyphrase prediction to judge the correctness of GenQA. |
| Outcome: | The proposed metric has a significantly higher correlation with human judgments than existing metrics in various datasets. |
StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer (2021.naacl-main)
Copied to clipboard
Yiwei Lyu, Paul Pu Liang, Hai Pham, Eduard Hovy, Barnabás Póczos, Ruslan Salakhutdinov, Louis-Philippe Morency
| Challenge: | Existing methods for text style transfer focus on individual high-level semantic changes but do not offer fine-grained control of sentence structure, emphasis, and content. |
| Approach: | They propose a large-scale text style transfer benchmark with 21 fine-grained stylistic changes across atomic lexical, syntactic, semantic, and thematic transfers. |
| Outcome: | The proposed method allows modeling fine-grained changes as building blocks for more complex, high-level transfers. |
Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge (2021.naacl-main)
Copied to clipboard
| Challenge: | Cant is important for understanding advertising, comedies and dogwhistle politics . currently, there are very few resources available for the research of cant . |
| Approach: | They propose a large and diverse dataset for creating and understanding cant from a computational linguistics perspective. |
| Outcome: | The proposed dataset can be used to test word embedding similarity and pretrained language models. |
COVID-19 Named Entity Recognition for Vietnamese (2021.naacl-main)
Copied to clipboard
| Challenge: | a new dataset is being developed to help fight the COVID-19 pandemic . the dataset is annotated for the named entity recognition task with newly-defined entity types . |
| Approach: | They present the first manually-annotated COVID-19 domain-specific dataset for Vietnamese . their dataset is annotated for the named entity recognition task with newly-defined entity types . |
| Outcome: | The proposed dataset is the first manually-annotated COVID-19 domain-specific dataset for Vietnamese. |
Framing Unpacked: A Semi-Supervised Interpretable Multi-View Model of Media Frames (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for news analysis lack transparency in their predictions. |
| Approach: | They propose a semi-supervised model that embeds local information into news articles . it can be used to improve automatic news analysis, authors argue . |
| Outcome: | The proposed model outperforms previous models and can be used with unlabeled training data. |
Automatic Classification of Neutralization Techniques in the Narrative of Climate Change Scepticism (2021.naacl-main)
Copied to clipboard
| Challenge: | neutralisation is used to justify lack of action or promote an alternative view of climate change . action on climate change has become an increasingly partisan issue with strong opposition voices discrediting scientists and spreading scepticism and misinformation. |
| Approach: | They propose to use neutralisation techniques to introduce the problem to the nlp community and to collect manual annotations of neutralised techniques in text relating to climate change. |
| Outcome: | The proposed models are supervised and semi-supervised by a team of researchers from the nlp and the ccsc. |
Suicide Ideation Detection via Social and Temporal User Representations using Hyperbolic Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies indicate that individuals exhibiting suicidal ideation increasingly turn to social media rather than mental health practitioners. |
| Approach: | They propose a framework leveraging a user’s emotional history and social information from a users neighborhood in a network to contextualize the interpretation of the latest tweet of a Twitter user. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on this task, showing the benefits of both socially and personally contextualized representations. |
WikiTalkEdit: A Dataset for modeling Editors’ behaviors on Wikipedia (2021.naacl-main)
Copied to clipboard
| Challenge: | Using the WikiTalkEdit dataset, we show how positive emotion and the use of first-person pronouns predict a positive emotional change in a Wikipedia contributor. |
| Approach: | They introduce and analyze WikiTalkEdit, a dataset of conversations and edit histories from Wikipedia, for research in online cooperation and conversation modeling. |
| Outcome: | The proposed dataset supports the classic understanding of style matching, where positive emotion and the use of first-person pronouns predict a positive emotional change in a Wikipedia contributor. |
The structure of online social networks modulates the rate of lexical change (2021.naacl-main)
Copied to clipboard
| Challenge: | lexical change is a prevalent process, as new words are added, thrive, and decline in day-to-day usage. |
| Approach: | They conduct a large-scale analysis of over 80k neologisms in 4420 online communities over a decade and found that the community’s network structure plays a significant role in lexical change. |
| Outcome: | The results show that the community’s network structure plays a significant role in lexical change. |
Modeling Framing in Immigration Discourse on Social Media (2021.naacl-main)
Copied to clipboard
| Challenge: | Using a dataset of immigration-related tweets, we examine how ordinary people on social media frame political issues. |
| Approach: | They propose to use a dataset of immigration-related tweets labeled for multiple framing typologies from political communication theory to analyze framers. |
| Outcome: | The proposed model enables comparisons between different types of frames on social media and a dataset of immigration-related tweets. |
Modeling the Severity of Complaints in Social Media (2021.naacl-main)
Copied to clipboard
| Challenge: | Complaining is a speech act used by humans to communicate a negative mismatch between reality and expectations . recent work on modeling complaints in natural language processing (NLP) has focused on distinguishing complaints from non-complaints in social media. |
| Approach: | They propose to classify complaints into various severity levels based on the face-threat that the complainer is willing to undertake and their purpose. |
| Outcome: | The proposed model achieves 55.7 macro F1 on binary complaint classification and 88.2 macro F1. |
What About the Precedent: An Information-Theoretic Analysis of Common Law (2021.naacl-main)
Copied to clipboard
| Challenge: | In common law, the outcome of a new case is determined mostly by precedent cases, rather than by existing statutes. |
| Approach: | They propose to model the argumentation of precedent cases and compare them to a case out-come classification task to determine how the precedent influences the outcome of a new case. |
| Outcome: | The proposed method compared arguments of two longstanding jurisprudential views on the European Court of Human Rights (ECtHR) and the precedent cases. |
Introducing CAD: the Contextual Abuse Dataset (2021.naacl-main)
Copied to clipboard
| Challenge: | Detecting and classifying online abuse is a complex and nuanced task, despite many advances in the power and availability of computational tools. |
| Approach: | They propose to annotate a reddit conversation thread with six distinct primary and secondary categories and an expert-driven group-adjudication process for high quality annotations. |
| Outcome: | The proposed dataset contains six distinct primary and secondary categories and uses an expert-driven group-adjudication process for high quality annotations. |
Lifelong Learning of Hate Speech Classification on Social Media (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on automated hate speech classification assumes that the dataset is fixed and the classes are pre-defined. |
| Approach: | They propose to use Variational Representation Learning and a load-balancing self-organizing inductive neural network to learn hate speech classification on social media. |
| Outcome: | The proposed model improves on the lifelong learning techniques on social media. |
Learning to Recognize Dialect Features (2021.naacl-main)
Copied to clipboard
| Challenge: | linguistics do not characterize dialects as simple categories, but as collections of correlated features. |
| Approach: | They propose two multitask learning approaches based on pretrained transformers to detect dialect features in speech and text. |
| Outcome: | The proposed models learn to recognize many features with high accuracy on 22 dialect features of Indian English. |
It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners (2021.naacl-main)
Copied to clipboard
| Challenge: | Pretraining ever-larger language models on massive corpora requires enormous amounts of compute. |
| Approach: | They propose to convert textual inputs into cloze questions that contain a task description . they also exploit unlabeled data to improve their performance . |
| Outcome: | The proposed model outperforms GPT-3 with PET/iPET with cloze questions and unlabeled data. |
Static Embeddings as Efficient Knowledge Bases? (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent research investigates factual knowledge stored in large pretrained language models . masked sentences such as “Paris is the capital of [MASK]” are used as probes . |
| Approach: | They use masked sentences to test whether a language model can capture factual knowledge . they show that static embeddings perform better than PLMs when restricted to a candidate set . |
| Outcome: | The results show that static embeddings perform better than PLMs when restricted to a candidate set . |
Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis (2021.naacl-main)
Copied to clipboard
| Challenge: | Knowledge Graph Embeddings (KGEs) have been explored in recent years due to their promise for a wide range of applications. |
| Approach: | They propose a KGE framework which can reduce the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches. |
| Outcome: | The proposed framework reduces the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches while producing competitive performance. |
Rethinking Network Pruning – under the Pre-train and Fine-tune Paradigm (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pruning results on benchmark transformers, such as BERT, are not as remarkable as those of convolutional neural networks. |
| Approach: | They propose to apply a knowledge-aware pruning process to transformer-based pre-trained language models to reduce model size and model weight. |
| Outcome: | The proposed pruning method outperforms the leading competitors with a 20-times weight/FLOPs compression and neglectable loss in prediction accuracy. |
Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models are not fully considered for societal biases . pre-training models can be useful for many NLP tasks, but they can be harmful when used at scale. |
| Approach: | They investigate gender and racial bias across pre-trained language models . they evaluate bias within pre-trainers using three metrics: WEAT, sequence likelihood, and pronoun ranking. |
| Outcome: | The proposed model fails to detect gender and racial biases in pre-trained models . the model is ineffective when word embedding, demonstrating the need for more robust bias testing in transformers. |
Detoxifying Language Models Risks Marginalizing Minority Voices (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing detoxification techniques have been proposed to mitigate toxic LM generations . e.g., detoxification makes LMs more brittle to distribution shift, especially on language used by marginalized groups . |
| Approach: | They propose to use detoxification techniques to reduce toxic LM generations without affecting perplexity or generation quality on nontoxic inputs. |
| Outcome: | The proposed methods hurt equity on language used by marginalized groups, the authors show . they show that detoxification makes LMs more brittle to distribution shift, they say . |
HONEST: Measuring Hurtful Sentence Completion in Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | 4.3% of the time, language models complete a sentence with a hurtful word . authors propose a score to quantify the amount of hurtful sentence completions in a language model. |
| Approach: | They propose a score to measure hurtful sentence completions in language models . they use a template- and lexicon-based bias evaluation methodology for six languages . |
| Outcome: | The proposed score measures the amount of hurtful sentences in language models. |
EaSe: A Diagnostic Tool for VQA based on Answer Diversity (2021.naacl-main)
Copied to clipboard
| Challenge: | EASE is a diagnostic tool for Visual Question Answering (VQA) it quantifies the difficulty of an image, question sample. |
| Approach: | They propose a diagnostic tool which quantifies the difficulty of an image, question sample. |
| Outcome: | The proposed tool can be used to select the most-informative samples for training/fine-tuning. |
DeCEMBERT: Learning from Noisy Instructional Videos via Dense Captions and Entropy Minimization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to train models on unlabeled web videos are noisy and temporally misaligned . authors propose a method that adds captions and constrained attention loss to improve performance . |
| Approach: | They propose a method that adds captions from video frames as auxiliary text input to provide visual cues for learning better video and language associations. |
| Outcome: | The proposed method outperforms state-of-the-art methods on video-and-language tasks . it adds captions and constrained attention loss to improve model performance . |
Improving Generation and Evaluation of Visual Stories via Semantic Consistency (2021.naacl-main)
Copied to clipboard
| Challenge: | Story visualization is an underexplored task that requires a generative model to generate images . prior work has focused on image generation but there is room for improvement . |
| Approach: | They propose to add a dual learning framework to reinforce semantic alignment between story and generated images and a copy-transform mechanism to model sequentially-consistent story visualization. |
| Outcome: | The proposed models outperform text-to-image synthesis models on the story visualization task . the proposed models also improve visual quality, coherence and relevance . |
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | a new study examines zero-shot cross-lingual transfer of vision-language models . we study multilingual text-to-video search in non-English languages without annotations . |
| Approach: | They propose a Transformer-based model that learns contextual multilingual multimodal embeddings . they propose 'zero-shot cross-lingual transfer' to improve multilingual search . |
| Outcome: | The proposed model outperforms baselines on multilingual text-to-video search and multilingual image search on VTT and VATEX. |
Video Question Answering with Phrases via Semantic Roles (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing VidQA evaluation metrics limit the models’ application scenario to a single-word answer or selecting a phrase from a fixed set of phrases. |
| Approach: | They propose to leverage video descriptions to mask out certain phrases to enable evaluation of answer phrases. |
| Outcome: | The proposed model reduces the influence of language bias on VidQA datasets by retrieving a video having a different answer for the same question. |
From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding (2021.naacl-main)
Copied to clipboard
Rob van der Goot, Ibrahim Sharaf, Aizhan Imankulova, Ahmet Üstün, Marija Stepanović, Alan Ramponi, Siti Oryza Khairunnisa, Mamoru Komachi, Barbara Plank
| Challenge: | Lack of publicly available evaluation data for low-resource languages limits progress in SLU . despite advances in neural modeling for slot and intent detection, datasets for SLU remain limited. |
| Approach: | They propose a joint learning approach with English SLU training data and non-English auxiliary tasks from raw text, syntax and translation for transfer. |
| Outcome: | The proposed model can learn English SLU training data and non-English auxiliary tasks from raw text, syntax and translation for transfer. |
WEC: Deriving a Large-scale Cross-document Event Coreference dataset from Wikipedia (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for cross-document event coreference resolution are limited and small . authors present a method for identifying clusters of text mentions that refer to the same event . |
| Approach: | They propose a method for generating a large-scale Wikipedia event coreference dataset . they use a generic approach that adapts state-of-the-art models to the cross-document setting . |
| Outcome: | The proposed method outperforms existing models and can be applied to other languages. |
Challenging distributional models with a conceptual network of philosophical terms (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing philosophical data are not accurate enough to support philosophers . comparative research on concepts should follow a conceptual model approach, authors argue . |
| Approach: | They propose a ground truth for evaluation created by philosophy experts and a blueprint for using DS models in a sound methodological setup. |
| Outcome: | The proposed model does not perform well enough to directly support philosophers yet, but it yields promising directions for future work. |
KILT: a Benchmark for Knowledge Intensive Language Tasks (2021.naacl-main)
Copied to clipboard
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel
| Challenge: | Existing models for knowledge-intensive language tasks require access to large, external knowledge sources. |
| Approach: | They propose a benchmark for knowledge-intensive language tasks (KILT) they test a shared dense vector index coupled with a seq2seq model to generate disambiguated text. |
| Outcome: | The proposed model outperforms tailor-made approaches on fact checking, open-domain question answering and dialog by generating disambiguated text. |
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | a growing body of work is focused on improving performance in low-resource settings . a goal of this study is to explain how these methods differ in their requirements . |
| Approach: | They propose to analyze data-lean scenarios across different dimensions of data availability to understand which approaches are effective in a specific low-resource setting. |
| Outcome: | The proposed methods enable learning when training data is sparse. |
Temporal Knowledge Graph Completion using a Linear Temporal Regularizer and Multivector Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing knowledge graphs involve evolving data, e.g., the fact (The President of the United States is Barack Obama) is valid only from 2009 to 2017. |
| Approach: | They propose a time-aware knowledge graph embebdding approach which performs 4th-order tensor factorization of a Temporal knowledge graph using a Linear temporal regularizer and Multivector embeddings. |
| Outcome: | The proposed model achieves state-of-the-art performance over four well-established temporal knowledge graph completion benchmarks. |
UDALM: Unsupervised Domain Adaptation through Language Modeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing techniques for unsupervised domain adaptation (UDA) are limited by domain shift, which leads to performance degradation. |
| Approach: | They propose a fine-tuning procedure that uses a mixed classification and Masked Language Model loss to adapt to the target domain distribution in a robust and sample efficient manner. |
| Outcome: | The proposed procedure can adapt to the target domain distribution in a robust and sample efficient manner. |
Beyond Black & White: Leveraging Annotator Disagreement via Soft-Label Multi-Task Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior work shows that disagreement between annotators can be useful in training models. |
| Approach: | They propose to use disagreements as an auxiliary task in a multi-task neural network to incorporate disagreements into models. |
| Outcome: | The proposed method significantly improves performance on NLP tasks beyond the standard approach and prior work. |
Clustering-based Inference for Biomedical Entity Linking (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to linking entities ignore relationships between entities in biomedical knowledge bases. |
| Approach: | They propose a model which can link mentions of unseen entities using learned representations of entities. |
| Outcome: | The proposed model improves on the largest publicly available biomedical dataset by 3.0 points of accuracy and 2.3 points of reliability. |
Variance-reduced First-order Meta-learning for Natural Language Processing Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies show that meta-learning can overfit to some specific adaptation when we have heterogeneous tasks. |
| Approach: | They propose to reduce the variance of the gradient estimator used in task adaptation by adding a new variance reduction term to the gradient estimation. |
| Outcome: | Experiments on few-shot text classification and multi-domain dialog state tracking show that the proposed method outperforms existing methods. |
Diversity-Aware Batch Active Learning for Dependency Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | a high annotation cost for dependency parsers is a challenge . batch active learning (AL) is based on batch mode, which is more efficient for annotators to label in bulk. |
| Approach: | They propose to reduce the number of labeled examples needed to train a strong dependency parser using batch active learning. |
| Outcome: | The proposed approach improves on an English newswire corpus by enforcing diversity in the sampled batches. |
How many data points is a prompt worth? (2021.naacl-main)
Copied to clipboard
| Challenge: | Proponents of prompting argue that they provide a method for injecting task-specific guidance, which is beneficial in low-data regimes. |
| Approach: | They aim to quantify prompting's benefit by testing prompts in a fair setting . they propose to use a generic model head or a task-specific prompt for prediction . |
| Outcome: | The proposed approach is used in T5 fine-tuning leading to state-of-the-art results on the SuperGLUE benchmark. |
Can Latent Alignments Improve Autoregressive Machine Translation? (2021.naacl-main)
Copied to clipboard
| Challenge: | Latent alignment objectives improve non-autoregressive models, but can they improve autoregressive ones? e.g., we show that latent alignments are incompatible with teacher forcing. |
| Approach: | They propose latent alignment objectives that use a dynamic program to comb the space of monotonic alignments between the "gold" target sequence and token probabilities the model predicts. |
| Outcome: | The proposed models are incompatible with teacher forcing, the authors show . they show that latent alignment objectives reduce misalignments and focus on original error . |
Smoothing and Shrinking the Sparse Seq2Seq Search Space (2021.naacl-main)
Copied to clipboard
| Challenge: | entmax-based sparse sequence-to-sequence models give high scores to short hypotheses . ent max models can shrink the search space by assigning zero probability to bad hypothese . |
| Approach: | They propose entmax-based sparse sequence-to-sequence models that minimize cross-entropy and use softmax to compute local normalized probabilities over target sequences. |
| Outcome: | The proposed models remove a major source of model error for word-level tasks . the proposed models improve cross-lingual morphological inflection and machine translation . |
Unified Pre-training for Program Understanding and Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | PLUG is a programming language that is used for programming and language understanding and generation tasks. |
| Approach: | They propose a sequence-to-sequence model that performs a broad spectrum of program and language understanding and generation tasks. |
| Outcome: | The proposed model outperforms or rivals state-of-the-art models on code summarization, code generation, and code translation tasks in seven programming languages. |
Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing continual learning approaches suffer from low accuracy and performance fluctuation when the distributions of old and new data are significantly different. |
| Approach: | They propose a hyperparameter-free continual learning model for text data that can stably produce high performance under various environments. |
| Outcome: | The proposed model outperforms the best state-of-the-art method by 20% in average accuracy and each component contributes effectively to overall performance. |
On the Embeddings of Variables in Recurrent Neural Networks for Source Code (2021.naacl-main)
Copied to clipboard
| Challenge: | Source code processing heavily relies on the methods widely used in natural language processing (NLP) but requires specifics that need to be taken into account to achieve higher quality. |
| Approach: | They propose a recurrent mechanism that adjusts the learned semantics of a variable when it obtains more information about the variable’s role in the program. |
| Outcome: | The proposed method significantly improves the performance of the recurrent neural network, in code completion and bug fixing tasks. |
Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for building high-quality CLWEs learn mappings that minimise the l2 norm loss function but this optimisation objective has been shown to be sensitive to outliers. |
| Approach: | They propose a simple post-processing step to improve cross-lingual word embeddings using the Manhattan norm goodness-of-fit criterion. |
| Outcome: | The proposed approach outperforms four state-of-the-art baselines in bilingual lexicon induction and cross-lingual transfer tasks. |
Semantic Frame Forecast (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior work focused on predicting the immediate future of a story, such as one to a few sentences ahead. |
| Approach: | They propose a task that predicts the semantic frames that will occur in the next 10, 100, or even 1,000 sentences in a running story. |
| Outcome: | The proposed model outperforms random, prior, and replay baselines when the block size is over 150 sentences. |
MUSER: MUltimodal Stress detection using Emotion Recognition as an Auxiliary Task (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to detect stress have not explored the inter-dependence between emotion and stress. |
| Approach: | They propose a transformer-based model architecture and a novel multi-task learning algorithm with speed-based dynamic sampling strategy to improve stress detection. |
| Outcome: | The proposed model is effective with internal and external auxiliary tasks and achieves state-of-the-art results. |
Learning to Decompose and Organize Complex Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | Using a novel end-to-end pipeline, we propose a solution that consumes a complex task and induces 'dependency graphs' from unstructured text to represent sub-tasks and their relationships. |
| Approach: | They propose a pipeline that consumes a complex task and induces 'dependency graphs' from unstructured text to represent sub-tasks and their relationships. |
| Outcome: | The proposed pipeline outperforms state-of-the-art graph induction pipelines in a dataset of complex tasks with their sub-task graphs. |
Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing continual learning methods focus on preserving knowledge from previous tasks . Continual learning is a useful tool for learning over time, but it is not always possible to generalize to new tasks. |
| Approach: | They propose a disentanglement-based regularization method for continual learning on text classification that disentangles text hidden spaces into generic representations and regularizes them differently to constrain knowledge required to generalize. |
| Outcome: | The proposed method disentangles text hidden spaces into representations that are generic to all tasks and representations specific to each individual task. |
Learning from Executions for Semantic Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | Semantic parsing aims at translating natural language (NL) utterances onto machine-interpretable programs. |
| Approach: | They propose to encourage a parser to generate executable programs for unlabeled NL utterances. |
| Outcome: | The proposed training objectives outperform conventional methods on Overnight and GeoQuery. |
Learning to Synthesize Data for Semantic Parsing (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for synthesizing data for semantic parsing require handcrafted rules to synthesize new programs or utterance-program pairs. |
| Approach: | They propose to use a (non-neural) PCFG to model the composition of programs and a BART-based translation model to map a program to an utterance to learn a generative model from existing data. |
| Outcome: | The proposed model can be efficiently learned from existing data on benchmarks of GeoQuery and Spider. |
Edge: Enriching Knowledge Graph Embeddings with External Text (2021.naacl-main)
Copied to clipboard
| Challenge: | Knowledge graphs suffer from sparsity which degrades the quality of representations generated by various methods. |
| Approach: | They propose a knowledge graph enrichment framework called Edge to enhance knowledge graphs based on "hard" co-occurrence of words in knowledge graph entities and external text. |
| Outcome: | The proposed framework achieves "soft" augmentation by combining external text with knowledge graph entities. |
FLIN: A Flexible Natural Language Interface for Web Navigation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing semantic parsing and slot-filling techniques cannot adapt to many different websites without being constantly re-trained. |
| Approach: | They propose a natural language interface for web navigation that maps user commands to concept-level actions rather than low-level UI actions. |
| Outcome: | The proposed interface can adapt to new websites in a given domain. |
Game-theoretic Vocabulary Selection via the Shapley Value and Banzhaf Index (2021.naacl-main)
Copied to clipboard
| Challenge: | Using the full vocabulary results in less explainable and memory intensive models. |
| Approach: | They propose a vocabulary selection method that views words as members of a team trying to maximize the model's performance. |
| Outcome: | The proposed method outperforms baseline models on multiple tasks and datasets. |
Incorporating External Knowledge to Enhance Tabular Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models struggle with tabular inference due to contextualized embeddings of text. |
| Approach: | They propose easy and effective modifications to how information is presented to a model for tabular inference. |
| Outcome: | The proposed modifications significantly improve tabular inference performance on large datasets. |
Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention (2021.naacl-main)
Copied to clipboard
Pengcheng Yin, Hao Fang, Graham Neubig, Adam Pauls, Emmanouil Antonios Platanios, Yu Su, Sam Thomson, Jacob Andreas
| Challenge: | Existing approaches to compositional generalization in semantic parsers focus on word-level alignments, but they focus on spans. |
| Approach: | They propose a span-level supervised attention loss that improves compositional generalization in semantic parsers by focusing on spans. |
| Outcome: | The proposed method improves on three benchmarks of compositional generalization. |
Domain Adaptation for Arabic Cross-Domain and Cross-Dialect Sentiment Analysis from Contextualized Word Embedding (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies have classified dialectal Arabic into more fine-grained levels, including countries and cities. |
| Approach: | They propose to use Arabic domains to transfer knowledge from labeled source domains into unlabeled target domains by transferring the learned knowledge from a labele . |
| Outcome: | The proposed method outperforms other domain adaptation methods and improves performance by 20.8% over the zero-shot transfer learning from BERT. |
Multi-task Learning of Negation and Speculation for Targeted Sentiment Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Currently, most work on targeted sentiment analysis is focused on improving the overall results. |
| Approach: | They propose a multi-task learning method to incorporate information from syntactic and semantic auxiliary tasks to create English-language models that are more robust to linguistic phenomena. |
| Outcome: | The proposed method improves on negation and speculation datasets but there is room for improvement. |
A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing topic models may extract topics associated with writers’ subjective opinions mixed with those related to factual descriptions. |
| Approach: | They propose a neural topic model combined with adversarial training to disentangle opinion topics from plot and neutral ones. |
| Outcome: | The proposed model shows improved coherence and variety of topics, consistent disentanglement rate, and superior sentiment classification performance to other supervised topic models. |
Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on aspect-level sentiment classification has shown that syntactic information is effective in capturing long-range syntaktic relations that are obscure from the surface form. |
| Approach: | They propose a graph ensemble technique that integrates syntactic structures with GNNs to better leverage syntaktic information in the face of parsing errors. |
| Outcome: | The proposed model outperforms models with single dependency tree and beats other models without adding model parameters. |
Emotion-Infused Models for Explainable Psychological Stress Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | a new study examines the use of emotion detection for detecting psychological stress in online posts . traditional multi-task learning and emotion-based language model fine-tuning are used to improve the model . |
| Approach: | They propose to use a semantically related task, emotion detection, for detecting psychological stress in online posts . they propose multi-task learning and emotion-based language model fine-tuning to improve the model . |
| Outcome: | The proposed model is more explainable and human-like than a black-box model . the proposed model mirrors psychological components of stress, the authors show . |
Aspect-based Sentiment Analysis with Type-aware Graph Convolutional Networks and Layer Ensemble (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies only leverage dependency relations without considering their dependency types . a valid and effective approach is demonstrated on six English benchmark datasets . |
| Approach: | They propose to explicitly utilize dependency types for ABSA with type-aware graph convolutional networks . attention is used in T-GCN to distinguish different edges in the graph and attentive layer ensemble to comprehensively learn from different layers of T-gCN. |
| Outcome: | The proposed approach performs well on six English benchmark datasets. |
Supertagging-based Parsing with Linear Context-free Rewriting Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | a new supertagging-based parser for linear context-free rewriting systems is developed for discontinuous constituents . discontinuous constituencies span non-contiguous sets of positions in a sentence, and can be modelled by CFG . |
| Approach: | They propose a supertagging-based parser for linear context-free rewriting systems . they propose an efficient procedure which induces a lexical LCFRS from any discontinuous treebank . |
| Outcome: | The proposed method outperforms previous LCFRS-based parsers in accuracy and speed by a wide margin. |
Outside Computation with Superior Functions (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing generalized inside-outside algorithm would violate Strong Exponential Time Hypothesis (SETH) |
| Approach: | They propose a framework for efficient outside computation that would yield a sub-exponential time algorithm for SAT, violating the Strong Exponential Time Hypothesis (SETH). |
| Outcome: | The proposed algorithm would violate the Strong Exponential Time Hypothesis (SETH) . |
Learning Syntax from Naturally-Occurring Bracketings (2021.naacl-main)
Copied to clipboard
| Challenge: | a new method for learning naturally-occurring bracketings is developed . it uses noisy and incomplete data to induce syntactic structures . |
| Approach: | They propose a partial-brackets-aware structured ramp loss in learning to address this challenge . they show that distantly-supervised models trained on naturally-occurring bracketing data are more accurate . constituency is a foundational building block for phrase-structure grammars, they argue . |
| Outcome: | The proposed model achieves an unlabeled F1 score for constituency parsing on the English WSJ corpus. |
Bot-Adversarial Dialogue for Safe Conversational Agents (2021.naacl-main)
Copied to clipboard
| Challenge: | a new method for evaluating chatbot safety is proposed to mimic human-generated data . a bot-adversarial dialogue model learns undesirable features from this data, a study finds . |
| Approach: | They propose a human-and-model-in-the-loop framework for evaluating toxicity of chatbots . they propose two methods for safe conversational agents by either training on data or ”baking-in” safety to the generative model itself. |
| Outcome: | The proposed methods are safer than existing models while maintaining usability metrics, the authors say . they show that the proposed methods can be used to make safer models with human-model interactions . |
Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog (2021.naacl-main)
Copied to clipboard
| Challenge: | Semantic parsing using sequence-to-sequence models is stymied by higher compute requirements and higher latency. |
| Approach: | They propose a non-autoregressive approach to predict semantic parse trees with an efficient seq2seq model architecture. |
| Outcome: | The proposed architecture achieves an 81% reduction in latency on TOP dataset and retains competitive performance over non-pretrained models on three different semantic parsing datasets. |
Example-Driven Intent Prediction with Observers (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior work has shown that BERT-like models attribute a significant amount of attention to the [CLS] token, which results in diluted representations. |
| Approach: | They propose two approaches to improve generalizability of dialog system intent classification models by using observers and example-driven training. |
| Outcome: | The proposed models achieve state-of-the-art on three intent prediction datasets in both the full data and few-shot settings. |
Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing research on taskoriented dialog systems mainly includes pipeline and end-to-end methods due to its non-differentiable nature. |
| Approach: | They propose a multi-level reward modeling approach that factorizes a reward into a three-level hierarchy: domain, act, and slot. |
| Outcome: | The proposed approach significantly improves performance and speed of training in a wide range of dialog systems. |
Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing goal-oriented dialogue datasets focus on identifying slots and values, but in reality, customer service agents follow multi-step procedures derived from explicit company policies. |
| Approach: | They propose to use a fully-labeled dataset to study customer service dialogue systems in real-world scenarios. |
| Outcome: | The proposed dataset outperforms existing models but still lacks 50.8% absolute accuracy to reach human-level performance on the dataset. |
Controlling Dialogue Generation with Semantic Exemplars (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to control dialogue generation are manual labelling and manual editing of data. |
| Approach: | They propose a method to control dialogue generation using exemplar responses . they use semantic frames present in exemplars to guide response generation . |
| Outcome: | The proposed model improves coherence while preserving semantic meaning and conversation goals . exemplar responses are handwritten or strategically curated to promote highlevel goals without explicit labels . |
COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent neural IR models shift towards soft matching all query document terms, but they lose the computation efficiency of exact match systems. |
| Approach: | They propose a contextualized exact match retrieval architecture where scoring is based on overlapping query document tokens’ contextualized representations. |
| Outcome: | The proposed architecture outperforms classical lexical retrieval systems and state-of-the-art deep language models with smaller latency. |
X-Class: Text Classification with Extremely Weak Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | Weak supervision is a problem in text classification, but it requires corpusspecific knowledge. |
| Approach: | They propose a framework for extremely weak supervision that can be used to train a text classifier. |
| Outcome: | The proposed framework outperforms seed-driven weakly supervised methods on 7 benchmark datasets. |
Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Topic models can augment or replace bag-of-words inputs with pre-trained transformer-based word prediction models. |
| Approach: | They propose several methods for fine-tuning encoders to improve both monolingual and zero-shot polylingual topic modeling. |
| Outcome: | The proposed methods improve both monolingual and zero-shot polylingual topic modeling. |
Exploring the Relationship Between Algorithm Performance, Vocabulary, and Run-Time in Text Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Many text classification algorithms depend on the size of the corpus’ vocabulary due to their bag-of-words representation. |
| Approach: | They propose to evaluate how preprocessing techniques affect the run-time of models by evaluating ten techniques over four models and two datasets. |
| Outcome: | The proposed methods can reduce run-time with no loss of accuracy while sacrificing up to 65%. |
Faithfully Explainable Recommendation via Neural Logic Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for explainable recommendation have neglected faithfulness of KG reasoning . |
| Approach: | They propose to draw on interpretable logical rules to guide path-reasoning process for explanation generation. |
| Outcome: | The proposed method delivers high-quality recommendations and ascertains the faithfulness of the derived explanation. |
You Sound Like Someone Who Watches Drama Movies: Towards Predicting Movie Preferences from Conversational Interactions (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for conversational recommendation include collaborative filtering, content-based filtering and user reviews. |
| Approach: | They propose to map a conversational user to most similar external reviewers, whose preferences are known, and adapt collaborative filtering techniques to estimate the current user’s preferences for new movies. |
| Outcome: | The proposed method can improve the accuracy of predicting user ratings for new movies by exploiting conversation content and external data. |
Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has used text-based games as a testbed for developing autonomous agents that operate using natural language. |
| Approach: | They propose an inverse dynamics decoder to regularize representation space and encourage exploration to reduce the amount of semantic information available to a learning agent. |
| Outcome: | The proposed model achieves high scores even in the absence of language semantics on Zork I . |
SOrT-ing VQA Models : Contrastive Gradient Learning for Improved Consistency (2021.naacl-main)
Copied to clipboard
| Challenge: | Current visual question answering models are inconsistent in their understanding of the world . they answer difficult reasoning questions correctly but get associated sub-questions wrong . |
| Approach: | They propose a gradient-based interpretability approach to determine the questions most strongly correlated with the reasoning question on an image and a contrastive gradient learning based approach called Sub-question Oriented Tuning (SOrT). |
| Outcome: | The proposed approach improves model consistency by up to 6.5% points over existing approaches while improving visual grounding and robustness to rephrasings of questions. |
Semi-Supervised Policy Initialization for Playing Games with Language Hints (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for Reinforcement Learning only give out the achieved signal, making learning difficult. |
| Approach: | They propose a semi-supervised initialization that allows the agent to learn from various possible hints before training under different tasks. |
| Outcome: | The proposed method improves learning speed and success rate by 11% compared to other methods. |
Revisiting Document Representations for Large-Scale Zero-Shot Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for visual recognition use visual attributes carefully annotated by humans. |
| Approach: | They propose a semi-automatic mechanism for visual sentence extraction that leverages document section headers and clustering structure of visual sentences. |
| Outcome: | The proposed method improves on the ImageNet dataset with 10,000 unseen classes. |
Negative language transfer in learner English: A new dataset (2021.naacl-main)
Copied to clipboard
| Challenge: | This dataset contains annotated error causes for learner writing errors that tie learner mistakes to structures from their first language. |
| Approach: | They propose a learner English dataset enhanced with annotated error causes and concrete examples of learner errors that relate to their first languages. |
| Outcome: | The proposed dataset will be used to analyze learner errors related to language transfer from the learners’ first language. |
SentSim: Crosslingual Semantic Evaluation of Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Machine translation (MT) is currently evaluated in one of two ways: monolingually or trained crosslingually by building a supervised model to predict quality scores from human-labeled data. |
| Approach: | They propose an unsupervised model that directly compares the source and machine translated sentence using strong pretrained multilingual word and sentence representations. |
| Outcome: | The proposed model outperforms glass-box approaches to quality estimation that rely on a supervised model. |
Quality Estimation for Image Captions Based on Large-scale Human Evaluations (2021.naacl-main)
Copied to clipboard
| Challenge: | a problem with automatic image captioning is that it produces low quality captions when used in the wild. |
| Approach: | They propose to model caption quality from a human perspective and *without* access to ground-truth references. |
| Outcome: | The proposed model can detect and filter out low-quality captions on previously unseen images. |
CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing systems that negotiate with humans have broad applications in pedagogy and conversational AI. |
| Approach: | They propose to annotate persuasion strategies and perform correlation analysis to understand how dialogue behaviors are associated with the negotiation performance. |
| Outcome: | The proposed system improves negotiation performance for all strategies labeled as skewed . the proposed system is available on github.com/kushalchawla/ . |
News Headline Grouping as a Challenging NLU Task (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in Natural Language Understanding (NLU) have seen models outperform human performance on many standard tasks. |
| Approach: | They propose a task of HeadLine Grouping and a dataset consisting of 20,056 pairs of news headlines, each labeled with a binary judgement as to whether the pair belongs within the same group. |
| Outcome: | The proposed model outperforms human models on a task consisting of 20,056 pairs of headlines on HLGD and a dataset with a binary judgement. |
Olá, Bonjour, Salve! XFORMAL: A Benchmark for Multilingual Formality Style Transfer (2021.naacl-main)
Copied to clipboard
| Challenge: | XFORMAL benchmarks formal reformulations of informal text in Brazilian Portuguese, French, and Italian . most work on style transfer within English, while covering different languages has received disproportional interest. |
| Approach: | They create a benchmark of multiple formal reformulations of informal text in Brazil, Brazil, and Italy. |
| Outcome: | XFORMAL benchmarks formal reformulations of informal text in Brazilian Portuguese, French, and Italian . results show that state-of-the-art approaches perform close to simple baselines . |
Grouping Words with Semantic Diversity (2021.naacl-main)
Copied to clipboard
| Challenge: | a new approach to grouping input words based on their semantic diversity is proposed . high-dimensional inputs and learning complexity hinders deep learning generalization, authors say . |
| Approach: | They propose a way to group input words based on their semantic diversity to simplify input language representation with low ambiguity. |
| Outcome: | The proposed methods generalize NLP models and demonstrate improvements on medium-scale machine translation tasks. |
Noise Stability Regularization for Improving BERT Fine-tuning (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that fine-tuning pre-trained language models is unstable when there are only a small number of training samples available. |
| Approach: | They propose to use a method to regularize noise in deep nets to improve fine-tuning on NLP tasks. |
| Outcome: | The proposed method improves fine-tuning on natural language processing tasks by incorporating noise to the input and demonstrating generalizability and stability. |
FlowPrior: Learning Expressive Priors for Latent Variable Sentence Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing training strategies are not effective for learning rich priors, we propose adding the importance-sampled log marginal likelihood as a second term to the standard VAE objective. |
| Approach: | They propose to add importance-sampled log marginal likelihood to standard VAE objective to help when learning the prior. |
| Outcome: | The proposed model improves language modeling tasks compared to baselines. |
HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for hierarchical text classification do not consider statistical constraint on label representations learned by structure encoder. |
| Approach: | They propose a new hierarchical text classification model called HTCInfoMax which incorporates two modules to improve the model's representations. |
| Outcome: | The proposed model can model the interaction between each text sample and its ground truth labels explicitly which filters out irrelevant information. |
Knowledge Guided Metric Learning for Few-Shot Text Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to learn from limited examples are insufficient for many-shot text classification tasks. |
| Approach: | They propose to introduce external knowledge into few-shot learning to imitate human knowledge by creating a parameter generator network that generates different metrics for different tasks. |
| Outcome: | The proposed method outperforms the SoTA few-shot text classification models. |
Ensemble of MRR and NDCG models for Visual Dialog (2021.naacl-main)
Copied to clipboard
| Challenge: | BLEU scores favor correct syntax over semantics. |
| Approach: | They propose a non-parametric ranking method that integrates the ranks of two strong MRR and NDCG models into a single ranking that excels on both metrics. |
| Outcome: | The proposed model can keep the MRR and NDCG models state-of-the-art and the NDGC models state of the art. |
Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on structured prediction has produced very effective supervised clustering algorithms using linear classifiers. |
| Approach: | They propose to use latent structured prediction loss and Transformer models to approach supervised clustering. |
| Outcome: | The proposed approach outperforms the state-of-the-art in recreating intents from public question corpora. |
ConVEx: Data-Efficient and Few-Shot Slot Labeling (2021.naacl-main)
Copied to clipboard
| Challenge: | ConVEx is an efficient pretraining and fine-tuning neural approach for slot-labeling dialog tasks. |
| Approach: | They propose an efficient pretraining and fine-tuning neural approach for slot-labeling dialog tasks that uses a pairwise cloze task and reddit data. |
| Outcome: | The proposed approach is well aligned with its intended use on slot-labeling tasks and can be used across a range of domains and data sets. |
CREAD: Combined Resolution of Ellipses and Anaphora in Dialogues (2021.naacl-main)
Copied to clipboard
Bo-Hsiang Tseng, Shruti Bhargava, Jiarui Lu, Joel Ruben Antony Moniz, Dhivya Piraviperumal, Lin Li, Hong Yu
| Challenge: | Traditionally, anaphora resolution and ellipses resolution are limited in dialogues . despite rapid progress in dialogue systems, several difficulties remain . |
| Approach: | They propose a joint learning framework for modeling coreference resolution and query rewriting for complex, multi-turn dialogues. |
| Outcome: | The proposed model outperforms the state-of-the-art model on a rewritten dialogue dataset. |
Knowledge-Driven Slot Constraints for Goal-Oriented Dialogue Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Traditional goal-oriented dialogue systems allow execution of validation rules as a post-processing step after slots have been filled which can lead to error accumulation. |
| Approach: | They propose a task of constraint violation detection based on knowledge-driven slot constraints . they propose methods to integrate external knowledge into the system and compare it to traditional rule-based pipeline approach . |
| Outcome: | The proposed task compares to the existing system and a rule-based pipeline. |
Clipping Loops for Sample-Efficient Dialogue Policy Optimisation (2021.naacl-main)
Copied to clipboard
| Challenge: | In previous work, a large number of human dialogues are required to train dialogue agents. |
| Approach: | They propose loop-clipping policy optimisation to eliminate useless responses by clipping loops from dialogue history and clipping advantage to distinguish useless actions from others. |
| Outcome: | The proposed method achieves 80% success rate on a Cambridge restaurant dialogue system using 260 training dialogues compared to baseline of 2160 dialogues. |
Integrating Lexical Information into Entity Neighbourhood Representations for Relation Prediction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to predict knowledge base relations are limited by maintenance costs and text-based formats. |
| Approach: | They propose a system that can extend relational database tables with information extracted from a document corpus. |
| Outcome: | The proposed system outperforms existing methods by incorporating embeddings of text-based representations of the entities and relations. |
Noisy-Labeled NER with Confidence Estimation (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies in deep learning have shown significant progress in named entity recognition (NER) . however, most existing works assume clean data annotation, while real-world data typically involve a large amount of noises. |
| Approach: | They propose a confidence estimation approach for named entity recognition using noisy labels using local and global independence assumptions. |
| Outcome: | The proposed method marginalizes out labels of low confidence with a CRF model and integrates it into a self-training framework for boosting performance. |
TABBIE: Pretrained Representations of Tabular Data (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on tabular representation-learning jointly models tables and associated text using self-supervised objective functions derived from pretrained language models such as BERT. |
| Approach: | They propose a tabular representation-learning model that integrates tabular data with a pretraining objective function that detects corrupted cells. |
| Outcome: | The proposed model understands complex table semantics and numerical trends. |
Better Feature Integration for Named Entity Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition (NER) focus on stacking the LSTM and graph neural networks (GCNs) however, the exact interaction mechanism between the two types of features is not clear and the performance gain is not significant. |
| Approach: | They propose a model that incorporates both types of features with a Synergized-LSTM which captures how the two types of feature interact. |
| Outcome: | The proposed model achieves better performance than previous approaches while requiring fewer parameters. |
ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to relation extraction require labeled data, but labeling is difficult . Existing models cannot recognize rare instances that are never covered by training data . |
| Approach: | They propose a multi-task learning model that directly predicts unseen relations without hand-crafted attribute labeling and multiple pairwise classifications. |
| Outcome: | The proposed model outperforms existing methods by 13.54% on two well-known datasets. |
Graph Convolutional Networks for Event Causality Identification with Rich Document-level Structures (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for document-level Event Causality Identification (ECI) are limited to intra-sentence contexts where event mention pairs are presented in the same sentences. |
| Approach: | They propose a deep learning model that accepts inter-sentence event mention pairs . they use interaction graphs to capture relevant connections between important objects . |
| Outcome: | The proposed model achieves state-of-the-art on two benchmark datasets. |
A Context-Dependent Gated Module for Incorporating Symbolic Semantics into Event Coreference Resolution (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for event coreference resolution use symbolic features, but they are noisy and contain errors. |
| Approach: | They propose a context-dependent gated module to adaptively control the information flows from the input symbolic features. |
| Outcome: | The proposed model achieves state-of-the-art on two datasets: ACE 2005 and KBP 2016 . |
Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for style transfer require joint annotations across all stylistic dimensions, limiting their application to multiple styles. |
| Approach: | They initialize an encoder-decoder setup with transformer-based language model pre-trained on a generic corpus and enhance its re-writing capability to multiple target style dimensions by employing multiple style-aware language models as discriminators. |
| Outcome: | The proposed model can control styles across multiple style dimensions while preserving content of the input text. |
FUDGE: Controlled Text Generation With Future Discriminators (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models allow us to generate increasingly realistic text by modeling a distribution P (X) over natural language sequences X. |
| Approach: | They propose a flexible and modular method for controlled text generation that uses a Bayesian decomposition of the conditional distribution of G given an attribute predictor and can easily compose predictors for multiple desired attributes. |
| Outcome: | The proposed method can be easily composed and performs three tasks. |
Controllable Text Simplification with Explicit Paraphrasing (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing text simplification systems rely on deletion and do not paraphrase well. |
| Approach: | They propose a hybrid approach that leverages linguistically-motivated rules for splitting and deletion and couples them with a neural paraphrasing model to produce varied rewriting styles. |
| Outcome: | The proposed model improves paraphrasing capability and paraphrases more often than existing models. |
Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on data-to-text generation focused on domain-specific benchmark datasets. |
| Approach: | They use a KG-Wikipedia text aligned corpus to verbalize the entire English Wikidata KG . they show that this approach can be used to integrate structured KGs and natural language corpora . |
| Outcome: | The proposed method improves on open domain QA and the LAMA knowledge probe. |
Choose Your Own Adventure: Paired Suggestions in Collaborative Writing for Evaluating Story Generation Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Comparative evaluations have been shown to produce more reliable and consistent results than Likert scale ratings. |
| Approach: | They propose a collaborative writing setup where two models generate suggestions to people as they write a short story and then ask them to choose which model's suggestions they prefer. |
| Outcome: | The proposed model performs better in cases where the differences in generation methods are small (nucleus vs. top-k sampling) and large (GPT2 v. Fusion models). |
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training (2021.naacl-main)
Copied to clipboard
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou
| Challenge: | Existing methods for learning cross-lingual representations are lacking in the field of NLP. |
| Approach: | They propose a framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts. |
| Outcome: | The proposed approach improves cross-lingual transferability on benchmarks. |
Context-Interactive Pre-Training for Document Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document machine translation typically suffers from a lack of document-level bilingual data. |
| Approach: | They propose a document machine translation model that incorporates contextual information into the training signals by capturing cross-sentence dependency within the target document and cross sentence translation to make better use of contextual information. |
| Outcome: | The proposed model outperforms baselines on three benchmark datasets and significantly outperformed previous approaches. |
Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots (2021.naacl-main)
Copied to clipboard
| Challenge: | Multilingual models have demonstrated impressive cross-lingual transfer abilities. |
| Approach: | They propose two strong adversarial attacks that target multilingual models that can handle code-mixed sentences using bilingual dictionaries. |
| Outcome: | The proposed model has a success rate of 89.75% against XLM-R-large, bringing its average accuracy of 79.85 down to 8.18 on XNLI. |
X-METRA-ADA: Cross-lingual Meta-Transfer learning Adaptation to Natural Language Understanding and Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Multilingual models have gained popularity for their zero-shot cross-lingual transfer learning capabilities, but their generalization ability is inconsistent for typologically diverse languages. |
| Approach: | They propose a meta-learning approach that adapts MAML to learn to adapt to new languages . they extensively evaluate two cross-lingual NLU tasks using English as source and spanish as target . |
| Outcome: | The proposed approach outperforms naive fine-tuning on cross-lingual tasks for most languages. |
Explicit Alignment Objectives for Multilingual Bidirectional Encoders (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained cross-lingual encoders have proven impressively effective at enabling transfer-learning of NLP systems from high-resource languages to low-resourced languages. |
| Approach: | They propose a method to align multilingual encoders using two explicit alignment objectives that align the multilingual representations at different granularities. |
| Outcome: | The proposed method achieves gains of up to 1.1 average F1 score on sequence tagging and 27.3 average accuracy on retrieval over the XLM-R-large model. |
Cross-lingual Cross-modal Pretraining for Multimodal Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent pretrained vision-language models have achieved impressive performance on cross-modal retrieval tasks in English. |
| Approach: | They propose a new approach to learn cross-lingual cross-modal representations for matching images and captions in multiple languages using an annotated corpus. |
| Outcome: | The proposed model achieves impressive performance on two multimodal multilingual image caption benchmarks: Multi30k with German captions and MSCOCO with Japanese captions. |
Wikipedia Entities as Rendezvous across Languages: Grounding Multilingual Language Models by Predicting Wikipedia Hyperlinks (2021.naacl-main)
Copied to clipboard
| Challenge: | Masked language models have become the de facto standard when processing text . however, these models are evaluated in a monolingual setting only . |
| Approach: | They propose a language-independent entity prediction task as an intermediate training procedure to ground word representations on entity semantics and bridge the gap between different languages. |
| Outcome: | The proposed approach bridges the gap between word representations and knowledge graphs by using a shared vocabulary of entities. |
multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work to generate proof graphs for formal reasoning over explicit knowledge is not unique and there may be multiple ways of reaching the correct answer. |
| Approach: | They propose to generate multiple proof graphs for reasoning over natural language rules and facts . they propose to combine all proofs and exploit correlations between them . |
| Outcome: | The proposed model outperforms PRover on multiple gold proofs on synthetic, zero-shot, and human-paraphrased datasets. |
Adaptable and Interpretable Neural MemoryOver Symbolic Knowledge (2021.naacl-main)
Copied to clipboard
| Challenge: | Past research has shown that large neural language models encode surprising amounts of factual information, but augmenting or modifying this information requires modifying a corpus and retraining, which is computationally expensive. |
| Approach: | They propose a neural LM that includes an interpretable neuro-symbolic KB in the form of a "fact memory" their LM improves performance on knowledge-intensive question-answering tasks, sometimes dramatically . |
| Outcome: | The proposed model improves on knowledge-intensive question-answering tasks, including a 27 point increase in one setting of WebQuestionsSP over a state-of-the-art open-book model. |
CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Images (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing research on visual question answering is limited to information explicitly present in an image or a video. |
| Approach: | They propose a vision-language question answering task based on a CLEVR dataset . they modify existing methods and propose baseline solvers for this task . |
| Outcome: | The proposed model motivates the development of better vision-language models . it provides insights about the capability of diverse architectures to perform joint reasoning over image-text modality. |
Refining Targeted Syntactic Evaluation of Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models’ syntaktic knowledge using hand-crafted minimal pairs of sentences that differ only in the main verb’s conjugation. |
| Approach: | They propose to use templates to evaluate language models' syntactic knowledge to assess their ability to conjugate arbitrary verbs and their likely behavior to measure their likelihood of conjugating grammatical sentences. |
| Outcome: | The proposed evaluations overestimate systematicity of language models, but score up to 40% better on verbs that they predict are likely in context. |
Universal Adversarial Attacks with Natural Triggers for Text Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has demonstrated the vulnerability of modern text classifiers to universal adversarial attacks, which are input-agnostic sequences of words added to text processed by classifier. |
| Approach: | They propose a gradient-based search that aims to maximize the downstream classifier’s prediction loss by using an adversarially regularized autoencoder to generate triggers and propose heuristics to spot such attacks. |
| Outcome: | The proposed algorithms reduce model accuracy while being less identifiable than prior models as per automatic detection metrics and human-subject studies. |
QuadrupletBERT: An Efficient Model For Embedding-Based Large-Scale Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for large-scale query-document retrieval are expensive and require sparse handcrafted features. |
| Approach: | They propose a quadrupletBERT model for effective and efficient retrieval using pre-trained language models like BERT. |
| Outcome: | The proposed model improves retrieval phase and leverages distances between simple negative and hard negative instances to obtain better embeddings. |
Dynamically Disentangling Social Bias from Task-Oriented Representations with Adversarial Attack (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to learn representations from text often reflect social biases . previous methods rely on pre-specified direction or suffer from unstable training . |
| Approach: | They propose an adversarial disentangled debiasing model to decouple social bias attributes from intermediate representations trained on the main task. |
| Outcome: | The proposed model decouples social bias attributes from intermediate representations trained on the main task. |
An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing research focuses on textual elements of financial disclosures but ignores the rich acoustic features in the executives’ speech. |
| Approach: | They propose to use a multimodal approach that leverages the verbal and vocal cues of speakers in financial disclosures to predict volatility and risk. |
| Outcome: | The proposed models outperform existing models in the financial realm but still underrepresent the diverse communities spanning demographics, gender, and native speech. |
Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing (2021.naacl-main)
Copied to clipboard
| Challenge: | Ethical considerations regarding the use of crowdworkers are limited to labor conditions . the Final Rule did not anticipate the use online crowdsourcing platforms for data collection . |
| Approach: | They propose to reopen discussion regarding ethical use of crowdworkers in NLP research . they propose to use online crowdsourcing platforms to evaluate risk of harm . |
| Outcome: | The proposed study identifies common scenarios where crowdworkers performing NLP tasks are at risk of harm. |
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021.naacl-main)
Copied to clipboard
| Challenge: | PTLMs can exhibit biases against protected groups in a host of modeling tasks . but, fine-tuned LMs may propagate bias to downstream classifiers . |
| Approach: | They propose to use upstream bias mitigation techniques to reduce bias on downstream tasks by fine-tuning an upstream model and applying it to a downstream model. |
| Outcome: | The proposed model reduces bias on hate speech detection, toxicity detection and coreference resolution tasks over bias factors. |
Case Study: Deontological Ethics in NLP (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work in natural language processing (NLP) has focused on ethical challenges . ethical foundations of NLP systems have not been explored . |
| Approach: | They propose to use deontological ethics to analyze ethical issues in natural language processing from the perspective of NLP. |
| Outcome: | The proposed ethical frameworks are based on the generalization principle and respect for autonomy through informed consent. |
Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels (2021.naacl-main)
Copied to clipboard
Fatemehsadat Mireshghallah, Huseyin Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, Robert Sim
| Challenge: | Neural language models have a high capacity for memorization of training samples . however, this can cause privacy degradation and disparate impact on subgroups of users . |
| Approach: | They propose two privacy-preserving regularization methods for training language models that enable joint optimization of utility and privacy. |
| Outcome: | The proposed methods have favorable utility-privacy trade-off, faster training and uniform treatment of under-represented subgroups. |
On the Impact of Random Seeds on the Fairness of Clinical Classifiers (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has shown that fine-tuning large networks is surprisingly sensitive to changes in random seed(s). |
| Approach: | They explore the implications of this phenomenon for model fairness across demographic groups in clinical prediction tasks over electronic health records (EHR) they find that jointly optimizing for high overall performance and low disparities does not yield statistically significant improvements. |
| Outcome: | The proposed model fairness is based on the MIMIC-III dataset, the standard dataset in clinical NLP research. |
Topic Model or Topic Twaddle? Re-evaluating Semantic Interpretability Measures (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for topic model evaluation use automated measures modeled on human evaluation tests that are dissimilar to applied usage. |
| Approach: | They propose to use a novel experimental framework to evaluate topic models and assess their coherence for specialized collections in an applied setting. |
| Outcome: | The proposed framework is reflective of human evaluations using open labeling, typical of applied research. |
Discourse Probing of Pretrained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on probing of pretrained language models has focused on sentence-level syntactic tasks. |
| Approach: | They introduce document-level discourse probing to evaluate the ability of pretrained LMs to capture document- level relations. |
| Outcome: | The proposed model performs best in encoder, but only in the encoder layer. |
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the performance of natural language processing models are over-parameterized and overfitted. |
| Approach: | They propose an approach to integrate dropout techniques into the training of Transformer models. |
| Outcome: | The proposed approach can achieve 1.5 BLEU improvement on IWSLT14 translation tasks and better accuracy for the classification even using strong pre-trained RoBERTa as backbone. |
tWT–WT: A Dataset to Assert the Role of Target Entities for Detecting Stance of Tweets (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing stance detection datasets are complex deep neural networks, making them difficult to interpret. |
| Approach: | They propose a new large dataset free of such biases and demonstrate its aptness on existing stance detection systems. |
| Outcome: | The proposed model achieves human-level performance on the WT–WT dataset and more than two-third accuracy on other datasets. |
Learning to Learn to be Right for the Right Reasons (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work shows that models trained on held-out data perform poorly on hard instances . previous methods have resorted to manual methods of encouraging models not to overfit to superficial cues . |
| Approach: | They propose to explicitly learn a model that does well on both easy and hard tests . they use Choice of Plausible Alternatives and Commonsense Explanation to evaluate the model . |
| Outcome: | The proposed model performs well on easy and hard tests with superficial cues but performs poorly on hard ones without superficial cuings. |
Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that NLP models are vulnerable to adversarial perturbations such as synonym substitutions or syntax-guided paraphrasing. |
| Approach: | They propose a “double perturbation” framework to uncover model weaknesses beyond the test dataset. |
| Outcome: | The proposed attack achieves high success rates on both original and robustly trained CNNs and Transformers. |
Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to explain neural network models are computationally inefficient for text inputs. |
| Approach: | They propose a method to implicitly detect word correlations by grouping correlated words from input text pairs together and measuring their contribution to corresponding NLP tasks. |
| Outcome: | The proposed method is evaluated with two different model architectures across four datasets. |
Almost Free Semantic Draft for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Empirical experiments show that the presented method can achieve competitive performance in common language pairs with a clear advantage in inference efficiency. |
| Approach: | They propose a method to sample and consider a semantic draft as global information from semantic space for decoding with almost free of cost. |
| Outcome: | Empirical results show that the proposed method can achieve competitive performance in common language pairs with a clear advantage in inference efficiency. |
Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for domain adaptation suffer from catastrophic forgetting, large domain divergence, and model explosion. |
| Approach: | They propose a method which prunes the model and keeps the important neurons or parameters responsible for both general-domain and in-domain translation. |
| Outcome: | The proposed method improves on different language pairs and domains compared with strong baselines. |
Multi-Hop Transformer for Document-Level Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to document-level neural machine translation (NMT) simply introduce the representations of context sentences without explicitly characterizing the inter-sentence reasoning process. |
| Approach: | They propose a novel multi-hop Transformer which explicitly models the human-like draft-editing and reasoning process by attending to multiple antecedent sentences iteratively. |
| Outcome: | Experiments on four widely used document translation tasks show that the proposed model significantly improves document-level translation performance and tackles discourse phenomena such as coreference error and the problem of polysemy. |
Continual Learning for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models are data-driven and require large-scale training corpus . continual learning remains a big challenge for artificial intelligence systems and models . |
| Approach: | They propose a continual learning framework for NMT models that incorporates multiple stages of training to alleviate catastrophic forgetting problem. |
| Outcome: | The proposed framework achieves superior performance compared to baseline models in all settings. |
Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods that use monolingual corpora for translation are not suitable for low-resource languages such as Estonian. |
| Approach: | They propose unsupervised neural machine translation (UNMT) that relies on monolingual corpora to train a robust UNMT system and improve its performance. |
| Outcome: | The proposed methods outperform conventional UNMT systems on several language pairs. |
Smart-Start Decoding for Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing neural machine translation models adopt a monotonic decoding order of either left-to-right or right-to left. |
| Approach: | They propose a method that starts decoding target words from the right side of a median word and generates words on the left. |
| Outcome: | The proposed method outperforms baseline models on three datasets. |
Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive machine translation models have shown significant inference speedup but suffer from inferior translation accuracy. |
| Approach: | They propose to use AT as an auxiliary task to transfer AT knowledge to NAT models by knowledge distillation. |
| Outcome: | The proposed method achieves significant improvements over baseline non-Autoregressive machine translation models on WMT14 En-De and WMT16 En-Ro datasets. |
ER-AE: Differentially Private Text Generation for Authorship Anonymization (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies on privacy protection for textual data focus on removing explicit sensitive identifiers without considering the author's writing style. |
| Approach: | They propose a text generation model with an exponential mechanism for authorship anonymization that augments the semantic information through a REINFORCE training reward function. |
| Outcome: | The proposed model outperforms the state-of-the-art on semantic preservation, authorship obfuscation, and stylometric transformation on the real-life peer reviews and Yelp review datasets. |
Distantly Supervised Transformers For E-Commerce Product QA (2021.naacl-main)
Copied to clipboard
| Challenge: | e-commerce services often provide an instant QA system on product pages . however, user queries and CQA pairs differ significantly in language characteristics . |
| Approach: | They propose a transformer-based instant question answering system on product pages . for each user query, relevant community question answer (CQA) pairs are retrieved . their framework is able to scale to large e-commerce QA traffic . |
| Outcome: | The proposed model outperforms syntactic and semantic baselines on user queries and training with CQA pairs. |
Quantitative Day Trading from Natural Language using Reinforcement Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to stock prediction are not optimized to make profitable investment decisions. |
| Approach: | They propose a deep reinforcement learning approach that makes time-aware decisions to trade stocks while optimizing profit using textual data. |
| Outcome: | The proposed method outperforms state-of-the-art in terms of risk-adjusted returns on two benchmarks: Tweets (English) and financial news (Chinese) |
Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | voluminous historical records are difficult to fully utilize since they are written in ancient languages and some parts are damaged over time. |
| Approach: | They propose a multi-task learning approach to restore and translate historical documents using a self-attention mechanism. |
| Outcome: | The proposed approach improves the accuracy of the translation task over baselines without multi-task learning. |
Modeling Diagnostic Label Correlation for Automatic ICD Coding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work built a binary prediction for each label independently, ignoring the dependencies between labels. |
| Approach: | They propose a framework to capture the label correlation and train a reranking estimator to rescore the probability of each label set candidate generated by a base predictor. |
| Outcome: | The proposed framework improves on the best-performing predictors on MIMIC datasets. |
Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents (2021.naacl-main)
Copied to clipboard
| Challenge: | End-to-end deep learning methods that focus on user satisfaction are challenging due to the required annotation costs and turnaround times. |
| Approach: | They propose a self-supervised contrastive learning approach that leverages the pool of unlabeled data to learn user-agent interactions. |
| Outcome: | The proposed approach reduces the required number of annotations while improving generalization on unseen skills. |
A recipe for annotating grounded clarifications (2021.naacl-main)
Copied to clipboard
| Challenge: | In order to interpret communicative intents of an utterance, it needs to be grounded in world modalities. |
| Approach: | They propose a recipe for obtaining grounding annotations for dialogue clarification mechanisms that make explicit the process of interpreting communicative intents of an utterance. |
| Outcome: | The proposed method is based on the definitions of perceptual and collaborative grounding and on the classification of clarification phenomena. |
Grey-box Adversarial Attack And Defence For Sentiment Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks have created applications for a range of different domains. |
| Approach: | They propose a grey-box adversarial attack and defence framework for sentiment classification . they show that the framework produces an improved classifier that is robust in defending . |
| Outcome: | The proposed framework produces an improved classifier that is robust in defending against multiple adversarial attacking methods. |
How low is too low? A monolingual take on lemmatisation in Indian languages (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior work on ML based lemmatization focused on high resource languages, where data sets (word forms) are readily available. |
| Approach: | They propose to use neural methods to relate inflected forms of words to their dictionary form to reduce the sparse data problem. |
| Outcome: | The proposed methods can give competitive accuracy even in low resource setting. |
Causal Effects of Linguistic Properties (2021.naacl-main)
Copied to clipboard
| Challenge: | Social scientists have long been interested in the causal effects of language, studying questions like: How should political candidates describe their personal history to appeal to voters? |
| Approach: | They propose an algorithm for estimating causal effects of linguistic properties that leverages distant supervision and a pre-trained language model to adjust for the text. |
| Outcome: | The proposed method outperforms other methods when estimating the effect of Amazon review sentiment on semi-simulated sales figures. |
Dynabench: Rethinking Benchmarking in NLP (2021.naacl-main)
Copied to clipboard
Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, Adina Williams
| Challenge: | Dynabench is an open-source platform for dynamic dataset creation and model benchmarking. |
| Approach: | They propose an open-source platform for dynamic dataset creation and model benchmarking. |
| Outcome: | The proposed platform can be used to create models that fail on simple challenges and falter in real-world scenarios. |
Translational NLP: A New Paradigm and General Principles for Natural Language Processing Research (2021.naacl-main)
Copied to clipboard
| Challenge: | Natural language processing research is often assumed to emerge naturally . many innovations go unapplied and important questions remain unstudied . |
| Approach: | They propose a new paradigm to structure and facilitate the processes by which basic and applied NLP research inform one another. |
| Outcome: | The proposed framework provides a roadmap for developing Translational NLP as a dedicated research area. |
Predicting Discourse Trees from Transformer-based Neural Summarizers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing extractive summarization tasks use only neural approaches to learn discourse information, but recent work has shown that it is beneficial for summarizing discourse information. |
| Approach: | They propose to generate document-level discourse trees from pre-trained neural summarizers that encode dependency- and constituency-style discourse information. |
| Outcome: | The proposed model learns both, dependency- and constituency-style discourse information, consistent with pre-neural results. |
Probing for Bridging Inference in Transformer Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained transformer language models are capable of bridging inference, but they lack the commonsense knowledge to capture syntactic information. |
| Approach: | They investigate whether pre-trained transformer language models capture bridging inference . they use a masked token prediction task to investigate attention heads in BERT . |
| Outcome: | The proposed model significantly captures bridging inference, the authors show . the distance between anaphor-antecedent and context plays an important role in the inference . |
Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | a common approach to coherence evaluation is shuffling the sentence order of a text, creating incoherent text samples that need to be discriminated from the original. |
| Approach: | They propose an extendable set of test suites addressing different aspects of discourse and dialogue coherence. |
| Outcome: | The proposed evaluation paradigm is suited to evaluate linguistic qualities that contribute to the notion of coherence. |
Stay Together: A System for Single and Split-antecedent Anaphora Resolution (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on single-antecedent anaphora has greatly improved . attention has now turned to more complex cases of anaphorisms such as split-antevore anaprs . |
| Approach: | They propose a system that resolves both single and split-antecedent anaphors and evaluates it in a more realistic setting that uses predicted mentions. |
| Outcome: | The proposed system resolves both single and split-antecedent anaphora and evaluates it in a more realistic setting that uses predicted mentions. |
Redefining Absent Keyphrases and their Effect on Retrieval Effectiveness (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural keyphrase generation models can output absent keyphrases, which are keyphrase that do not appear in the source text. |
| Approach: | They propose a finer-grained categorization scheme that sheds more light on the impact of absent keyphrases on scientific document retrieval. |
| Outcome: | The proposed model shows that only 20% of the words that make up keyphrases actually serve as document expansion, but this small fraction behind much of the gains observed in retrieval effectiveness. |
CoRT: Complementary Rankings from Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent approaches to information retrieval mitigate computational costs by using a multi-stage ranking pipeline. |
| Approach: | They propose a ranking model that leverages contextual representations from pre-trained language models to complement term-based ranking functions while causing no significant delay at query time. |
| Outcome: | The proposed model significantly increases candidate recall by complementing BM25 with missing candidates while causing no significant delay at query time. |
Multi-source Neural Topic Modeling in Multi-view Embedding Spaces (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has used pre-trained word embeddings to address data sparsity in short-text or small document collections. |
| Approach: | They propose a neural topic modeling framework using multi-view embedding spaces to improve topic quality and deal with polysemy. |
| Outcome: | The proposed framework improves topic quality and deal with polysemy. |
Inductive Topic Variational Graph Auto-Encoder for Text Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for text classification do not assume explicit latent semantic structure of documents, making them less effective and difficult to interpret. |
| Approach: | They propose a model that integrates a topic model into variational graph-auto-encoder to capture hidden semantic information between documents and words. |
| Outcome: | The proposed model outperforms existing models on supervised and semi-supervised text classification and unsupervised representation learning. |
Self-Alignment Pretraining for Biomedical Entity Representations (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to self-supervised learning of biomedical entities are limited in the biomedic domain. |
| Approach: | They propose a pretraining scheme that self-aligns the representation space of biomedical entities. |
| Outcome: | The proposed framework achieves state-of-the-art on six MEL benchmarking datasets. |
TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names (2021.naacl-main)
Copied to clipboard
| Challenge: | Hierarchical multi-label text classification (HMTC) aims to assign each text document to a set of relevant classes from a taxonomy. |
| Approach: | They propose to conduct HMTC based on only class surface names as supervision signals to mimic human experts. |
| Outcome: | The proposed framework outperforms the best existing method by 25% on two challenging datasets. |
MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding (2021.naacl-main)
Copied to clipboard
| Challenge: | a new method for generating metaphors is proposed to generate literal sentences . human evaluations show that our best model generates metaphors better than three well-crafted baselines 66% of the time on average. |
| Approach: | They propose a method to automatically construct a parallel corpus by transforming literal sentences to metaphorical ones using commonsense inference and masked language modeling. |
| Outcome: | The proposed method generates metaphors better than baselines 66% of the time on average. |
On Learning Text Style Transfer with Direct Rewards (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for text style transfer lack parallel corpora, which makes it impossible to train supervised models. |
| Approach: | They propose to use semantic similarity metrics to explicitly assess the preservation of content between system outputs and inputs. |
| Outcome: | The proposed methods provide significant gains in automatic and human evaluation over strong baselines. |
Focused Attention Improves Document-Grounded Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document grounded generation is the task of using the information provided in a document to improve text generation. |
| Approach: | They propose two new document grounded generation tasks that use information provided in a document to improve text generation. |
| Outcome: | The proposed models outperform existing methods on automated and human evaluation for closeness to reference and relevance to the document. |
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints (2021.naacl-main)
Copied to clipboard
| Challenge: | Conditional text generation often requires lexical constraints, i.e., which words should or shouldn't be included in the output text. |
| Approach: | They propose an algorithm that enables neural language models to generate fluent text while satisfying complex lexical constraints. |
| Outcome: | The proposed algorithm outperforms existing methods on four benchmarks and shows that it handles any set of lexical constraints expressible under predicate logic while its asymptotic runtime is equivalent to conventional beam search. |
Ask what’s missing and what’s useful: Improving Clarification Question Generation using Global Knowledge (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models that generate clarification questions fail to identify useful information in contexts . human ability to generate fluent and relevant questions is important in reducing ambiguity . |
| Approach: | They propose a model that first identifies what is missing and then generates a question about it. |
| Outcome: | The proposed model outperforms baselines as judged by automatic metrics and humans. |
Progressive Generation of Long Text with Pretrained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for "long" text generation are limited to outputs of 50-200 tokens . however, our proposed ProGen generates coherent long passages of text in a progressive manner . |
| Approach: | They propose a method for generating coherent long passages of text in a progressive manner . they first produce domain-specific content keywords and then refine them into complete passages . human evaluation validates that their proposed generation is more coherent . |
| Outcome: | The proposed method produces domain-specific content keywords and refines them into complete passages in multiple stages. |
SOCCER: An Information-Sparse Discourse State Tracking Collection in the Sports Commentary Domain (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for state tracking are limited and state changes are less densely distributed over utterances. |
| Approach: | They propose to turn to simplified, fully observable systems that show some of these properties. |
| Outcome: | The proposed system shows that state changes occur infrequently while messages are "chatter" it allows for rich descriptions of state while avoiding the complexities of other settings. |
Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to generate implausible stories using plots are unnatural and oversimplify the characteristics of implusible machine-generated stories. |
| Approach: | They propose to generate a more comprehensive set of implausible stories using plots . plots are structured representations of controllable factors used to generate stories . |
| Outcome: | The proposed model improves the quality of generated implausible stories using plots . it shows that the evaluation metrics trained on the generated data correlate better with human judgments compared to baselines. |
MultiOpEd: A Corpus of Multi-Perspective News Editorials (2021.naacl-main)
Copied to clipboard
| Challenge: | a news editorial is a genre of persuasive text where argumentation structure is usually implicit. |
| Approach: | They propose an open-domain news editorial corpus that supports automatic perspective discovery by identifying and abstracting natural language perspectives from editorials. |
| Outcome: | The proposed system supports automatic perspective discovery tasks in news editorials. |
Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing benchmarks for lexical substitution depend on human recall as the only source of data, authors say . existing benchmarks lack coverage of the appropriate substitutes that would be most helpful to humans . |
| Approach: | They propose a benchmark for lexical substitution to find appropriate substitutes for a target word in context . existing benchmarks depend on human recall as the only source of data, they argue . |
| Outcome: | The new benchmark for lexical substitution uses a context-free thesaurus . it has 3x as many substitutes per target word for the same quality, and substitutes are 1.4x more appropriate . |
“I’m Not Mad”: Commonsense Implications of Negation and Contradiction (2021.naacl-main)
Copied to clipboard
| Challenge: | a new commonsense knowledge graph for negated and contradicted events is developed to help humans reason about their underlying causes and effects. |
| Approach: | They propose a new commonsense knowledge graph with 624K if-then rules focusing on negated and contradictory events. |
| Outcome: | The proposed model can be used to analyze negated and contradicted statements in natural language. |
Identifying Medical Self-Disclosure in Online Communities (2021.naacl-main)
Copied to clipboard
| Challenge: | a new dataset of health-related posts from online social platforms is available for analysis . medical self-disclosure may be useful for early detection and treatment of medical issues . |
| Approach: | They propose to analyze medical self-disclosure in online health conversations . they release a dataset of health-related posts from online social platforms with high inter-annotator agreement . |
| Outcome: | The proposed model achieves an accuracy of 81.02% and sets a strong performance benchmark. |
Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction (2021.naacl-main)
Copied to clipboard
| Challenge: | Scholarly work in this area uses toy worlds and synthetic linguistic data, but grounded language learning offers several practical and scientific advantages. |
| Approach: | They propose to model teacher-learner dynamics through natural interactions occurring between users and search engines. |
| Outcome: | The proposed model is better than non-grounded models on compositionality and zero-shot inference tasks. |
Finding Concept-specific Biases in Form–Meaning Associations (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to detect cross-linguistic associations are not effective, but their effects are minor. |
| Approach: | They propose a method to measure cross-linguistic associations by controlling for the influence of language family and geographic proximity within a large concept-aligned, cross-lingual lexicon. |
| Outcome: | The proposed method shows that it is small, but it is unsurprisingly small (less than 0.5% on average). |
How (Non-)Optimal is the Lexicon? (2021.naacl-main)
Copied to clipboard
| Challenge: | lexical meanings are mapped to wordforms by usage pressures and constraints on sequences of symbols. |
| Approach: | They propose a coding-theoretic view of the lexicon and a novel generative statistical model to quantify its compressibility under various constraints. |
| Outcome: | The proposed model shows that (compositional) morphology and graphotactics can account for most of the complexity of natural codes—as measured by code length. |
Word Complexity is in the Eye of the Beholder (2021.naacl-main)
Copied to clipboard
| Challenge: | Lexical complexity is a subjective notion, yet it is often neglected in lexical simplification and readability systems which use a ”one-size-fits-all” approach. |
| Approach: | They propose to use a dataset of complex words annotated by readers with different backgrounds to investigate which aspects contribute to the notion of lexical complexity. |
| Outcome: | The proposed approach can be replicated in a dataset of complex words annotated by readers with different backgrounds. |
Linguistic Complexity Loss in Text-Based Therapy (2021.naacl-main)
Copied to clipboard
| Challenge: | linguistic complexity loss in text-based therapy can be used to identify patterns of mental health . authors: clients who reported more anxiety used less lexically diverse language . |
| Approach: | They analyze linguistic complexity loss in online therapy conversations as it relates to mental health . they find that clients used less lexically diverse language when they were more anxious . |
| Outcome: | The proposed analysis shows that therapists use more complex language when clients are anxious . the authors show that analyzing linguistic complexity can identify meaningful patterns in mental health . |
Ab Antiquo: Neural Proto-language Reconstruction (2021.naacl-main)
Copied to clipboard
| Challenge: | Historical linguists have identified regularities in the process of historic sound change. |
| Approach: | They propose a method to reconstruct proto-words based on cognates in daughter languages . they use a dataset of 8,000 comparative entries to analyze phonological changes . |
| Outcome: | The proposed method outperforms conventional methods in a proto-word reconstruction task. |
On Biasing Transformer Attention Towards Monotonicity (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work has focused on learning monotonic attention behavior via specialized attention functions or pretraining. |
| Approach: | They introduce a monotonicity loss function compatible with standard attention mechanisms and test it on sequence-to-sequence tasks. |
| Outcome: | The proposed monotonicity loss function can achieve largely monotonic behavior on grapheme-to-phoneme conversion, morphological inflection, transliteration, and dialect normalization tasks. |
Extracting a Knowledge Base of Mechanisms from COVID-19 Papers (2021.naacl-main)
Copied to clipboard
Tom Hope, Aida Amini, David Wadden, Madeleine van Zuylen, Sravanthi Parasa, Eric Horvitz, Daniel Weld, Roy Schwartz, Hannaneh Hajishirzi
| Challenge: | COVID-19 has spawned a diverse body of scientific literature that is challenging to navigate . researchers are using automated tools to help find useful knowledge . |
| Approach: | They develop a schema to extract mechanism relations from scientific papers . their search engine, dataset and code are publicly available . |
| Outcome: | The proposed schema outperforms PubMed search in clinical trials. |
Constrained Multi-Task Learning for Event Coreference Resolution (2021.naacl-main)
Copied to clipboard
| Challenge: | a neural event coreference model is based on a task of determining whether two event mentions refer to the same event . event coreferent tasks require nontrivial tasks such as identifying potential arguments and linking arguments to their event mention. |
| Approach: | They propose a neural event coreference model in which event coreference is jointly trained with five tasks. |
| Outcome: | The proposed model achieves state-of-the-art on the KBP 2017 event coreference dataset. |
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality (2021.naacl-main)
Copied to clipboard
| Challenge: | In human-level NLP tasks, the number of observations is often smaller than the standard 768+ hidden state sizes of each layer within transformer-based language models. |
| Approach: | They propose to use dimension reduction methods to fine-tune large models with limited data and to use pre-trained dimension reduction regimes to improve model performance. |
| Outcome: | The proposed model outperforms other models in human-level NLP tasks with a pre-trained dimension reduction regime. |
Leveraging Deep Representations of Radiology Reports in Survival Analysis for Predicting Heart Failure Patient Mortality (2021.naacl-main)
Copied to clipboard
| Challenge: | Current models for survival analysis are limited in scope and require a large amount of data and expert annotations for training. |
| Approach: | They propose to use BERT-based hidden layer representations of clinical texts as covariates for proportional hazards models to predict patient survival outcomes. |
| Outcome: | The proposed method outperforms the baseline model by 5.7% across C-index and time-dependent AUC. |
On the Use of Context for Predicting Citation Worthiness of Sentences in Scholarly Articles (2021.naacl-main)
Copied to clipboard
| Challenge: | citation worthiness is an emerging research topic in the natural language processing domain . citation recommendation systems are often approached as ranking problems . |
| Approach: | They propose a hierarchical biLSTM-based model that uses two adjacent sentences to solve a citation worthiness problem. |
| Outcome: | The proposed approach can be applied to a dataset of over two million sentences and their labels. |
Data and Model Distillation as a Solution for Domain-transferable Fact Verification (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural networks depend heavily on lexicalized information, which transfers poorly between domains. |
| Approach: | They propose a method to delexicize lexicalized data and a model distillation technique to prevent aggressive data distillation. |
| Outcome: | The proposed method improves performance on lexicalized data and out of domain models. |
Adapting Coreference Resolution for Processing Violent Death Narratives (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing coreference models suffer from poortransferability due to domain gaps . existing models are not robust enough to handle text data about LGBT individuals . |
| Approach: | They propose to use a dataaugmentation rule to improve coreference resolution in an administrative database written in English to better handle LGBT data. |
| Outcome: | The proposed model improves perfor-mance and accuracy of coreference resolution in a violent death nar-rative from the Centers for Disease Control's (CDC) national Violent Death Re-porting System. |
Time-Stamped Language Model: Teaching Language Models to Understand The Flow of Events (2021.naacl-main)
Copied to clipboard
| Challenge: | Using transformer-based language models to track entities is challenging due to dynamic nature of the world described in the text. |
| Approach: | They propose to use transformer-based language models to track entities throughout a procedure . they propose to introduce timestamp encoding to encode event information in LMs . |
| Outcome: | The proposed model improves on the state-of-the-art model with a 3.1% increase in F1 score on the Propara dataset and better results on the location prediction task on the NPN-Cooking dataset. |
If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | et al. : evidence retrieval is highly dependent on partial, incorrect or no supporting knowledge. |
| Approach: | They propose a method that retrieves and reranks evidence facts jointly . they propose to account for links between sentences and coverage with the given query . |
| Outcome: | The proposed approach achieves state-of-the-art evidence retrieval performance on two multi-hop question answering datasets. |
SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on the spatial reasoning capabilities of modern language models (LMs) however, there has been limited research into the spatial thinking capabilities of LMs. |
| Approach: | They propose a question-answering (QA) benchmark for spatial reasoning on natural language text which contains more realistic spatial phenomena not covered by prior work. |
| Outcome: | The proposed method significantly improves LMs' ability on spatial understanding, which in turn helps solve two external datasets, bAbI, and boolQ. |
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing information-seeking question answering datasets do not perform well on answering these questions . existing models that do well on other QA tasks do not do well answering these tasks . |
| Approach: | They present a dataset of 5049 questions over 1585 NLP papers . they use a question-seeking QA model that seeks information in the full text . |
| Outcome: | The proposed dataset underperforms existing models on other QA tasks by 27 F1 points . the focus is on document-grounded, information-seeking QA . |
Differentiable Open-Ended Commonsense Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing commonsense reasoning models work by scoring a question-candidate pair, but new approaches are needed to answer multiple-choice questions. |
| Approach: | They propose to use a corpus of commonsense facts to answer a commonsensical question without any pre-defined choices as a resource. |
| Outcome: | The proposed model outperforms baseline methods by a large margin in the open-ended commonsense reasoning task. |
Does Structure Matter? Encoding Documents for Machine Reading Comprehension (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing Transformer-based models for machine reading comprehension treat documents as flat sequences. |
| Approach: | They propose a Transformer-based method that reads a document as tree slices and jointly trains and consults the modules at inference time. |
| Outcome: | The proposed method outperforms several baseline approaches on two datasets from varied domains. |
Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | Current methods for complex question answering use structured knowledge and unstructured text. |
| Approach: | They propose a multi-step retrieval approach that iteratively forms an evidence chain through beam search in dense representations. |
| Outcome: | The proposed method is competitive to state-of-the-art systems without using semi-structured information. |
Scalable and Interpretable Semantic Change Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Several cluster-based methods for word usage change detection are unscalable in terms of memory consumption and computation time. |
| Approach: | They propose a scalable method for word usage-change detection that uses contextual embeddings to aggregate word usages into clusters. |
| Outcome: | The proposed method offers high performance and interpretability while being unscalable. |
Scalar Adjective Identification and Multilingual Ranking (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on scalar adjective ranking have focused on English due to the availability of datasets for evaluation. |
| Approach: | They propose a binary classification task to examine the models’ ability to distinguish scalar from relational adjectives in English. |
| Outcome: | The proposed task compares the models' ability to distinguish scalar from relational adjectives in English using monolingual and multilingual models. |
ESC: Redesigning WSD with Extractive Sense Comprehension (2021.naacl-main)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is a historical NLP task aimed at linking words in contexts to discrete sense inventories. |
| Approach: | They propose a transformer-based neural architecture for extractive Sense Comprehension to solve a span extraction problem and a new state of the art English WSD task. |
| Outcome: | The proposed model outdoes all of its competitors while relying on three times fewer annotations. |
Recent advances in neural metaphor processing: A linguistic, cognitive and social perspective (2021.naacl-main)
Copied to clipboard
| Challenge: | Metaphor processing systems have benefited from recent studies on the role of metaphor in communication and deep learning for natural language processing. |
| Approach: | They present a review of automated metaphor processing and discuss their results from downstream NLP tasks. |
| Outcome: | The proposed system is based on the findings of a systematic and comprehensive survey of metaphor processing systems published five years ago. |
Constructing Taxonomies from Pretrained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | A variety of NLP tasks use taxonomic information, including question answering and information retrieval. |
| Approach: | They propose a method for constructing taxonomic trees using pretrained language models by incorporating web-retrieved glosses into the model. |
| Outcome: | The proposed model achieves 66.7 ancestor F1, a 20.0% relative increase over the previous best published model on English WordNet. |
Event Representation with Sequential, Semi-Supervised Discrete Variables (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for event modeling take discrete, external knowledge into account . obtaining fully accurate structured knowledge can be difficult . |
| Approach: | They propose a method that takes partially-observed sequences of discrete, external knowledge into account. |
| Outcome: | The proposed method outperforms baselines and state-of-the-art in script induction and converges faster. |
Seq2Emo: A Sequence to Multi-Label Emotion Classification Model (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for multi-label emotion classification are based on binary relevance and classifier chain (CC) |
| Approach: | They propose a sequence-to-emotion approach which implicitly models emotion correlations in a bi-directional decoder. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on a SemEval’18 and GoEmotions dataset. |
Knowledge Enhanced Masked Language Model for Stance Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Detecting stance on Twitter is difficult because of the short length of each tweet . Twitter content is dynamic, constantly coining new terminology and hashtags . |
| Approach: | They propose a BERT-based fine-tuning method that enhances stance detection models . they use weighted log-odds-ratio to identify words with high stance distinguishability . |
| Outcome: | The proposed method outperforms the state-of-the-art for stance detection on Twitter data about the 2020 US presidential election. |
Learning Paralinguistic Features from Audiobooks through Style Voice Conversion (2021.naacl-main)
Copied to clipboard
| Challenge: | Paralinguistics, the non-lexical components of speech, play a crucial role in human-human interaction. |
| Approach: | They propose a framework that enables a neural network to learn to extract paralinguistic attributes from speech using data that are not annotated for emotion. |
| Outcome: | The proposed framework improves on emotion recognition and speaking style detection tasks. |
Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on continual learning of a sequence of aspect sentiment classification tasks have not addressed these issues. |
| Approach: | They propose a capsule network based model called B-CL to address these issues . it uses continual learning adapters and capsule networks to encourage knowledge transfer . |
| Outcome: | The proposed model improves the performance on both the new task and the old tasks via forward and backward knowledge transfer. |
Adversarial Learning for Zero-Shot Stance Detection on Social Media (2021.naacl-main)
Copied to clipboard
| Challenge: | a new model for zero-shot stance detection on Twitter uses adversarial learning to generalize across topics . previous work on zero- shot stance detector on English social media focuses on cross-target stances . |
| Approach: | They propose a model that uses adversarial learning to generalize across topics on Twitter . their model achieves state-of-the-art performance on unseen test topics . |
| Outcome: | The proposed model achieves state-of-the-art performance on unseen topics with minimal computational costs. |
Efficiently Summarizing Text and Graph Encodings of Multi-Document Clusters (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive multi-document summarization (MDS) is a task that has seen advances with the introduction of large-scale datasets and powerful Transformer-based models. |
| Approach: | They propose an efficient graph-enhanced approach to multi-document summarization with an encoder-decoder Transformer model. |
| Outcome: | The proposed model scales to large input documents and improves on a multi-document dataset. |
Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization (2021.naacl-main)
Copied to clipboard
Yichen Jiang, Asli Celikyilmaz, Paul Smolensky, Paul Soulos, Sudha Rao, Hamid Palangi, Roland Fernandez, Caitlin Smith, Mohit Bansal, Jianfeng Gao
| Challenge: | Abstractive summarization is the task of generating a concise summary of input documents . a middle-aged man and a young girl died after they were unable to avoid the plane . |
| Approach: | They propose a model that enriches the original Transformer with a Tensor Product Representation for abstractive summarization. |
| Outcome: | The proposed model outperforms the Transformer and the original TP-Transformer significantly on several datasets. |
What’s in a Summary? Laying the Groundwork for Advances in Hospital-Course Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to summarize clinical narratives are lacking. |
| Approach: | They propose to generate a paragraph that tells the story of a patient's hospitalization . they analyze a dataset of 109,000 hospitalizations and their corresponding summary proxy . |
| Outcome: | The proposed model is based on a dataset of 109,000 hospitalizations and their corresponding summary proxy. |
Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics (2021.naacl-main)
Copied to clipboard
| Challenge: | Modern summarization models generate fluent but often factually unreliable outputs. |
| Approach: | They propose to use human annotations to identify different categories of factual errors and benchmark factuality metrics to improve summarization evaluation. |
| Outcome: | The proposed method identifies the proportion of different categories of factual errors and benchmarks their human judgements as well as their specific strengths and weaknesses. |
GSum: A General Framework for Guided Neural Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models are flexible, but they can be difficult to control. |
| Approach: | They propose a general and extensible guided summarization framework that takes different kinds of guidance as input and perform experiments across different varieties. |
| Outcome: | The proposed framework can generate more faithful summaries and different types of guidance generate qualitatively different summary. |
What Will it Take to Fix Benchmarking in Natural Language Understanding? (2021.naacl-main)
Copied to clipboard
| Challenge: | Evaluation for many natural language understanding (NLU) tasks is broken due to unreliable and biased systems scoring so high on standard benchmarks. |
| Approach: | They argue that current benchmarks fail at four criteria for evaluation . they argue that adversarial data collection does not address the causes of failures . |
| Outcome: | The proposed frameworks fail at four criteria, and adversarial data collection does not address the causes of these failures, the authors argue . restoring a healthy evaluation ecosystem will require significant progress in the design of benchmark datasets, reliability with which they are annotated, their size, and the ways they handle social bias. |
TuringAdvice: A Generative and Dynamic Evaluation of Language Use (2021.naacl-main)
Copied to clipboard
| Challenge: | Empirical results show that today’s language models struggle at TuringAdvice . language models are getting ever-larger, and are being trained on ever-increasing quantities of text . |
| Approach: | They propose a task task that requires models to generate helpful advice in natural language. |
| Outcome: | The proposed model outperforms even multibillion parameter models on 600k in-domain training examples. |
Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior work on identifying narratives related to sexual abuse disclosures did not consider this as an independent task. |
| Approach: | They propose to identify narratives related to sexual abuse disclosures as a joint modeling task that leverages their emotional attributes through multitask learning. |
| Outcome: | The proposed model leverages emotional attributes of textual conversations to identify narratives related to sexual abuse disclosures in homogeneous and heterogeneously settings. |
Self Promotion in US Congressional Tweets (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior studies have found that women self-promote less than men due to gender stereotypes. |
| Approach: | They built a BERT-based NLP model to predict whether a Congressional tweet shows self-promotion and then used it to examine whether he gender gap exists among Congressional Tweets. |
| Outcome: | The model predicts whether a Congressional tweet shows self-promotion and then tests it against 2 million tweets from 2017 to 2021. |
Profiling of Intertextuality in Latin Literature Using Word Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | a new study examines the intertextual relationships between authors in classical Latin literature . a large corpus of lemmatized Latin is used to train word embeddings . |
| Approach: | They propose to train an optimized word2vec model on a large corpus of Latin . they then replicate a previous study of the Roman historian Livy using hand-crafted stylometric features. |
| Outcome: | The proposed model outperforms a widely used lexical search method on Latin epic poetry . it advances the development of core computational resources for a major premodern language . |
Identifying inherent disagreement in natural language inference (2021.naacl-main)
Copied to clipboard
| Challenge: | Natural language inference is the task of determining whether text is entailed, contradicted or unrelated to another piece of text. |
| Approach: | They propose to tease systematic inferences from disagreement items by capturing modes in annotations to simulate uncertainty in the annotation process. |
| Outcome: | The proposed approach performs statistically better than baselines on the CommitmentBank corpus in English. |
Modeling Human Mental States with an Entity-based Narrative Graph (2021.naacl-main)
Copied to clipboard
| Challenge: | Understanding narrative text requires capturing characters’ motivations, goals, and mental states. |
| Approach: | They propose an Entity-based Narrative Graph (ENG) to model the internal-states of characters in a story and evaluate it on two narrative understanding tasks. |
| Outcome: | The proposed model is based on two narrative understanding tasks: predicting character mental states, and desire fulfillment. |
A Simple and Efficient Multi-Task Learning Approach for Conditioned Dialogue Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have focused on conditioned dialogue generation, but there is a scarcity of labeled responses. |
| Approach: | They propose a multi-task learning approach to leverage labeled dialogue and text data to generate conditioned dialogues. |
| Outcome: | The proposed approach outperforms the state-of-the-art models by leveraging the labeled texts and obtains larger improvement compared to the previous methods to leverage text data. |
Hurdles to Progress in Long-form Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Long-form question answering (LFQA) involves retrieving documents relevant to a given question and using them to generate a paragraph-length answer. |
| Approach: | They propose a long-form question answering system that relies on sparse attention and contrastive retriever learning to achieve state-of-the-art performance on the ELI5 LFQA dataset. |
| Outcome: | The proposed system tops the public leaderboard on the ELI5 LFQA dataset, but it has several troubling issues. |
ENTRUST: Argument Reframing with Language Models and Entailment (2021.naacl-main)
Copied to clipboard
| Challenge: | Public opinion has been shown to be significantly influenced by framing effects. |
| Approach: | They propose a method for reframing arguments that combines controllable text generation with a post-decoding entailment component to achieve the same denotation. |
| Outcome: | The proposed method is effective compared to baselines along the dimensions of fluency, meaning, and trustworthiness/reduction of fear. |
Paragraph-level Simplification of Medical Texts (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for simplification of medical texts are limited due to jargon and technical content. |
| Approach: | They propose to automate the simplification of medical texts by penalizing decoders for producing "jargon" terms. |
| Outcome: | The proposed method improves on existing heuristics by penalizing the decoder for producing "jargon" terms. |
An Empirical Study on Neural Keyphrase Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent years have seen a flourishing of neural keyphrase generation (KPG) works, including the release of several large-scale datasets and a host of new models to tackle them. |
| Approach: | They propose to compare the generalizability of KPG models with other models by analyzing the most crucial factors that may affect their generalizarability. |
| Outcome: | The proposed model can be used to predict keyphrases from a set of input sequences, and it can be compared with existing models. |
Attention Head Masking for Inference Time Content Selection in Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies show that multi-heads attentions at the same layer collectively guide the summarization. |
| Approach: | They propose an inference-time attention head masking mechanism that works on encoder-decoder attentions to pinpoint salient content at inference time. |
| Outcome: | The proposed technique outperforms state-of-the-art models on CNN/DailyMail and New York Times datasets and is data-efficient. |
Factual Probing Is [MASK]: Learning vs. Learning to Recall (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for factual probing can interpret the model’s prediction accuracy as a lower bound on the amount of factual information it encodes. |
| Approach: | They propose a method which directly optimizes in continuous embedding space and can predict an additional 6.4% of facts in the LAMA benchmark. |
| Outcome: | The proposed method outperforms the best previous prompt method by 6.4% on the LAMA benchmark. |
Evaluating Saliency Methods for Neural Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | a general complaint of neural network models is that their internal decision mechanisms are hard to understand. |
| Approach: | They evaluate the quality of prediction interpretations from two perspectives: plausibility and faithfulness. |
| Outcome: | The evaluation of saliency methods on neural language models shows they can be trusted . the methods can be used to interpret the same prediction, but they disagree on interpretations . |
Contextualized Perturbation for Textual Adversarial Attack (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing techniques for generating adversarial examples are driven by local heuristic rules that are agnostic to the context, resulting in unnatural and ungrammatical outputs. |
| Approach: | They propose a ContextuaLized AdversaRial Example generation model that generates fluent and grammatical outputs through a mask-then-infill procedure. |
| Outcome: | The proposed model outperforms baseline models in terms of attack success rate, textual similarity, fluency and grammaticality. |
DirectProbe: Studying Representations without Classifiers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches for probing opaque representations often use training classifiers and use the accuracy, mutual information, or complexity as a proxy for the representation’s goodness. |
| Approach: | They propose a heuristic that directly studies the geometry of a representation by building upon the notion of 'version space' they argue that doing so can be unreliable because different representations may need different classifiers . |
| Outcome: | Experiments with linguistic tasks and contextualized embeddings show that even without training classifiers, DirectProbe can shine lights on how an embeddable space represents labels and anticipate the classifier performance for the representation. |
Evaluating the Values of Sources in Transfer Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Transfer learning is a form of learning that adapts a model trained on data-rich sources to low-resource targets. |
| Approach: | They propose a source valuation framework that quantifies the usefulness of the sources in transfer learning by using the Shapley value method. |
| Outcome: | The proposed framework is effective in choosing useful transfer sources and the source values match the intuitive source-target similarity. |
Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have shown that word embeddings do not occupy a narrow cone, but rather drift in common directions. |
| Approach: | They show that anisotropy can be restored using a simple transformation of word embeddings. |
| Outcome: | The proposed model can restore anisotropy using a simple transformation. |
On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies suggest that pretrained language models acquire useful inductive biases through masks that implicitly act as cloze reductions for downstream tasks. |
| Approach: | They construct cloze-like masks using task-specific lexicons to explain their results . they show that the majority of performance gains come from generic masks that are not associated with the lexical . |
| Outcome: | The proposed method outperforms a classic method for unsupervised parsing. |
Limitations of Autoregressive Models and Their Alternatives (2021.naacl-main)
Copied to clipboard
| Challenge: | Standard autoregressive language models only perform polynomial-time computation to compute probability of next symbol. |
| Approach: | authors propose alternative to standard autoregressive language models that use polynomial-time computation to compute probability of next symbol. |
| Outcome: | a large model size can grow superpolynomially in length, allowing it to store precomputed results and verify solutions. |
On the Transformer Growth for Progressive BERT Training (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods only conduct network growth in a single dimension, but compound growth operators are beneficial for multiple dimensions. |
| Approach: | They propose a method to train BERT progressively using a Transformer model and explore alternative growth operators in each dimension via controlled comparison. |
| Outcome: | The proposed method speeds up BERT pre-training by 73.6% and 82.2% for the base and large models respectively while achieving comparable performances. |
Revisiting Simple Neural Probabilistic Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in language modeling have been driven not only by advances in neural architectures, but also through hardware and optimization improvements. |
| Approach: | They revisit the neural probabilistic language model (NPLM) of Bengio et al. (2003) which simply concatenates word embeddings within a fixed window and passes the result through a feed-forward network to predict the next word. |
| Outcome: | The proposed model performs better on word-level language model benchmarks than a baseline Transformer with short input contexts but struggles to handle long-term dependencies. |
ReadTwice: Reading Very Large Documents with Memories (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to model long-range dependencies in text are limited to 512 tokens . however, the amount of compute in attention depends quadratically on the number of tokens in an input text passage. |
| Approach: | They propose a technique that summarises text into a memory table to be used in a second read of the text. |
| Outcome: | The proposed method outperforms models of comparable size on several question answering datasets and sets a new state of the art on the NarrativeQA task, with questions about entire books. |
SCRIPT: Self-Critic PreTraining of Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for representation learning of text are masked language modeling (MLM) a language model is trained to learn universal contextual embeddings, which are fine-tuned on a down-stream task. |
| Approach: | They propose a self-critic pretraining transformer for representation learning of text . they demonstrate improved sample-efficiency and improved performance over strong baselines . |
| Outcome: | The proposed model improves sample-efficiency and performance over strong baselines. |
Learning How to Ask: Querying LMs with Mixtures of Soft Prompts (2021.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models retain factual knowledge that can be extracted with a sentential prompt. |
| Approach: | They propose to learn prompts by gradient descent, either fine-tuning prompts or starting from random initialization. |
| Outcome: | The proposed approach outperforms existing methods on English LMs and tasks. |
Nutri-bullets Hybrid: Consensual Multi-document Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for generating comparative summaries that highlight similarities and contradictions in input documents are lacking large parallel training data for their training. |
| Approach: | They propose a method for generating comparative summaries that highlight similarities and contradictions in input documents by using a neural interpretation of traditional concept-to-text generation systems. |
| Outcome: | The proposed model is compared with conventional methods in the domain of nutrition and health, where the existing models lack large parallel training data. |
AVA: an Automatic eValuation Approach for Question Answering Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | AVA is an automatic evaluation approach for question answering . it uses transformer-based language models to encode question, answer, and reference texts . |
| Approach: | They propose an automatic evaluation approach for Question Answering that uses Transformer-based language models to encode question, answer, and reference texts. |
| Outcome: | AVA can estimate system Accuracy with an error lower than 7% at 95% confidence level . the proposed approach achieves 74.7% F1 score in predicting human judgment for single answers . |
SpanPredict: Extraction of Predictive Document Spans with Neural Attention (2021.naacl-main)
Copied to clipboard
| Challenge: | identifying predictive text in clinical notes can be as important as the predictions themselves . identifying specific content in clinical note descriptions may illuminate previously unknown risk factors . |
| Approach: | They propose a method for identifying predictive text in clinical notes . they use linear attention to formalize the problem as predictive extraction . |
| Outcome: | The proposed model preserves differentiability and allows scalable inference via stochastic gradient descent. |
Text Editing by Command (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has focused on making such models more controllable and factually grounded. |
| Approach: | They propose a novel interactive text generation setting in which the user interacts with the system by issuing commands to edit existing text. |
| Outcome: | The proposed model outperforms baseline models and obtains positive results in automatic and human evaluations. |
A Deep Metric Learning Approach to Account Linking (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to identify abusive content may fail to adapt to new trends, and individual posts may fail . |
| Approach: | They propose a method that embeds variable-sized samples of user activity into a vector space, where samples by the same author map to nearby points. |
| Outcome: | The proposed model outperforms several competitive baselines under a new evaluation framework modeled after established benchmarks in other domains. |
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing report generation systems suffer from incomplete and inconsistent generation, despite achieving high performance on natural language metrics such as CIDEr and BLEU. |
| Approach: | They propose two new rewards that encourage the generation of factually complete and consistent radiology reports by using an existing semantic equivalence metric. |
| Outcome: | The proposed system significantly improves the F1 score of a clinical information extraction performance on two open radiology report datasets. |
Multimodal End-to-End Sparse Model for Emotion Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work in emotion recognition uses a two-phase pipeline, but the extracted features are fixed and cannot be fine-tuned on different tasks. |
| Approach: | They propose a two-phase pipeline for emotion recognition and personality recognition . they propose restructured datasets to enable fully end-to-end training . |
| Outcome: | The proposed model outperforms the current state-of-the-art models on emotion recognition and personality recognition tasks with half less computation in the feature extraction part. |
MIMOQA: Multimodal Input Multimodal Output Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Multimodal research has picked up significantly in the space of question answering with the task being extended to visual question answering, charts question answering as well as multimodal input question answering. |
| Approach: | They propose a multimodal question-answering task that produces a unimodal textual output as the answer through human experiments. |
| Outcome: | The proposed framework outperforms existing frameworks on both automatic and human metrics. |
OCID-Ref: A 3D Robotic Dataset With Embodied Language For Clutter Scene Grounding (2021.naacl-main)
Copied to clipboard
| Challenge: | Visual grounding (VG) is a crucial task in natural language processing, computer vision, and robotics. |
| Approach: | They propose a visual grounding task with referring expressions of occluded objects in a OCID-Ref dataset with 2,300 scenes and a point cloud input. |
| Outcome: | The proposed dataset shows that it can handle 2D and 3D signals but referring to occluded objects remains challenging for the modern visual grounding systems. |
Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models require large amounts of image-caption data for pre-training . existing models require expensive data collection and curation . |
| Approach: | They propose to conduct "mask-and-predict" pre-training on text-only and image-only corpora and introduce the object tags detected by an object recognition model as anchor points to bridge two modalities. |
| Outcome: | The proposed approach achieves performance close to a model pre-trained with aligned data, on four English benchmarks. |
Multitasking Inhibits Semantic Drift (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have found that LLP training is prone to semantic drift (use of messages inconsistent with their natural language meanings) |
| Approach: | They propose to use latent language policies to train neural LLPs to eliminate semantic drift in a well-studied family of signaling games to reduce drift and improve sample efficiency. |
| Outcome: | The proposed model eliminates semantic drift in a well-studied family of signaling games while improving sample efficiency. |
Probing Contextual Language Models for Common Ground with Visual Representations (2021.naacl-main)
Copied to clipboard
| Challenge: | Contextual language models have attracted great interest in probing what is encoded in their representations. |
| Approach: | They propose a probing model that evaluates how effective are text-only representations in distinguishing between matching and non-matching visual representations. |
| Outcome: | The proposed model outperforms text-only language models in instance retrieval, but underperform humans. |
BBAEG: Towards BERT-based Biomedical Adversarial Example Generation for Text Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent efforts to generate adversaries using rule-based synonyms and BERT-MLMs have been witnessed in general domain, but the ever-increasing biomedical literature poses unique challenges. |
| Approach: | They propose a black-box attack algorithm for biomedical text classification that uses rule-based synonyms and BERT-MLMs to generate adversarial examples. |
| Outcome: | The proposed algorithm performs stronger with better language fluency and semantic coherence than previous work. |
Targeted Adversarial Training for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing adversarial training approaches focus on making adversarials less expensive or regularizing rather than replacing the standard training objective. |
| Approach: | They propose an algorithm to introspect current mistakes and prioritize adversarial training steps to where the model errs the most. |
| Outcome: | The proposed algorithm improves adversarial training for natural language understanding by introspecting mistakes and prioritizing training steps to where the model errs the most. |
Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for sarcasm detection are limited due to the difficulty in acquiring ground-truth annotations. |
| Approach: | They propose a generalized latent optimization strategy that allows different losses to accommodate each other and improves training dynamics. |
| Outcome: | The proposed approach outperforms transfer learning and meta-learning baselines and achieves 10.02% performance gain on the iSarcasm dataset. |
Self-training Improves Pre-training for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Veselin Stoyanov, Alexis Conneau
| Challenge: | Unsupervised pretraining has led to improvements in natural language understanding . a data augmentation method can be used to generate labels for unlabeled examples . |
| Approach: | They propose a semi-supervised method which uses unlabeled data to retrieve sentences from a database of billions of unlabed sentences crawled from the web. |
| Outcome: | The proposed method improves on standard text classification benchmarks by 2.6% and knowledge distillation by few shots. |
Supporting Clustering with Contrastive Learning (2021.naacl-main)
Copied to clipboard
Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
| Challenge: | Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space, but different categories overlap with each other at the beginning of the learning process. |
| Approach: | They propose a framework to leverage contrastive learning to promote better separation between different categories by optimizing a clustering objective defined in the representation space. |
| Outcome: | The proposed framework improves state-of-the-art accuracy and normalized mutual information on short text clustering and combines top-down and bottom-up instance discrimination to achieve better distances. |
TITA: A Two-stage Interaction and Topic-Aware Text Matching Model (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for keyword-document matching do not define topic-aware relevance clearly. |
| Approach: | They propose a two-stage interaction and topic-aware text matching model to solve this problem . they propose to combine latent topic of document with deep neural representation to model complex interactions between keyword and document. |
| Outcome: | The proposed model outperforms other well-designed baselines and shows excellent performance in the recommendation system. |
Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing GEC models produce spurious corrections or fail to detect lots of errors. |
| Approach: | They propose a neural network for GEC quality estimation with multiple hypotheses . VERNet establishes interactions among hypothese based on reasoning graph . |
| Outcome: | The proposed model achieves state-of-the-art grammatical error detection performance and best quality estimation results on four GEC datasets. |
Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing neural network tuning methods cause instance-wise side effects . et al., 2018: a new approach to perform neural network surgery . |
| Approach: | They propose to perform neural network surgery by only changing 10-5 parameters . they propose to use a dynamic selecting method to achieve the best overall performance . |
| Outcome: | The proposed method achieves the best overall performance and induces fewer instance-wise side effects by changing only 10-5 of the parameters. |
Discrete Argument Representation Learning for Interactive Argument Pair Identification (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing research on monological argumentation covers claims generation, argument structure prediction, and essay scoring. |
| Approach: | They propose to identify argument pairs from two posts with opposite stances to a certain topic. |
| Outcome: | The proposed framework outperforms competing models on a large-scale dataset . it also proves that it is useful for analyzing argument pairs from two posts . |
On Unifying Misinformation Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | On any given day, 2.5 quintillion bytes of information are created on the Internet, a figure that is only expected to increase in the coming years. |
| Approach: | They propose a general-purpose misinformation model that jointly models multiple domains of misinformation with a single, unified setup. |
| Outcome: | The proposed model is useful for few-shot learning of unseen misinformation tasks/datasets and generalizability to unseense events. |
Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Model (2021.naacl-main)
Copied to clipboard
| Challenge: | linguistic steganography is the practice of concealing a secret message in some cover data such that an eavesdropper is not even aware of the existence of the secret message. |
| Approach: | They propose to use edit-based linguistic steganography to generate genuine-looking texts by using a masked language model that eliminates painstaking rule construction and has a high payload capacity. |
| Outcome: | The proposed method eliminates painstaking rule construction and has a high payload capacity for an edit-based model. |
Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | a few-shot text classification task requires a large number of output classes, with few training examples per class. |
| Approach: | They propose a data augmentation technique suitable for training with limited data for few-shot, highly-multiclass text classification scenarios. |
| Outcome: | The proposed technique improves performance on four classification tasks by 3.0% on average. |
Do RNN States Encode Abstract Phonological Alternations? (2021.naacl-main)
Copied to clipboard
| Challenge: | Sequence-to-sequence models have been successful in word formation tasks, but the opacity of the models makes it difficult to determine whether complex generalizations are learned or whether there is some level of generalization across related sound changes. |
| Approach: | They propose to train character-based sequence-to-sequence models for inflection of Finnish nouns into the genitive case, an inflation type which is encoded in the hidden states of an LSTM encoderdecoder trained to perform word infference. |
| Outcome: | The proposed models encode 17 different consonant gradation processes in a handful of dimensions in the RNN. |
Pre-training with Meta Learning for Chinese Word Segmentation (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained models are beneficial to Chinese Word Segmentation (CWS). However, these models lack task-specific prior segmentation knowledge. |
| Approach: | They propose a pre-trained Chinese word segmentation model MetaSeg which incorporates meta learning into a multi-criteria pre-training task. |
| Outcome: | Empirical results show that MetaSeg can achieve new state-of-the-art performance on twelve widely-used CWS datasets and significantly improve model performance in low-resource settings. |
Decompose, Fuse and Generate: A Formation-Informed Method for Chinese Definition Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing definition generation methods take the source word as an indecomposable semantic unit, but in parataxis languages like Chinese, word meanings can be composed using the word formation process. |
| Approach: | They propose to use word formation features to enhance Definition Generation (DG) in Chinese to generate an explanatory text. |
| Outcome: | The proposed model enhances Definition Generation (DG) in Chinese by decomposing the word meaning into different semantic components. |
User-Generated Text Corpus for Evaluating Japanese Morphological Analysis and Lexical Normalization (2021.naacl-main)
Copied to clipboard
| Challenge: | Morphological analysis (MA) and lexical normalization (LN) are important tasks for Japanese user-generated text. |
| Approach: | They construct a publicly available Japanese UGT corpus annotated with morphological and normalization information. |
| Outcome: | The proposed corpus shows low performance for non-general words and non-standard forms . morphological analysis is an important task in Japanese user-generated text . |
GPT Perdetry Test: Generating new meanings for new words (2021.naacl-main)
Copied to clipboard
| Challenge: | We create a set of nonce words and prompt GPT-3 to generate their dictionary definitions. |
| Approach: | They create a set of nonce words and prompt GPT-3 to generate their dictionary definitions. |
| Outcome: | The proposed model can process new words and make them 'neologisms' . it can also adapt to and extend a changing vocabulary, the authors found . |
Universal Semantic Tagging for English and Mandarin Chinese (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to generating semantic annotations for different languages are attracting more and more interest. |
| Approach: | They propose to extend Universal Semantic Tagging to Mandarin Chinese and evaluate its performance. |
| Outcome: | The proposed scheme is only tested in four Indo–European languages . accuracies are 92.7% and 94.6% for Chinese and English respectively . |
ShadowGNN: Graph Projection Neural Network for Text-to-SQL Parser (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing semantic parsing models struggle to adapt to unseen database schemas . a new architecture, ShadowGNN, processes schemas at abstract and semantic levels . |
| Approach: | They propose a new architecture which processes schemas at abstract and semantic levels. |
| Outcome: | The proposed architecture outperforms state-of-the-art models on a text-to-sql benchmark . it uses domain-independent representations to extract logical linking between question and schema . |
Contextualized and Generalized Sentence Representations by Contrastive Self-Supervised Learning: A Case Study on Discourse Relation Analysis (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to learn contextualized and generalized sentence representations are limited by the size of manually annotated data. |
| Approach: | They propose a method to learn contextualized and generalized sentence representations using contrastive self-supervised learning. |
| Outcome: | The proposed method outperforms baseline methods based on BERT, XLNet, and RoBERTa in English and Japanese and outperformed strong baseline methods. |
AMR Parsing with Action-Pointer Transformer (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation parsing is a sentence-to-graph prediction task . graph nodes are semantically based on one or more sentence tokens, so implicit alignments can be derived. |
| Approach: | They propose a transition-based system that decouples hard-attention over sentences with a target-side action pointer mechanism to decouple source tokens from node representations and address alignments. |
| Outcome: | The proposed system achieves the second best Smatch score on AMR 2.0 (81.8) it decouples source tokens from node representations and addresses alignments, but lacks expressiveness. |
NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction (2021.naacl-main)
Copied to clipboard
Ahmed Elgohary, Christopher Meek, Matthew Richardson, Adam Fourney, Gonzalo Ramos, Ahmed Hassan Awadallah
| Challenge: | Existing systems frame semantic parsing as a one-shot translation from a natural language question to the logical form. |
| Approach: | They propose a model that uses natural language feedback to correct parsers . they show that NL-EDIT can boost the accuracy of existing parser by 20% . |
| Outcome: | The proposed model can boost parsers' accuracy by 20% with just one turn of correction. |
Unsupervised Concept Representation Learning for Length-Varying Text Similarity (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing document similarity approaches suffer from the information gap caused by context and vocabulary mismatches when comparing varying-length texts. |
| Approach: | They propose an unsupervised concept representation learning approach to address this issue . they propose a concept-based document matching method to leverage recognition of local phrase features . |
| Outcome: | The proposed method achieves a better F1 score than baseline models on real-world data sets. |
Augmenting Knowledge-grounded Conversations with Sequential Knowledge Transition (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue models lack fine-grained control over knowledge selection and integration with dialogues. |
| Approach: | They propose to explicitly model the knowledge transition in sequential multi-turn conversations by abstracting knowledge into topic tags. |
| Outcome: | The proposed model outperforms baseline models on knowledge-grounded dialogue benchmarks. |
Adversarial Self-Supervised Learning for Out-of-Domain Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-domain (OOD) intents are unsupervised and require extensive labeled data. |
| Approach: | They propose a self-supervised contrastive learning framework to model discriminative semantic features from unlabeled data. |
| Outcome: | The proposed framework outperforms baseline methods on two public benchmark datasets with a statistically significant margin. |
Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking (2021.naacl-main)
Copied to clipboard
Zhaojiang Lin, Bing Liu, Seungwhan Moon, Paul Crook, Zhenpeng Zhou, Zhiguang Wang, Zhou Yu, Andrea Madotto, Eunjoon Cho, Rajen Subba
| Challenge: | Existing models for zero-shot cross-domain dialogue state tracking require in-domain data to model a new domain. |
| Approach: | They propose a slot descriptions enhanced generative approach for zero-shot cross-domain DST by encoding a dialogue context and a slots with a pre-trained encoder and generating slot value in auto-regressive manner. |
| Outcome: | The proposed model significantly improves state-of-the-art results in zero-shot cross-domain setting. |
Hierarchical Transformer for Task Oriented Dialog Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for dialog generation are challenging to train using the standard Seq2Seq models. |
| Approach: | They propose a framework for Hierarchical Transformer Encoders that can be morphed into any hierarchical transformer by using specially designed attention masks and positional encodings. |
| Outcome: | The proposed framework can be morphed into any hierarchical encoder, including HRED and HIBERT like models, by using specially designed attention masks and positional encodings. |
Measuring the ‘I don’t know’ Problem through the Lens of Gricean Quantity (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on analyzing chatbots for generic, safe responses has not addressed the ‘I don’t know’ problem, but lack of analysis leaves it unclear if a method improves chatbot models by mitigating this problem, or another. |
| Approach: | They propose to use Relative Utterance Quantity to diagnose the ‘I don’t know’ problem, in which a dialog system produces generic responses. |
| Outcome: | The proposed method allows for the direct analysis of the ‘I don’t know’ problem, which has been addressed but not analyzed by prior work. |
RTFE: A Recursive Temporal Fact Embedding Framework for Temporal Knowledge Graph Completion (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for static knowledge graph embedding (SKGE) ignore the continuity of states of TKGs in time evolution. |
| Approach: | They propose a Recursive Temporal Fact Embedding framework to transplant SKGE models to TKGs and enhance the performance of existing TKGE models. |
| Outcome: | The proposed framework can be used to transplant SKGE models to TKGs and improve existing models for TKG completion. |
Open Hierarchical Relation Extraction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing OpenRE methods cast different relation types in isolation without considering their hierarchical dependency. |
| Approach: | They propose a framework to establish bidirectional connections between OpenRE and relation hierarchies by integrating hierarchy information into relation representations. |
| Outcome: | The proposed framework outperforms state-of-the-art models on relation clustering and hierarchy expansion. |
Jointly Extracting Explicit and Implicit Relational Triples with Reasoning Pattern Enhanced Binary Pointer Network (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for relational triple extraction ignore implicit triples that lack explicit expressions, leading to incomplete knowledge graphs. |
| Approach: | They propose a binary pointer network to extract explicit and implicit relational triples from sentences and to retain the information of extracted triples in an external memory. |
| Outcome: | The proposed framework extracts overlapping triples relevant to each word sequentially and retains the information of extracted triples in an external memory. |
Multi-Grained Knowledge Distillation for Named Entity Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained big models have delivered top performance in Seq2seq modeling, but their deployments in real-world applications are often hindered by excessive computations and memory demands. |
| Approach: | They propose a distillation scheme to efficiently transfer knowledge from big models to their cheaper counterparts. |
| Outcome: | The proposed scheme maximizes the assimilation of knowledge from the teacher model to the student model. |
SGG: Learning to Select, Guide, and Generate for Keyphrase Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing keyphrase generation approaches synchronously generate present and absent keyphrases without explicitly distinguishing these two categories. |
| Approach: | They propose to deal with present and absent keyphrases separately with different mechanisms by using a hierarchical neural network with a pointing-based selector and a selection-guided generator. |
| Outcome: | The proposed model outperforms baselines on four keyphrase generation tasks and shows extensibility in natural language generation tasks. |
Towards Sentiment and Emotion aided Multi-modal Speech Act Classification in Twitter (2021.naacl-main)
Copied to clipboard
| Challenge: | Speech Act Classification determining the communicative intent of an utterance has been investigated widely over the years as a standalone task. |
| Approach: | They propose a multi-modal, emotion-TA dataset called EmoTA from open-source Twitter dataset and a Dyadic Attention Mechanism framework that integrates intra-modal and inter-modal attention to fuse multiple modalities. |
| Outcome: | The proposed framework boosts the performance of the primary task, i.e., TA classification (TAC), by benefitting from the two secondary tasks, namely, Sentiment and Emotion Analysis compared to its uni-modal and single task TAC variants. |
Generative Imagination Elevates Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing multimodal neural machine translation methods require triplets of bilingual sentence - image for training and tuples of source sentence . Existing methods require truncated images for inference, but ImagiT uses both source sentence and “imagined representation” to produce a target translation. |
| Approach: | They propose a multimodal machine translation method using visual imagination to generate a target translation from a sentence in a source language. |
| Outcome: | The proposed method significantly outperforms the existing text-only neural machine translation baselines and improves translation quality. |
Non-Autoregressive Translation by Learning Target Categorical Codes (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive text generation models still fall behind in translation quality . authors propose a model that learns implicitly categorical codes as latent variables . |
| Approach: | They propose a non-autoregressive Transformer model that implicitly categorizes latent variables into decoding . they find it improves translation quality by introducing more informative decoder inputs . |
| Outcome: | The proposed model achieves comparable or better performance in machine translation tasks than strong baselines. |
Training Data Augmentation for Code-Mixed Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | We show a 5.8 point increase in BLEU on heavily code-mixed sentences . code-mixing is becoming more commonplace in several bilingual communities . |
| Approach: | They propose a method to convert existing parallel data sources into code-mixed parallel data. |
| Outcome: | The proposed method shows a 5.8 point increase in BLEU on heavily code-mixed sentences on a Hindi-English code-mixed translation task. |
Rethinking Perturbations in Encoder-Decoders for Fast Training (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have proposed various regularization methods to avoid over-fitting. |
| Approach: | They propose to use scheduled sampling and adversarial perturbations to regularize neural models but they are not efficient enough for training time. |
| Outcome: | The proposed methods achieve comparable scores even though they are faster. |
Context-aware Decoder for Neural Machine Translation using a Target-side Document-Level Language Model (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural machine translation models that incorporate inter-sentential contexts can be trained only in document-level parallel data with sentential alignments. |
| Approach: | They propose a method to perform context-aware decoding with any pre-trained translation model . their method uses sentence-level parallel data and target-side document-level monolingual data . |
| Outcome: | The proposed method performs context-aware decoding on English to Russian translation using BLEU and contrastive tests. |
Machine Translated Text Detection Through Text Similarity with Round-Trip Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing detectors for translating texts fail to detect a text from a strange translator . Existing methods for detection of translated texts use text structure and complex words to detect translations . |
| Approach: | They propose a detector using text similarity with round-trip translation (TSRT) TSRT achieves 86.9% accuracy in detecting a translated text from a strange translator . Existing detectors have been built around a specific translator but fail to detect a translation from skeptics . |
| Outcome: | Existing detectors fail to detect translated texts from a strange translator . a detector achieves 86.9% accuracy in detecting a translated text from skeptic translators . |
TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models (PLMs) are expensive in inference, making them impractical in resource-limited real-world applications. |
| Approach: | They propose a dynamic token reduction approach to accelerate PLMs' inference by adapting the layer number of each token to avoid redundant calculation. |
| Outcome: | The proposed approach speeds up BERT by 2-5 times and improves performance in long-text tasks with less computation. |
Breadth First Reasoning Graph for Multi-hop Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent Graph Neural Network (GNN) has been used as a promising tool in multi-hop question answering task. |
| Approach: | They propose a model of Breadth First Reasoning Graph that passes to next nodes hop by hop until all edges have been passed. |
| Outcome: | The proposed model achieves state-of-the-art on answer span prediction on hotpotQA leaderboard. |
Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to solve multilingual question answering over knowledge graph (KGQA) use of bilingual lexicon induction to map training questions into those in target language circumvents language inconsistency . |
| Approach: | They propose to use bilingual lexicon induction to map training questions in source and target languages as augmented training data to minimize syntax-disorder. |
| Outcome: | The proposed model narrows the gap in zero-shot cross-lingual transfer between source and target languages. |
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering (2021.naacl-main)
Copied to clipboard
Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, Haifeng Wang
| Challenge: | Open-domain question answering uses dense passage retrieval to find answers . however, it is difficult to effectively train a dual-encoder due to discrepancy between training and inference . |
| Approach: | They propose an optimized training approach to improve dense passage retrieval using RocketQA . they propose cross-batch negatives, denoised hard negatives and data augmentation . |
| Outcome: | The proposed approach outperforms state-of-the-art models on both MSMARCO and Natural Questions. |
DAGN: Discourse-Aware Graph Network for Logical Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent QA with logical reasoning questions requires passage-level relations among the sentences. |
| Approach: | They propose a discourse-aware graph network that aggregates passage-level clues for QA by using discourse-based information. |
| Outcome: | The proposed model achieves competitive results on two logical reasoning QA datasets. |
Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Open-domain question answering is a task of finding answers to generic factoid questions. |
| Approach: | They propose to use a retrieve-and-read mechanism to drastically reduce the footprint of an open-domain QA system by up to 160x. |
| Outcome: | The proposed method achieves better accuracy than a parametric model with comparable docker-level system size. |
Unsupervised Multi-hop Question Answering by Question Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing training data for multi-hop question answering (QA) is time-consuming and resource-intensive. |
| Approach: | They propose an unsupervised framework that generates human-like multi-hop training data from homogeneous and heterogeneously data sources. |
| Outcome: | The proposed framework achieves 61% and 83% of the supervised learning performance for the HybridQA and HotpotQA datasets. |
Sliding Selector Network with Dynamic Memory for Extractive Summarization of Long Documents (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing summarization models suffer from the length limitation of text encoder, which results in huge loss of summary-relevant contents. |
| Approach: | They propose a sliding selector network with dynamic memory for extractive summarization of long-form documents that employs a window to extract summary sentences segment by segment. |
| Outcome: | The proposed model outperforms state-of-the-art models on two large-scale datasets showing that it is highly efficient and fluent. |
AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art abstractive summarization models rely on extensive labeled data, which lowers their generalization ability on domains where such data are not available. |
| Approach: | They propose to use domain adaptation methods to simulate the low-resource domain adaptation setting for abstractive summarization systems with existing datasets across six diverse target domains. |
| Outcome: | The proposed model can be used to adapt to a low-resource domain adaptation setting. |
QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization (2021.naacl-main)
Copied to clipboard
Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, Dragomir Radev
| Challenge: | Existing work on meeting summarization tasks is limited to short summaries that cover all the content of a meeting. |
| Approach: | They propose a query-based multi-domain meeting summarization task that generates a single short summary of meetings based on a transcript. |
| Outcome: | The proposed task is based on 1,808 query-summary pairs over 232 meetings in multiple domains. |
MM-AVS: A Full-Scale Dataset for Multi-modal Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Multimodal summarization materials lacking a holistic organization by integrating resources from various modalities. |
| Approach: | They propose a multimodal article and video summarization dataset that integrates resources from different modalities. |
| Outcome: | The proposed dataset validates the important assistance role of external information for multimodal summarization. |
MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets for dialogue summarization are limited to their small sizes and are built from a narrow domain. |
| Approach: | They propose a large-scale media interview dataset consisting of 463.6K transcripts with abstractive summaries. |
| Outcome: | The proposed dataset is larger and contains multi-party conversations from multiple domains. |
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that current models are prone to generating unfaithful summaries . a proposed method is effective in identifying and correcting extrinsic hallucinations . |
| Approach: | They propose a model-agnostic post-processing technique to correct unfaithful summaries . they generate alternative candidates where names and quantities are replaced with compatible ones . |
| Outcome: | The proposed method corrects extrinsic hallucinations in unfaithful summaries. |
Inference Time Style Control for Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to generate summaries of different styles without training separate models are lacking parallel data and expensive (re)training. |
| Approach: | They propose two methods that can be deployed during summary decoding on any pre-trained Transformer-based summarization model. |
| Outcome: | The proposed methods generate news headlines with various ideological leanings while still informative. |
ReinforceBug: A Framework to Generate Adversarial Textual Examples (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that ML Models are sensitive to Adversarial Examples (AEs) AEs are generated by perturbingining examples that preserve the intrinsic utility of the ML solutions but influence the classifier's predictions between original and modified inputs. |
| Approach: | They propose a reinforcement learning framework that learns a policy that is transferable on unseen datasets and generates utility-preserving and transferable AEs. |
| Outcome: | The proposed framework is 10% more successful than the state-of-the-art attack TextFooler. |