Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Detecting Attackable Sentences in Arguments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work in NLP studies focus on argument quality and making counterarguments toward the main claim, without investigating what parts of an argument are attackable for successful persuasion. |
| Approach: | They propose to use machine learning to find attackable sentences in online arguments by analyzing driving reasons for attacks and identifying relevant characteristics of sentences. |
| Outcome: | The proposed model can detect attackable sentences significantly better than baselines and comparably well to laypeople. |
Extracting Implicitly Asserted Propositions in Argumentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Argumentation is a rhetorical device that asserts propositions implicitly, but few studies have examined the issue. |
| Approach: | They propose a computational method for extracting propositions that are implicitly asserted in questions, reported speech, and imperatives in argumentation. |
| Outcome: | The proposed models are based on a corpus of 2016 debates and online commentary. |
Quantitative argument summarization and beyond: Cross-domain key point analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on multi-document summarization lacks quantitative aspect of summarizing views, arguments or opinions . authors develop method for automatic extraction of key points, which is comparable to a human expert . |
| Approach: | They propose to map arguments to a small set of expert-generated key points . they demonstrate that the applicability of key point analysis goes well beyond argumentation data . |
| Outcome: | The proposed method outperforms arguments in municipal surveys and user reviews . it is shown that the extraction of key points is comparable to a human expert . |
Unsupervised stance detection for arguments from consequences (2020.emnlp-main)
Copied to clipboard
| Challenge: | Social media platforms are becoming an essential venue for online deliberation . stance detection is a task to determine whether a text is in favor of, against, or unrelated to a given topic. |
| Approach: | They propose an unsupervised method to detect the stance of argumentative claims with respect to a topic. |
| Outcome: | The proposed method outperforms BERT and can be comparable to other methods. |
BLEU might be Guilty but References are not Innocent (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using a method to collect references and compare their value with human evaluations, we show that multi-reference BLEU does not improve the correlation for high quality output. |
| Approach: | They propose a method to compare the quality of automated metrics by analyzing references and comparing them with human evaluations. |
| Outcome: | The proposed method improves correlation with all modern evaluation metrics including embedding-based methods. |
Statistical Power and Translationese in Machine Translation Evaluation (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent paper argues that translationese has been used to describe features of translated text . a translationed text can be more explicit than the original source, authors say . authors recommend reverse-created test data be omitted from future evaluations . |
| Approach: | They propose to omit translationese from future machine translation evaluations . they also re-evaluate a past evaluation claiming human-parity of MT . |
| Outcome: | The proposed analysis shows that translationese does not affect machine translation evaluations. |
Simulated multiple reference training improves low-resource machine translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing valid translations for a given sentence are limited by a single reference translation, causing data sparsity in low-resource settings. |
| Approach: | They propose a method that approximates the full space of possible translations by sampling a paraphrase of the reference sentence from a MT model and training it to predict the paraphraser’s distribution over possible tokens. |
| Outcome: | The proposed method improves in low-resource settings and is complementary to back-translation. |
Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for machine translation evaluation are causing the correlation between human judgments and automatic metrics to break down. |
| Approach: | They propose to train a multilingual NMT system to score machine translation output with a sequence-to-sequence paraphraser, conditioned on a human reference. |
| Outcome: | The proposed model outperforms or statistically ties with all prior metrics on the WMT 2019 segment-level shared metrics task in all languages (excluding Gujarati where the model had no training data). |
PRover: Proof Generation for Interpretable Reasoning over Rules (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that transformers can act as “soft theorem provers” by answering questions over explicitly provided knowledge in natural language. |
| Approach: | They propose a transformer-based model that answers binary questions over rule-bases and generates the corresponding proofs. |
| Outcome: | The proposed model generates proofs with an accuracy of 87% while maintaining or improving performance on the QA task. |
Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite rapid progress in multihop question-answering, models still have trouble explaining why an answer is correct. |
| Approach: | They propose three explanation datasets in which explanations from corpus facts are annotated . they first annotate multiple candidate explanations for each answer, then use crowd-sourcing perturbations to test generalization . |
| Outcome: | The proposed datasets improve explanation quality but still behind the upper bound . the proposed dataset can be used to improve explanations using a BERT-based classifier . |
Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current supervised Question Answering methods rely on expensive data annotations and can introduce unintended annotator bias. |
| Approach: | They propose a self-supervised task over knowledge graphs that can be supervised by a data annotation tool. |
| Outcome: | The proposed task performs better than pre-trained language models on a large dataset. |
More Bang for Your Buck: Natural Perturbation for Robust Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Creating large datasets to train NLP models is becoming increasingly expensive. |
| Approach: | They propose to use a question-answering dataset to expand a training set using human-driven perturbations instead of rule-based machine perturbations. |
| Outcome: | The proposed approach improves on a question-answering dataset with human-driven perturbations. |
A matter of framing: The impact of linguistic formalism on probing results (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained contextualized encoders have had a major impact on the field of natural language processing. |
| Approach: | They conduct an in-depth cross-formalism layer probing study in role semantics to investigate the linguistic knowledge implicitly learned by pre-trained contextualized encoders. |
| Outcome: | The proposed model outperforms pre-trained models on a range of downstream tasks. |
Information-Theoretic Probing with Minimum Description Length (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite widespread adoption of probes, differences in their accuracy fail to adequately reflect differences in representations. |
| Approach: | They propose an alternative to the standard probes, information-theoretic probing with minimum description length (MDL). |
| Outcome: | The proposed method agrees in results and is more informative and stable than the standard probes. |
Intrinsic Probing through Dimension Selection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research on probing for linguistic structure in word embeddings has focused on intrinsic probing, but what these representations encode about linguistic structures remains unclear. |
| Approach: | They propose a framework that allows us to determine whether linguistic information in word embeddings is dispersed or focal. |
| Outcome: | The proposed framework allows us to determine whether linguistic information in word embeddings is dispersed or focal. |
Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually) (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretraining on self-supervised linguistic tasks is effective for learning features helpful for language understanding, but it requires more data to learn to prefer linguistic generalizations over surface ones. |
| Approach: | They propose a set of 20 ambiguous binary classification tasks to test whether a pretrained model prefers linguistic or surface generalizations. |
| Outcome: | The proposed model can learn to represent linguistic features with little pretraining data, but requires far more data to learn to prefer linguistic generalizations over surface ones. |
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference (2020.emnlp-main)
Copied to clipboard
Bang An, Jie Lyu, Zhenyi Wang, Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu, Changyou Chen
| Challenge: | Existing studies show that multi-head attention is an effective module in deep neural networks, but there are no explicit mechanisms guaranteeing this property. |
| Approach: | They propose a non-parametric approach that explicitly improves the repulsiveness in multi-head attention and consequently strengthens model’s expressiveness. |
| Outcome: | The proposed approach improves the repulsiveness in multi-head attention and strengthens model’s expressiveness. |
KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations (2020.emnlp-main)
Copied to clipboard
Fabio Massimo Zanzotto, Andrea Santilli, Leonardo Ranaldi, Dario Onorati, Pierfrancesco Tommasino, Francesca Fallucchi
| Challenge: | Syntactic parsers are losing their centrality in downstream tasks due to the success of large-scale textual representation learners. |
| Approach: | They propose to embed symbolic syntactic parse trees into artificial neural networks to visualize how syntax is used in inference. |
| Outcome: | The proposed encoder can visualize how syntax is used in inference. |
ETC: Encoding Long and Structured Inputs in Transformers (2020.emnlp-main)
Copied to clipboard
Joshua Ainslie, Santiago Ontanon, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, Li Yang
| Challenge: | Existing models for natural language processing (NLP) have been challenging to scale attention to longer inputs. |
| Approach: | They propose an extended Transformer construction architecture that scales attention to longer inputs by combining global-local attention with relative position encodings and a "Contrastive Predictive Coding" objective. |
| Outcome: | The proposed architecture scales attention to longer inputs and encodes structured inputs. |
Pre-Training Transformers as Energy-Based Cloze Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | elucidates close connection between cloze modeling and representation learning over text. |
| Approach: | They propose an energy-based cloze model for representation learning over text . they assign a scalar energy score to each input token indicating how likely it is given context . |
| Outcome: | The proposed model performs better than masked language models and faster than cloze models. |
Calibration of Pre-trained Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained Transformers dominate benchmark tasks but use a large number of self-attention heads across many layers in a way that is difficult to unpack. |
| Approach: | They analyze pre-trained Transformer models' posterior probabilities to determine whether they are calibrated for three tasks: natural language inference, paraphrase detection, and commonsense reasoning. |
| Outcome: | The models are calibrated in-domain and out-of-domain, and their calibration error out-domain can be as much as 3.5x lower. |
Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Linguistic steganography studies how to hide secret messages in natural language cover texts. |
| Approach: | They propose a method which encodes secret messages using self-adjusting arithmetic coding based on a neural language model. |
| Outcome: | The proposed method outperforms the state-of-the-art methods on four datasets by 15.3% and 38.9% in terms of bits/word and KL metrics. |
Multi-Dimensional Gender Bias Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | a novel framework decomposes gender bias in text along several pragmatic and semantic dimensions . language is a primary means by which people communicate, express identities and categorize themselves . unwanted gender biases can affect downstream applications, leading to poor user experiences . |
| Approach: | They propose a framework that decomposes gender bias in text along several dimensions . they annotate eight large scale datasets with gender information and collect a benchmark . |
| Outcome: | The proposed framework decomposes gender bias in text along several pragmatic and semantic dimensions. |
FIND: Human-in-the-Loop Debugging Deep Text Classifiers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models are limited in the number of available datasets and lack the necessary tools to improve them. |
| Approach: | They propose a framework which enables humans to debug deep learning text classifiers by disabling irrelevant hidden features. |
| Outcome: | Experiments show that using FIND, humans can improve CNN text classifiers trained on different types of imperfect datasets. |
Conversational Document Prediction to Assist Customer Care Agents (2020.emnlp-main)
Copied to clipboard
Jatin Ganhotra, Haggai Roitman, Doron Cohen, Nathaniel Mills, Chulaka Gunasekara, Yosi Mass, Sachindra Joshi, Luis Lastras, David Konopnicki
| Challenge: | Using a conversational search system, the agent/system can ask clarification questions and interactively modify the search results as the conversation progresses. |
| Approach: | They propose to use a public dataset to analyze the task of predicting the documents that customer care agents can use to facilitate users’ needs. |
| Outcome: | The proposed model is more efficient than existing models and is more cost-effective than existing ones. |
Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU (2020.emnlp-main)
Copied to clipboard
| Challenge: | a number of languages are processed incrementally, but the best ones do not . we test five models on various datasets and compare their performance using three incremental evaluation metrics. |
| Approach: | They investigate how bidirectional LSTMs and Transformers behave under incremental interfaces . they propose to use bidirectional encoders in incremental mode while retaining non-incremental quality . |
| Outcome: | The proposed models perform better under incremental interfaces than the "omni-directional" BERT model, which achieves better non-incremental performance, but is impacted more by the incremental access. |
Augmented Natural Language for Generative Sequence Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | generative framework for joint sequence labeling and sentence-level classification is general purpose, performing well on few-shot learning, low resource, and high resource tasks. |
| Approach: | They propose a generative framework for joint sequence labeling and sentence-level classification . their framework incorporates label semantics and shares knowledge across tasks . |
| Outcome: | The proposed model performs on few-shot learning, slot labeling, and intent classification benchmarks. |
Dialogue Response Ranking Training with Large-Scale Human Feedback Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain dialog models can minimize the perplexity of target human responses . however, some human responses are more engaging than others, spawning more followup interactions . |
| Approach: | They train open-domain dialog models to minimize perplexity of target human responses . they use social media feedback data to train models to predict engaging dialog turns . |
| Outcome: | The proposed model outperforms existing models on 133M human feedback pairs . it also outperformed the conventional dialog perplexity baseline model . |
Semantic Evaluation for Text-to-SQL with Distilled Test Suites (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate semantic accuracy of Text-to-SQL models are not accurate. |
| Approach: | They propose a test suite accuracy method to approximate semantic accuracy for Text-to-SQL models. |
| Outcome: | The proposed method evaluates 21 models submitted to the Spider leader board and manually checks on 100 examples. |
Cross-Thought for Sentence Encoder Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models to pretrain sentence encoders with large unlabeled corpus are lacking in linguistic information retrieval. |
| Approach: | They propose a novel approach to pre-training sequence encoder using transformers . they propose to train a Transformer-based sequence encoded over a large set of short sequences based on a set of masked words . |
| Outcome: | The proposed approach outperforms state-of-the-art encoders on hotpotQA by improving intermediate information retrieval performance. |
AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate semantic parsers that answer questions on databases require large amounts of annotated data. |
| Approach: | They propose a method to generate semantic parsers that answer questions on databases . they use automatic paraphrasing and template-based parsing to find alternative expressions . |
| Outcome: | The proposed method achieves 69.8% answer accuracy on natural questions, 16.4% higher than state-of-the-art models and 5.2% lower than the same model trained with human data. |
A Spectral Method for Unsupervised Multi-Document Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | a spectral-based hypothesis is proposed for the unsupervised task of multi-document summarization. |
| Approach: | They propose a spectral-based hypothesis that a summary candidate's spectral impact is closely linked to its spectre. |
| Outcome: | The proposed method has a competitive result compared to state-of-the-art systems. |
What Have We Achieved on Text Summarization? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text summarization have been investigated, but there are still gaps between them and human professionals. |
| Approach: | They analyze 8 major sources of errors on 10 representative summarization models manually. |
| Outcome: | Aiming to gain more understanding of summarization systems with respect to their strengths and limitations on a fine-grained syntactic and semantic level, we use 8 major sources of errors on 10 representative summarizing models. |
Q-learning with Language Model for Edit-based Unsupervised Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised text summarization methods are promising, but their performance is still behind that of state-of-the-art supervised methods. |
| Approach: | They propose a method based on Q-learning with an edit-based summarization that uses an Editorial Agent and Language Model converter to predict edit actions. |
| Outcome: | The proposed method delivers competitive performance even with zero paired data, while requiring no validation set. |
Friendly Topic Assistant for Transformer Based Abstractive Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive document summarization is a comprehensive task in natural language processing. |
| Approach: | They propose a topic assistant that rearranges and learns document semantics . they propose TA that is compatible with Transformer-based models and user-friendly . |
| Outcome: | The proposed model is compatible with Transformer-based models and user-friendly. |
Contrastive Distillation on Intermediate Representations for Language Model Compression (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to compress language models use a simple L_2 loss to distill knowledge in the intermediate representations of a large BERT model to a smaller one. |
| Approach: | They propose a method that uses knowledge distillation to distill knowledge through intermediate layers of the teacher via a contrastive objective. |
| Outcome: | The proposed method outperforms state-of-the-art methods on the GLUE benchmark. |
TernaryBERT: Distillation-aware Ultra-low Bit BERT (2020.emnlp-main)
Copied to clipboard
| Challenge: | Transformer-based pre-training models like BERT are computationally expensive and limited to resource-constrained devices. |
| Approach: | They propose a method which ternarizes the weights in a fine-tuned BERT model. |
| Outcome: | The proposed method outperforms the other methods on the GLUE and SQUAD benchmarks while being 14.9x smaller. |
Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for supervised meta-learning require many training tasks to generalize . cloze-style objectives can be used to generate a large, rich, meta-training task distribution from unlabeled text. |
| Approach: | They propose a self-supervised approach to generate a large, rich, meta-learning task distribution from unlabeled text. |
| Outcome: | The proposed approach generates a large, rich, meta-learning task distribution from unlabeled text. |
Efficient Meta Lifelong-Learning with Limited Memory (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing natural language learning models fail to continuously learn new tasks as they are re-trained throughout their lifetime. |
| Approach: | They propose a meta-lifelong framework that combines three common lifelong learning principles . they propose to store past examples in episodic memory and replay them at training and inference time . |
| Outcome: | The proposed framework achieves state-of-the-art performance using 1% memory size and narrows the gap with multi-task learning. |
Don’t Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning. |
| Approach: | They show that English dev accuracy makes it difficult to obtain reproducible results . they recommend providing oracle scores alongside zero-shot results if possible . |
| Outcome: | mBERT and XLM have shown strong performance on cross-lingual recognition, text classification, dependency parsing, and other tasks. |
A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT (2020.emnlp-main)
Copied to clipboard
| Challenge: | supervised word alignment tools such as GIZA++, MGIZA (Gao and Vogel, 2008) and FastAlign remain stagnant in terms of word alignment accuracy. |
| Approach: | They propose a supervised word alignment method based on cross-language span prediction by formalizing a word alignment problem as a collection of independent predictions from a token in the source sentence to a span in the target sentence. |
| Outcome: | The proposed method significantly outperforms previous supervised and unsupervised word alignment methods without any bitexts for pretraining. |
Accurate Word Alignment Induction from Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work suggests that Transformer captures poor word alignments through its attention mechanism. |
| Approach: | They propose two new word alignment induction methods that use attention weights to capture accurate word alignments. |
| Outcome: | The proposed methods outperform baselines on three publicly available datasets and are significantly better than GIZA++. |
ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cherokee is a highly endangered Native American language spoken by the Cherokee people . there are only 2,000 fluent first language Cherokee speakers remaining in the world . |
| Approach: | They propose a Cherokee-English parallel dataset to facilitate machine translation between Cherokee and English. |
| Outcome: | The proposed dataset compares Cherokee-English and English-Cherokee machine translation systems . the results show that the datasets are low-resource and low-cost compared to other datasets . |
Unsupervised Discovery of Implicit Gender Bias (2020.emnlp-main)
Copied to clipboard
| Challenge: | Social biases are difficult to identify because human judgements in this domain can be unreliable. |
| Approach: | They propose an unsupervised approach to detecting implicit gender bias in text . their main challenge is forcing the model to focus on signs of implicit bias . |
| Outcome: | The proposed model reduces the influence of confounds by focusing on signs of implicit bias rather than other artifacts in the data. |
Condolence and Empathy in Online Communities (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using computational tools, we examine the dynamics of condolence online. |
| Approach: | They develop computational tools to analyze 11.4M distress expressions and 2.8M condolence offerings in a massive dataset of 11.4 million people. |
| Outcome: | The proposed model reveals that condolence features differ from those seen in interpersonal settings and that the features of condolance individuals find most helpful differ from the features seen in social media. |
An Embedding Model for Estimating Legislative Preferences from the Frequency and Sentiment of Tweets (2020.emnlp-main)
Copied to clipboard
| Challenge: | Legislator preferences are typically estimated as general ideology using roll call votes on legislation, but these measures fail to capture aspects of preferences not reflected in legislation, such as attitudes towards a sitting president. |
| Approach: | They propose an embedding-based method for measuring legislator attitudes using tweets . they model legislators' attitudes towards president Donald Trump as vector embeddables that interact with embeddibles for Trump himself constructed using a neural network from the text of his daily tweets. |
| Outcome: | The proposed model predicts the frequency and sentiment of tweets by comparing it to traditional measures of legislator preferences. |
Measuring Information Propagation in Literary Social Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | a gap in computational work to support the "Miss Havisham is dead" "She died" research focuses on the representation of social networks in literature . |
| Approach: | They propose a pipeline for measuring information propagation in literature . they analyze the dynamics of information propagations in over 5,000 works of fiction . |
| Outcome: | The proposed pipeline analyzes the dynamics of information propagation in 5,000 works of fiction and finds that women fill structural holes connecting different communities more frequently than men. |
Social Chemistry 101: Learning to Reason about Social and Moral Norms (2020.emnlp-main)
Copied to clipboard
| Challenge: | SOCIAL CHEMISTRY is a conceptual formalism to study people’s everyday social norms and moral judgments over a rich spectrum of real life situations described in natural language. |
| Approach: | They propose a new conceptual formalism to study people's everyday social norms and moral judgments over a rich spectrum of real life situations described in natural language. |
| Outcome: | The proposed model can be used to model people's everyday social norms and moral judgments over a rich spectrum of real life situations. |
Event Extraction by Answering (Almost) Natural Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work in event argument extraction relies heavily on entity recognition as a preprocessing/concurrent step, causing error propagation. |
| Approach: | They propose a question answering task that extracts event arguments in an end-to-end manner. |
| Outcome: | The proposed framework outperforms prior work on the ACE 2005 task on event argument extraction. |
Connecting the Dots: Event Graph Schema Induction with Path Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to automate event extraction focus on uncertainty, re-occurring events and multiple hypotheses. |
| Approach: | They propose a new Event Graph Schema where two event types are connected through multiple paths involving entities that fill important roles in a coherent story. |
| Outcome: | The proposed model is highly effective at inducing salient and coherent schemas. |
Joint Constrained Learning for Event-Event Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Understanding natural language involves recognizing how multiple event mentions structurally and temporally interact with each other. |
| Approach: | They propose a joint constrained learning framework that enforces logical constraints within and across multiple temporal and subevent relations of events by converting constraints into differentiable learning objectives. |
| Outcome: | The proposed framework outperforms SOTA methods on benchmarks for temporal relation extraction and event hierarchy construction. |
Incremental Event Detection via Knowledge Consolidation Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to event detection require a fixed set of pre-defined event types . existing methods cannot handle semantic ambiguity and training data imbalance problems . |
| Approach: | They propose a Knowledge Consolidation Network to address these issues . they propose to use a prototype enhanced retrospection and hierarchical distillation to mitigate the adverse effects of semantic ambiguity and class imbalance. |
| Outcome: | The proposed method outperforms the state-of-the-art model by 19% and 13.4% of whole F1 score on ACE and TAC benchmarks. |
Semi-supervised New Event Type Induction and Event Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing event extraction studies assume a set of event types and corresponding annotations are given, which could be expensive. |
| Approach: | They propose a semi-supervised task of event type induction to learn seen and unseen types . they use seen type annotations to optimize the process and enforce the reconstruction . |
| Outcome: | The proposed method achieves state-of-the-art on supervised event detection and discovers high-quality new types. |
Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches that integrate commonsense knowledge into pre-trained language models simply transfer relational knowledge while ignoring rich connections within the knowledge graph. |
| Approach: | They propose a method that leverages structural and semantic information of the knowledge graph to generate commonsense-aware text. |
| Outcome: | The proposed method outperforms baseline models on three text generation tasks that require reasoning over commonsense knowledge. |
Reformulating Unsupervised Style Transfer as Paraphrase Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for style transfer warp the input’s meaning through attribute transfer, which changes semantic properties such as sentiment. |
| Approach: | They propose a method for fine-tuning pretrained language models on automatically generated paraphrase data to improve the efficiency of style transfer. |
| Outcome: | The proposed method outperforms state-of-the-art style transfer systems on human and automatic evaluations and proposes fixed variants. |
De-Biased Court’s View Generation with Causality (2020.emnlp-main)
Copied to clipboard
Yiquan Wu, Kun Kuang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Jun Xiao, Yueting Zhuang, Luo Si, Fei Wu
| Challenge: | Existing approaches to court’s view generation can be used to address this problem, but neglecting the confounding bias in data can limit the model performance and pollute learning outcomes. |
| Approach: | They propose a novel Attentional and Counterfactual based Natural Language Generation method consisting of an attentional encoder and a pair of innovative counterfactual decoders to generate judgment-discriminative court's views. |
| Outcome: | The proposed method is able to generate judgment-discriminative court's views (both supportive and non-supportive views) under both quantitative and qualitative evaluation metrics. |
PAIR: Planning and Iterative Refinement in Pre-trained Transformers for Long Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | We present a content-controlled text generation framework for pre-trained Transformers . large pre-train models are the cornerstone of many state-of-the-art models in natural language understanding and generation tasks. |
| Approach: | They propose a content-controlled text generation framework that adds content planning to large pre-trained Transformers without modifying model architecture. |
| Outcome: | The proposed framework improves the quality of the outputs on three domains. |
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning (2020.emnlp-main)
Copied to clipboard
Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D. Hwang, Ronan Le Bras, Antoine Bosselut, Yejin Choi
| Challenge: | Existing methods for integrating past and future contexts are limited and require manual input. |
| Approach: | They propose an unsupervised decoding algorithm that incorporates past and future contexts using off-the-shelf, left-to-right language models and no supervision. |
| Outcome: | The proposed method outperforms unsupervised methods on abductive and counterfactual reasoning tasks. |
Where Are You? Localization from Embodied Dialog (2020.emnlp-main)
Copied to clipboard
| Challenge: | Observer and Locator perform a cooperative localization task in a 3D environment. |
| Approach: | They propose a dataset of 6k dialogs in which two humans complete a cooperative localization task. |
| Outcome: | The proposed model achieves 32.7% success at identifying the Observer’s location within 3m in unseen buildings, vs. 70.4% for human Locators. |
Learning to Represent Image and Text with Denotation Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in learning representations of visual and language information have been a problem with many applications. |
| Approach: | They propose to extract visual expressions from images aligned with linguistic expressions that describe the images to learn representations from implicit expressions. |
| Outcome: | The proposed representations lead to stronger empirical results on downstream tasks of cross-modal image retrieval, referring expression, and compositional attribute-object recognition. |
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Observable changes in the scene are reflected in captions, but actions are also linked to social aspects such as intentions, effects, and attributes that describe the agent. |
| Approach: | They propose to generate captions from videos that describe latent aspects of the human agent's actions. |
| Outcome: | The proposed model can be used to describe latent aspects of human actions in video clips and answer questions about videos. |
Does my multimodal model learn cross-modal interactions? It’s harder to tell than you might think! (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new tool for evaluating expressive cross-modal interactions is needed . empirical multimodally-additive function projection is a tool for isolating unimodal structure . |
| Approach: | They propose a tool that modifies model predictions so that cross-modal interactions are eliminated . they propose to use EMAP to evaluate models' ability to leverage cross-module interactions . |
| Outcome: | The proposed tool can be used to evaluate models on image+text classification tasks . it finds that removing cross-modal interactions results in little to no performance degradation . |
MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Availability of large-scale datasets has enabled statistical machine learning in vision and language understanding. |
| Approach: | They propose a training paradigm that exposes models to perceptually similar mutations of input . they show a 10.57% improvement in the VQA-CP challenge . |
| Outcome: | The proposed training paradigm improves on the visual question answering challenge with 10.57% accuracy. |
Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research shows that dialogue systems trained on human conversation data are biased and can produce responses that reflect people’s gender prejudice. |
| Approach: | They propose a novel adversarial learning framework Debiased-Chat to train dialogue models free from gender bias while keeping their performance. |
| Outcome: | The proposed framework significantly reduces gender bias in dialogue models while maintaining the response quality. |
Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for improving consistency often train with additional NLI labels or attach trained extra modules to the generative agent. |
| Approach: | They propose to encode personas into dialogue embeddings and a persona-conditioned dialogue dataset to improve persona consistency. |
| Outcome: | The proposed approach can enforce dialogue agents to refrain from contradictions and improve consistency of existing models. |
TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models with self-attention encoder architectures are less useful in practice. |
| Approach: | They propose to use user and system tokens to model dialogue behavior during pre-training . they propose a contrastive objective function to simulate the response selection task . |
| Outcome: | The proposed model outperforms baseline models on four downstream tasks . it also has a few-shot ability that can mitigate the data scarcity problem . |
RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | RiSAWOZ contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues spanning over 12 domains . despite of substantial progress made, there are challenges in creating challenging datasets in terms of size, multiple domains, semantic annotations and complexity. |
| Approach: | They propose a large-scale multi-domain Chinese Wizard-of-Oz dataset with rich semantic annotations that captures discourse phenomena for task-oriented dialogue modeling. |
| Outcome: | The proposed dataset contains 11.2K human-to-human (H2H) multi-turn semantically annotated dialogues with more than 150K utterances spanning over 12 domains. |
Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness (2020.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale dialogue datasets contain a non-negligible number of unacceptable utterance pairs . previous studies have identified such flaws and reported that the corpus is noisy . |
| Approach: | They propose a method for scoring the quality of utterance pairs based on their connectivity and relatedness. |
| Outcome: | The proposed method has a good correlation with human judgment of dialogue quality and is applied to training data filtered by the proposed method. |
Latent Geographical Factors for Analyzing the Evolution of Dialects in Contact (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to analyze the evolution of dialects use admixture analysis, but such ancestral populations are hardly interpretable in the context of the tree model. |
| Approach: | They propose a probabilistic generative model that represents latent factors as geographical distributions and a tree model that can be alternatively represented as a set of geographical distribution. |
| Outcome: | The proposed model has higher affinity with the tree model because a tree can alternatively be represented as a set of geographical distributions. |
Predicting Reference: What do Language Models Learn about Discourse Models? (2020.emnlp-main)
Copied to clipboard
| Challenge: | a growing literature that probes neural language models to assess their latent acquisition of grammatical knowledge has not investigated their acquisition of discourse modeling ability. |
| Approach: | They draw on a psycholinguistic literature that has established how different contexts affect referential biases concerning who is likely to be referred to next. |
| Outcome: | The proposed models do not resemble human language users, the authors show . their models capture the linguistic knowledge required to perform discourse modeling . |
Word class flexibility: A deep contextualized approach (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on word class flexibility have been fraught with difficulties in quantifying it accurately and at scale. |
| Approach: | They propose a method to quantify word class flexibility in 37 languages using contextualized word embeddings. |
| Outcome: | The proposed method builds on recent work in contextualized word embeddings to quantify semantic shift between word classes and uncovers shared tendencies in class flexibility across languages. |
Shallow-to-Deep Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that deep training is 1:4 faster than training from scratch. |
| Approach: | They propose a shallow-to-deep training method that learns deep models by stacking shallow models. |
| Outcome: | The proposed method is 1:4 faster than training from scratch and achieves BLEU scores of 30:33 and 43:29 on two translation tasks. |
Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive inference procedures that refine in token space often require computational overhead. |
| Approach: | They propose an efficient inference procedure that iteratively refines translation purely in the continuous space using a latent variable instead of the latent variables. |
| Outcome: | The proposed procedure is twice as efficient and more effective than the existing EM-like inference procedure. |
Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation techniques are not suitable for deep learning tasks due to memory constraints. |
| Approach: | They propose to combine knowledge from a large teacher network into a student network (S) they propose to use a combinatorial mechanism to inject layer-level supervision from T to S . |
| Outcome: | The proposed model outperforms existing models in PortugueseEnglish, TurkishEnglish and EnglishGerman directions and students trained using it have 50% fewer parameters and can deliver comparable results to 12-layer teachers. |
Multi-task Learning for Multilingual Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual neural machine translation systems rely on bitext training data, which is limited and costly to collect. |
| Approach: | They propose a multi-task learning framework that trains the model with the translation task on bitext data and two denoising tasks on monolingual data. |
| Outcome: | The proposed framework outperforms pre-training models for both NMT and cross-lingual transfer learning NLU tasks. |
Token-level Adaptive Training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing token imbalance phenomenon in natural language as different tokens appear with different frequencies, which leads to different learning difficulties for tokens in Neural machine translation (NMT). |
| Approach: | They propose to assign tokens with different frequencies to target tokens during training to encourage the model to pay more attention to low-frequency tokens. |
| Outcome: | The proposed model yields consistent improvements on ZH-EN, EN-RO, and EN-DE translation tasks, especially on sentences that contain more low-frequency tokens. |
Multi-Unit Transformers for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that the MUTE models outperform the Transformer-Base by up to +1.52, +1.99 and +1.00 BLEU points, with only a mild drop in inference speed (about 3.1%). |
| Approach: | They propose to use multiple parallel units to promote the expressiveness of the Transformer by introducing diverse and complementary units. |
| Outcome: | The proposed models outperform the Transformer-Base model with only a mild drop in inference speed (about 3.1%). |
On the Sparsity of Neural Machine Translation Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern neural machine translation models employ a large number of parameters, which leads to serious over-parameterization. |
| Approach: | They propose to prune parameters to improve the model by +0.8 BLEU points and to reallocate them to enhance the ability of modeling low-level lexical information. |
| Outcome: | The pruned parameters improve the model by +0.8 BLEU points and the rejuvenated parameters enhance the ability to model low-level lexical information. |
Incorporating a Local Translation Mechanism into Non-autoregressive Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to capture local dependencies among output tokens are not efficient, causing errors of repeated translation. |
| Approach: | They propose a local autoregressive translation mechanism that predicts a short sequence of tokens for each target decoding position instead of one token. |
| Outcome: | Empirical results show that the proposed method achieves comparable or better performance with fewer decoding iterations, bringing a 2.5x speedup. |
Self-Paced Learning for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that the training of neural machine translation (NMT) rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. |
| Approach: | They propose to train NMT model using a self-paced learning approach that allows it to quantify the learning confidence over training examples and flexibly govern its learning via regulating the loss in each iteration step. |
| Outcome: | The proposed model outperforms baseline models and those trained with human-designed curricula on translation quality and convergence speed. |
Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to significant improvement of document-level neural machine translation (NMT). |
| Approach: | They propose a long-short term masking self-attention on top of the standard transformer to capture the long-range dependence and reduce the propagation of errors. |
| Outcome: | The proposed model captures the long-range dependence and reduces errors on two publicly available document-level datasets. |
Generating Diverse Translation from Model Distribution with Dropout (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural machine translation models lack diversity in their generation. |
| Approach: | They propose to generate diverse translations by deriving Bayesian models and sampling models from them for inference. |
| Outcome: | The proposed method makes a better trade-off between diversity and accuracy. |
Non-Autoregressive Machine Translation with Latent Alignments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive machine translation methods are lacking in the field of latent alignments. |
| Approach: | They propose two strong methods for non-autoregressive machine translation that model latent alignments with dynamic programming. |
| Outcome: | The proposed models achieve state-of-the-art on the WMT’14 EnDe task, compared with the autoregressive Transformer baseline. |
Look at the First Sentence: Position Bias in Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Extractive question answering models are trained to predict start and end positions of answers . recent QA models outperform humans in some datasets due to their simplicity and effectiveness. |
| Approach: | They propose to use prior distribution of answer positions as a bias model to reduce position bias. |
| Outcome: | The proposed model outperforms BERT from 37.48% to 81.64% when trained on a biased SQUAD dataset. |
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing question answering datasets for common sense reasoning are lacking for prototypical situations. |
| Approach: | They propose a question answering dataset for training and evaluating common sense reasoning capabilities of artificial intelligence systems in such prototypical situations. |
| Outcome: | The proposed model outperforms existing models on all evaluation metrics with a meaningful gap. |
IIRC: A Dataset of Incomplete Information Reading Comprehension Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing reading comprehension tasks focus on questions for which the contexts provide all the information required to answer them, thus not evaluating a system’s performance at identifying a potential lack of sufficient information and locating sources for that information. |
| Approach: | They propose to use a dataset with 13K questions over paragraphs from English Wikipedia that provide only partial information to answer them, with the missing information occurring in one or more linked documents. |
| Outcome: | The proposed model achieves 31.1% F1 on the reading comprehension task, while estimated human performance is 88.4%. |
Unsupervised Adaptation of Question Answering Systems via Generative Self-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Supervised self-training methods have transformed applied machine learning . however, adapting to target data has received little attention . |
| Approach: | They propose a method to generate synthetic QA pairs for unsupervised self adaptation . they use massive amounts of data to simulate self-supervised tasks . |
| Outcome: | The proposed method improves QA systems significantly by using less data and training computation than existing augmentation approaches. |
TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current machine reading comprehension benchmarks have no questions that test temporal phenomena . a new study studies reading comprehension for temporal relations . |
| Approach: | They propose a reading comprehension benchmark built on news snippets and 21k human-generated questions querying temporal relationships. |
| Outcome: | The new reading comprehension benchmark TORQUE achieves an exact-match score of 51% on the test set . the benchmark is built on 3.2k news snippets with 21k human-generated questions . |
ToTTo: A Controlled Table-To-Text Generation Dataset (2020.emnlp-main)
Copied to clipboard
Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, Dipanjan Das
| Challenge: | Existing methods for data-to-text generation often hallucinate phrases not supported by the Wikipedia table. |
| Approach: | They propose a controlled task where annotators directly revise existing Wikipedia sentences to generate a one-sentence description. |
| Outcome: | The proposed task produces a one-sentence description from a Wikipedia table and highlighted cells. |
ENT-DESC: Entity Description Generation by Exploring Knowledge Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for knowledge-to-text generation use RDF triples or key-value pairs to generate a natural language description. |
| Approach: | They propose a large-scale dataset to facilitate the study of KG-to-text . they propose MGCN model architecture that incorporates aggregation methods to extract the rich graph information. |
| Outcome: | The proposed model can represent the original graph information more comprehensively and integrates multiple aggregation methods to extract the rich graph information. |
Small but Mighty: New Benchmarks for Split and Rephrase (2020.emnlp-main)
Copied to clipboard
| Challenge: | Split and Rephrase is a text simplification task that requires a strong evaluation benchmark and metric . despite its relatively new nature, the benchmark dataset contains easily exploitable syntactic cues . |
| Approach: | They propose to use crowdsourced datasets to evaluate split and rephrase models . they find that the widely used benchmark dataset universally contains exploitable syntactic cues . |
| Outcome: | The proposed model performs better than the state-of-the-art model, the authors say . they show that the datasets contain significantly more diverse syntax . |
Online Back-Parsing for AMR-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstract meaning representation (AMR) is a semantic graph representation that abstracts meaning away from a sentence. |
| Approach: | They propose a decoder that back predicts projected AMR graphs on target sentences . their results show superiority over previous state-of-the-art decoded graph Transformer . |
| Outcome: | The proposed model outperforms the state-of-the-art model on two AMR benchmarks. |
Reading Between the Lines: Exploring Infilling in Visual Narratives (2020.emnlp-main)
Copied to clipboard
| Challenge: | Generating long form narratives from multiple modalities requires a model to learn surrounding contextual information by masking spans of input while decoding attempts in generating the entire text. |
| Approach: | They propose to use infilling techniques to generate textual descriptions from images that are rich in contextual dependencies. |
| Outcome: | The proposed model outperforms existing models in visual storytelling by generating text from a large scale dataset of 46,200 procedures and 340k pairwise images and textual descriptions. |
Acrostic Poem Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Acrostic poems contain a hidden message; typically, the first letter of each line spells out a word or short phrase. |
| Approach: | They propose a task for acrostic poem generation in English with multiple constraints . they define the task as a generation task with multiple constraint constraints based on a conditional neural language model and a neural rhyming model . |
| Outcome: | The proposed task is based on a baseline model and a neural rhyming model. |
Local Additivity Based Data Augmentation for Semi-supervised NER (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is one of the first stages in deep language understanding yet current NER models heavily rely on human-annotated data. |
| Approach: | They propose a Local Additivity based Data Augmentation method for semi-supervised Named Entity Recognition (NER) that creates virtual samples by interpolating sequences close to each other. |
| Outcome: | The proposed method improves both entity and context learning by adding to training data and extending it to semi-supervised setting. |
Grounded Compositional Outputs for Adaptive Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language models are a key component of natural language processing, but their size is a problem because they are typically trained with a closed output vocabulary derived from the training data. |
| Approach: | They propose a fully compositional output embedding layer for language models that is grounded in semantically related words and free-text definitions. |
| Outcome: | The proposed model outperforms state-of-the-art methods and adaptation approaches on cross-domain modeling and cross-learning tasks. |
SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness (2020.emnlp-main)
Copied to clipboard
| Challenge: | Data augmentation is a common method used to improve out-of-domain (OOD) generalization. |
| Approach: | They propose a data augmentation method that uses corruption and reconstruction functions to move randomly on a manifold to generate training examples. |
| Outcome: | The proposed method outperforms existing methods and baseline models on both in-domain and OOD data and achieves gains of 0.8% on OOD Amazon reviews, 1.8% accuracy on OOO MNLI, and 1.4 BLEU on in- domain IWSLT14 German-English. |
SetConv: A New Approach for Learning from Imbalanced Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for classification are biased towards the majority class when the Imbalance Ratio (IR) is high. |
| Approach: | They propose a set convolution operation and an episodic training strategy to extract a single representative for each class, so that classifiers can later be trained on a balanced class distribution. |
| Outcome: | The proposed algorithm is permutation-invariant despite the order of inputs and shows superiority on multiple large-scale benchmark text datasets. |
Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on augmenting question answering models with external knowledge (e.g., knowledge graphs) lacks transparency into the model’s prediction rationale. |
| Approach: | They propose a knowledge-aware approach that equips pre-trained language models with a multi-hop relational reasoning module that performs multi-relational reasoning over subgraphs extracted from external knowledge graphs. |
| Outcome: | The proposed model performs multi-hop, multi-relational reasoning over subgraphs extracted from external knowledge graphs. |
Improving Bilingual Lexicon Induction for Low Frequency Words (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate word-level translations from non-parallel corpora are based on word embeddings. |
| Approach: | They propose two methods to address two factors that degrade bilingual lexicon induction accuracy . they propose a method that assumes a seeding dictionary is available . |
| Outcome: | The proposed method improves bilingual lexicon induction significantly for rare words. |
Learning VAE-LDA Models with Rounded Reparameterization Trick (2020.emnlp-main)
Copied to clipboard
| Challenge: | No reparameterization form of Dirichlet distributions is known to date for topic models . |
| Approach: | They propose a method to reparameterize Dirichlet distributions for the learning of VAE-LDA models by using a latent Dirichlets prior. |
| Outcome: | The proposed method outperforms existing neural topic models on benchmark datasets and on a synthetic dataset. |
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models suffer from severe miscalibration for both in-distribution and out-of-difference data due to over-parameterization. |
| Approach: | They propose a regularized method to improve in-distribution and out-of-distance calibrations by using on-manifold regularization and off-manfold regularisation. |
| Outcome: | The proposed method outperforms existing methods for text classification in terms of expectation calibration error, misclassification detection, and OOD detection on six datasets. |
Scaling Hidden Markov Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Hidden Markov models are a fundamental tool for sequence modeling that separates the hidden state from the emission structure. |
| Approach: | They propose methods for scaling hidden Markov models to massive state spaces while maintaining efficient exact inference and effective regularization. |
| Outcome: | The proposed methods are much more accurate than previous HMMs and n-gram-based methods, making progress towards the performance of state-of-the-art NN models. |
Coding Textual Inputs Boosts the Accuracy of Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new approach to natural language processing uses arbitrary symbols to represent meaning . Soundex, MetaPhone, NYSIIS, logogram are used as inputs for NLP . |
| Approach: | They propose to use arbitrary symbols to represent linguistic meaning of a word . they propose to integrate codewords with text to provide more reliable inputs . |
| Outcome: | The proposed approach outperforms state-of-the-art models on machine translation, language modeling, and part-of speech tagging. |
Learning from Task Descriptions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in supervised NLP has shown significant progress in learning tasks from examples. |
| Approach: | They propose a framework for developing NLP systems that solve new tasks after reading their descriptions, synthesizing prior work in this area. |
| Outcome: | The proposed model achieves 12% on the new dataset, leaving a significant challenge for NLP researchers. |
Hashtags, Emotions, and Comments: A Large-Scale Dataset to Understand Fine-Grained Social Emotions to Online Topics (2020.emnlp-main)
Copied to clipboard
| Challenge: | A large-scale dataset is collected from Chinese microblog Sina Weibo with over 13 thousand trending topics, emotion votes in 24 fine-grained types from massive participants, and user comments to allow context understanding. |
| Approach: | They use a large-scale dataset from Chinese microblog Sina Weibo to examine readers' responses to online discussion topics. |
| Outcome: | The proposed model outperforms the human model in predicting social emotions in a multilabel classification setting. |
Named Entity Recognition for Social Media Texts with Semantic Augmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for named entity recognition suffer from data sparsity problems when conducted on short and informal texts. |
| Approach: | They propose a neural-based approach to named entity recognition for social media texts . they obtain augmented semantic information from a large-scale corpus and encode it . |
| Outcome: | The proposed approach outperforms existing approaches on three social media datasets. |
Coupled Hierarchical Transformer for Stance-Aware Rumor Verification in Social Media Conversations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to rumor verification and stance classification fail to exploit intertask dependencies . |
| Approach: | They propose a Hierarchical Transformer model which uses BERT to obtain thread representations . they propose 'coupled' transformer modules to capture intertask interactions and a post-level attention layer to use predicted stance labels for RV. |
| Outcome: | The proposed model outperforms existing methods on two benchmark datasets. |
Social Media Attributions in the Context of Water Crisis (2020.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we analyze social media discussions to identify attribution factors for natural disasters/collective misfortunes. |
| Approach: | They propose a task of attribution tie detection to identify factors held responsible for a water crisis in a social media document. |
| Outcome: | The proposed task can be performed on a dataset constructed from YouTube comments on 2,500 videos relevant to the 2019 Chennai water crisis. |
On the Reliability and Validity of Detecting Approval of Political Actors in Tweets (2020.emnlp-main)
Copied to clipboard
| Challenge: | Social media sites have the potential to complement surveys that measure political opinions and, more specifically, political actors’ approval. |
| Approach: | They propose to compare untargeted sentiment, targeted sentiment, and stance detection methods to a set of custom models trained on minimal custom data. |
| Outcome: | The proposed methods have low generalizability on unseen and familiar targets, while low-resource custom models are more robust. |
Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | MRC has achieved significant progress on the open domain in recent years due to large-scale pre-trained language models. |
| Approach: | They propose a machine reading comprehension model which exploits structural medical knowledge and reference medical plain text to improve the exam's accuracy. |
| Outcome: | The proposed model outperforms existing models with a large margin and passes the exam with 61.8% accuracy rate on the test set. |
Generating Radiology Reports via Memory-driven Transformer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Medical imaging reports are time-consuming and can be error-prone for inexperienced radiologists. |
| Approach: | They propose to generate radiology reports with memory-driven Transformer using relational memory and memory-based conditional layer normalization. |
| Outcome: | The proposed method outperforms existing models on IU X-Ray and MIMIC-CXR . it generates long reports with medical terms and meaningful image-text attention mappings . |
Planning and Generating Natural and Diverse Disfluent Texts as Augmentation for Disfluency Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to disfluency detection heavily depend on labeled data. |
| Approach: | They propose a Planner-Generator based disfluency generation model that generates natural disfluent texts as augmented data. |
| Outcome: | The proposed model outperforms baselines and leads to state-of-the-art performance on Switchboard corpus. |
Predicting Clinical Trial Results by Implicit Evidence Integration (2020.emnlp-main)
Copied to clipboard
| Challenge: | Clinical trials are expensive and time-consuming, and inappropriately designed studies can be devastating in a pandemic. |
| Approach: | They propose a model that takes a PICO-formatted clinical trial proposal and predicts the outcome from it. |
| Outcome: | The proposed model outperforms baseline models on a benchmark dataset with 10.7% relative gain over BioBERT. |
Explainable Clinical Decision Support from Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | Clinical prediction models often use structured variables and provide outcomes that are not readily interpretable by clinicians. |
| Approach: | They propose a hierarchical CNN-transformer model with explicit attention as an interpretable, multi-task clinical language model. |
| Outcome: | The proposed model achieves AUROCs of 0.75 and 0.78 on sepsis and mortality prediction. |
A Knowledge-driven Generative Model for Multi-implication Chinese Medical Procedure Entity Normalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Medical entity normalization (NEN) is a task that links medical mentions to entities in knowledge bases. |
| Approach: | They propose a sequence generative framework to generate Chinese medical procedure entity normalization by constraint decoding and category-based model refining. |
| Outcome: | The proposed model improves on baselines especially in the case of multi-implication Chinese medical procedures. |
Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to label radiology text reports rely on feature engineering based on medical domain knowledge or manual annotations by experts. |
| Approach: | They propose a BERT-based approach to medical image report labeling that exploits the scale of available rule-based systems and the quality of expert annotations. |
| Outcome: | The proposed model outperforms the previous best rules-based labeler with statistical significance on one of the largest datasets of chest x-rays. |
Benchmarking Meaning Representations in Neural Semantic Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on meaning representations is not comprehensively evaluated due to the lack of readily-available execution engines. |
| Approach: | They propose a unified benchmark on meaning representations by integrating existing semantic parsing datasets, completing the missing logical forms, and implementing the missing execution engines. |
| Outcome: | The proposed benchmark combines existing parsing datasets, completes missing logical forms, and implements missing execution engines. |
Analogous Process Structure Induction for Sub-event Sequence Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on event understanding is focusing on procedural (or horizontal) tasks such as predicting the next event given an observed sequence. |
| Approach: | They propose an Analogous Process Structure Induction framework which leverages analogies among processes and conceptualization of sub-event instances to predict the whole sub- sequence of previously unseen open-domain processes. |
| Outcome: | The proposed framework can predict the whole sub-event sequence of previously unseen open-domain processes. |
SLM: Learning a Discourse Language Representation with Sentence Unshuffling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent models for learning discourse language representations focus on bottom or top-level representations, but they do not capture intermediate-size structures in natural languages such as sentences and the relationships among them. |
| Approach: | They propose a new objective for learning a discourse language representation in a self-supervised manner by shuffling the sequence of input sentences and training a hierarchical transformer model to reconstruct the original ordering. |
| Outcome: | The proposed model improves the original BERT model on downstream tasks by large margins. |
Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank (2020.emnlp-main)
Copied to clipboard
| Challenge: | Detecting fine-grained differences in content conveyed in different languages is expensive and hard to scale. |
| Approach: | They propose a training strategy for multilingual BERT models by learning to rank divergent examples of varying granularity. |
| Outcome: | The proposed model improves the prediction and annotation of fine-grained semantic divergences. |
A Bilingual Generative Transformer for Semantic Sentence Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Semantic sentence embedding models encode natural language sentences into vectors, such that closeness in embeddable space indicates closeness of semantics between the sentences. |
| Approach: | They propose a deep latent variable model that attempts to perform source separation on parallel sentences, isolating what they have in common in a latent semantic vector, and explaining what is left over with language-specific latent vectors. |
| Outcome: | The proposed model outperforms the state-of-the-art on a standard suite of unsupervised semantic similarity evaluations. |
Semantically Inspired AMR Alignment for the Portuguese Language (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) parsers require alignment between nodes and words of the sentence. |
| Approach: | They propose to use a more semantically matched word-concept pair to align graphs with words in Portuguese . they performed intrinsic and extrinsic evaluations and found it outperforms the English alignment strategies. |
| Outcome: | The proposed method outperforms the existing methods for English and achieves competitive results with a parser designed for the Portuguese language. |
An Unsupervised Sentence Embedding Method by Mutual Information Maximization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Sentence BERT is inefficient for sentence-pair tasks as it needs to evaluate combinatorially many sentence pairs which is very time-consuming. |
| Approach: | They propose a lightweight extension on top of BERT and a self-supervised learning objective to derive meaningful sentence embeddings in an unsupervised manner. |
| Outcome: | The proposed method outperforms baselines on common semantic textual similarity tasks and downstream supervised tasks and achieves performance competitive with supervised methods on various tasks. |
Compositional Phrase Alignment and Beyond (2020.emnlp-main)
Copied to clipboard
| Challenge: | Phrase alignment is the basis for sentence pair interactions, such as paraphrase identification and textual entailment recognition. |
| Approach: | They propose a phrase alignment model that embeds similarity distributions into powerful contextualized representations that can be used to model sentence pair interactions. |
| Outcome: | The proposed method significantly outperforms that used in a previous study and achieves a performance competitive with that of experienced human annotators. |
Table Fact Verification with Structure-Aware Transformer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained models cannot be used to encode semi-structured data because of their nature. |
| Approach: | They propose a Structure-Aware Transformer which injects table structural information into mask . method could combine symbolic and linguistic reasoning, they propose . |
| Outcome: | The proposed method outperforms baseline on a large scale table verification dataset. |
Double Graph Based Reasoning for Document-level Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction fail to recognize relations between entities across sentences. |
| Approach: | They propose a method to recognize relations for long paragraphs by a Graph Aggregation-and-Inference Network (GAIN) they propose to use a heterogeneous mention-level graph and an entity-level EG graph to analyze the relationships. |
| Outcome: | The proposed method achieves a significant performance improvement (2.85 on F1) over the previous state-of-the-art. |
Event Extraction as Machine Reading Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Event extraction (EE) is a crucial information extraction task that aims to extract event information in texts. |
| Approach: | They propose a new learning paradigm for event extraction by explicitly casting it as a machine reading comprehension problem. |
| Outcome: | The proposed model achieves state-of-the-art performance on the data-scarce scenario, achieving 49.8% in F1 for event argument extraction with only 1% data, compared with 2.2% of the previous method. |
MAVEN: A Massive General Domain Event Detection Dataset (2020.emnlp-main)
Copied to clipboard
Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, Jie Zhou
| Challenge: | Existing datasets exhibit data scarcity and limited coverage of general-domain events. |
| Approach: | They present a MAssive eVENt detection dataset which contains 4,480 Wikipedia documents and 168 event types. |
| Outcome: | The proposed dataset shows that existing methods cannot achieve promising results on the small datasets. |
Knowledge Graph Alignment with Entity-Pair Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for Knowledge Graph (KG) alignment are not satisfactory. |
| Approach: | They propose a method that directly learns embeddings of entity-pairs for KG alignment. |
| Outcome: | The proposed approach can achieve state-of-the-art on five real-world datasets. |
Adaptive Attentional Network for Few-Shot Knowledge Graph Completion (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent attempts to learn static representations of entities and references ignore their dynamic properties. |
| Approach: | They propose to learn static representations of entities and references ignoring their dynamic properties . a neighbor encoder learns entities' roles while a query-aware aggregator learns references' contributions . |
| Outcome: | The proposed approach achieves state-of-the-art results with different few-shot sizes. |
Pre-training Entity Relation Encoder with Intra-span and Inter-span Information (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models do not handle text spans and relation among text span pairs. |
| Approach: | They propose to integrate span-related information into pre-trained encoder for entity relation extraction task. |
| Outcome: | The proposed pre-training method outperforms distantly supervised pre-trained models on two entity relation extraction benchmark datasets. |
Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition and relation extraction are two important fundamental problems. |
| Approach: | They propose to design two separate encoders to capture two different types of information in the representation learning process. |
| Outcome: | The proposed methods show significant improvements on standard datasets. |
Beyond [CLS] through Ranking by Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on generative ranking models for Information Retrieval has focused on discriminative methods that learn a similarity function to compare questions and candidates answers. |
| Approach: | They propose to use a language model to train a ranking function that model the semantic similarity of documents and queries instead of discriminative ranking functions. |
| Outcome: | The proposed approaches are as effective as state-of-the-art discriminative models for the answer selection task and show unlikelihood losses are reduced for IR. |
Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too! (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing topic models rely on probabilistic models to uncover themes within document collections, but are they the only option? |
| Approach: | They propose a way to cluster pre-trained word embeddings while incorporating document information for weighted clustering and reranking top words. |
| Outcome: | The proposed approach performs as well as classical topic models, but with lower runtime and computational complexity. |
Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on single-document summarization (SDS) benefit from advances in neural sequence learning, but they produce unsatisfactory results on multi-document summary (MDS). |
| Approach: | They propose a neural sequence learning method that unifies advanced neural SDS methods and statistical measures used in classical MDS. |
| Outcome: | The proposed method achieves state-of-the-art performance on benchmark MDS datasets. |
Improving Neural Topic Models using Knowledge Distillation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current paradigms for transfer learning use general knowledge as a foundation for more specialized endeavors. |
| Approach: | They propose to combine probabilistic topic models and pretrained transformers to improve topic quality by using knowledge distillation. |
| Outcome: | The proposed framework improves topic quality over all estimated topics and in head-to-head comparisons of aligned topics. |
Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder (2020.emnlp-main)
Copied to clipboard
| Challenge: | Topic models for short texts suffer from data sparsity because of limited word co-occurrences. |
| Approach: | They propose a neural topic model with a new topic distribution quantization approach that generates peakier distributions that are more appropriate for modeling short texts. |
| Outcome: | The proposed model outperforms both strong traditional and neural baselines under extreme data sparsity scenes, producing high-quality topics. |
Querying Across Genres for Medical Claims in News (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using cross-genre query-based biomedical information retrieval, we find the research publication that supports the primary claim made in a news article. |
| Approach: | They propose a query-based biomedical information retrieval task where the goal is to find the research publication that supports the primary claim made in a news article. |
| Outcome: | The proposed approach compares classical IR with more recent transformer-based models and shows that it is feasible but requires domain-specific knowledge. |
Incorporating Multimodal Information in Open-Domain Web Keyphrase Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Open-domain Keyphrase extraction (KPE) is a fundamental yet complex NLP task . effective designs encode within layout and formatting signals that point to where the important information can be found. |
| Approach: | They propose a multi-modal approach to open-domain keyphrase extraction (KPE) on the Web that leverages layout and formatting signals to aid in the task. |
| Outcome: | The proposed model outperforms state-of-the-art models on the open-domain keyphrase extraction task. |
CMU-MOSEAS: A Multimodal Language Dataset for Spanish, Portuguese, German and French (2020.emnlp-main)
Copied to clipboard
AmirAli Bagher Zadeh, Yansheng Cao, Simon Hessner, Paul Pu Liang, Soujanya Poria, Louis-Philippe Morency
| Challenge: | Existing datasets in multimodal language are limited and disproportionately affect native speakers of other languages . authors propose a large-scale dataset for Spanish, Portuguese, German and French . |
| Approach: | They propose a large-scale multimodal language dataset for Spanish, Portuguese, German and French. |
| Outcome: | The proposed dataset is the largest of its kind with 40,000 total labelled sentences . it covers a diverse set topics and speakers and carries supervision of 20 labels including sentiment, emotions, and attributes. |
Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to disfluency detection rely on human annotations, which are expensive to obtain. |
| Approach: | They propose an unsupervised learning paradigm which can work with unlabeled text corpora. |
| Outcome: | The proposed method performs better than existing supervised systems using word embeddings. |
Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent multimodal learning models with strong performances on human-centric tasks are often black-box with very limited interpretability. |
| Approach: | They propose a multimodal routing algorithm which dynamically adjusts weights between input and output modalities for each input sample. |
| Outcome: | The proposed model can interpret modality-prediction relationships globally and locally for each input sample while keeping competitive performance compared to state-of-the-art methods. |
Multistage Fusion with Forget Gate for Multimodal Summarization in Open-Domain Videos (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for multimodal summarization for open-domain videos lack fine-grained interactions between multisource inputs. |
| Approach: | They propose a multistage fusion network with a forget gate module to integrate multimodal information into a fluent textual summary. |
| Outcome: | The proposed model achieves state-of-the-art on multiple encoder-decoder architectures and low noise transcripts. |
BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to video-grounded dialogues focus on superficial temporal-level visual cues, but neglect more fine-grained spatial signals from videos. |
| Approach: | They propose a vision-language neural framework for high-resolution queries in videos based on textual cues that exploits both spatial and temporal-level information. |
| Outcome: | The proposed approach outperforms previous approaches on the TGIF-QA benchmark and significantly outperformed previous approaches. |
UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to training dialogue agents separately are not optimized for multi-domain task-oriented dialogues. |
| Approach: | They propose a unified neural architecture for end-to-end conversational systems in multi-domain task-oriented dialogues that jointly trains a bi-level state tracker and a joint dialogue act and response generator. |
| Outcome: | The proposed system outperforms existing systems on the MultiWOZ2.1 benchmark in dialogue state tracking, context-to-text, and end-to end settings. |
GraphDialog: Integrating Graph Knowledge into End-to-End Task-Oriented Dialogue Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | End-to-end task-oriented dialogue systems aim to generate system responses directly from plain text inputs. |
| Approach: | They propose a recurrent cell architecture which exploits the structural information in dialogue history . they propose recursive cell architecture to allow representation learning on graphs . |
| Outcome: | The proposed model improves on two different datasets on task-oriented dialogues. |
Structured Attention for Unsupervised Dialogue Structure Induction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using structured attention, a model can learn dialogue structure in unsupervised fashion. |
| Approach: | They propose to incorporate structured attention layers into a Variational Recurrent Neural Network model with discrete latent states to learn dialogue structure in an unsupervised fashion. |
| Outcome: | The proposed model learns semantic structures similar to templates used to generate a dialogue corpus on two-party datasets and on multi-party dialogues, disentangling dialogues without human annotation. |
Cross Copy Network for Dialogue Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite the success of sequence-to-sequence models, dialogue logics are often ignored. |
| Approach: | They propose a network architecture to explore the current dialog context and similar dialogue instances’ logical structure simultaneously. |
| Outcome: | The proposed network architecture is superior to existing state-of-the-art models. |
Multi-turn Response Selection using Dialogue Dependency Relations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for multi-turn response selection ignore the dependencies between the turns. |
| Approach: | They propose a dialogue extraction algorithm to transform a dialog history into threads based on their dependency relations. |
| Outcome: | The proposed model outperforms the state-of-the-art models on DSTC7 and DSTF8* with competitive results on UbuntuV2 . |
Parallel Interactive Networks for Multi-Domain Dialogue State Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models do not account for the dependencies between system and user utterances in the same turn and across different turns. |
| Approach: | They propose to integrate an interactive encoder to jointly model in-turn dependencies and cross-turn dependents. |
| Outcome: | The proposed model is superior to existing models and can be used to selectively copy words from historical system utterances or historical user utterrances. |
SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Slot filling and intent detection are two main tasks in spoken language understanding systems. |
| Approach: | They propose a non-autoregressive slot filling model with two-pass iteration mechanism to handle uncoordinated slots problem. |
| Outcome: | The proposed model significantly outperforms previous models in slot filling task while speeding up decoding. |
An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to condition models on a concise rationale are less accurate than models that can use the entire context. |
| Approach: | They propose a method to optimize a bound on the Information Bottleneck objective to extract concise rationales from a binary mask and an end-task predictor that uses only the residual sentences. |
| Outcome: | The proposed model outperforms existing norm-minimization techniques in task performance and agreement with human rationales in the ERASER benchmark. |
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models use cultural biases implicitly, causing harm . identifying and quantifying learnt biase enables us to measure progress . |
| Approach: | They propose a benchmark to measure social bias in pretrained language models . they use 1508 examples that cover stereotypes dealing with nine types of bias . |
| Outcome: | The proposed benchmark focuses on stereotypes about historically disadvantaged groups and contrasts them with advantaged groups. |
LOGAN: Local Group Bias Detection by Clustering (2020.emnlp-main)
Copied to clipboard
| Challenge: | a number of machine learning models inherit and amplify the societal biases in data. |
| Approach: | a new bias detection technique based on clustering is proposed to detect local biases in data . authors propose to use LOGAN to analyze local bias in data. |
| Outcome: | The proposed technique detects bias in a local region and allows better analysis of model predictions. |
RNNs can generate bounded hierarchical languages with optimal memory (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that RNNs can efficiently generate bounded hierarchical languages with high syntactic fidelity, but their success is not well-understood theoretically. |
| Approach: | They propose a language of well-nested brackets and m-bounded nesting depth . they prove that an RNN with O(m log k) hidden units suffices, an exponential reduction in memory, by an explicit construction. |
| Outcome: | The proposed language is well-nested brackets and has m-bounded nesting depth . it shows that an RNN with O(m log k) hidden units suffices, an exponential reduction in memory, by an explicit construction. |
Detecting Independent Pronoun Bias with Partially-Synthetic Data Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | linguistic differences between English pronouns that are not inherently biased can become biases in some machine learning models. |
| Approach: | They propose a method to detect bias by alternating pronouns in different contexts. |
| Outcome: | The proposed method can be used to detect bias in language models and for text generation more broadly. |
Visually Grounded Continual Learning of Compositional Phrases (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern NLP systems rely on offline training and are inefficient for new tasks. |
| Approach: | They propose a visually grounded ContinuaL learning task which simulates the continual acquisition of compositional phrases from streaming visual scenes. |
| Outcome: | The proposed system improves on existing systems, but it's infeasible to store all possible compositions. |
MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on phrase localization uses caption-image datasets as weak supervision . existing work on supervised phrase localisation uses a large-scale annotated dataset . |
| Approach: | They develop a multimodal alignment framework to leverage more widely available caption-image datasets to model phrase relevance. |
| Outcome: | The proposed model improves on the widely-adopted Flickr30k dataset . it also improves the previous best unsupervised result by 5.56% . |
Domain-Specific Lexical Grounding in Noisy Visual-Textual Documents (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing image-text grounding approaches require detailed annotations, authors say . existing methods are difficult to adapt to unlabeled multi-image, multi-sentence documents, they say . |
| Approach: | They propose a method that can learn contextual meanings from unlabeled documents . they demonstrate that a simple unsupervised clustering-based method can be useful . |
| Outcome: | The proposed method is particularly effective for local contextual meanings of a word . existing image-text grounding methods are difficult to adapt to unlabeled multi-image, multi-sentence documents . |
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | HERO is a framework for large-scale video+language omni-representation learning. |
| Approach: | They propose a framework for large-scale video+language omni-representation learning that encodes multimodal inputs in a hierarchical structure and uses Masked Language Modeling and Masked Frame Modeling to train models. |
| Outcome: | The proposed framework achieves state-of-the-art on multiple benchmarks over text-based video/video-moment retrieval, video question answering (QA), Video-and-language Inference and video Captioning tasks across different domains. |
Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing language pretraining frameworks only take the language context as selfsupervision . current frameworks do not take grounding information from the external visual world . |
| Approach: | They propose a visually-supervised language model that extrapolates multimodal alignments to language-only data by contextually mapping language tokens to related images. |
| Outcome: | The proposed model improves on multiple pure-language tasks. |
Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to defend against fake news are limited to text and metadata . authors identify weaknesses that adversaries can exploit by manipulating such technology . |
| Approach: | They propose a more realistic defense mechanism to defend against machine-generated news . they use a NeuralNews dataset to identify weaknesses that adversaries can exploit . |
| Outcome: | The proposed approach detects visual-semantic inconsistencies and provides a useful first line of defense against machine-generated disinformation. |
Enhancing Aspect Term Extraction with Soft Prototypes (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on designing neural sequence taggers to extract linguistic features from token level. |
| Approach: | They propose to correlating aspects with each other through soft prototypes . they propose to combine ATE with almost all sequence taggers to extract aspect terms . |
| Outcome: | The proposed model boosts the performance of three typical ATE methods on four SemEval datasets. |
FedED: Federated Learning via Ensemble Distillation for Medical Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing relation extraction methods require centralizing training data from different medical platforms while holding the privacy-sensitive data puts patients' privacy at risk. |
| Approach: | They propose a federated relation extraction model that trains a central model without sharing or exchange of private local data. |
| Outcome: | The proposed model trains a central model without uploading local parameters, and it performs well on three publicly available datasets. |
Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product (2020.emnlp-main)
Copied to clipboard
| Challenge: | In the real world, product attribute values are incomplete and vary over time, which hinders practical applications. |
| Approach: | They propose a multimodal method to jointly predict product attributes and extract values from product images using multimodal product information. |
| Outcome: | The proposed method can predict product attributes and extract values from product images with the help of product images. |
A Predicate-Function-Argument Annotation of Natural Language for Open-Domain Information eXpression (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing OIE (Open Information Extraction) algorithms are redundant and not reusable. |
| Approach: | They propose a pipeline where an Open-domain Information eXpression task provides a platform for all OIE strategies. |
| Outcome: | The proposed pipeline provides a platform for all OIE strategies. |
Retrofitting Structure-aware Transformer Language Model for End Tasks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that structure-aware Transformer language model achieves improved perplexity, meanwhile inducing accurate syntactic phrases. |
| Approach: | They propose to exploit syntactic distance to encode phrasal constituency and dependency connection into Transformer language model and leverage it for structure integration. |
| Outcome: | The proposed model achieves significant improvements for both semantic- and syntactic-dependent tasks. |
Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | AMR-to-text generation is used to transduce Abstract Meaning Representation structures (AMRs) Graph Convolution Networks (GCNs) are not able to capture non-local information and follow a local (first-order) information aggregation scheme. |
| Approach: | They propose a dynamic fusion mechanism that captures richer non-local interactions . they propose weight tied convolutions and group graph convolution to reduce memory usage . |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets with significantly fewer parameters while maintaining the model capacity. |
If beam search is the answer, what was the question? (2020.emnlp-main)
Copied to clipboard
| Challenge: | surprisingly, beam search results on language generation tasks are low-quality . despite its high error rate, beam searches can be used to decode models with high probability . |
| Approach: | They frame beam search as the exact solution to a different decoding objective . they propose a set of decoding objectives that explicitly enforce this property . |
| Outcome: | The proposed method enforces uniform information density in text, a property motivated by cognitive science. |
Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Latent structure models can mitigate the error propagation and annotation bottleneck in pipeline systems, while uncovering linguistic insights about the data. |
| Approach: | They propose a latent structure model with a pullback of the downstream learning objective. |
| Outcome: | The proposed model outperforms the known and proposed model in the same family and yields new insights for practitioners and revealing intriguing failure cases. |
Is the Best Better? Bayesian Statistical Model Comparison for Natural Language Processing (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study raises concerns about the use of standard splits to compare models . we compare the performance of six English part-of-speech taggers to those of other models based on standard split analysis . |
| Approach: | They propose a Bayesian statistical model comparison technique using k-fold cross-validation . they rank six English part-of-speech taggers across two data sets and three evaluation metrics . |
| Outcome: | The proposed method ranks English part-of-speech taggers on two data sets and three evaluation metrics. |
Exploring Logically Dependent Multi-task Learning with Causal Inference (2020.emnlp-main)
Copied to clipboard
| Challenge: | Hierarchical multi-task learning models can utilize task dependencies by stacking encoders and outperform democratic ones. |
| Approach: | They propose a model that utilizes the labels of all lower-level tasks and a Gumbel sampling model to deal with cascading errors. |
| Outcome: | The proposed model outperforms democratic models on six out of seven subtasks and achieves state-of-the-art on the two English and one Chinese datasets. |
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Extensive evaluations of masking BERT, RoBERTa, and DistilBERT on eleven diverse NLP tasks show that our binary masked language models encode information necessary for solving downstream tasks. |
| Approach: | They propose an efficient method of utilizing pretrained language models where selective binary masks are learned instead of finetuning. |
| Outcome: | Extensive evaluations of masking BERT, RoBERTa, and DistilBERT on eleven diverse NLP tasks show that the proposed method yields comparable performance to finetuning, but has a much smaller memory footprint when multiple tasks need to be solved. |
Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing document-level neural machine translation methods use all context sentences in a fixed scope. |
| Approach: | They propose an approach to select dynamic context so that document-level neural machine translation models can utilize more useful selected context sentences. |
| Outcome: | The proposed approach can select adaptive context sentences for different source sentences and significantly improves translation quality over sentences in a document. |
Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale training datasets make training neural machine translation models difficult. |
| Approach: | They propose to identify inactive training examples which contribute less to the model performance and introduce data rejuvenation to improve NMT models' training. |
| Outcome: | The proposed framework stabilizes and accelerates the training process of NMT models, resulting in models with better generalization capability. |
Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses (2020.emnlp-main)
Copied to clipboard
| Challenge: | Popular machine translation model training uses backtranslation to improve BLEU scores . we use generative-discriminative hybrid losses to fine-tune a trained model . |
| Approach: | They propose a class of conditional generative-discriminative hybrid losses to fine-tune a machine translation model. |
| Outcome: | The proposed model improves on a sentence-level and contextual model without additional data. |
Learning Adaptive Segmentation Policy for Simultaneous Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that adaptive segmentation policies for simultaneous translation are more accurate than current methods . if translation starts before adequate source content is delivered, the quality of translation degrades . waiting for too much source text increases latency, which would hurt accuracy . |
| Approach: | They propose a new adaptive segmentation policy for simultaneous translation based on human interpreters . it learns to segment the source text by considering possible translations produced by the translation model . |
| Outcome: | Experimental results show that the proposed method achieves better accuracy-latency trade-off over state-of-the-art methods. |
Learn to Cross-lingual Transfer with Meta Graph Learning Across Heterogeneous Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing mPLM-based methods focus on designing costly model pre-training while ignoring equally crucial downstream adaptation. |
| Approach: | They propose a meta graph learning method that extracts meta-knowledge from historical CLT experiences to learn to cross-lingual transfer. |
| Outcome: | The proposed method can learn to cross-lingual transfer by extracting meta-knowledge from historical CLT experiences (tasks) it can also capture intrinsic language relationships to explicitly guide cross-linguistic transfer. |
UDapter: Language Adaptation for Truly Universal Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cross-language interference and restrained model capacity remain major obstacles in multilingual dependency parsing. |
| Approach: | They propose a multilingual task adaptation approach based on contextual parameter generation and adapter modules that learn adapters via language embeddings while sharing model parameters across languages. |
| Outcome: | The proposed approach outperforms strong monolingual and multilingual baselines on most languages on high-resource and low-resourced (zero-shot) languages. |
Uncertainty-Aware Label Refinement for Sequence Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Conditional random fields (CRF) for label decoding have been a problem for many tasks. |
| Approach: | They propose a two-stage label decoding framework that model long-term label dependencies while being much more computationally efficient. |
| Outcome: | The proposed method outperforms the CRF-based methods and greatly accelerates the inference process. |
Adversarial Attack and Defense of Structured Prediction Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to building effective adversarial attackers focus on classification problems. |
| Approach: | They propose a framework that learns to attack a structured prediction model with feedbacks from multiple reference models. |
| Outcome: | The proposed framework is able to attack state-of-the-art models and boost them with training . it is based on a sequence-to-sequence model with feedbacks from multiple reference models . |
Position-Aware Tagging for Aspect Sentiment Triplet Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research efforts focus on extracting the triplets of target entities, their associated sentiment, and opinion spans explaining the reason for the sentiment. |
| Approach: | They propose a position-aware tagging scheme that can extract triplets using a sequence tapping approach. |
| Outcome: | The proposed model improves performance on multiple datasets and compares with existing models. |
Simultaneous Machine Translation with Visual Context (2020.emnlp-main)
Copied to clipboard
| Challenge: | Simultaneous machine translation (SiMT) aims to reproduce human interpretation, where an interpreter translates spoken utterances as they are produced. |
| Approach: | They propose to add visual context to siMT to compensate for the missing source context . they show visual-grounded models are much better than commonly used global features . |
| Outcome: | The proposed models reach up to 3 BLEU points improvement under low latency scenarios. |
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | XCOPA dataset provides a typologically diverse dataset for commonsense reasoning in 11 languages . current methods for evaluating commonsensible reasoning in resource-poor languages are weak compared to translation-based transfer. |
| Approach: | They propose a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages. |
| Outcome: | The proposed model performs better than current methods on a resource-poor dataset compared to translation-based transfer in the 11 languages studied . |
The Secret is in the Spectra: Predicting Cross-lingual Task Performance with Spectral Similarity Measures (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have suggested that bilingual lexicon induction is influenced by the (dis)similarity of the languages at hand. |
| Approach: | They propose to measure the isomorphism of monolingual embedding spaces based on their spectra and introduce isometric measures to measure their similarity. |
| Outcome: | The proposed measures outperform standard isomorphism measures while being more tractable and easier to interpret. |
Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies consider linguistic typology as a potential source of knowledge to support multilingual natural language processing (NLP) tasks. |
| Approach: | They propose to fuse both views using canonical correlation analysis and use it to infer typological features and language phylogenies to construct a multi-view language vector space for multilingual machine translation. |
| Outcome: | The proposed model achieves competitive translation accuracy in multilingual machine translation tasks without expensive retraining of massive multilingual or ranking models. |
AnswerFact: Fact Checking in Product Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | a product-related community question answering platform is widely employed in many E-commerce sites . however, the misinformation in the answers on those platforms poses unprecedented challenges for users to obtain reliable and truthful product information. |
| Approach: | They propose a large scale fact checking dataset from product question answering forums to predict the answer veracity . each answer is accompanied by its veraity label and associated evidence sentences . |
| Outcome: | The proposed model outperforms baselines on the question veracity prediction task. |
Context-Aware Answer Extraction in Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Extractive QA models have shown promising performance in predicting the correct answer to a given question. |
| Approach: | They propose a BLANC-based context prediction task that learns the context prediction tasks. |
| Outcome: | The proposed model outperforms the state-of-the-art models on reading comprehension and hotpotQA. |
What do Models Learn from Question Answering Datasets? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models have outperformed humans on question answering datasets, but they have yet to outperform humans on the task of question answering itself. |
| Approach: | They evaluate BERT-based question answering models on their generalizability to out-of-domain examples, responses to missing or incorrect data, and ability to handle question variations. |
| Outcome: | The proposed models outperform human baselines on the widely-used SQuAD 1.1 and SQu AD 2.0 datasets. |
Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading (2020.emnlp-main)
Copied to clipboard
Yifan Gao, Chien-Sheng Wu, Jingjing Li, Shafiq Joty, Steven C.H. Hoi, Caiming Xiong, Irwin King, Michael Lyu
| Challenge: | Document interpretation and dialog understanding are the two major challenges for conversational machine reading. |
| Approach: | They propose a discourse-aware entailment reasoning network to strengthen the connection and enhance the understanding of document and dialog. |
| Outcome: | The proposed model improves document interpretation and dialog understanding on the ShARC benchmark. |
A Method for Building a Commonsense Inference Dataset based on Basic Events (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to acquire commonsense are limited by the general-purpose language models. |
| Approach: | They propose a method for building a commonsense inference dataset using crowdsourcing and automatic extraction from a corpus. |
| Outcome: | The proposed method can solve 104k commonsense inference problems in a Japanese corpus with high accuracy, but low bias. |
Neural Deepfake Detection with Factual Structure of Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to deepfake detection typically represent documents with coarse-grained representations, but they struggle to capture factual structures of documents. |
| Approach: | They propose a graph-based model that captures factual structures of documents for deepfake detection. |
| Outcome: | The proposed model improves strong base models built with RoBERTa on two public deepfake datasets. |
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines the zero-shot transfer capabilities of text matching models on a massive scale. |
| Approach: | They propose to integrate self-supervised with supervised multi-task learning on all available source domains to study the zero-shot transfer capabilities of text matching models on a massive scale. |
| Outcome: | The proposed model outperforms in-domain BERT and the previous state of the art on six benchmarks. |
XL-AMR: Enabling Cross-Lingual AMR Parsing with Transfer Learning Techniques (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a popular formalism of natural language. |
| Approach: | They develop a cross-lingual AMR parser that can be trained on the produced data . they use transfer learning techniques to produce automatic AMR annotations across languages . |
| Outcome: | The proposed parser significantly surpasses those reported in Chinese, German, Italian and Spanish. |
Improving AMR Parsing with Sequence-to-Sequence Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstract meaning representation (AMR) parsing is limited by the size of curated datasets. |
| Approach: | They propose a seq2seq pre-training approach to build pre-trained models on three relevant tasks. |
| Outcome: | The proposed model improves performance on three relevant tasks while maintaining the response of pre-trained models. |
Hate-Speech and Offensive Language Detection in Roman Urdu (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research on hate-speech and offensive language detection in social media content is mainly focused on the English language. |
| Approach: | They propose to use an annotated dataset to detect hate-speech and offensive language in social media content . they propose to transfer five existing embedding models to Roman Urdu to test their performance . |
| Outcome: | The proposed model outperforms existing methods on RUHSOLD dataset and train domain-specific embeddings on more than 4.7 million tweets. |
Suicidal Risk Detection for Military Personnel (2020.emnlp-main)
Copied to clipboard
| Challenge: | a dataset of 2,791 posts with 13,955 expert annotations of suicidal risk levels is available for research . Suicide is one of the major causes of death in the military. |
| Approach: | They analyze posts related to military service in the Republic of Korea and annotate them with military experts and mental health experts. |
| Outcome: | The proposed method predicts the level of suicide risk, reaching .88 F1 for classifying the risks. |
Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has shown that data collection is neglected by ignoring the quality of data. |
| Approach: | They propose to use latent semantics to evaluate selection bias in hate speech . they compare latent Dirichlet Allocation (LDA) to eleven hate speech corpora . |
| Outcome: | The proposed method could be revisable before focusing on classification performance. |
HENIN: Learning Heterogeneous Neural Interaction Networks for Explainable Cyberbullying Detection on Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting cyberbullying rely on text analysis of social media sessions. |
| Approach: | They propose a deep model that uses a comment encoder and a post-comment co-attention sub-network to explain why a media session is identified as cyberbullying. |
| Outcome: | The proposed model outperforms existing models on real datasets and shows evidential comments in the model explainability of cyberbullying detection. |
Reactive Supervision: A New Method for Collecting Sarcasm Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | sarcasm detection requires large amounts of labeled data, with a high cost and noisy labels. |
| Approach: | They propose a method that uses the dynamics of online conversations to collect sarcasm data. |
| Outcome: | The proposed method can be adapted to other affective computing domains, opening up new research opportunities. |
Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on curriculum learning focus on selecting the best distribution of data to train a system. |
| Approach: | They propose a self-supervised neural machine translation model that self-selects data without being told to do so. |
| Outcome: | The proposed model self-selects samples of increasing complexity and task relevance without being told to do so, and performs a denoising curriculum. |
Towards Reasonably-Sized Character-Level Transformer NMT by Finetuning Subword Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train character-level models require very deep architectures that are difficult and slow to train. |
| Approach: | They propose to fine tune a Transformer token-based model to get a model without token segmentation. |
| Outcome: | The proposed model improves translation quality and robustness to noise while requiring less token segmentation. |
Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that results from high-resource languages cannot be easily transferred to realistic, low-resourced scenarios. |
| Approach: | They analyse performance of multilingual transformer models using available resources for Hausa, isiXhosa and NER and topic classification. |
| Outcome: | The proposed models can achieve with as little as 10 or 100 labeled sentences the same performance as baselines with much more supervised training data. |
Translation Quality Estimation by Jointly Learning to Score and Rank (2020.emnlp-main)
Copied to clipboard
| Challenge: | The translation quality estimation (QE) task aims to evaluate the general quality of a translation without using reference translations. |
| Approach: | They propose a translation quality estimation task that uses translations as reference . they propose supervised learning using cross-lingual sentence embeddings from pre-trained multilingual models. |
| Outcome: | The proposed model outperforms sentBLEU on the WMT 2019 QE as a Metric task and outperformed sentBLUE on the QE in a multilingual language task. |
Direct Segmentation Models for Streaming Speech Translation (2020.emnlp-main)
Copied to clipboard
Javier Iranzo-Sánchez, Adrià Giménez Pastor, Joan Albert Silvestre-Cerdà, Pau Baquero-Arnal, Jorge Civera Saiz, Alfons Juan
| Challenge: | Existing approaches to stream ST combine advances in ASR and MT to achieve high quality translations without compromising the speed of the system. |
| Approach: | They propose to concatenate an Automatic Speech Recognition system followed by a Machine Translation system. |
| Outcome: | The proposed models improve on the Europarl-ST dataset on the BLEU score. |
Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation (2020.emnlp-main)
Copied to clipboard
Tahmid Hasan, Abhik Bhattacharjee, Kazi Samin, Masum Hasan, Madhusudan Basak, M. Sohel Rahman, Rifat Shahriyar
| Challenge: | despite being the seventh most widely spoken language, Bengali has received little attention in machine translation due to being low in resources. |
| Approach: | They propose a customized sentence segmenter for Bengali and two new methods for parallel corpus creation on low-resource setups. |
| Outcome: | The proposed method improves Bengali-English parallel corpus by 9 BLEU over previous approaches . the results will pave the way for future research on Bengali and other low-resource languages . |
CSP:Code-Switching Pre-training for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation models are data-hungry and low-resource . et al., 2018; Radford e.t., 2019; Yang ee.,2019) proposes a new pre-training method for NMT . |
| Approach: | They propose a new pre-training method which randomly replaces some words in the input sentence with their translation words in target language. |
| Outcome: | The proposed method improves on unsupervised and supervised NMT models by making full use of monolingual corpora. |
Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias (2020.emnlp-main)
Copied to clipboard
| Challenge: | English challenge datasets highlight gender-ambiguous occurrences of ‘doctor’ as male doctors, but they are not useful for other languages. |
| Approach: | They propose to build multi-task challenge datasets for detecting gender bias that lead to unambiguously wrong model predictions for languages with type B reflexivization. |
| Outcome: | The proposed dataset can detect gender bias in languages with type B reflexivization and spans four languages and four NLP tasks. |
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-training methods are not effective for machine translation tasks. |
| Approach: | They propose a method to pre-train a universal multilingual neural machine translation model . they use random aligned substitution technique to bring words and phrases with similar meanings closer in the representation space. |
| Outcome: | The proposed approach improves translation quality on low, medium, rich resource languages. |
Losing Heads in the Lottery: Pruning Transformer Attention in Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research shows that attention heads are not confident in their decisions and can be pruned. |
| Approach: | They apply the lottery ticket hypothesis to prune heads in early training . they find that the pruned model is 1.5 times faster at inference . |
| Outcome: | The proposed method is 1.5 times faster at inference, but at the cost of longer training. |
Towards Enhancing Faithfulness for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has achieved great success due to the ability to generate high-quality sentences. |
| Approach: | They propose a training strategy with a multi-task learning paradigm to build a faithfulness enhanced NMT model. |
| Outcome: | The proposed model can generate high-quality sentences that are very close to natural language. |
COMET: A Neural Framework for MT Evaluation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Historically, metrics for evaluating the quality of machine translation (MT) have relied on basic, lexical-level features such as counting the number of matching n-grams between the MT hypothesis and the reference translation. |
| Approach: | They propose a neural framework for training multilingual machine translation evaluation models which exploits human judgements to obtain new state-of-the-art levels of correlation with MT quality. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the WMT 2019 Metrics shared task and demonstrate robustness to high-performing systems. |
Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) models with limited data are ineffective when the two languages are not available for one language. |
| Approach: | They propose an approach that reuses a language model that is pretrained on two languages with large monolingual data to initialize an unsupervised neural machine translation system. |
| Outcome: | The proposed method outperforms a competitive cross-lingual pretraining model in English-Macedonian (En-Mk) and English-Albanian (En Sq) it yields more than +8.3 BLEU points for all four translation directions. |
LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Space (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction are mapping-based, but they do not hold for closely related languages. |
| Approach: | They propose a semi-supervised method to learn cross-lingual word embeddings for BLI using a linear mapping function and a latent space of two independently trained autoencoders. |
| Outcome: | The proposed method outperforms existing models on 15 different language pairs on both directions. |
Uncertainty-Aware Semantic Augmentation for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for neural machine translation only observe one source sentence at training time . this discrepancy in data distribution leads to a formidable learning challenge . |
| Approach: | They propose an uncertainty-aware semantic augmentation approach to capture universal semantic information among multiple source sentences and enhance hidden representations with this information. |
| Outcome: | The proposed approach outperforms baseline and existing methods on translation tasks. |
Can Automatic Post-Editing Improve NMT? (2020.emnlp-main)
Copied to clipboard
| Challenge: | APE has been successful with statistical machine translation systems but has not been as successful over neural machine translation (NMT) systems. |
| Approach: | They propose to train neural APE models on a corpus of human post-edits of NMT and compile a larger corpus to test their hypothesis. |
| Outcome: | The proposed model can improve a strong in-domain NMT system, challenging the current understanding in the field. |
Parsing Gapping Constructions Based on Grammatical and Semantic Roles (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for parsing sentences with gapping recover elided elements from redundant elements . grammatical and semantic tags are used to identify gaps in a coordinated structure . |
| Approach: | They propose a method of parsing sentences with gapping to recover elided elements . they use constituent trees annotated with grammatical and semantic roles . |
| Outcome: | The proposed method outperforms the previous method in terms of F-measure and recall. |
Span-based discontinuous constituency parsing: a family of exact chart-based algorithms with time complexities from O(nˆ6) down to O(nˆ3) (2020.emnlp-main)
Copied to clipboard
| Challenge: | a novel chart-based parser for discontinuous constituency trees is proposed for span-based span parsing . it can process discontinuous constituent trees of block degree two, including ill-nested structures . |
| Approach: | They propose a chart-based algorithm for span-based parsing of discontinuous constituency trees . they build variants with smaller search spaces and time complexities ranging from O(n6) down to O(N3) . |
| Outcome: | The proposed algorithm can process 98% of constituents in linguistic treebanks while having the same complexity as continuous constituency parsers. |
Some Languages Seem Easier to Parse Because Their Treebanks Leak (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cross-language differences in (universal) dependency parsing performance are mostly attributed to treebank size, average sentence length, average dependency length, morphological complexity, and domain differences. |
| Approach: | They compute graph isomorphisms and find that treebank size is a factor that influences parsing performance. |
| Outcome: | The results show that the overlap between training and test graphs explain more of the observed variation than standard explanations such as the above. |
Discontinuous Constituent Parsing as Sequence Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to discontinuous parsing are complex and low-level. |
| Approach: | They propose to encode discontinuities as nearly ordered permutations of the input sequence. |
| Outcome: | The proposed model is fast and accurate under the right representation. |
Modularized Syntactic Neural Networks for Sentence Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for sentence classification use local information of sub-trees, but new models use global context . |
| Approach: | They propose a tree-parallel mini-batch strategy for efficient training and predicting sentences . they propose to use syntax category labels to model sub-trees . |
| Outcome: | The proposed model outperforms state-of-the-art tree-based methods on the sentence classification task. |
TED-CDB: A Large-Scale Chinese Discourse Relation Dataset on TED Talks (2020.emnlp-main)
Copied to clipboard
| Challenge: | TED-CDB dataset is a unique corpus of spoken discourse in Chinese . TED is based on the concept that discourse relations are grounded in an identifiable set of discourse connectives or Altlex expressions. |
| Approach: | They have created a dataset that annotates TED talks in Chinese . they propose to adapt the dataset to Chinese news text to improve its performance . |
| Outcome: | The TED-CDB dataset can improve the performance of systems for languages other than Chinese . it is adapted to features that are not present in English and can extract discourse semantic features . |
QADiscourse - Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines (2020.emnlp-main)
Copied to clipboard
| Challenge: | Discourse relations describe how two propositions relate to one another . annotating discourse relations requires expert annotators . |
| Approach: | They propose a new representation of discourse relations as question-and-answer pairs that crowd-sources wide-coverage data annotated with discourse relations. |
| Outcome: | The proposed representation of discourse relations as QA pairs allows crowd-sourcing wide-coverage datasets annotated with discourse relations. |
Discourse Self-Attention for Discourse Element Identification in Argumentative Student Essays (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite its importance, discourse element identification is challenging due to the ambiguity of sentences . the number of elaboration sentences could be 10 times more than the number edna sentences. |
| Approach: | They propose to use sentence positional encodings to explicitly represent sentence positions and inter-sentence attentions to capture sentence interactions and enhance sentence representation. |
| Outcome: | The proposed model improves on a Chinese and English dataset. |
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models (2020.emnlp-main)
Copied to clipboard
Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
| Challenge: | Existing pre-trained large language models have shown unparalleled generative capabilities, but they are not controllable. |
| Approach: | They propose a framework that uses large-scale language models and adds control to text generation by incorporating an external knowledge base. |
| Outcome: | The proposed model generates more fluent, consistent, and coherent stories with less repetition and higher diversity compared to previous work on the ROC story dataset. |
Incomplete Utterance Rewriting as Semantic Segmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on the task of incomplete utterance rewriting as a machine translation task. |
| Approach: | They propose a semantic segmentation task which incorporates edit operations into the problem and predicts a word-level edit matrix. |
| Outcome: | The proposed approach outperforms existing baselines on several datasets and is four times faster than the standard approach in inference. |
Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for grammatical error correction are data-hungry and it is hard to train a seq2seq model with good performance without suf-Clean. |
| Approach: | They propose a method inspired by adversarial training to generate more meaningful and valuable training examples by continually identifying weak spots of a model and to enhance the model by gradually adding adversarials to the training set. |
| Outcome: | The proposed method improves generalization and robustness of GEC models by adding adversarial examples to the training set. |
Homophonic Pun Generation with Lexically Constrained Rewriting (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new method for generating puns using two homophones is needed to generate creative puns . early models for pun generation rely on templates and lack novelty. |
| Approach: | They propose a neural approach to generate homophonic puns with two meanings . they use constraint words to find the semantic incongruity and explicit negative constraints . |
| Outcome: | The proposed model achieves state-of-the-art in automatic and human evaluations. |
How to Make Neural Natural Language Generation as Reliable as Templates in Task-Oriented Dialogue (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural Natural Language Generation (NLG) systems are well known for their unreliability. |
| Approach: | They propose a data augmentation approach which restricts the output of a neural network and guarantees reliability. |
| Outcome: | The proposed approach scored 100% in semantic accuracy on the E2E NLG Challenge dataset, the same as a template system. |
Multilingual AMR-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on generating text from structured data into English has focused on bridging the gap between structure and natural language (NL) and semantically underspecified input and fully specified output. |
| Approach: | They propose a multilingual approach that can decode into 21 different languages . they leverage advances in cross-lingual embeddings and pretraining to generate multilingual models . |
| Outcome: | The proposed model surpasses baselines that generate into one language in eighteen languages. |
Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for gender bias mitigation for word embeddings are based on pre-trained word embeds . however, the assumption that the bias subspace is linear is untested . |
| Approach: | They propose a method to isolate gender bias in word embeddings using pre-trained word embeds. |
| Outcome: | The proposed method eliminates gender bias in word embeddings but assumes bias subspace is linear . the proposed method has some drawbacks, but it is a good one for a non-linear analysis. |
Lifelong Language Knowledge Distillation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to perform lifelong language learning (LLL) on stream of different tasks are challenging . Existing models face catastrophic forgetting problem, which can be mitigated by lifelong learning . |
| Approach: | They propose a method that can be easily applied to existing LLL architectures to mitigate degradation. |
| Outcome: | The proposed method improves state-of-the-art models and reduces degradation compared to multi-task models. |
Sparse Parallel Training of Hierarchical Dirichlet Process Topic Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | To scale non-parametric extensions of probabilistic topic models, practitioners rely increasingly on parallel and distributed systems. |
| Approach: | They propose a data-parallel sampler that utilizes all available sources of sparsity found in natural language to control memory requirements and computational complexity. |
| Outcome: | The proposed sampler is able to train a hierarchical Dirichlet process topic model on a well-known corpus (PubMed) with 8m documents and 768m tokens, using a single multi-core machine in under four days. |
Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Few/zero-shot learning is a big challenge of many classification tasks, where a classifier is required to recognise instances of classes that have very few or even no training samples. |
| Approach: | They propose a multi-graph aggregation model that fuses knowledge from multiple label graphs encoding different semantic label relationships to improve multi-label zero/few-shot document classification. |
| Outcome: | The proposed model improves on two large clinical datasets and the EU legislation dataset on few/zero-shot labels. |
Word Rotator’s Distance (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to measure textual similarity are inconsistent with the word alignment and are empirically inferior to the simple cosine similarity between general-purpose sentence vectors. |
| Approach: | They propose to decouple word vectors into their norm and direction and then grow the norm and directions of word vector. |
| Outcome: | The proposed methods outperform alignment-based approaches on several benchmarks and strong baselines on the semantic textual similarity task. |
Disentangle-based Continual Graph Representation Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing graph embedding methods overlook streaming nature of incoming data in real-world applications. |
| Approach: | They propose a disentangle-based continual graph representation learning framework inspired by the human’s ability to learn procedural knowledge. |
| Outcome: | The proposed framework outperforms state-of-the-art continual graph representation learning framework and alleviate catastrophic forgetting problem. |
Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing semisupervised methods do not fully utilize the knowledge hidden in annotated and nonannotated data, which hinders further improvement of their performance. |
| Approach: | They propose a semi-supervised BLI framework to encourage interaction between supervised signal and unsupervised alignment. |
| Outcome: | The proposed framework can incorporate any supervised and unsupervised BLI methods based on optimal transport and bi-directional lexicon update. |
Wasserstein Distance Regularized Sequence Representation for Text Matching in Asymmetrical Domains (2020.emnlp-main)
Copied to clipboard
| Challenge: | Asymmetrical text matching is a fundamental problem in information retrieval and natural language processing. |
| Approach: | They propose a method that regularizes features vectors projected from different domains . WD-Match can be used to improve different text matching methods . |
| Outcome: | The proposed method outperforms existing methods and benchmarks on four datasets. |
A Simple Approach to Learning Unsupervised Multilingual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised cross-lingual embeddings in the bilingual setting has given the impetus to learning a shared embeddable space for several languages. |
| Approach: | They propose to solve two sub-problems together to learn a shared embedding space for several languages. |
| Outcome: | The proposed approach outperforms existing methods in bilingual lexicon induction, cross-lingual word similarity, multilingual document classification, and multilingual dependency parsing tasks. |
Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games (2020.emnlp-main)
Copied to clipboard
Subhajit Chaudhury, Daiki Kimura, Kartik Talamadupula, Michiaki Tatsubori, Asim Munawar, Ryuki Tachibana
| Challenge: | Reinforcement Learning methods for text-based games fail to generalize on unseen games, especially in small data regimes. |
| Approach: | They propose a Context Relevant Episodic State Truncation method for irrelevant token removal in observation text for improved generalization. |
| Outcome: | The proposed method shows that it can generalize on unseen games using 10x-20x fewer training games compared to previous state-of-the-art methods despite requiring fewer number of training episodes. |
BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover’s Distance (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been proposed and applied to many NLP tasks, yielding state-of-the-art performance, but high storage and computational costs obstruct them to be effectively deployed on resource-constrained devices and real-time applications. |
| Approach: | They propose a BERT distillation method which allows each intermediate student layer to learn from any intermediate teacher layers. |
| Outcome: | The proposed method can learn from different teacher layers adaptively for different NLP tasks. |
Slot Attention with Value Normalization for Multi-Domain Dialogue State Tracking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing dialogue state tracking approaches rely on ontology already defined, where all slots and their possible values are given. |
| Approach: | They propose a new architecture to exploit domain ontology by using Slot Attention and Value Normalization . they supplement the annotation of supporting span for MultiWOZ 2.1, which is the shortest span in utterances to support the labeled value. |
| Outcome: | The proposed architecture exploits ontology and can convert supporting spans to values. |
Don’t Read Too Much Into It: Adaptive Computation for Open-Domain Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to Open-Domain Question Answering assume all passages are of equal importance and allocate computation to them. |
| Approach: | They propose to use adaptive computation to control the computational budget allocated for the passages to be read. |
| Outcome: | The proposed approach reduces computational cost by 4.3x over strong static and adaptive methods while retaining 95% performance of the full model. |
Multi-Step Inference for Reasoning Over Paragraphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for complex reasoning use symbols or black-box transformers . a compositional model can chain together free-form predicates and logical connectives . |
| Approach: | They propose a compositional model that finds relevant sentences and then chains them together using neural modules. |
| Outcome: | The proposed model improves performance on a recently-introduced dataset. |
Learning a Cost-Effective Annotation Policy for Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art question answering systems require large amounts of training data for which labeling is time consuming and thus expensive. |
| Approach: | They propose a framework for annotating QA datasets that entails learning a cost-effective annotation policy and a semi-supervised annotation scheme. |
| Outcome: | The proposed approach can reduce up to 21.1% of the annotation cost compared with traditional methods . the proposed approach is based on a cost-effective annotation policy and semi-supervised annotation scheme . |
Scene Restoring for Narrative Machine Reading Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Narrative passages describe a chain of events, which helps the machine understand the passage comprehensively. |
| Approach: | They propose a method to let machine read narrative passages with their prior knowledge . they build a scene graph using Atomic as external knowledge and encode it with GDIN . |
| Outcome: | The proposed method achieves state-of-the-art on a Story Cloze Test and CosmosQA datasets. |
A Simple and Effective Model for Answering Multi-span Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for reading comprehension restrict output space to a set of single contiguous spans . multi-span questions are problematic because they require multiple inputs - a task that requires a sequence tagging problem . |
| Approach: | They propose a simple architecture for answering multi-span questions by casting the task as a sequence tagging problem. |
| Outcome: | The proposed model significantly improves performance on span extraction questions from DROP and Quoref by 9.9 and 5.5 EM points respectively. |
Top-Rank-Focused Adaptive Vote Collection for the Evaluation of Domain-Specific Semantic Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Embedding-based models are increasingly needed for domain-specific evaluation datasets. |
| Approach: | They propose a protocol for the construction of a relatedness-based evaluation dataset based on adaptive pairwise comparisons and appropriate metrics to evaluate a semantic model via the aforementioned dataset. |
| Outcome: | The proposed protocol is particularly accurate in top-rank evaluation. |
Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained neural language models improve learning for various NLP tasks by fine-tuning them on task-specific training sets. |
| Approach: | They propose a meta-learning procedure to fine-tune neural language models on task-specific training sets. |
| Outcome: | The proposed procedure solves a group of similar NLP tasks on a text mining dataset. |
Incorporating Behavioral Hypotheses for Query Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has focused on extending standard Seq2Seq models but literature often leaves out the influence of clickthrough actions. |
| Approach: | They propose a generic encoder-decoder Transformer framework to generate query suggestions from user inputs. |
| Outcome: | The proposed approach improves top-k word error rate and Bert F1 score compared to a recent BART model. |
Conditional Causal Relationships between Emotions and Causes in Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the causal relationships between emotions and causes focus on extracting causally related clauses from documents, but none considers whether context clauses are indispensable for extracted clauses to be causally linked. |
| Approach: | They propose a task to determine whether an input pair of emotion and cause has a valid causal relationship under different contexts. |
| Outcome: | The proposed task identifies whether an input pair of emotion and cause has a valid causal relationship under different contexts and then fine-tunes the prediction results based on the characteristics of the input clauses. |
COMETA: A Corpus for Medical Entity Linking in the Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for Entity Linking (EL) fail to address the complex nature of health terminology in layman’s language. |
| Approach: | They propose to use a corpus of 20k English biomedical entity mentions from Reddit expert-annotated with links to a widely-used medical knowledge graph to investigate the ability of these systems to perform complex inference on entities and concepts. |
| Outcome: | The proposed corpus satisfies a combination of desirable properties, from scale and coverage to diversity and quality, that to the best of our knowledge has not been met by existing resources in the field. |
Pareto Probing: Trading Off Accuracy for Complexity (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks are a pillar of modern NLP systems, but their inner workings are poorly understood. |
| Approach: | They propose a probe metric that reflects the trade-off between probe complexity and performance: the Pareto hypervolume. |
| Outcome: | The proposed probe metric conforms to accepted rankings among contextual representations, and is more complex than other probe tasks. |
Interpretation of NLP models through input marginalization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to interpret NLP predictions replace each token with a predefined value, resulting in misleading interpretations. |
| Approach: | They propose to marginalize each token out of the training data distribution to demystify the "black box" property of deep neural networks for natural language processing. |
| Outcome: | The proposed method marginalizes each token out of the training data distribution. |
Generating Label Cohesive and Well-Formed Adversarial Claims (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on adversarial triggers for fact checking models reveals weaknesses and flaws of models . universal adversarials often inadvertently invert the meaning of instances they are inserted in . |
| Approach: | They propose a method for automatically generating highly potent, well-formed, label cohesive claims for FC using universal adversarial triggers. |
| Outcome: | The proposed method maintains the directionality and semantic validity of the claim better than previous work on the FEVER dataset. |
Are All Good Word Vector Spaces Isomorphic? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing algorithms for aligning cross-lingual word vector spaces assume that vector spaces are approximately isomorphic. |
| Approach: | They propose to find out whether non-isomorphism is also crucially a sign of degenerate word vector spaces. |
| Outcome: | The proposed method performs poorly on non-isomorphic spaces, but it is not . it is also crucially a sign of degenerate word vector spaces, the authors show . |
Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks typically need large labeled data for training and are not easily interpretable. |
| Approach: | They propose a type of recurrent neural networks that combine neural networks and regular expression rules. |
| Outcome: | The proposed recurrent neural networks outperform previous neural approaches in low- and zero-shot scenarios and remain very competitive in rich-resource settings. |
When BERT Plays the Lottery, All Tickets Are Winning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Large Transformer-based models are reduced to a smaller number of self-attention heads and layers. |
| Approach: | They propose to prune BERT self-attention heads and layers to find subnetworks with comparable performance . they also extend this technique to multi-layer perceptrons to find out if they are unstable . |
| Outcome: | The proposed models are able to achieve 90% of full model performance with structured pruning and similar-sized subnetworks sampled from the rest of the model perform worse. |
On the weak link between importance and prunability of attention heads (2020.emnlp-main)
Copied to clipboard
| Challenge: | a large fraction of attention heads can be randomly pruned with limited effect on accuracy, a new study finds . a second study finds no advantage in pruning attention heads identified to be important based on the location of a head . |
| Approach: | They examine the importance of pruning attention heads on a Transformer-based model . they find no advantage in pruning attention head positions on the BERT model based on location . |
| Outcome: | The results show that pruning strategies on Transformer and BERT models are not important based on location . the results suggest that interpretation of attention heads does not strongly inform pruning strategies. |
Towards Interpreting BERT for Reading Comprehension Based QA (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models such as ELMO and XLNet have achieved state-of-the-art performance on various NLP tasks. |
| Approach: | They propose to define a layer’s role or functionality using Integrated Gradients and perform preliminary analysis across all layers. |
| Outcome: | The proposed model performs better than existing models on RCQA and ELMO, but it lacks the human-level performance needed to perform the task. |
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Attribution methods assess the contribution of inputs to the model prediction. |
| Approach: | They propose a method which removes subsets of inputs and a model which is based on hidden layers to make the decision to include or disregard an input token. |
| Outcome: | The proposed method is efficient because it predicts rather than searches the inputs. |
A Diagnostic Study of Explainability Techniques for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing explainability techniques that can be produced post-hoc with already trained models are lacking a definitive guide on how to choose one given a particular task and model architecture. |
| Approach: | They propose to use a list of diagnostic properties to evaluate existing explainability techniques to compare them with human annotations of salient input regions. |
| Outcome: | The proposed list compares a set of explainability techniques on downstream text classification tasks and neural network architectures. |
STL-CQA: Structure-based Transformers with Localization and Encoding for Chart Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Chart Question Answering (CQA) is a task of answering natural language questions about visualisations in the chart image. |
| Approach: | They propose a method for Chart Question Answering which improves the question/answering through sequential elements localization, question encoding and then, a structural transformer-based learning approach. |
| Outcome: | The proposed method outperforms state-of-the-art methods on various chart Q/A datasets while outperforming even human baseline. |
Learning to Contrast the Counterfactual Samples for Robust Visual Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods of generating counterfactual samples are not fully utilized in the task of Visual Question Answering (VQA). |
| Approach: | They propose a self-supervised contrastive learning mechanism to learn the relationship between original samples, factual samples and counterfactual samples. |
| Outcome: | The proposed method surpasses state-of-the-art models on the VQA-CP dataset, a diagnostic benchmark for assessing the VQ model’s robustness. |
Learning Physical Common Sense as Knowledge Graph Completion via BERT Data Augmentation and Constrained Tucker Factorization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Physical commonsense learning is an essential part of human-robot interaction . existing methods of learning physical commons sense suffer from generalization . |
| Approach: | They propose to use physical commonsense learning as a knowledge graph completion problem to better use latent relationships among training samples. |
| Outcome: | The proposed method outperforms existing methods in the human-robot interaction problem. |
A Visually-grounded First-person Dialogue Dataset with Verbal and Non-verbal Responses (2020.emnlp-main)
Copied to clipboard
| Challenge: | In visual-grounded dialogue systems, first-person visual information about where the other speakers are and what they are paying attention to is crucial to understand their intentions. |
| Approach: | They propose a visually-grounded first-person dialogue (VFD) dataset with verbal and non-verbal responses. |
| Outcome: | The proposed dataset provides verbal and non-verbal responses for first-person visual information and recent neural network models. |
Cross-Media Keyphrase Prediction: A Unified Framework with Multi-Modality Multi-Head Attention and Image Wordings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on text modeling, ignoring the rich features embedded in the matching images. |
| Approach: | They propose a novel multi-modal multi-head attention model to capture cross-media interactions and image wordings to bridge the two modalities. |
| Outcome: | The proposed model outperforms the current state of the art based on text modeling and image matching . |
VD-BERT: A Unified Vision and Dialog Transformer with BERT (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work focused on attention mechanisms to model complex interactions in visual dialog . a new framework for visual dialog is based on pretrained BERT language models . |
| Approach: | They propose a framework for a vision-dialog Transformer that leverages pretrained BERT language models for Visual Dialog tasks. |
| Outcome: | The proposed framework achieves the top position on the visual dialog leaderboard without pretraining on external vision-language data. |
The Grammar of Emergent Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on emergent languages focus on semantics, but lack tools to analyse their properties. |
| Approach: | They propose to use unsupervised grammar induction techniques to analyse emergent languages and to examine their syntactic properties. |
| Outcome: | The proposed techniques are appropriate to analyse emergent languages and show that they exhibit syntactic properties similar to those observed in human language. |
Sub-Instruction Aware Vision-and-Language Navigation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite significant advances, few previous works are able to fully utilize the strong correspondence between visual and textual sequences. |
| Approach: | They propose to provide agents with fine-grained annotations during training and provide them with sub-instructions and their corresponding paths. |
| Outcome: | The proposed method improves the performance of four state-of-the-art agents in a room-to-room (R2R) benchmark dataset. |
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results indicate that pre-trained language models can significantly outperform state-of-the-art methods in both automatic evaluation and human judgment. |
| Approach: | They propose to equip a pre-trained language model with a knowledge selection module to generate knowledge-grounded dialogues. |
| Outcome: | The proposed model outperforms state-of-the-art methods in evaluation and human judgment. |
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learn dialogue state tracking and response generation are time-intensive and not transferable between domains. |
| Approach: | They propose a transfer learning framework that allows efficient dialogue state tracking with a minimal generation length. |
| Outcome: | The proposed framework improves the inference efficiency and improves state-of-the-art results on multi-domain multi-tasking systems. |
Variational Hierarchical Dialog Autoencoder for Dialog State Tracking Data Augmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that generative data augmentation, where synthetic samples generated from deep generative models complement the training dataset, benefit NLP tasks. |
| Approach: | They propose a Variational Hierarchical Dialog Autoencoder for modeling the complete aspects of goal-oriented dialogs using inter-connected latent variables and learns to generate coherent dialogs from the latent spaces. |
| Outcome: | The proposed model outperforms previous strong baselines on dialog response generation and user simulation tasks. |
Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue models lack prior and posterior knowledge selection . prior selection module may not learn to select knowledge properly because of lack of posterior information . |
| Approach: | They propose a knowledge distillation-based training strategy to remove the exposure bias of knowledge selection. |
| Outcome: | The proposed model improves on two knowledge-grounded dialogue datasets. |
Counterfactual Off-Policy Training for Neural Dialogue Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for open-domain dialogue generation suffer from data insufficiency . a potential response inferred in hindsight is called a counterfactual reasoning . |
| Approach: | They propose to explore potential responses by counterfactual reasoning . given an observed response, the model automatically infers the outcome of an alternative policy that could have been taken . |
| Outcome: | The proposed model outperforms the HRED model and conventional learning frameworks on the DailyDialog dataset. |
Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research has focused on training open-domain dialogue models using unpaired data. |
| Approach: | They propose a data-level distillation method for training open-domain dialogue models by utilizing unpaired data. |
| Outcome: | The proposed method produces high-quality dialogue pairs with diverse contents, and can improve competitive baselines. |
Task-Completion Dialogue Policy Learning via Monte Carlo Tree Search with Dueling Network (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models of reinforcement learning use background planning and may suffer from low-quality simulated experiences. |
| Approach: | They propose a Monte Carlo Tree Search with Double-q Dueling network framework for task-completion dialogue policy learning. |
| Outcome: | The proposed method outperforms the previous model-based reinforcement learning methods and is robust to simulation errors. |
Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multi-turn response generation for open-domain dialogues have a complexity problem . auxiliary tasks that relate to context understanding can guide the learning of the generation model . |
| Approach: | They propose a multi-turn response generation model that has a simple structure yet can effectively leverage conversation contexts for response generation. |
| Outcome: | The proposed model outperforms state-of-the-art models in response quality and human judgment . it also enjoys a faster decoding process . |
AttnIO: Knowledge Graph Exploration with In-and-Out Attention Flow for Knowledge-Grounded Dialogue (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for retrieving proper knowledge relevant to conversational context use only KG structure . empirical evaluations present a marked performance improvement of AttnIO compared to all baselines in OpenDialKG dataset . |
| Approach: | They propose a dialog-conditioned path traversal model that makes full use of rich structural information in KG . they show a marked performance improvement compared to baselines in OpenDialKG a KG dataset . |
| Outcome: | The proposed model makes full use of rich structural information in KG structure . it can be trained to generate an adequate knowledge path even when paths are not available . |
Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to task-oriented dialogue systems require a large number of handcrafted features and labels. |
| Approach: | They propose a "Two-Teacher One-Student" learning framework for task-oriented dialogue . the framework amalgamates knowledge from two teacher networks and provides guidance . |
| Outcome: | The proposed framework outperforms baseline methods on two benchmark datasets . it can retrieve accurate KB entities and generate human-like responses simultaneously . |
Task-oriented Domain-specific Meta-Embedding for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods neglect domain-specific knowledge and use the same word embedding for each word in all domain-specified datasets. |
| Approach: | They propose a method to incorporate domain-specific and task-oriented information into meta-embeddings by combining pre-trained word embeddings. |
| Outcome: | The proposed method performs well on four text classification datasets and shows that it is compatible with existing methods. |
Don’t Neglect the Obvious: On the Role of Unambiguous Words in Word Sense Disambiguation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing senseannotated corpora lack coverage of many instances in WordNet . however, unambiguous words make up a large portion of WordNet while being poorly covered in existing senseannnotated . |
| Approach: | They propose a method to provide annotations for most unambiguous words in a large corpus by using a dataset. |
| Outcome: | The proposed method improves on the original results on Word Sense Disambiguation (WSD) using pre-trained language models and propagation algorithms. |
Within-Between Lexical Relation Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for recognizing lexical-semantic relations between words are path-based and distributional. |
| Approach: | They propose a novel Within-Between Relation model for recognizing lexical-semantic relations between words. |
| Outcome: | The proposed model outperforms baselines across various benchmarks and is competitive and competitive. |
With More Contexts Comes Better Performance: Contextualized Sense Embeddings for All-Round Word Sense Disambiguation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Contextualized word embeddings have been used effectively across several tasks in Natural Language Processing, but it is difficult to link them to structured sources of knowledge. |
| Approach: | They propose a semi-supervised approach to producing sense embeddings for the lexical meanings within a lexicon that is comparable to that of contextualized word vectors. |
| Outcome: | The proposed approach outperforms state-of-the-art models in the English Word Sense Disambiguation task and in the multilingual one while training on sense-annotated data in English only. |
Convolution over Hierarchical Syntactic and Lexical Graphs for Aspect Level Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for aspect-level sentiment classification ignore corpus level word co-occurrence information . a novel architecture convolutes over hierarchical syntactic and lexical graphs . |
| Approach: | They propose a novel architecture which convolutes over hierarchical syntactic and lexical graphs . they employ a global lexical graph to encode corpus level word co-occurrence information . |
| Outcome: | The proposed architecture outperforms the state-of-the-art methods on five bench- mark datasets. |
Multi-Instance Multi-Label Learning Networks for Aspect-Category Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect sentiment toward aspect categories ignore the fact that the sentiment of an aspect category mentioned in a sentence is an aggregation of the sentiments of the words indicating the aspect category in the sentence, which leads to suboptimal performance. |
| Approach: | They propose a multi-instance multi-label learning network for Aspect-Category sentiment analysis that treats sentences as bags, words as instances, and the words indicating an aspect category as key instances of the aspect category. |
| Outcome: | The proposed model is based on three public datasets showing that it performs well. |
Aspect Sentiment Classification with Aspect-Specific Opinion Spans (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing attention-based models for sentiment analysis are not able to capture opinion spans as a whole or variable-length opinion span. |
| Approach: | They propose a model that extracts aspect-specific opinion spans and evaluates sentiment polarity by exploiting extracted opinion features. |
| Outcome: | The proposed model extracts aspect-specific opinion spans and evaluates sentiment polarity using extracted opinion features. |
Emotion-Cause Pair Extraction as Sequence Labeling Based on A Novel Tagging Scheme (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to extract emotions and causes from unannotated emotion texts are labor intensive and limited applications in real-world scenarios. |
| Approach: | They propose a novel task to find emotions and corresponding causes in unannotated emotion texts. |
| Outcome: | The proposed model outperforms the state-of-the-art method by 2.26% (p0.001) in F1 measure. |
End-to-End Emotion-Cause Pair Extraction based on Sliding Window Multi-Label Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to extract potential pairs of emotions ignore the fact that the cause and the emotion it triggers are inseparable. |
| Approach: | They propose two frameworks that combine multi-label learning and multi-labeled learning to extract emotion clauses . they evaluate a benchmark emotion cause corpus and find the best performance . |
| Outcome: | The proposed frameworks achieve the best performance among all compared systems on the ECPE task. |
Multi-modal Multi-label Emotion Detection with Modality and Label Dependence (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on multi-label emotion detection focus on one modality . current studies focus on label dependence, but there is no consensus on the model . |
| Approach: | They propose a multi-modal sequence-to-set approach to model label dependence and modality dependence in a multiple-modal scenario. |
| Outcome: | The proposed approach is able to model the label dependence and the modality dependence in a multi-modal scenario. |
Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing ABSA test sets cannot be used to distinguish the sentiment of the target aspect from the non-target aspect. |
| Approach: | They propose a simple but effective approach to enrich ABSA test sets by disentangle the confounding sentiments of non-target aspects from the target aspect’s sentiment. |
| Outcome: | The proposed model can distinguish the sentiment of the non-target aspects from the target aspect’s sentiment by using the Aspect Robustness Test Set (ARTS). |
Modeling Content Importance for Summarization with Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on content importance do not consider semantics and context when evaluating importance. |
| Approach: | They apply information theory to pre-trained language models to define the concept of importance from the perspective of information amount. |
| Outcome: | Experiments on CNN/Daily Mail and New York Times show that the proposed model can model the importance of content better than previous methods based on F1 and ROUGE scores. |
Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document summarization consider the informativeness of the assessed summary and require human-generated references for each test summary. |
| Approach: | They propose to evaluate summary qualities without reference summaries by unsupervised contrastive learning. |
| Outcome: | The proposed method outperforms other evaluation metrics even without reference summaries. |
Neural Extractive Summarization with Hierarchical Attentive Heterogeneous Graph Network (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing extractive summarization methods focus on balancing salience and redundancy between sentences. |
| Approach: | They propose a hierarchical attentive heterogeneous graph for text summarization that models sentences . they propose to iteratively refine the sentence representations and deliver the labels by message passing . |
| Outcome: | The proposed method outperforms existing extractive summarization methods on large corpus. |
Coarse-to-Fine Query Focused Multi-Document Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on query focused multi-document summarization relies heavily on retrieval-style methods. |
| Approach: | They propose a query-cluster-based model which uses more accurate modules for estimating whether text segments are relevant, likely to contain an answer, and central. |
| Outcome: | The proposed framework outperforms strong comparison systems on benchmark datasets across domains and query types. |
Pre-training for Abstractive Document Summarization by Reinstating Source Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive document summarization models are often trained on limited supervised data . authors present three objectives for pretraining abstractive summarizing models . |
| Approach: | They propose to pre-train a SEQ2SEQ based abstractive summarization model on unlabeled text. |
| Outcome: | The proposed method improves on two benchmark summarization datasets with 19GB of text . the goal is sentence reordering, next sentence generation and masked document generation . |
Learning from Context or Names? An Empirical Study on Neural Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets may leak shallow heuristics via entity mentions, thus contributing to the high performance on RE benchmarks. |
| Approach: | They propose an entity-masked contrastive framework for relation extraction to gain a deeper understanding on textual context and type information while avoiding rote memorization of entities. |
| Outcome: | The proposed framework improves the effectiveness and robustness of neural models in different RE scenarios. |
SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for relation extraction use heuristics or distant-supervised annotations, but distant supervised methods make strong assumptions on entity cooccurrence without sufficient contexts. |
| Approach: | They propose a framework that exploits weak, self-supervised signals by leveraging large pretrained language models for adaptive clustering on contextualized relational features. |
| Outcome: | The proposed framework exploits weak, self-supervised signals on open-domain Relation Extraction . it bootstraps the self-supervised signals by improving contextualized features in relation classification . |
Denoising Relation Extraction from Document-level Distant Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate auto-labeled sentences for relation extraction (RE) are difficult to extend to document-level relation extraction as noise from DS may be even multiplied in documents. |
| Approach: | They propose a pre-trained model which de-emphasizes noisy DS data via multiple pre-training tasks. |
| Outcome: | The proposed model can capture useful information from noisy data and achieve promising results on the large-scale DocRE benchmark. |
Let’s Stop Incorrect Comparisons in End-to-end Relation Extraction! (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on Relation Extraction (RE) uses multiple evaluation setups to compare performance. |
| Approach: | They propose to quantify the most common comparison mistake and evaluate it leads to overestimating the final RE performance by around 5% on ACE05. |
| Outcome: | The proposed meta-analysis overestimates the final RE performance by around 5% on ACE05. |
Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for relation extraction (RE) use shallow heuristics that do not generalize to challenge-set data. |
| Approach: | They propose to annotate a dataset to test whether relation extraction models are generalized to the challenge-set data. |
| Outcome: | The proposed model performs better on the challenge-set compared with the SOTA models on the same dataset. |
Global-to-Local Neural Networks for Document-Level Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Relation extraction (RE) aims to identify the semantic relations between named entities in text. |
| Approach: | They propose a novel relation extraction model that encodes document information in terms of entity global and local representations and context relation representations. |
| Outcome: | The proposed model achieves superior performance on two public datasets for document-level RE. |
Recurrent Interaction Network for Jointly Extracting Entities and Classifying Relations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to solve the extraction problem learn interactions between the two tasks through a shared network . |
| Approach: | They propose to use multi-task learning to address the joint extraction of entity and relation . they exploit correlation between ER and relation classification tasks to improve performance . |
| Outcome: | Empirical results show that the proposed model improves on two real-world datasets. |
Temporal Knowledge Base Completion: New Algorithms and Evaluation Protocols (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing TKBC models heavily overestimate link prediction performance due to imperfect evaluation mechanisms. |
| Approach: | They propose a method that integrates entities, relations and time into a uniform space . they propose improved evaluation protocols for link and time prediction . |
| Outcome: | The proposed method exploits the recurrent nature of some facts/events and temporal interactions between pairs of relations yielding state-of-the-art results. |
OpenIE6: Iterative Grid Labeling and Coordination Analysis for Open Information Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | OpenIE generates extractions iteratively, requiring repeated encoding of partial outputs. |
| Approach: | They propose an iterative open information extraction system that generates extractions iterativly, requiring repeated encoding of partial outputs. |
| Outcome: | The proposed system beats the previous systems by as much as 4 pts in F1 while being much faster. |
Public Sentiment Drift Analysis Based on Hierarchical Variational Auto-encoder (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting public sentiment drift are not designed for sentiment drift detection. |
| Approach: | They propose a Hierarchical Variational Auto-Encoder model to learn better distribution representation and a new drift measure to directly evaluate distribution changes between historical and new data. |
| Outcome: | The proposed model performs better than three existing state-of-the-art methods. |
Point to the Expression: Solving Algebraic Word Problems using the Expression-Pointer Transformer Model (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that generate solution equations using ‘Op (operator/operand) tokens suffered expression fragmentation and operand-context separation. |
| Approach: | They propose a pure neural model, Expression-Pointer Transformer, which uses (1) ‘Expression’ token and (2) operand-context pointers when generating solution equations. |
| Outcome: | The proposed model achieves comparable performance accuracy to state-of-the-art models and achieves better performance than existing models by at most 40%. |
Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models focus on one-unknown linear MWPs. |
| Approach: | They propose a universal expression tree-structured solver that integrates multiple expression trees underlying a MWP into a single expression tree. |
| Outcome: | The proposed method outperforms state-of-the-art models on a MWPs dataset and generates a universal expression tree explicitly by deciding which symbol to generate . |
Neural Topic Modeling by Incorporating Document Relationship Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Graph Topic Models (GNNs) capture relationships between graph nodes via message passing . recent research has focused on topic modeling using latent Dirichlet Allocation . |
| Approach: | They propose a Graph Topic Model (GTM) that captures relationships between graph nodes via message passing. |
| Outcome: | The proposed model captures the relationships between nodes via message passing . the results demonstrate that the proposed model is effective in generating documents . |
Routing Enforced Generative Model for Recipe Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to generate recipes with information about ingredients are difficult to use in practice. |
| Approach: | They propose a routing method to dive into the content selection under the internal restrictions. |
| Outcome: | The proposed model improves on BLEU, F1 and human evaluation. |
Assessing the Helpfulness of Learning Materials with Inference-Based Learner-Like Agent (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work uses hand-crafted scores to recommend sentences but has difficulty adopting such scores to all the near-synonyms as near-near-sonyms differ in various ways. |
| Approach: | They propose an inference-based learner-like agent to mimic learner behavior and identify good learning materials by examining the agent's performance. |
| Outcome: | The proposed agent achieves the best performance in fill-in-the-blank and good example sentence selection tasks. |
Selection and Generation: Learning towards Multi-Product Advertisement Post Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | E-commerce websites have billions of products, so it is impossible to write all copywriting manually. |
| Approach: | They propose a model to generate an AD post using a select network and a MGenNet network to generate a post including selected products. |
| Outcome: | The proposed model achieves impressive performance on a large-scale real-world AD post dataset. |
Form2Seq : A Framework for Higher-Order Form Structure Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Document structure extraction is a widely researched area for decades due to image resolution and poor semantics. |
| Approach: | They propose a sequence-to-sequence framework for document structure extraction using text . they use a text-based framework to classify low-level constituent elements into ten types . |
| Outcome: | The proposed framework outperforms existing methods for document structure extraction on ICDAR 2013 dataset. |
Domain Adaptation of Thai Word Segmentation Models using Stacked Ensemble (2020.emnlp-main)
Copied to clipboard
Peerat Limkonchotiwat, Wannaphong Phatthiyaphaibun, Raheem Sarwar, Ekapol Chuangsuwanich, Sarana Nutanong
| Challenge: | Thai word segmentation is domain-dependent, and researchers have been relying on transfer learning to adapt existing models to new domains. |
| Approach: | They propose a filter-and-refine solution to address Thai word segmentation as a domain-dependent problem. |
| Outcome: | The proposed method is an effective domain adaptation method and has similar performance as the transfer learning method. |
DagoBERT: Generating Derivational Morphology with a Pretrained Language Model (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) generate derivationally complex words, but it is unclear what they learn about other aspects of language. |
| Approach: | They propose to use BERT to examine its derivational capabilities in different settings, from unmodified pretrained models to full finetuning. |
| Outcome: | The proposed model outperforms the state-of-the-art in derivation generation. |
Attention Is All You Need for Chinese Word Segmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on Chinese word segmentation has been concerned about the following three perspectives. |
| Approach: | They propose to use a greedy decoding algorithm to improve Chinese word segmentation model. |
| Outcome: | The proposed model achieves state-of-the-art or comparable performance against strong baselines in strict closed test setting. |
A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for word-level segmentation (CWS) for the Chinese language have been successful in large-scale annotated corpora. |
| Approach: | They propose a method that integrates different segmentation criteria into one model . they use a transfer learning method to improve the performance of OOV words . |
| Outcome: | The proposed method achieves state-of-the-art performance on multiple benchmark datasets . it shows a competitive practicability and generalization ability for the CWS task . |
Alignment-free Cross-lingual Semantic Role Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to semantic role labeling rely on word alignments, translation engines or preprocessing tools. |
| Approach: | They propose a cross-lingual semantic role labeling model which only requires annotations in a source language and access to raw text in . |
| Outcome: | The proposed model minimizes the effort required to construct annotations or models for a new target language. |
Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-grained propaganda detection are not based on input-output data, but instead use declarative knowledge to detect propagandistic text fragments. |
| Approach: | They propose a method to inject declarative knowledge of fine-grained propaganda techniques into training data to get better representations of propagandistic texts. |
| Outcome: | The proposed method achieves superior performance on a large dataset for propaganda detection. |
X-SRL: A Parallel Cross-Lingual Semantic Role Labeling Dataset (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual SRL datasets contain disparate annotation styles or come from different domains, hampering generalization in multilingual learning. |
| Approach: | They propose to automatically construct an SRL corpus that is parallel in four languages with unified predicate and role annotations that are fully comparable across languages. |
| Outcome: | The proposed method improves performance for English SRL in weaker languages. |
Graph Convolutions over Constituent Trees for Syntax-Aware Semantic Role Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Semantic role labeling (SRL) is the task of identifying predicates and labeling argument spans with semantic roles. |
| Approach: | They propose to use graph convolutional networks to encode constituents and inform an SRL system by combining word representations of the first and last words in a constituent tree. |
| Outcome: | The proposed model is compared with other models and shows that it is more efficient than dependency trees. |
Fast semantic parsing with well-typedness guarantees (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing algorithms for AM dependency parsing are slow and do not support linguistic principles. |
| Approach: | They propose an A* parser and a transition-based parsing algorithm which guarantee well-typedness and improve parse speed by up to 3 orders of magnitude. |
| Outcome: | The proposed algorithms guarantee well-typedness and improve parsing speed by up to 3 orders of magnitude while maintaining or improving accuracy. |
Improving Out-of-Scope Detection in Intent Classification by Using Embeddings of the Word Graph Space of the Classes (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for intent classification use one-class classification or inverse dictionary. |
| Approach: | They propose to represent class labels as a vector space where word graphs are mapped . they use inverse dictionary to take in account inter-class similarities provided by repeated occurrences . |
| Outcome: | The proposed method beats the state-of-the-art method in the Larson dataset by about 31 percentage points. |
Supervised Seeded Iterated Learning for Interactive Language Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has focused on word-based conversational agents that tend to invent their language rather than leveraging natural language. |
| Approach: | They propose two methods to counter language drift by combining S2P and Seeded Iterated Learning to minimize their weaknesses. |
| Outcome: | The proposed methods reduce late-stage training collapses and higher negative likelihood when evaluated on human corpus. |
Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems (2020.emnlp-main)
Copied to clipboard
Jan Deriu, Don Tuggener, Pius von Däniken, Jon Ander Campos, Alvaro Rodrigo, Thiziri Belkacem, Aitor Soroa, Eneko Agirre, Mark Cieliebak
| Challenge: | Lack of time efficient and reliable evalu-ation methods is hampering the development of conversational dialogue systems (chatbots). |
| Approach: | They propose a framework that replaces human-bot conversations with conversations between bots and an annotation tool that ranks chatbots based on their ability to mimic human behaviour. |
| Outcome: | The proposed evaluation framework replaces human-bot conversations with bot conversations and allows for frequent evaluations of chatbots during their evaluation cycle. |
Human-centric dialog training via offline reinforcement learning (2020.emnlp-main)
Copied to clipboard
Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, Rosalind Picard
| Challenge: | a novel offline RL method can train dialog models to produce better conversations without the risk of humans teaching it harmful chat behaviors. |
| Approach: | They develop offline reinforcement learning algorithms that use human feedback to train dialog models . they use language similarity, laughter, sentiment, and more to identify positive feedback . |
| Outcome: | The proposed method improves on existing methods with 80 users in an open-domain setting. |
Speakers Fill Lexical Semantic Gaps with Context (2020.emnlp-main)
Copied to clipboard
| Challenge: | Lexical ambiguity is widespread in language, allowing for the reuse of economical word forms and thus making language more efficient. |
| Approach: | They propose two ways to estimate lexical ambiguity as the entropy of meanings a word can take . they validate this hypothesis by using WordNet and BERT . |
| Outcome: | The proposed method shows that on six high-resource languages, there are significant correlations between the estimate and the number of synonyms a word has in WordNet. |
Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models of crosslinguistic adjective ordering have relied on native speakers' intuitive judgment, not corpus data. |
| Approach: | They propose a latent-variable model that can order adjectives across 24 languages . they use tools and techniques to find universal, cross-linguistic, hierarchical ordering tendencies . |
| Outcome: | The proposed model can order adjectives across 24 languages even when languages are different . similar ordering preferences have been found to apply universally across languages . |
Surprisal Predicts Code-Switching in Chinese-English Bilingual Text (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines the propensity of bilinguals to switch languages . word surprisal and word entropy are important predictors of code-switching . |
| Approach: | They propose high cognitive effort as a reason for code-switching . they use a computational model of surprisal and word entropy to model code-changing . |
| Outcome: | The proposed model shows that word surprisal, but not entropy, is a significant predictor . sentence length is also a predictor, which has been related to sentence complexity . |
Word Frequency Does Not Predict Grammatical Knowledge in Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural language models learn the grammatical properties of natural languages to varying degrees of accuracy. |
| Approach: | They focus on subject-verb agreement and reflexive anaphora to investigate whether there are systematic sources of variation in the language models’ accuracy. |
| Outcome: | The proposed model can learn grammatical properties from training data. |
Improving Word Sense Disambiguation with Translations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing WSD systems rarely consider multilingual information for word sense disambiguation (WSD). |
| Approach: | They propose a method that leverages multilingual information to improve a base WSD system by generating translations. |
| Outcome: | The proposed method improves performance of a base WSD system in English and multilingual WSD on several languages. |
Towards Better Context-aware Lexical Semantics:Adjusting Contextualized Representations through Static Anchors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has shown that contextualized models generate dynamic embeddings for words in context, but static embedds are often overlooked in this trend towards contextualized modeling. |
| Approach: | They propose a method that learns a transformation through static anchors and requires only another pre-trained model. |
| Outcome: | The proposed method improves a range of benchmark tasks that test contextual variations of meaning across different usages of a word and across different words as they are used in context. |
Compositional Demographic Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are usually derived from corpora containing text from many individuals . however, they cannot account for user-specific word preferences, such as using the same word in different ways across contexts. |
| Approach: | They propose a new form of personalized word embeddings that use demographic-specific word representations derived compositionally from full or partial demographic information for a user. |
| Outcome: | The proposed representations outperform generic representations on two English language tasks. |
Are “Undocumented Workers” the Same as “Illegal Aliens”? Disentangling Denotation and Connotation in Vector Spaces (2020.emnlp-main)
Copied to clipboard
| Challenge: | popular pretrained models encode both denotation and connotation as one entangled representation . a researcher using a pretrained representation can confuse words with connotations . |
| Approach: | They propose a nerual netowrk that decomposes a pretrained representation as independent denotation and connotation representations. |
| Outcome: | The proposed model improves document rankings by comparing denotation and connotation representations with extrinsic representations. |
Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on text summarization focus on single-speaker docs, scientific publications and encyclopedia articles. |
| Approach: | They propose a multi-view sequence-to-sequence model that extracts conversational structures from unstructured daily chats and incorporates different views to generate dialogue summaries. |
| Outcome: | The proposed model outperforms state-of-the-art models via automatic evaluation and human judgment on a large-scale dialogue summarization corpus. |
Few-Shot Learning for Opinion Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that abstractive summarization models fail to capture their essential properties due to the high cost of summary production. |
| Approach: | They propose a few-shot framework for abstractive opinion summarization that bootstraps the output of an unsupervised model. |
| Outcome: | The proposed framework outperforms extractive and abstractive methods on Amazon and Yelp datasets. |
Learning to Fuse Sentences with Transformers for Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization systems that fuse sentences are not rewarded for correctly fusing sentences. |
| Approach: | They propose to leverage the knowledge of points of correspondence between sentences to enhance their ability to fuse sentences. |
| Outcome: | The proposed algorithms improve the ability of the proposed summarization systems to fuse sentences and show that they can fuse sentences in a way that retains the original meaning. |
Stepwise Extractive Summarization and Planning with Structured Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to extractive summarization use transformers to learn the structure of long inputs. |
| Approach: | They propose encoder-centric stepwise models for extractive summarization using structured transformers – HiBERT and Extended Transformers . |
| Outcome: | The proposed models outperform previous models on CNN/DailyMail extractive summarization and Rotowire table-to-text generation. |
CLIRMatrix: A massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cross-Lingual Information Retrieval (CLIR) is a retrieval task in which search queries and candidate documents are written in different languages. |
| Approach: | They present a massively large collection of bilingual and multilingual datasets for Cross-Lingual Information Retrieval extracted automatically from Wikipedia. |
| Outcome: | The proposed datasets are the largest and most comprehensive CLIR dataset to date. |
SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Search (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing search methods for COVID-19 are not based on scientific data, but use a neural re-ranking model pre-trained on scientific text. |
| Approach: | They propose a zero-shot ranking algorithm that adapts to COVID-related scientific literature . they use a neural re-ranking model pre-trained on scientific text and filters the target document . |
| Outcome: | The proposed algorithm outperforms models on the TREC COVID Round 1 leaderboard . it outperformed models that do not rely on TREC-COVID data . |
Modularized Transfomer-based Ranking Framework (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent innovations in Transformer-based ranking models have advanced the state-of-the-art in information retrieval. |
| Approach: | They propose to modularize a Transformer ranker into separate modules for text representation and interaction. |
| Outcome: | The proposed model is faster than previous models and is easier to interpret and understand. |
Ad-hoc Document Retrieval using Weak-Supervision with BERT and GPT2 (2020.emnlp-main)
Copied to clipboard
| Challenge: | a weakly-supervised method is used for document retrieval tasks . traditional methods are used for ad-hoc querying, but they require large amounts of labeled data . |
| Approach: | They propose a weakly-supervised method for training deep learning models for ad-hoc document retrieval using weak-supervision from the documents in the corpus. |
| Outcome: | The proposed method outperforms state-of-the-art methods on a COVID-19 dataset and two news datasets without the need for labeling data. |
Adversarial Semantic Collisions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate semantic collisions for NLP tasks are vulnerable to adversarial examples. |
| Approach: | They propose gradient-based approaches for generating semantic collisions given white-box access to a model and deploy them against several NLP tasks. |
| Outcome: | The proposed approaches evade perplexity-based filtering and discuss other potential mitigations. |
Learning Explainable Linguistic Expressions with Neural Inductive Logic Programming for Sentence Classification (2020.emnlp-main)
Copied to clipboard
Prithviraj Sen, Marina Danilevsky, Yunyao Li, Siddhartha Brahma, Matthias Boehm, Laura Chiticariu, Rajasekar Krishnamurthy
| Challenge: | Existing approaches to explain models are difficult to interpret and have undesirable biases. |
| Approach: | They propose a neural network architecture for learning transparent sentences . they use linguistic expressions built on top of predicates extracted using shallow natural language understanding . |
| Outcome: | The proposed model outperforms statistical relational learning and other neuro-symbolic methods and performs better than black-box recurrent neural networks. |
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models have been successful when finetuned to downstream tasks . however, it is difficult to determine whether the knowledge that finetuning LMs contain is learned during the pretraining or the finetailing process. |
| Approach: | They propose a method to create prompts for a diverse set of tasks using a gradient-guided search. |
| Outcome: | The proposed method performs sentiment analysis and natural language inference without additional parameters and finetuning. |
Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for improving model interpretability require prior information or human annotations as additional inputs. |
| Approach: | They propose a variational word mask method to automatically learn task-specific important words and reduce irrelevant information on classification, which ultimately improves model interpretability. |
| Outcome: | The proposed method improves model prediction accuracy and interpretability on seven datasets. |
Sparse Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current text generators require sampling from a modified softmax to avoid degenerate text . entmax sampling creates a mismatch between training and testing conditions . |
| Approach: | They propose to use entmax transformation to train and sample from a sparse language model to avoid degenerate text. |
| Outcome: | The proposed model improves fluency and consistency, fewer repetitions, and n-gram diversity closer to human text. |
PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking (2020.emnlp-main)
Copied to clipboard
| Challenge: | a novel task is to generate a coherent narrative consistent with an outline . large-scale language models are not sufficient in generating coherent narratives for the given outline despite their impressive generation performance . |
| Approach: | They propose a task of outline-conditioned story generation that generates a coherent narrative . they propose 'plotmachines' that tracks dynamic plot states and learns different writing styles . |
| Outcome: | The proposed model can generate a coherent story by tracking the dynamic plot states while conditioning on the input outline while generating the full story. |
Do sequence-to-sequence VAEs learn global features of sentences? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Autoregressive language models are often trained without explicit conditioning labels . authors question claim that latent vectors can capture global features in unsupervised manner . |
| Approach: | They propose to use a sequence-to-sequence architecture to learn latent variables . they find that VAEs are prone to memorizing the first words and sentence length . |
| Outcome: | The proposed model is prone to memorizing the first words and sentence length, the authors show . et al., 2016: a new model learns latent variables that are more global, more predictive of topic or topic labels . authors question this claim, say it is a waste of time and money . |
Content Planning for Neural Story Generation with Aristotelian Rescoring (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to narrative composition are plagued by difficulty in mastering structure, will veer between topics, and lack long-range cohesion. |
| Approach: | They propose a plot-generation language model and a set of rescoring models that implement an aspect of good story-writing as detailed in Aristotle's Poetics. |
| Outcome: | The proposed system improves the quality of the narrative generated from the proposed model and improves its relevance to a given prompt and quality of stories written with our principled plot structure. |
Generating Dialogue Responses from a Semantic Latent Space (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for dialogue generation are unable to integrate information from multiple semantically similar valid responses of a given prompt. |
| Approach: | They propose to learn the pair relationship between the prompts and responses as a regression task instead of the end-to-end classification on vocabulary. |
| Outcome: | The proposed model learns the pair relationship between the prompts and responses on a latent space instead of the end-to-end classification on vocabulary. |
Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Subsequent references exploit the common ground accumulated by the interlocutors and tend to be shorter and reuse expressions that were effective in previous mentions. |
| Approach: | They propose a model that generates first and subsequent references in visually grounded dialogue . they also implement a reference resolution system to assess the referring effectiveness . |
| Outcome: | The proposed model produces better, more effective referring utterances than one not grounded in the dialogue context. |
Visually Grounded Compound PCFGs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on visual groundings for language understanding has been drawing much attention. |
| Approach: | They propose to use an extension of probabilistic context-free grammar model to do fully-differentiable end-to-end visually grounded learning. |
| Outcome: | The proposed model outperforms the previous grounded model and significantly outperformed the previous model on the MSCOCO test captions. |
ALICE: Active Learning with Contrastive Natural Language Explanations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Annotating a large dataset with annotations is costly and infeasible. |
| Approach: | They propose an expert-in-the-loop training framework that utilizes contrastive natural language explanations to improve data efficiency in learning. |
| Outcome: | The proposed framework outperforms baseline models trained with 40-100% more training data on bird species classification and social relationship classification tasks. |
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Room-Across-Room (RxR) is a vision-and-language navigation dataset that addresses gaps in existing ones by addressing known biases in paths and eliciting more references to visible entities. |
| Approach: | They introduce a new Vision-and-Language Navigation (VLN) dataset that addresses biases in paths and elicits more references to visible entities. |
| Outcome: | The proposed model learns from synchronized pose traces by focusing only on portions of the panorama attended to in human demonstrations. |
SSCR: Iterative Language-Based Image Editing via Self-Supervised Counterfactual Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Iterative language-based image editing (ILBIE) tasks follow iterative instructions to edit images step by step. data scarcity makes learning the association between vision and language challenging. |
| Approach: | They propose a framework that incorporates counterfactual thinking to overcome data scarcity by combining out-of-distribution instructions with previous images. |
| Outcome: | The proposed model improves the correctness of ILBIE on two IBLIE datasets, even with only 50% of the training data. |
Identifying Elements Essential for BERT’s Multilinguality (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual BERT (mBERT) does not use any crosslingual signal during training. |
| Approach: | They propose a multilingual pretraining setup that modifies the masking strategy using VecMap to allow for fast experimentation. |
| Outcome: | The proposed setup with pretrained models with three languages shows that it works well. |
On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern multilingual models are trained on concatenated text from multiple languages in hopes of conferring benefits to each (positive transfer) however, recent work has shown that this approach can degrade performance on high-resource languages, a phenomenon known as negative interference. |
| Approach: | They propose a meta-learning algorithm that adds language-specific parameters as meta-parameters and trains them in a manner that explicitly improves shared layers’ generalization on all languages. |
| Outcome: | The proposed model improves cross-lingual transferability and generalization on all languages, and improves on the language-specific parameters. |
Pre-tokenization of Multi-word Expressions in Cross-lingual Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multi-Word Expressions (MWEs) are common in every language, but they are not translated by cross-lingual word embeddings. |
| Approach: | They propose a method for word translation of Multi-Word Expressions (MWEs) they compile lists of MWEs in each language and tokenize them as single tokens before training word embeddings. |
| Outcome: | The proposed method can translate multi-word expressions to and from English in 10 languages. |
Monolingual Adapters for Zero-Shot Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adapter layers are more parameter-efficient and provide better performance than bilingual ones. |
| Approach: | They propose to use monolingual adapter layers instead of bilingual ones to compose them and generalize to unseen language pairs. |
| Outcome: | The proposed adapter layer formalism achieves a median improvement of +2.77 BLEU points over a 20-language multilingual Transformer baseline trained on TED talks. |
Do Explicit Alignments Robustly Improve Multilingual Encoders? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Explicit alignment objectives based on bitexts like Europarl and MultiUN have been shown to improve cross-lingual representations. |
| Approach: | They propose a new contrastive alignment objective that can better utilize bitexts . they propose to use a random sample of 1 million pair subset of OPUS data . |
| Outcome: | The proposed objective outperforms existing alignment objectives on a random 1 million pair subset of the OPUS dataset. |
From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that multilingual transformers are less effective in resource-lean scenarios and for distant languages. |
| Approach: | They propose to use massively multilingual transformers to pretrain languages . they show that MMTs are less effective in resource-lean scenarios and distant languages if they are pre-trained via language modeling . |
| Outcome: | The proposed model is less effective in resource-lean scenarios and for distant languages than cross-lingual word embeddings. |
Distilling Multiple Domains for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation is a powerful tool for high-resource domains, but performance suffers when the input domain is low-resourced. |
| Approach: | They propose a framework for training a single multi-domain neural machine translation model that can translate multiple domains without increasing inference time or memory usage. |
| Outcome: | The proposed model improves translation on both high- and low-resource domains over strong multi-domain baselines and is robust under noisy data conditions. |
Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing sentence embeddings models are monolingual, and only for English . a new method allows to create multilingual versions from monolingual models . |
| Approach: | They propose a method to extend existing sentence embedding models to new languages . they use a translated sentence to generate sentence embeds for the source language . |
| Outcome: | The proposed method improves accuracy for multilingual setups and languages. |
A Streaming Approach For Efficient Batched Beam Search (2020.emnlp-main)
Copied to clipboard
| Challenge: | During decoding, candidates terminate or are pruned according to heuristics, a streaming method is used to "refill" the batch after it finishes translating some fraction of the current batch. |
| Approach: | They propose an efficient batching strategy for variable-length decoding on GPU architectures by streamlining the batching process. |
| Outcome: | The proposed method reduces runtime by 71% compared to a fixed-width beam search baseline and 17% compared with a variable-widness baseline while matching baselines’ BLEU. |
Improving Multilingual Models with Language-Clustered Vocabularies (2020.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual models depend on vocabularies that cover all languages . but the methods for generating those vocalaries are not ideal for massively multilingual applications. |
| Approach: | They propose a procedure for multilingual vocabulary generation that combines separately trained vocabularies of several automatically derived language clusters. |
| Outcome: | The proposed procedure shows improvements across languages on multilingual benchmark tasks . the proposed procedure reduces out-of-vocabulary rate by a factor of 8 . |
Zero-Shot Cross-Lingual Transfer with Meta Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | There are more than 7,000 languages spoken in the world, over 90 of which have more than 10 million native speakers each. |
| Approach: | They propose to use meta-learning to train a model on multiple languages at the same time . they use standard supervised, zero-shot cross-lingual, and few-shot crosses-lingual settings for different natural language understanding tasks. |
| Outcome: | The proposed setup improves on the state-of-the-art for a total of 15 languages. |
The Multilingual Amazon Reviews Corpus (2020.emnlp-main)
Copied to clipboard
| Challenge: | The corpus contains reviews in English, Japanese, German, French, Spanish, and Chinese, which were collected between 2015 and 2019 . |
| Approach: | They propose to use mean absolute error (MAE) instead of classification accuracy for this task since MAE accounts for ordinal nature of the ratings. |
| Outcome: | The proposed model uses mean absolute error (MAE) instead of classification accuracy since MAE accounts for ordinal nature of the ratings. |
GLUCOSE: GeneraLized and COntextualized Story Explanations (2020.emnlp-main)
Copied to clipboard
Nasrin Mostafazadeh, Aditya Kalyanpur, Lori Moon, David Buchanan, Lauren Berkowitz, Or Biran, Jennifer Chu-Carroll
| Challenge: | Existing knowledge resources and pretrained language models do not include or readily predict GLUCOSE’s rich inferential content. |
| Approach: | They propose a platform for crowdsourcing GLUCOSE data at scale that uses semi-structured templates to elicit causal explanations. |
| Outcome: | The proposed model can be trained on human-readable stories and build similar models on unseen stories. |
Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new method of analysis based on semantic tags demonstrates that character-level representations improve performance across a subset of selected semantic phenomena. |
| Approach: | They combine character-level and contextual language model representations to improve performance on Discourse Representation Structure parsing. |
| Outcome: | The proposed model improves performance on a subset of selected semantic phenomena. |
Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to augment pre-trained language models with disease knowledge are lacking. |
| Approach: | They propose a method to augment BERT-like pre-trained language models with disease knowledge. |
| Outcome: | The proposed method improves on a suite of BERT models over three tasks. |
Unsupervised Commonsense Question Answering with Self-Talk (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current systems rely on pre-trained language models or external knowledge bases to incorporate additional relevant knowledge. |
| Approach: | They propose an unsupervised framework based on self-talk to improve commonsense performance by asking language models to ask information seeking questions. |
| Outcome: | Empirical results show that the proposed framework improves on four out of six commonsense benchmarks and competes with models that obtain knowledge from external KBs. |
Reasoning about Goals, Steps, and Temporal Ordering with WikiHow (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets focus on relation between procedural events, but little attention has been paid to relation between events. |
| Approach: | They propose a set of reasoning tasks targeting goal-step relations and step-step temporal relations based on wikiHow articles . their automatically-generated training set allows models to transfer to out-of-domain tasks requiring knowledge of procedural events . |
| Outcome: | The proposed dataset improves on SWAG, Snips, and Story Cloze Test in zero- and few-shot settings. |
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it. |
| Approach: | They develop controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training. |
| Outcome: | The proposed models can make syntactic generalizations for tokens seen as few as two times during training and transfer them to transformed contexts. |
Investigating representations of verb bias in neural language models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Languages typically provide more than one grammatical construction to express certain types of messages. |
| Approach: | They propose a large benchmark dataset containing 50K human judgments for 5K distinct sentence pairs in the English dative alternation. |
| Outcome: | The proposed model outperforms recurrent architectures even under comparable parameter and training settings. |
Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze (2020.emnlp-main)
Copied to clipboard
| Challenge: | a long tradition of cognitive studies shows that the interplay between language and vision is complex. |
| Approach: | They propose an approach to image description generation where visual processing is modelled sequentially. |
| Outcome: | The proposed model exploits gaze-driven attention to produce better descriptions . it sheds light on human cognitive processes by comparing different ways of aligning gaze with language production. |
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for language understanding and understanding can be trained to provide contextualized representations of words based on text data. |
| Approach: | They propose a large-scale language VAE model Optimus that is pre-trained on large text corpus and fine-tuned for various language generation and understanding tasks. |
| Outcome: | The proposed model achieves new state-of-the-art on VAE language modeling benchmarks. |
BioMegatron: Larger Biomedical Domain Language Model (2020.emnlp-main)
Copied to clipboard
Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, Raghav Mani
| Challenge: | Existing studies on domain language models do not study the factors affecting performance on domain languages. |
| Approach: | They empirically evaluate factors that can affect performance on domain language applications . sub-word vocabulary set, model size, pre-training corpus, and domain transfer are important . |
| Outcome: | The results show language models trained on biomedical text perform better on biomedicine benchmarks than those trained on general domain text corpora. |
Text Segmentation by Cross Segment Attention (2020.emnlp-main)
Copied to clipboard
| Challenge: | Document and discourse segmentation are two fundamental NLP tasks pertaining to breaking up text into constituents. |
| Approach: | They propose three transformer-based NLP models that break up text into constituents and compare them to previous approaches. |
| Outcome: | The proposed architectures reduce errors by a large margin on three datasets and improve performance on real-world datasets. |
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark (2020.emnlp-main)
Copied to clipboard
Tatiana Shavrina, Alena Fenogenova, Emelyanov Anton, Denis Shevelev, Ekaterina Artemova, Valentin Malykh, Vladislav Mikhailov, Maria Tikhonova, Andrey Chertok, Andrey Evlampiev
| Challenge: | Modern scientific methodology is beginning to explore universal transformers as an independent object of study. |
| Approach: | They propose a Russian general language understanding evaluation benchmark - Russian SuperGLUE . they provide a benchmark of nine tasks, human level evaluation and a leaderboard for the Russian language . |
| Outcome: | The proposed benchmark provides nine tasks for the Russian language and human level evaluation and leaderboard of transformer models. |
An Empirical Study of Pre-trained Transformers for Arabic Information Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual pre-trained Transformers have been shown to enable effective cross-lingual zero-shot transfer, but their performance on Arabic information extraction tasks is not well studied. |
| Approach: | They pre-train a bilingual BERT that is designed specifically for Arabic NLP and English-to-Arabic zero-shot transfer learning. |
| Outcome: | The pre-trained model significantly outperforms mBERT, XLM-RoBERTa, and AraBERT in both the supervised and zero-shot transfer settings. |
TNT: Text Normalization based Pre-training of Transformers for Content Moderation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language model pre-training (self-supervised or unsupervised learning) has been widely used in a multitude of language processing tasks such as named entity recognition, sentiment analysis, question answering and content moderation. |
| Approach: | They propose a new language pre-training model TNT for content moderation that uses a combination of masking strategy and text normalization to learn from text. |
| Outcome: | The proposed model outperforms baselines on hate speech classification task and is a potential approach to misspelling correction. |
Methods for Numeracy-Preserving Word Embeddings (2020.emnlp-main)
Copied to clipboard
Dhanasekar Sundararaman, Shijing Si, Vivek Subramanian, Guoyin Wang, Devamanyu Hazarika, Lawrence Carin
| Challenge: | Word embedding models capture semantic relationships between words but fail to capture numerical properties associated with numbers. |
| Approach: | They propose a method to assign and learn embeddings for numbers using word embedders. |
| Outcome: | The proposed model outperforms pre-trained word embedding models across multiple examples of two tasks. |
An Empirical Investigation of Contextualized Number Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | a large scale empirical investigation of contextualized number prediction in running text is needed. |
| Approach: | They propose a suite of output distribution parameterizations that incorporate latent variables to add expressivity and better fit the natural distribution of numeric values in running text. |
| Outcome: | The proposed models outperform flow-based models on two numeric datasets in the financial and scientific domain. |
Modeling the Music Genre Perception across Language-Bound Cultures (2020.emnlp-main)
Copied to clipboard
| Challenge: | a prevalent approach to culturally study music genres assumes that the same music genre is associated with the items in all cultures. |
| Approach: | They propose to use distributed concept embeddings and ontologies to obtain cross-lingual music genre annotations using language-specific semantic representations. |
| Outcome: | The proposed model can be compared with existing models using domain-dependent cross-lingual corpus. |
Joint Estimation and Analysis of Risk Behavior Ratings in Movie Scripts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to identify portrayals of risk behaviors from audio-visual cues are limited in their applicability to films in post-production, where modifications might be prohibitively expensive. |
| Approach: | They propose a model that estimates content ratings based on the language use in movie scripts and leverages the co-occurrence of risk behaviors following a multi-task approach. |
| Outcome: | The proposed model improves state-of-the-art by adapting novel techniques to learn better movie representations from the semantic and sentiment aspects of a character’s language use and by leveraging the co-occurrence of risk behaviors, following a multi-task approach. |
Keep it Surprisingly Simple: A Simple First Order Graph Based Parsing Model for Joint Morphosyntactic Parsing in Sanskrit (2020.emnlp-main)
Copied to clipboard
| Challenge: | Morphologically rich languages benefit from joint processing of morphology and syntax, as compared to pipeline architectures. |
| Approach: | They propose a graph-based model for joint morphological parsing and dependency parser in Sanskrit using the Energy based model framework. |
| Outcome: | The proposed model outperforms standalone morphological parsers in morphology and syntax parsing, and in dependency parser. |
Unsupervised Parsing via Constituency Tests (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised parsing rely on constituency tests . linguists can judge a sentence's grammatical validity by modifying it via some transformation . |
| Approach: | They propose a method for unsupervised parsing based on a constituency test . they specify a set of transformations and use an unsupervised neural acceptability model to make grammaticality decisions. |
| Outcome: | The proposed method achieves 62.8 F1 on the Penn Treebank test set, an improvement of 7.6 points over the previous best results. |
Please Mind the Root: Decoding Arborescences for Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | a dependency tree has a root constraint, but only one edge may emanate from the root node. |
| Approach: | They propose an algorithm which enforces a root constraint without compromising the original runtime. |
| Outcome: | The proposed algorithm satisfies the constraint without compromising the original runtime. |
Unsupervised Cross-Lingual Part-of-Speech Tagging for Truly Low-Resource Scenarios (2020.emnlp-main)
Copied to clipboard
| Challenge: | a limited set of translations into one or more high-resource languages are available for POS tagging . a bi-LSTM architecture that uses contextualized word embeddings improves performance . |
| Approach: | They propose an unsupervised cross-lingual transfer approach for part-of-speech tagging . they use the Bible as parallel data to learn POS taggers for target languages . |
| Outcome: | The proposed approach improves accuracy on 12 diverse languages . the Bible is used as a parallel corpus for the study . |
Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | Syntactic parse trees are valuable intermediate features for many NLP pipelines. |
| Approach: | They propose an improved version of DIORA that encodes a single tree rather than a softly-weighted mixture of trees by employing a hard argmax operation and a beam at each cell in the chart. |
| Outcome: | The proposed model improves state-of-the-art in constituency parsing on the English WSJ Penn Treebank by 2.2-6% F1, depending on the data used for fine-tuning. |
Utility is in the Eye of the User: A Critique of NLP Leaderboards (2020.emnlp-main)
Copied to clipboard
| Challenge: | Performance-based evaluation has been at the expense of other attributes valued by the NLP community, such as compactness and energy efficiency. |
| Approach: | They propose to frame both the leaderboard and NLP practitioners as consumers and the benefit they get from a model as its utility to them. |
| Outcome: | The proposed model size and energy efficiency benchmarks have been successful in driving the creation of more accurate models, but have been at the expense of other attributes valued by the NLP community. |
An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-training large language models is a standard practice in the natural language processing community. |
| Approach: | They propose to use elastic weight consolidation to mitigate catastrophic forgetting when pre-trained large language models are evaluated on generic benchmarks. |
| Outcome: | The proposed model achieves state-of-the-art on out-of domain tasks with minimal pre-training . elastic weight consolidation provides best overall scores yielding only a 0.33% drop in performance across seven generic tasks while remaining competitive in bio-medical tasks. |
Analyzing Individual Neurons in Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that deep NLP models capture linguistic knowledge but little attention is paid to individual neurons. |
| Approach: | They conduct a neuron-level analysis of pre-trained neural language models to determine linguistic properties. |
| Outcome: | The proposed model is more localized and disjoint when predicting properties than BERT and others. |
Dissecting Span Identification Tasks with Performance Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Span identification tasks are a staple of applied NLP, but there is little insight on how their properties influence their difficulty. |
| Approach: | They propose to build a model to predict span ID performance for unseen span ID tasks that can support architecture choices. |
| Outcome: | The proposed model predicts span ID tasks for unseen span ID task in English, and the meta model predictable span ID performance. |
Assessing Phrasal Representation and Composition in Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for deep transformers are able to combine word meanings into phrase meanings, but they lack a clear understanding of how they handle complex linguistic inputs. |
| Approach: | They propose to analyze phrasal representations in pre-trained transformers to determine whether they reflect sophisticated composition of phrase meaning. |
| Outcome: | The proposed models are able to combine meaning units into larger units, a phenomenon known as composition, and reflects human understanding of meaning. |
Analyzing Redundancy in Pretrained Transformer Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that transformer-based deep NLP models are over-parameterized and do not require all the representational power lent by the rich architectural choices during inference. |
| Approach: | They define a notion of Redundancy and propose a feature-based transfer learning procedure which maintains 97% performance while using at-most 10% of the original neurons. |
| Outcome: | The proposed model maintains 97% performance while using 10% of the original neurons. |
Be More with Less: Hypergraph Attention Networks for Inductive Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Text classification is a critical research topic with broad applications in natural language processing. graph neural networks (GNNs) have received increasing attention but their performance is jeopardized in practice. |
| Approach: | They propose a model which captures long-distance interactions between words and a graph-based model which can be used to perform text classification. |
| Outcome: | The proposed model can achieve more expressive power with less computational consumption on the text classification task. |
Entities as Experts: Sparse Memory Access with Entity Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Unlike previous attempts to integrate entity knowledge into sequence models, EaE’s entity representations are learned directly from text. |
| Approach: | They propose a model that can access distinct memories of entities mentioned in a piece of text and a new architecture that can do this. |
| Outcome: | The proposed model outperforms an encoder-generator Transformer model with 10x the parameters on a task that requires 10x more parameters to answer. |
Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have been used to obtain state-of-the-art results on pronoun resolution. |
| Approach: | They propose to use pre-trained language models to train pronoun resolution models . they compare performance and seed-wise stability of four models that represent four objectives . |
| Outcome: | The proposed model performs best in-domain, while the other objectives perform poorly out-of-domain. |
On Losses for Modern Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Devlin et al. ( 2018) released a transformer network (BERT) pre-training over two tasks: masked language modelling (MLM) and next sentence prediction (NSP). |
| Approach: | They clarify NSP's effect on BERT pre-training and explore ways to include multiple tasks into pre-train. |
| Outcome: | The proposed framework outperforms BERTBase on the GLUE benchmark using fewer than a quarter of training tokens. |
We Can Detect Your Bias: Predicting the Political Ideology of News Articles (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines the role of media in predicting political ideology or bias in news articles . systematic exposure to bias in the news can foster intolerance and ideological segregation . |
| Approach: | They propose an adversarial media adaptation and a specially adapted triplet loss for predicting political ideology in news articles. |
| Outcome: | The proposed model improves over state-of-the-art models in this challenging setup. |
Semantic Label Smoothing for Sequence to Sequence Problems (2020.emnlp-main)
Copied to clipboard
Michal Lukasik, Himanshu Jain, Aditya Menon, Seungyeon Kim, Srinadh Bhojanapalli, Felix Yu, Sanjiv Kumar
| Challenge: | Existing methods for seq2seq regularization use label smoothing, but it is difficult to extend it to other datasets. |
| Approach: | They propose a method that smooths over well formed relevant sequences that are semantically similar to the target sequence. |
| Outcome: | The proposed method shows a consistent and significant improvement over the state-of-the-art methods on different datasets. |
Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on simple approaches to model the output label space . graphical models are often limited to (heuristic) greedy search and its variants . |
| Approach: | They propose an approach for efficiently training and decoding hybrids of graphical and graphical models based on Gibbs sampling. |
| Outcome: | The proposed approach improves on Dutch and Dutch with graphical models . the proposed model improves over a strong baseline on three languages . |
Multilevel Text Alignment with Cross-Document Attention (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing alignment methods operate at a single, predefined level and cannot learn to align texts at sentence and document levels. |
| Approach: | They propose a learning approach that equips hierarchical attention encoders for representing documents with a cross-document attention component, enabling structural comparisons across different levels. |
| Outcome: | The proposed model outperforms existing hierarchical, attention encoders on citation recommendation and plagiarism detection tasks. |
Conversational Semantic Parsing (2020.emnlp-main)
Copied to clipboard
Armen Aghajanyan, Jean Maillard, Akshat Shrivastava, Keith Diedrick, Michael Haeger, Haoran Li, Yashar Mehdad, Veselin Stoyanov, Anuj Kumar, Mike Lewis, Sonal Gupta
| Challenge: | Structured representations for task-oriented assistant systems are limited due to the limitations of the representation. |
| Approach: | They propose a semantic representation for task-oriented conversational systems that can represent co-reference and context carryover. |
| Outcome: | The proposed model improves the best results on ATIS, SNIPS, TOP and DSTC2 by up to 5 points for slot-carryover. |
Probing Task-Oriented Dialogue Representation from Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using pre-trained language models, we find out which model has the most informative representation for task-oriented dialogue tasks. |
| Approach: | They propose a supervised classifier probe and unsupervised mutual information probe to investigate the mutual dependence between a real clustering and a representation clustering. |
| Outcome: | The proposed model is a supervised classifier probe and unsupervised mutual information probe. |
End-to-End Slot Alignment and Recognition for Cross-Lingual NLU (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to expand an NLU system to new languages use machine translation with slot label projection from source to translated utterances. |
| Approach: | They propose a novel end-to-end model that learns to align and predict target slot labels jointly for cross-lingual transfer. |
| Outcome: | The proposed model outperforms a simple label projection method on most languages and achieves competitive performance to the more complex, state-of-the-art projection method with only half the training time. |
Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference (2020.emnlp-main)
Copied to clipboard
Jianguo Zhang, Kazuma Hashimoto, Wenhao Liu, Chien-Sheng Wu, Yao Wan, Philip Yu, Richard Socher, Caiming Xiong
| Challenge: | Existing work on few-shot intent classification without OOS has focused on the few-shot intent classification with out-of-scope intents. |
| Approach: | They propose to use BERT-style pairwise encoding to train a binary classifier that estimates the best matched training example for a user input. |
| Outcome: | The proposed approach achieves more stable and accurate in-domain and OOS detection accuracy than RoBERTa-based classifiers and embedding-based nearest neighbor approaches. |
Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on DA tagging focus on human-human social conversations, which is less applicable for task-oriented setting. |
| Approach: | They propose a controllable mechanism that augments text input by leveraging the pre-trained Mask token from BERT model. |
| Outcome: | The proposed mechanism augments text input by leveraging the pre-trained Mask token from BERT model. |
Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have enabled several approaches to successfully parse more complex queries, but these models require a large amount of annotated training data to parser on new domains (e.g. reminder, music). |
| Approach: | They propose a method that adapts task-oriented semantic parsers to low-resource domains and outperforms a supervised neural model at a 10-fold data reduction. |
| Outcome: | The proposed method outperforms baseline methods on a newly collected multi-domain task-oriented semantic parsing dataset (TOPv2) . |
Sound Natural: Content Rephrasing in Dialog Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, virtual assistants work in the paradigm of intent-slot tagging and the slot values are directly passed as-is to the execution engine. |
| Approach: | They propose to use BART to rephrase a query to make it more natural . they propose to add a copy-pointer and copy loss to it to improve performance . |
| Outcome: | The proposed model improves on existing models by adding a copy-pointer and copy loss. |
Zero-Shot Crosslingual Sentence Simplification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to simplification have shown promising results with encoder-decoder models trained on large amounts of parallel data which often only exists in English. |
| Approach: | They propose a model which transfers simplification knowledge from English to another language while generalizing across languages and tasks. |
| Outcome: | Empirical results show that the proposed model performs better than unsupervised and pivot-based methods. |
Facilitating the Communication of Politeness through Fine-Grained Paraphrasing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using technology, people are increasingly able to communicate across geographical, cultural and language barriers, but they also face new challenges, as they need to adapt their communication approaches to increasingly diverse circumstances. |
| Approach: | They propose a method for suggesting paraphrases that achieve the intended level of politeness under a given communication circumstance and evaluate it in two realistic communication scenarios. |
| Outcome: | The proposed method reduces misalignment between the speaker’s intentions and listener’s perceptions in two realistic communication scenarios and is able to communicate with people from different backgrounds in diverse contexts. |
CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial text generation approaches can lead to generation lacking diversity or fluency, whereas perturbing in the intermediate representation space can lead a model to generate generations that are not related to the input. |
| Approach: | They propose to generate adversarial texts through controllable attributes that are known to be invariant to task labels. |
| Outcome: | The proposed model generates more diverse and fluent adversarial examples, compared to existing approaches, and is more robust against model re-training and different model architectures. |
Seq2Edits: Sequence Transduction Using Span-level Edit Operations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Seq2Edits is an open-vocabulary approach to sequence editing for natural language processing tasks with a high degree of overlap between input and output texts. |
| Approach: | They propose an open-vocabulary approach to sequence editing for NLP tasks with a high degree of overlap between input and output texts. |
| Outcome: | The proposed approach speeds up inference by up to 5.2x compared to full sequence models . it improves explainability by associating each edit operation with a human-readable tag. |
Controllable Meaning Representation to Text Generation: Linearization and Data Augmentation Strategies (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using task-oriented dialogue generation benchmarks, we compare the effect of four input linearization strategies on controllability and faithfulness. |
| Approach: | They compare the effect of four input linearization strategies on controllability and faithfulness . they also evaluate how a phrase-based data augmentation method can improve performance . |
| Outcome: | The proposed model can generate utterances whose phrases follow the order of the provided plan. |
Blank Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches focus on adapting left-to-right language models for text infilling. |
| Approach: | They propose a model that generates sequences by dynamically creating and filling in blanks. |
| Outcome: | Experiments on style transfer and damaged ancient text restoration show that the proposed model outperforms baseline models in terms of accuracy and fluency. |
COD3S: Diverse Generation with Discrete Semantic Signatures (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating semantically diverse sentences are based on locality-sensitive hash (LSH)-based semantic sentence codes that explicitly capture meaningful semantic differences. |
| Approach: | They propose a method for generating semantically diverse sentences using neural sequence-to-sequence models by conditioned on locality-sensitive hash-based semantic sentence codes whose Hamming distances correlate with human judgments of semantic textual similarity. |
| Outcome: | The proposed method improves output diversity without degrading performance on causal generation tasks. |
Automatic Extraction of Rules Governing Morphological Agreement (2020.emnlp-main)
Copied to clipboard
Aditi Chaudhary, Antonios Anastasopoulos, Adithya Pratapa, David R. Mortensen, Zaid Sheikh, Yulia Tsvetkov, Graham Neubig
| Challenge: | Creating a descriptive grammar is an indispensable step for language documentation but it is tedious and time-consuming. |
| Approach: | They propose a framework for extracting a first-pass grammatical specification from raw text in a concise, human- and machine-readable format. |
| Outcome: | The proposed framework extracts a grammatical specification that is nearly equivalent to those created with large amounts of gold-standard annotated data. |
Tackling the Low-resource Challenge for Canonical Segmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | morphological segmentation is a task of dividing words into their constituting morphemes . we compare two new approaches for the task when training data is limited . |
| Approach: | They propose to use an LSTM pointer-generator and a sequence-to-sequence model to perform canonical segmentation when training data is limited. |
| Outcome: | The proposed models outperform existing models on German, English, and Indonesian in low-resource scenarios by 11.4% accuracy. |
IGT2P: From Interlinear Glossed Texts to Paradigms (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for learning morphology have limited their use to languages with publicly available structured data, such as online dictionaries like Wiktionary. |
| Approach: | They propose a task that generates entire morphological paradigms from IGT input and a language expert cleaning noisy IGT data. |
| Outcome: | The proposed task speeds up the process and generates entire morphological paradigm tables from IGT input. |
A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empathy measurement has predominantly occurred in synchronous, face-to-face settings, and may not translate to asynchronous, text-based contexts. |
| Approach: | They propose a computational approach to understanding how empathy is expressed in online mental health platforms. |
| Outcome: | The proposed model can identify empathic conversations and extract rationales from them. |
Modeling Protagonist Emotions for Emotion-Aware Storytelling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cognitive scientists have pinpointed the central role of emotions in storytelling. |
| Approach: | They propose to use Emotion Supervision and two Emotion-Reinforced models to generate stories that follow the desired emotion arcs for the protagonist. |
| Outcome: | The proposed models generate stories that follow the desired emotion arcs without sacrificing story quality. |
Help! Need Advice on Identifying Advice (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained systems are able to capture advice better than rule-based systems, but advice identification is challenging. |
| Approach: | They analyze a dataset of advice posts on two reddit forums and annotate whether they contain advice. |
| Outcome: | The proposed models show that pre-trained models capture advice better than rule-based systems, but advice identification is challenging. |
Quantifying Intimacy in Language (2020.emnlp-main)
Copied to clipboard
| Challenge: | Intimacy is a fundamental aspect of how we relate to others in social settings. |
| Approach: | They propose a computational framework for studying the intimacy in language with a dataset and a deep learning model for accurately predicting the intimacy level of questions. |
| Outcome: | The proposed framework enables the analysis of 80.5M questions across social media, books, and films to quantify the intimacy expressed in language and to predict the intimacy level. |
Writing Strategies for Science Communication: Data and Computational Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing science communication guides do not provide empirical evidence for how their strategies are used in practice. |
| Approach: | They propose to use prescriptive writing strategies to identify and train human-readable annotations that can be automatically recognized by a corpus of 128k science writing documents in English. |
| Outcome: | The proposed system can be used to detect and suggest writing strategies for scientists by allowing them to automatically recognize them. |
Weakly Supervised Subevent Knowledge Acquisition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Subevents elaborate an event and exist in event descriptions. |
| Approach: | They propose a weakly supervised approach to extract subevent relation tuples from text . they then use the initial seed subeven pairs to train a contextual classifier . |
| Outcome: | The proposed method is high quality and covers a wide range of event types. |
Biomedical Event Extraction as Sequence Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results show that BeeSL’s speed and accuracy makes it a viable approach for large-scale real-world scenarios. |
| Approach: | They propose a joint end-to-end neural information extraction model that recasts the task as sequence labeling and jointly models intermediate tasks via multi-task learning. |
| Outcome: | Empirical results show that BeeSL outperforms the current best system on the Genia 2011 benchmark by 1.57% absolute F1 score reaching 60.22% F1 . |
Annotating Temporal Dependency Graphs via Crowdsourcing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing temporal annotation schemes have been limited due to the complexity of temporal relations between events. |
| Approach: | They propose to build a corpus of Wikinews articles annotated with temporal dependency graphs . they also propose a crowdsourcing strategy to annotate TDGs based on the corpus . |
| Outcome: | The proposed method achieves a good trade-off between completeness and practicality in temporal annotation. |
Introducing a New Dataset for Event Detection in Cybersecurity Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | a large amount of text data is produced to report and discuss cyber vulnerabilities . detecting cybersecurity events is necessary to keep us informed about the fast growing number of such events reported in text. |
| Approach: | They propose a dataset characterizing the manual annotation for 30 important cybersecurity event types and a large dataset to develop deep learning models. |
| Outcome: | The proposed dataset characterizes the manual annotation for 30 important event types and supports the modeling of document-level information to improve the performance. |
CHARM: Inferring Personal Attributes from Conversations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Personal Knowledge Bases (PKBs) capture individual user traits for customizing downstream applications like chatbots or recommenders. |
| Approach: | They propose a method that leverages keyword extraction and document retrieval to predict attribute values that were never seen during training. |
| Outcome: | The proposed method can predict attributes that were never seen during training. |
Event Detection: Gate Diversity and Syntactic Importance Scores for Graph Convolution Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on event detection (ED) have shown that the syntactic dependency graph can be employed in graph convolutional neural networks (GCNs) but the computation of the hidden vectors in such graph-based models is agnostic to the trigger candidate words, leaving irrelevant information for the trigger candidates. |
| Approach: | They propose a mechanism to filter noisy information in the hidden vectors of graph-based models based on the information from the trigger candidate. |
| Outcome: | The proposed model achieves state-of-the-art on two ED datasets. |
Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events (2020.emnlp-main)
Copied to clipboard
Miguel Ballesteros, Rishita Anubhai, Shuai Wang, Nima Pourdamghani, Yogarshi Vyas, Jie Ma, Parminder Bhatia, Kathleen McKeown, Yaser Al-Onaizan
| Challenge: | Existing models for temporal ordering of events rely on pretrained representations, transfer and multitask learning, and self-training techniques. |
| Approach: | They propose a neural architecture and a set of training methods for ordering events by predicting temporal relations by pre-training models. |
| Outcome: | The proposed models can predict temporal relations between two pairs of events within a span of text and identify temporal relationships between them. |
How Much Knowledge Can You Pack Into the Parameters of a Language Model? (2020.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we show that large neural language models trained on unstructured text can attain competitive results on open-domain question answering benchmarks without access to external knowledge. |
| Approach: | They propose to fine-tune pre-trained neural language models to answer questions without external knowledge . they show that this approach scales with model size and performs competitively . |
| Outcome: | The proposed approach scales with model size and performs competitively with open-domain systems that explicitly retrieve answers from an external knowledge source when answering questions. |
EXAMS: A Multi-subject High School Examinations Dataset for Cross-lingual and Multilingual Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | EXAMS is a benchmark dataset for cross-lingual and multilingual question answering for high school examinations. |
| Approach: | They propose to use EXAMS to evaluate cross-lingual and multilingual question answering for high school examinations. |
| Outcome: | The proposed model can be used to explore multilingual reasoning and knowledge transfer methods and pre-trained models in schools in different languages, which was not possible by now. |
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems (2020.emnlp-main)
Copied to clipboard
Siamak Shakeri, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang
| Challenge: | Existing approaches for synthetic QA data generation have limited or no success in improving the downstream Reading Comprehension task. |
| Approach: | They propose an end-to-end approach for synthetic QA data generation using a transformer-based encoder-decoder network that is trained end- to-end to generate both answers and questions. |
| Outcome: | The proposed model outperforms current state-of-the-art methods in the domain adaptation of QA models. |
Multi-Stage Pre-training for Low-Resource Domain Adaptation (2020.emnlp-main)
Copied to clipboard
Rong Zhang, Revanth Gangi Reddy, Md Arafat Sultan, Vittorio Castelli, Anthony Ferritto, Radu Florian, Efsun Sarioglu Kayi, Salim Roukos, Avi Sil, Todd Ward
| Challenge: | Existing approaches to transfer learning target data to in-domain text . prior work has adapted pre-trained LMs to specific domains . |
| Approach: | They extend the vocabulary of a pretrained language model with domain-specific terms to create synthetic tasks that help it transfer to downstream tasks. |
| Outcome: | The proposed approaches show significant performance gains on extractive reading comprehension, document ranking and duplicate question detection tasks. |
ISAAQ - Mastering Textbook Questions with Pre-trained Transformers and Bottom-Up and Top-Down Attention (2020.emnlp-main)
Copied to clipboard
| Challenge: | Textbook Question Answering is a complex task that requires reasoning with multimodal information from text and diagrams. |
| Approach: | They propose to use transformer language models and bottom-up and top-down attention to tackle the language and visual understanding challenges of text and diagrams. |
| Outcome: | The proposed system achieves unprecedented accuracies on all TQA question types . the system also obtains state-of-the-art results in other demanding datasets . |
SubjQA: A Dataset for Subjectivity and Review Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Subjectivity is the expression of internal opinions or beliefs which cannot be objectively observed or verified. |
| Approach: | They develop a dataset which investigates subjectivity in question answering . they find that subjectivity is an important feature in the case of QA . |
| Outcome: | The proposed dataset shows that subjectivity is an important feature in question answering (QA) it also shows that subjective questions and answers can have more complex interactions than previously thought. |
Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing tools for examining and fixing missing captions are lacking in mobile UIs. |
| Approach: | They propose a task for automatically generating language descriptions for UI elements from multimodal input including both the image and structural representations of user interfaces. |
| Outcome: | The proposed task can generate captions from image and structural representations of UI elements. |
Unsupervised Natural Language Inference via Decoupled Multimodal Contrastive Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that humans are not supervised by the natural language inference . |
| Approach: | They propose to solve the natural language inference problem via task-agnostic multimodal pretraining. |
| Outcome: | The proposed network outperforms fully-supervised BiLSTM and BiLS+ELMO on plain text inference datasets. |
Digital Voicing of Silent Speech (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using electromyography, we can convert silently mouthed words into audible speech . prior work focused on training speech synthesis models from vocalized data . |
| Approach: | They propose a method of training on silent EMG by transferring audio targets from vocalized to silent signals and propose voicing task using muscle sensor measurements. |
| Outcome: | The proposed method greatly improves intelligibility of audio generated from silent EMG compared to baseline that only trains with vocalized data. |
Imitation Attacks and Defenses for Black-box Machine Translation Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using simulated experiments, we demonstrate that MT systems can be stolen even when imitation models have different input data or architectures than their target models. |
| Approach: | They propose a defense that modifies translation outputs to misdirect optimization of imitation models. |
| Outcome: | The proposed defense degrades the adversary’s BLEU score and attack success rate at some cost in the defender’s performance and inference speed. |
Sequence-Level Mixed Sample Data Augmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite their empirical success, neural networks still have difficulty capturing compositional aspects of natural language. |
| Approach: | They propose a data augmentation approach to encourage compositional behavior in neural networks . they propose to softly combine input/output sequences from the training set . |
| Outcome: | The proposed approach yields 1.0 BLEU improvement on translation datasets over baselines. |
Consistency of a Recurrent Language Model With Respect to Incomplete Decoding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural sequence models trained with maximum likelihood have been shown to exhibit issues such as length bias and degenerate repetition. |
| Approach: | They propose to use a recurrent language model to address inconsistency in decoding algorithms that are inconsistent despite the fact that recursive language models are trained to produce sequences of finite length. |
| Outcome: | The proposed methods prevent inconsistency in the proposed models. |
An Exploration of Arbitrary-Order Sequence Labeling via Energy-Based Inference Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that conditional random fields (CRFs) perform well in sequence labeling tasks. |
| Approach: | They propose several high-order energy terms to capture dependencies among labels in sequence labeling . they use convolutional, recurrent, and self-attention networks to construct these energy terms . |
| Outcome: | The proposed approach improves on four sequence labeling tasks while having the same decoding speed as simple classifiers. |
Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast—Choose Three (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern neural networks do not always produce wellcalibrated predictions . post-hoc calibration methods require a held-out calibration dataset, which may not be available in all circumstances. |
| Approach: | They validate ensemble distillation framework for producing well-calibrated structured prediction models without the prohibitive inference-time cost of ensembles. |
| Outcome: | The proposed framework produces well-calibrated predictions without the prohibitive inference-time cost of ensembles. |
Inducing Target-Specific Latent Structures for Aspect Sentiment Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Aspect-level sentiment analysis aims to classify the sentiment polarity of an aspect or a target in a comment . graph convolutional networks can be used to classifice aspect terms in syllables . |
| Approach: | They propose to combine word dependency graphs and latent graphs to create latent models . they propose to model the interaction between the aspect and its surrounding contexts . |
| Outcome: | The proposed model can complement syntactic features with latent semantic dependencies. |
Affective Event Classification with Discourse-enhanced Self-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on recognizing affective events focused on producing lexical resources of verbs or event phrases with corresponding affective polarity values. |
| Approach: | They propose a BERT-based model for affective event classification and a discourse-enhanced self-training method that iteratively improves the classifier with unlabeled data. |
| Outcome: | The proposed model outperforms existing models with unlabeled data and improves recall and precision. |
Deep Weighted MaxSAT for Aspect-based Opinion Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing deep learning models lack the capability to encode explicit domain knowledge to model complex causal relationships among variables. |
| Approach: | They propose a model that uses a weighted version of MaxSAT to model logic inference . they propose to use this model to rectify erroneous predictions from deep neural networks . |
| Outcome: | The proposed model combines the benefits of high-level feature learning, knowledge reasoning, and structured learning with observable performance gain for aspect-based opinion extraction. |
Multi-view Story Characterization from Movie Plot Synopses and Reviews (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for characterizing stories by generating tags from synopses suffer from coverage issues. |
| Approach: | They propose to use synopses and reviews to characterize stories by inferring attributes such as theme and style from written synopsis and reviews. |
| Outcome: | The proposed model improves over methods that only use synopses and reviews . it can extract a complementary set of story attributes from reviews without supervision . |
Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Inflectional variation is a common feature of World Englishes such as Colloquial Singapore English and African American Vernacular English. |
| Approach: | They propose a method to tokenize English text by reducing inflected words to their base forms before reinjecting the grammatical information as special symbols. |
| Outcome: | The proposed method defends against inflectional adversaries while maintaining performance on clean data. |
Measuring the Similarity of Grammatical Gender Systems by Comparing Partitions (2020.emnlp-main)
Copied to clipboard
| Challenge: | A grammatical gender system divides a lexicon into a small number of fixed categories with fixed usage across speakers. |
| Approach: | They propose to define gender systems extensionally to reduce comparisons to cluster evaluation by comparing pairwise overlaps between gender systems. |
| Outcome: | The proposed measures are based on a phylogenetic tree over extant Indo-European languages. |
RethinkCWS: Is Chinese Word Segmentation a Solved Task? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have seen remarkable success in the use of deep neural networks on Chinese word segmentation (CWS) however, the performance of CWS systems has gradually reached a plateau with the rapid development of deep networks. |
| Approach: | They propose a fine-grained evaluation for existing Chinese word segmentation systems that allows us to diagnose the strengths and weaknesses of existing models. |
| Outcome: | The proposed model can diagnose strengths and weaknesses of existing models and alleviate negative transfer problem when doing multi-criteria learning. |
Learning to Pronounce Chinese Without a Pronunciation Dictionary (2020.emnlp-main)
Copied to clipboard
| Challenge: | EM method achieves a test-set accuracy of 71%, vector-based method achieve 81%. |
| Approach: | They propose a program that learns to pronounce Chinese text in Mandarin without a pronunciation dictionary. |
| Outcome: | The proposed program deciphers Chinese text in Mandarin without a pronunciation dictionary. |
Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing reasoning methods for sparse KGs are incomplete and lack of evidential paths to target entities makes multi-hop reasoning difficult. |
| Approach: | They propose a multi-hop reasoning model over sparse KGs to solve this problem . they use latent prediction of embedding-based models to make the model perform more potential path search over sparses . |
| Outcome: | The proposed method outperforms state-of-the-art models on five datasets from Freebase, NELL and Wikidata. |
Knowledge Association with Hyperbolic Knowledge Graph Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge graphs (KGs) depend on high embedding dimensions and hierarchical structures to achieve expressiveness. |
| Approach: | They propose a hyperbolic relational graph neural network for KG embedding and capture knowledge associations with a high-dimensional transformation. |
| Outcome: | Experiments on entity alignment and type inference show the proposed method is effective and efficient. |
Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to extract event temporal relations from text data are limited by hard constraints and large datasets. |
| Approach: | They propose a framework that enhances deep neural network with distributional constraints constructed by probabilistic domain knowledge to improve the baseline neural network models. |
| Outcome: | The proposed framework improves baseline models with strong statistical significance on two widely used datasets in news and clinical domains. |
TeMP: Temporal Message Passing for Temporal Knowledge Graph Completion (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for static knowledge graphs do not explicitly leverage multi-hop structural information and temporal facts from recent time steps to enhance their predictions. |
| Approach: | They propose a framework to leverage time-dependent temporal information to infer missing facts in temporal knowledge graphs. |
| Outcome: | The proposed framework achieves 10.7% improvement in Hits@10 across three standard benchmarks. |
Understanding the Difficulty of Training Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Admin (Adaptive model initialization) is more stable, converges faster, and leads to better performance. |
| Approach: | They propose a model initialization algorithm to stabilize early training and unleash its full potential in the late stage. |
| Outcome: | The proposed model initialization method stabilizes early training and unleashes full potential in late stage. |
An Empirical Study of Generation Order for Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study of generation order for machine translation shows it does not affect output quality . Neural sequence models have been successfully applied to a broad range of tasks in recent years . |
| Approach: | They propose a soft order-reward framework that enables models to follow arbitrary oracle generation policies. |
| Outcome: | The proposed framework explores a wide variety of generation orders including uninformed orders, location-based orders, frequency-based or model-based orderings, and model-driven orders. |
Inference Strategies for Machine Translation with Conditional Masking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Conditional masked language model training has proven successful for non-autoregressive and semi-auto-regressively sequence generation tasks. |
| Approach: | They propose a conditional masked language model (CMLM) that is a factorization of conditional probabilities of partial sequences and propose heuristics to improve performance. |
| Outcome: | The proposed algorithm is more efficient than the standard “mask-predict” algorithm on machine translation tasks. |
AmbigQA: Answering Ambiguous Open-domain Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain question answering systems assume questions have a single welldefined answer. |
| Approach: | They propose an open-domain question answering task which involves finding every plausible answer and rewriting the question for each one to resolve the ambiguity. |
| Outcome: | The proposed task is based on a dataset covering 14,042 open-domain questions . it shows that strong models benefit from weakly supervised learning . |
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing data augmentation techniques for natural language processing tasks are difficult to design. |
| Approach: | They propose a controllable rewriting based question data augmentation method for machine reading comprehension, question generation and question-answering natural language inference tasks. |
| Outcome: | The proposed method generates high-quality, high-quality question data samples on machine reading comprehension, question generation, and question-answering natural language inference tasks. |
Training Question Answering Models From Synthetic Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on question and answer generation aims to improve question answering models given limited amount of labeled data. |
| Approach: | They synthesize questions and answers from a synthetic text corpus generated by an 8.3 billion parameter GPT-2 model and achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. |
| Outcome: | The proposed model achieves higher accuracy than the SQUAD1.1 training set questions using synthetic questions and answers than the training set question. |
Few-Shot Complex Knowledge Base Question Answering via Meta Reinforcement Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to complex question-answering (CQA) exhibit uneven performance when questions have different types, harboring inherently different characteristics, e.g., difficulty level. |
| Approach: | They propose a meta-reinforcement learning approach to program induction in CQA to tackle the potential distributional bias in questions. |
| Outcome: | The proposed method achieves state-of-the-art performance on the CQA dataset while using only five trial trajectories for the top-5 retrieved questions in each support set. |
Multilingual Offensive Language Identification with Cross-lingual Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Several studies investigating methods to detect offensive content in social media use English data. |
| Approach: | They apply cross-lingual contextual embeddings and transfer learning to make predictions in languages with less resources. |
| Outcome: | The proposed method compares favorably to the best systems submitted to recent shared tasks on Bengali, Hindi, and Spanish. |
Solving Historical Dictionary Codes with a Neural Language Model (2020.emnlp-main)
Copied to clipboard
| Challenge: | a dictionary-based substitution code is common, but no automatic decipherment algorithms exist. |
| Approach: | They propose a decoding lattice and a neural language model to solve word-based substitution codes . they apply their method to letters exchanged between general James Wilkinson and agents of the Spanish Crown . |
| Outcome: | The proposed method decrypts letters written by general James Wilkinson and agents of the Spanish Crown in the late 1700s and early 1800s using a neural language model. |
Toward Micro-Dialect Identification in Diaglossic and Code-Switched Environments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on dialect prediction is limited to coarse-grained varieties . a new language model, MARBERT, can predict micro-dialects with 9.9% F1, 76 better than a majority class baseline. |
| Approach: | They propose a new task of Micro-Dialect Identification (MDI) that can predict a fine-grained variety given a single message. |
| Outcome: | The proposed model predicts micro-dialects with 9.9% F1, 76 better than a majority class baseline. |
Investigating African-American Vernacular English in Transformer-Based Text Generation (2020.emnlp-main)
Copied to clipboard
Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, William Yang Wang
| Challenge: | Recent work in Natural Language Generation (NLG) uses a Transformer-based language model to generate high-quality, coherent text when prompted by arbitrary input. |
| Approach: | They evaluate the performance of a Transformer-based model that generates high-quality, coherent text when prompted by arbitrary input. |
| Outcome: | The proposed model improves on AAVE and SAE text with pretrained sentiment classifiers. |
Iterative Domain-Repaired Back-Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that NMT models perform poorly in specific domains when in-domain parallel corpora are scarce or nonexistent. |
| Approach: | They propose an iterative domain-repaired back-translation framework to refine translations in bilingual data by round-trip translating monolingual sentences. |
| Outcome: | The proposed framework achieves 15.79 and 4.47 BLEU improvements over unadapted models and back-translation in domain-specific translations. |
Dynamic Data Selection and Weighting for Iterative Back-Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results demonstrate that our methods achieve improvements of up to 1.8 BLEU points over competitive baselines. |
| Approach: | They propose a data selection and weighting strategy to iterate back-translation models and apply it to it . they use a target language to back-transcribe monolingual data, which is of high quality and reflect the target domain. |
| Outcome: | The proposed approach achieves 1.8 BLEU points over baselines on domain adaptation, low-resource, and high-resourced MT settings and on two language pairs. |
Revisiting Modularized Multilingual NMT to Meet Industrial Demands (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, the complete sharing of parameters for multilingual translation (1-1) is the most popular approach because of its compactness. |
| Approach: | They propose to use a multilingual neural machine translation model that only shares modules among the same languages as 1-1 to satisfy industrial requirements. |
| Outcome: | The proposed model can enjoy the benefits of multi-way training without the capacity bottleneck and low maintainability. |
LAReQA: Language-Agnostic Answer Retrieval from a Multilingual Pool (2020.emnlp-main)
Copied to clipboard
| Challenge: | LAReQA tests for “strong” cross-lingual alignment, requiring semantically related cross-language pairs to be closer in representation space than unrelated same-language pair. |
| Approach: | They propose a new benchmark for language-agnostic answer retrieval from a multilingual candidate pool that tests for "strong" cross-lingual alignment . they augment training data via machine translation and find that model performance is improved by augmenting training data through machine translation . |
| Outcome: | The proposed task is based on multilingual BERT (mBERT) and XLM-R. |
OCR Post Correction for Endangered Language Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, there is little to no data available to build natural language processing models for endangered languages. |
| Approach: | They propose a benchmark dataset of transcriptions for scanned books in three critically endangered languages and a method to improve OCR in these data-scarce settings. |
| Outcome: | The proposed method reduces the recognition error rate by 34% across the three endangered languages. |
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language models (LMs) capture factual knowledge by filling in the blanks of cloze-style prompts. |
| Approach: | They propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
| Outcome: | The proposed method improves the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other. |
| Approach: | They exploit the signals embedded in URLs to label web documents at scale with an average precision of 94.5% across different language pairs. |
| Outcome: | The proposed method can label documents at 94.5% across languages with high precision . the proposed method is useful for low-resource languages with limited resources . |
Localizing Open-Ontology QA Semantic Parsers in a Day Using Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new toolkit for localizing a semantic parser for a language is proposed . the proposed approach is based on a method for question answering systems . |
| Approach: | They propose a toolkit that leverages Neural Machine Translation systems to localize a semantic parser for a new language. |
| Outcome: | The proposed approach outperforms state-of-the-art methods in 10 new languages . it can be deployed in restaurants and hotels in less than 24 hours . |
Interactive Refinement of Cross-Lingual Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings transfer knowledge between languages to models trained on resource-rich languages can predict in low-resource languages. |
| Approach: | They propose an interactive system to quickly refine cross-lingual word embeddings for a given classification problem. |
| Outcome: | The proposed system improves on identifying health-related text in four low-resource languages. |
Exploiting Sentence Order in Document Alignment (2020.emnlp-main)
Copied to clipboard
| Challenge: | a document alignment method that exploits sentence order information is beneficial even when the end goal is sentence-level bitext. |
| Approach: | They propose a document alignment method that incorporates sentence order information in both candidate generation and candidate re-scoring. |
| Outcome: | The proposed method outperforms the most recent document alignment method on Sinhala–English documents. |
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)
Copied to clipboard
Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, Ming Zhou
| Challenge: | XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios . |
| Approach: | They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora. |
| Outcome: | The proposed dataset is labeled in English and includes only natural language understanding tasks. |
AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to sequence labeling require sequential computation that makes parallelization impossible. |
| Approach: | They propose to employ a parallelizable approximate variational inference algorithm for the CRF model. |
| Outcome: | The proposed approach improves decoding speed and accuracy with long sentences and is parallelizable for faster training and prediction. |
HIT: Nested Named Entity Recognition via Head-Tail Pair and Token Interaction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental task in natural language processing due to the nature of the named entity. |
| Approach: | They propose a nested NER model that leverages two key properties pertaining to the named entity, including explicit boundary tokens and tight internal connection between tokens within the boundary. |
| Outcome: | The proposed model achieves state-of-the-art on three public NER datasets. |
Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on supertagging but have not tapped into contextual information. |
| Approach: | They propose to build a graph from chunks extracted from a lexicon and apply attention over it to enhance supertagging by leveraging contextual information. |
| Outcome: | The proposed approach outperforms previous studies in terms of supertagging and parsing. |
DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks (2020.emnlp-main)
Copied to clipboard
Bosheng Ding, Linlin Liu, Lidong Bing, Canasai Kruengkrai, Thien Hai Nguyen, Shafiq Joty, Luo Si, Chunyan Miao
| Challenge: | Data augmentation techniques are widely used to improve machine learning performance . however, due to the complexity of language, it is difficult to generalize such rules for languages. |
| Approach: | They propose a method to generate high quality synthetic data for low-resource tagging tasks . they use unlabeled data only and unlabelled data plus a knowledge base . |
| Outcome: | The proposed method outperforms baselines on NER, part of speech and target based sentiment analysis tasks. |
Interpretable Multi-dataset Evaluation for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation methods for named entity recognition tasks are difficult to interpret . authors present a general methodology for interpretable evaluation for named entities . |
| Approach: | They propose a general methodology for interpretable evaluation for named entity recognition task. |
| Outcome: | The proposed evaluation method enables researchers to interpret differences in models and datasets . it makes it easy for future researchers to run similar analyses and drive progress in this area . |
Adversarial Semantic Decoupling for Recognizing Open-Vocabulary Slots (2020.emnlp-main)
Copied to clipboard
| Challenge: | Open-vocabulary slots degrade neural-based slot filling models because they can take on unlimited set of values and have no semantic restriction nor length limit. |
| Approach: | They propose a model-agnostic slot filling method that explicitly decouples local semantics inherent in open-vocabulary slot words from the global context. |
| Outcome: | The proposed method outperforms other models on open-vocabulary slots without deteriorating performance. |
Plug and Play Autoencoders for Conditional Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Text autoencoders are used for conditional generation tasks such as style transfer. |
| Approach: | They propose a plug-and-play method where any pretrained autoencoder can be used and only requires learning a mapping within the embedding space. |
| Outcome: | The proposed method performs better than or comparable to strong baselines while being up to four times faster. |
Structure Aware Negative Sampling in Knowledge Graphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for learning low-dimensional representations of entities and relations in knowledge graphs employing corruption distributions that generate hard negative samples. |
| Approach: | They propose a structure-aware negative sampling strategy that utilizes the rich graph structure by selecting negative samples from a node’s k-hop neighborhood. |
| Outcome: | The proposed method finds semantically meaningful negatives and is competitive with SOTA approaches while requires no additional parameters nor difficult adversarial optimization. |
Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train language models on diverse text corpora have brought up performance improvements on several natural language understanding (NLU) tasks. |
| Approach: | They propose a method to automatically generate domain- and task-adaptive maskings of a given text for self-supervised pre-training. |
| Outcome: | The proposed framework outperforms rule-based masking strategies on question answering and text classification datasets on which it outperformed rule-driven masking techniques. |
Autoregressive Knowledge Distillation through Imitation Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Autoregressive models are ubiquitous in natural language processing due to the sequential nature of text generation. |
| Approach: | They propose a compression technique for autoregressive models driven by an imitation learning perspective on knowledge distillation. |
| Outcome: | The proposed method outperforms other distillation algorithms on translation and summarization tasks while increasing inference speed 14 times. |
T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial examples can induce arbitrary errors to the target models, but they can be exploited to estimate robustness of NLP models. |
| Approach: | They propose a target-controllable adversarial attack framework T3 to handle adversarials . they use tree-based decoders to regularize the syntactic correctness of generated text . |
| Outcome: | The proposed framework can be used to estimate the robustness of NLP models. |
Structured Pruning of Large Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in language modeling have led to remarkable improvements on a variety of tasks. |
| Approach: | They propose a generic, structured pruning approach by parameterizing each weight matrix and adaptively removing rank-1 components during training. |
| Outcome: | The proposed method outperforms unstructured pruning and block pruning on language modeling tasks while achieving speedups during training and inference. |
Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has shown the importance of training contextualised word embedding models on the domain of the target task of interest. |
| Approach: | They propose a masking strategy which adversarially masks out those tokens which are harder to reconstruct by the underlying MLM. |
| Outcome: | The proposed training strategy outperforms random masking on six unsupervised domain adaptation tasks and achieves up to +1.64 F1 score improvements. |
BAE: BERT-based Adversarial Examples for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have exposed the vulnerability of text classification models to adversarial examples . perturbed versions of the original text are indiscernible by humans and misclassified by the model . |
| Approach: | They propose a black box attack for generating adversarial examples using contextual perturbations from a BERT-masked language model. |
| Outcome: | The proposed attack produces examples with improved grammaticality and semantic coherence compared to previous work. |
Adversarial Self-Supervised Data-Free Distillation for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation algorithms rely on the accessibility of the training dataset, which may be unavailable due to privacy issues. |
| Approach: | They propose a data-free distillation method for a pre-trained transformer-based model that uses plug & play Embedding Guessing to craft pseudo embeddings from the teacher's hidden knowledge. |
| Outcome: | The proposed method is the first data-free distillation framework designed for NLP tasks. |
BERT-ATTACK: Adversarial Attack Against BERT Using BERT (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to generate adversarial samples for discrete data are heuristic replacement strategies that are difficult to implement in continuous data. |
| Approach: | They propose a method to generate adversarial samples using pre-trained masked language models using BERT. |
| Outcome: | The proposed method outperforms state-of-the-art methods in success rate and perturb percentage while remaining fluent and semantically preserved. |
The Thieves on Sesame Street are Polyglots - Extracting Multilingual Models from Monolingual APIs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has demonstrated that deployed NLP models can be stolen by adversaries by querying victim models with gibberish input data that consists of random sequences of words. |
| Approach: | They propose to extract a local copy of a monolingual victim model from an API and query it with gibberish input data paired with the victim's labels. |
| Outcome: | The extracted model learns the task from the monolingual victim, but it generalizes far better than the victim to several other languages. |
When Hearst Is not Enough: Improving Hypernymy Detection from Corpus with Distributional Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | a taxonomy is a semantic hierarchy of words or concepts organized w.r.t. their hypernymy relationships. |
| Approach: | They propose a framework for hypernymy detection using large textual corpora . they quantify the non-negligible existence of specific sparsity cases . |
| Outcome: | The proposed framework quantifies the non-negligible existence of specific sparsity cases on several benchmark datasets. |
Interpreting Open-Domain Modifiers: Decomposition of Wikipedia Categories into Disambiguated Property-Value Pairs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge repositories rely on Wikipedia for core sets of topics and knowledge assertions. |
| Approach: | They propose an open-domain method for automatically annotating modifier constituents (20th-century’) within Wikipedia categories with properties (date of birth). |
| Outcome: | The proposed method improves precision and recall over a set of Wikipedia categories. |
A Synset Relation-enhanced Framework with a Try-again Mechanism for Word Sense Disambiguation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing sense embeddings fail to embed sense knowledge in semantic networks. |
| Approach: | They propose a Synset Relation-Enhanced Framework that leverages sense relations for sense embedding enhancement and a try-again mechanism that implements WSD again. |
| Outcome: | The proposed system outperforms knowledge-based systems with 20% SemCor data on all-words and lexical datasets. |
Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for news headline generation focus on producing a single short sentence . et al., 2017; Gehrmann e.t., 2018; Zhong ee., 2019) focus on single-headline generation. |
| Approach: | They propose a method to generate multiple headlines with keyphrases of user interests . they propose generating multiple keyphrase-relevant headlines using a transformer decoder . |
| Outcome: | The proposed method achieves state-of-the-art in terms of quality and diversity. |
Factual Error Correction for Abstractive Summarization Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are unable to ensure factual consistency of generated summaries. |
| Approach: | They propose a post-editing corrector module to identify and correct factual errors in generated summaries. |
| Outcome: | The proposed model outperforms existing models on CNN/DailyMail dataset on factual consistency evaluation. |
Compressive Summarization with Plausibility and Salience Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new method to learn which compressions to apply is based on syntactic rules for deleting spans . plausibility and salience are the two main criteria for determining which compression to apply . a recent study shows that the plausability model generally selects for grammatical and factual deletions compared to extractive methods . |
| Approach: | They propose to leave the decision about what to delete to two data-driven criteria . they show that plausibility and salience are the most important criteria if a span is deleted . |
| Outcome: | The proposed method achieves strong in-domain results on benchmark datasets and human evaluation shows that plausibility model generally selects for grammatical and factual deletions. |
Understanding Neural Abstractive Summarization Models via Uncertainty (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in abstractive summarization have been fueled by the advent of large-scale Transformers pre-trained on autoregressive language modeling objectives. |
| Approach: | They analyze summarization decoders in both blackbox and whitebox ways by studying on the entropy, or uncertainty, of the model’s token-level predictions. |
| Outcome: | The proposed model generates tokens in a free-form manner, but this flexibility makes it difficult to interpret their behavior. |
Better Highlighting: Creating Sub-Sentence Summary Highlights (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarizations are considered to be less reliable because they distort the original meaning and can be confusing for readers. |
| Approach: | They propose a method to generate summary highlights that are understandable on their own to avoid confusion. |
| Outcome: | The proposed method allows summaries to be understood in context and avoids misdirecting readers to false conclusions. |
Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on aspect-based abstractive summarization assume a small set of aspects and do not consider other diverse aspects. |
| Approach: | They propose a weak supervision construction method and an aspect modeling scheme to solve this problem. |
| Outcome: | The proposed method significantly expands the application of the task in practice. |
BERT-enhanced Relational Sentence Ordering Network (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve coherence modeling for paragraphs have been developed. |
| Approach: | They propose a BERT-enhanced Relational Sentence Ordering Network to capture better dependency relationship among sentences and exploit it with a deep relational module. |
| Outcome: | The proposed model shows significant improvement over the state-of-the-art on six datasets. |
Online Conversation Disentanglement with Pointer Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for disentangling textual conversations rely on dataset specific features that hinder generalization and adaptability. |
| Approach: | They propose an end-to-end online framework for conversation disentanglement that embeds the whole utterance that comprises timestamp, speaker, and message text. |
| Outcome: | The proposed method performs state-of-the-art on the Ubuntu IRC dataset and on other social and organizational platforms. |
VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for definition modeling combine distributional and lexical semantics in an implicit rather than direct way. |
| Approach: | They propose a model that introduces a continuous latent variable to model the relationship between a phrase and its definition. |
| Outcome: | The proposed model achieves state-of-the-art performance on four challenging benchmarks and the first non-English corpus. |
Coarse-to-Fine Pre-training for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a task of discovering information entities and identifying their corresponding categories. |
| Approach: | They propose a NER-specific framework to inject coarse-to-fine named entity knowledge into pre-trained models by using a remote supervision strategy. |
| Outcome: | The proposed framework achieves significant improvements against several pre-trained base-lines, demonstrating its effectiveness in label-few and low-resource scenarios. |
Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment (2020.emnlp-main)
Copied to clipboard
| Challenge: | Entity alignment (EA) aims at building a Knowledge Graph (KG) of rich content by linking the equivalent entities from various KGs. |
| Approach: | They propose to use an attributed value encoder to partition a Knowledge Graph into subgraphs to model the various types of attribute triples efficiently. |
| Outcome: | The proposed method achieves significant improvements over 12 baselines in cross-lingual and monolingual datasets. |
Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is widely adopted in several domains, such as news, medical, and social media. |
| Approach: | They propose a few-shot named entity recognition system based on nearest neighbor learning and structured inference. |
| Outcome: | The proposed method improves F1 scores on standard few-shot NER evaluation tasks by 6% to 16% relative to previous methods. |
Learning Structured Representations of Entity Names using Active Learning and Weak Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Structured representations of entity names are useful for many entity-related tasks such as entity normalization and variant generation. |
| Approach: | They propose a framework that combines active learning and weak supervision to solve this problem. |
| Outcome: | The proposed framework enables learning of high-quality models from a dozen labeled examples. |
Entity Enhanced BERT Pre-training for Chinese NER (2020.emnlp-main)
Copied to clipboard
| Challenge: | Character-level BERT pre-trained in Chinese suffers from lacking lexicon information, which shows effectiveness for Chinese NER. |
| Approach: | They propose a semi-supervised method to integrate lexicon into pre-trained LMs in Chinese . they extract an entity lexiconal from raw text and integrate it into BERT . |
| Outcome: | The proposed method is highly effective and achieves the best results on a news dataset and two datasets annotated by the authors. |
Scalable Zero-shot Entity Linking with Dense Entity Retrieval (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for entity linking use manually curated mention tables and incoming Wikipedia link popularity. |
| Approach: | They propose a BERT-based entity linking model with a bi-encoder that embeds the mention context and the entity descriptions and then re-ranked the candidate with . they also evaluate the accuracy-speed trade-off inherent to large pre-trained models. |
| Outcome: | The proposed model is state-of-the-art on recent zero-shot benchmarks and established non-zero-shot evaluations. |
A Dataset for Tracking Entities in Open Domain Procedural Text (2020.emnlp-main)
Copied to clipboard
Niket Tandon, Keisuke Sakaguchi, Bhavana Dalvi, Dheeraj Rajagopal, Peter Clark, Michal Guerquin, Kyle Richardson, Eduard Hovy
| Challenge: | Existing tasks require only a small set of attributes to track state changes in procedural text. |
| Approach: | They propose a task where given a procedural text as input, the task is to generate a set of state change tuples for each step. |
| Outcome: | The proposed task generates state change tuples from a set of pre-defined attributes for each step and predicts them from an open vocabulary. |
Design Challenges in Low-resource Cross-lingual Entity Linking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing techniques for grounding mentions of entities in a foreign language do not rise to the challenges introduced by text in low-resource languages (LRL) and fail to generalize to text not taken from Wikipedia, on which they are usually trained. |
| Approach: | They propose a cross-lingual XEL technique that uses search engines to locate and search for foreign language entries in Wikipedia. |
| Outcome: | The proposed system shows an increase of 25% in gold candidate recall and 13% in end-to-end linking accuracy over state-of-the-art baselines. |
Efficient One-Pass End-to-End Entity Linking for Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for entity linking are limited to entity disambiguation and require mention boundaries to be given in the input. |
| Approach: | They propose a fast end-to-end entity linking model that uses a biencoder to jointly detect mentions and link in one pass. |
| Outcome: | The proposed model outperforms the current state of the art on WebQSP and GraphQuestions with extended annotations that cover multiple entities per question. |
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for entity representations do not capture information in a knowledge base, and cannot represent entities that do not exist in the KB. |
| Approach: | They propose a pretrained contextualized representation of words and entities based on the bidirectional transformer. |
| Outcome: | The proposed model achieves impressive empirical performance on a wide range of entity-related tasks. |
Generating similes effortlessly like a Pro: A Style Transfer Approach for Simile Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Literary tropes are at the crux of human imagination and communication. |
| Approach: | They propose to automatically transform similes from reddit to their literal counterparts using common sense knowledge to generate simile models. |
| Outcome: | The proposed method generates 88% novel similes that do not share properties with training data. |
STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets lack rich enough contexts to guide models and evaluations are unreliable for long-form creative text. |
| Approach: | They propose a dataset and evaluation platform built from STORIUM . their dataset contains 6K lengthy stories with fine-grained natural language annotations . |
| Outcome: | The proposed model can be used to generate 6K long stories with fine-grained natural language annotations and a user-generated dataset. |
Substance over Style: Document-Level Targeted Content Transfer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models excel at writing from scratch, but many real-world scenarios require rewriting an entire document to fit a set of constraints. |
| Approach: | They propose a document-level targeted content transfer task that addresses the challenge of rewriting an entire document coherently by generating coherent and diverse rewrites that obey a constraint while remaining close to the original document. |
| Outcome: | The proposed model outperforms existing methods by generating coherent and diverse rewrites that obey the constraint while remaining close to the original document. |
Template Guided Text Generation for Task-Oriented Dialogue (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines the use of templates to generate natural language utterances for a large number of APIs. |
| Approach: | They propose a schema-guided approach which conditions the generation on a natural language schema. |
| Outcome: | The proposed method improves over strong baselines, is robust to out-of-domain inputs and shows improved sample efficiency. |
MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing reading comprehension metrics rely on token overlap and are agnostic to the nuances of reading comprehension. |
| Approach: | They propose a benchmark for training and evaluating generative reading comprehension metrics: MOdeling Correctness with Human Annotations. |
| Outcome: | The proposed benchmark outperforms baseline metrics by 10 to 36 absolute Pearson points on held-out annotations. |
Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite the success of contextualized language models, language models cannot capture textual coherence of a long, multi-sentence document. |
| Approach: | They propose a paragraph completion task that predicts masked sentences in a sentence . they propose SSPlanner that predict what to say first and guides the pretrained model . |
| Outcome: | The proposed model outperforms baseline generation models on the paragraph completion task in automatic and human evaluation. |
Inquisitive Question Generation for High Level Text Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing data-driven questions generate questions that fill gaps in knowledge . a dataset of 19K questions is used to generate meaningful questions . |
| Approach: | They propose a dataset of 19K questions that are elicited while a person is reading a document. |
| Outcome: | The proposed model generates reasonable questions, but the task is challenging. |
Towards Persona-Based Empathetic Conversational Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empathetic conversational models have been shown to improve user satisfaction and task outcomes in numerous domains. |
| Approach: | They propose a task towards persona-based empathetic conversations and propose e-learning model CoBERT that can be used to train persona on emmpathetic conversations. |
| Outcome: | The proposed model improves empathetic responding more when trained on e-mpathetic conversations than non-empathy ones. |
Personal Information Leakage Detection in Conversations (2020.emnlp-main)
Copied to clipboard
| Challenge: | 4.5 billion dollars will be invested in conversational assistants (chatbots) by 2021, according to Opus Research 2 . Among diverse types of chatbots, Google Duplex represents the kind of AI personal assistants that act on behalf of people to perform simple tasks. |
| Approach: | They propose to protect personal information by warning users of detected suspicious sentences . they propose to use a constrained alignment problem to perform an alignment optimization problem . |
| Outcome: | The proposed models outperform baseline models on the behavior of personalized chit-chat dialogue systems. |
Response Selection for Multi-Party Conversations with Dynamic Topic Tracking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing response selection methods focus on a two-party single-conversation scenario. |
| Approach: | They propose a multi-task learning framework that frames response selection as a dynamic topic tracking task to match the topic between the response and relevant conversation context. |
| Outcome: | The proposed framework outperforms existing methods on an Ubuntu IRC dataset in response selection and topic disentanglement tasks. |
Regularizing Dialogue Generation by Imitating Implicit Scenarios (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for dialogue generation lack the flexibility to handle such freedoms. |
| Approach: | They propose to take into account dialogue history and future conversation to implicitly reconstruct the scenario knowledge. |
| Outcome: | The proposed approach outperforms state-of-the-art models on diversity and relevance and expresses scenario-specific knowledge. |
MovieChats: Chat like Humans in a Closed Domain (2020.emnlp-main)
Copied to clipboard
| Challenge: | Currently, open-domain chatbots are far from satisfactory. |
| Approach: | They propose a unified, readily scalable neural approach which reconciles all subtasks like intent prediction and knowledge retrieval. |
| Outcome: | The proposed approach outperforms commercial systems replying on complex rules on static and interactive tests and shows that the results are remarkably good. |
Conundrums in Entity Coreference Resolution: Making Sense of the State of the Art (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite significant progress on entity coreference resolution, there is a general lack of understanding of what has been improved. |
| Approach: | They present an empirical analysis of entity coreference resolvers to provide an understanding of what has been improved. |
| Outcome: | The proposed model improves the performance of entity coreference resolvers. |
Semantic Role Labeling Guided Multi-turn Dialogue ReWriter (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing attentive models attend to all words without prior focus, which results in inaccurate concentration on some dispensable words. |
| Approach: | They propose to use semantic role labeling to provide additional guidance for multi-turn dialogue rewriting models. |
| Outcome: | The proposed model outperforms existing models on multi-turn dialogue rewriting tasks. |
Continuity of Topic, Interaction, and Query: Learning to Quote in Online Conversations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Quotations are crucial for successful explanations and persuasions in interpersonal communications. |
| Approach: | They propose to use an encoder-decoder neural framework to continue the context with a quotation via language generation to capture latent topics, interactions with the dialogue history, and coherence to the existing contents. |
| Outcome: | The proposed model outperforms state-of-the-art models on two large-scale datasets in English and Chinese and shows that topic, interaction, and query consistency are helpful to learn how to quote in online conversations. |
Profile Consistency Identification for Open-domain Dialogue Agents (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on improving attribute consistency focus on incorporating attribute information in responses, but few efforts have identified the consistency relations between response and attribute profile. |
| Approach: | They propose a key-value structure information enriched BERT model to identify the profile consistency . they propose to incorporate attribute information into the generated responses . |
| Outcome: | The proposed model improves over strong baselines on downstream tasks. |
An Element-aware Multi-representation Model for Law Article Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that using law articles as external knowledge can improve the performance of the Legal Judgment Prediction. |
| Approach: | They propose a Law Article Element-aware Multi-representation Model which makes full use of law article information and can be used for multi-label samples. |
| Outcome: | The proposed model improves the accuracy of 5.84%, macro F1 of 6.42%, and micro F1 by 4.28% compared with baseline models like TopJudge. |
Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for reasoning over temporal knowledge graphs focus on past timestamps and are not able to predict future interactions. |
| Approach: | They propose a novel autoregressive architecture for predicting future interactions using a recurrent event encoder and a neighborhood aggregator. |
| Outcome: | The proposed method achieves state-of-the-art on five public datasets. |
Multi-resolution Annotations for Emoji Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Emojis are able to express various linguistic components, such as emotions, sentiments, events, etc. emojis have the merit of preserving information more densely, compared to words, argues a new study. |
| Approach: | They propose to use passage-level and aspect-level emoji annotations to predict the proper emmojis associated with text. |
| Outcome: | The proposed method is heuristically generated and validated with a pre-trained BERT model. |
Less is More: Attention Supervision with Counterfactuals for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Specifically, we explore the advantage of counterfactual reasoning, over associative reasoning . Adding human supervision to attention has been shown to improve model predictions and explanations . |
| Approach: | They propose to use machine-augmented human attention supervision to enhance model quality. |
| Outcome: | The proposed method is more effective than existing methods requiring higher annotation cost . the proposed method can be trained to generate similar attention to human supervision . |
MODE-LSTM: A Parameter-efficient Recurrent Network with Multi-Scale for Sentence Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for sentence classification use linear convolution, which may not be sufficient to model the non-consecutive dependency of the phrase and may overfit the sequential information. |
| Approach: | They propose a model that extracts multi-scale n-gram features for understanding the semantic meaning of sentences by some key-phrases located at different positions. |
| Outcome: | The proposed model outperforms existing models on eight benchmark datasets and is competitive against state-of-the-art models. |
HSCNN: A Hybrid-Siamese Convolutional Neural Network for Extremely Imbalanced Multi-label Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to solve the data imbalance problem are limited in extremely imbalanced data. |
| Approach: | They propose a hybrid approach which adapts general networks for head categories and few-shot techniques for tail categories. |
| Outcome: | The proposed approach improves the performance of Single networks with diverse loss objectives on tail or entire categories. |
Multi-Stage Pre-training for Automated Chinese Essay Scoring (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for automatic essay scoring are based on hand-crafted surface-level features, but recent advances in representation learning have improved performance. |
| Approach: | They propose a pre-training based automated Chinese essay scoring method with weakly supervised pre- training, supervised cross- prompt fine-tuning and supervised target- prompt refine-tuneing. |
| Outcome: | The proposed method improves a state-of-the-art neural essay scorer in terms of effectiveness and domain adaptation ability, while in-depth analysis also reveals its limitations. |
Multi-hop Inference for Question-driven Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for summarizing source document for non-factoid questions are lacking in factoidic QA. |
| Approach: | They propose a question-driven abstractive summarization method that incorporates multi-hop reasoning into question-based summarizing. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two non-factoid QA datasets. |
Towards Interpretable Reasoning over Paragraph Effects in Situation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models ignore complex reasoning process and solve it with a one-step "black box" approach. |
| Approach: | They propose a sequential approach which explicitly models each step of the reasoning process with neural network modules. |
| Outcome: | The proposed model is more interpretable and more accurate than existing models. |
Question Directed Graph Attention Network for Numerical Reasoning over Text (2020.emnlp-main)
Copied to clipboard
Kunlong Chen, Weidi Xu, Xingyi Cheng, Zou Xiaochuan, Yuyu Zhang, Le Song, Taifeng Wang, Yuan Qi, Wei Chu
| Challenge: | Numerical reasoning requires both natural language understanding and arithmetic computation. |
| Approach: | They propose a graph representation for the context of the passage and question needed for numerical reasoning. |
| Outcome: | The proposed model achieves remarkable results in benchmark datasets such as DROP. |
Dense Passage Retrieval for Open-Domain Question Answering (2020.emnlp-main)
Copied to clipboard
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih
| Challenge: | Open-domain question answering relies on efficient passage retrieval to select candidate contexts. |
| Approach: | They propose a dual-encoder framework that can be implemented to retrieve passages from a small number of questions and passages. |
| Outcome: | The proposed system outperforms a strong Lucene-BM25 system in top-20 passage retrieval accuracy on multiple open-domain QA benchmarks. |
Distilling Structured Knowledge for Text-Based Relational Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-based relational reasoning models lack a symbolic representation of text . performance gap between NLP models and structured models remains . |
| Approach: | They first pre-train a GNN on a reasoning task using structured inputs and then incorporate its knowledge into an NLP model. |
| Outcome: | The proposed model improves on two state-of-the-art NLP models on 13 different inductive reasoning datasets from the CLUTRR benchmark. |
Asking without Telling: Exploring Latent Ontologies in Contextual Representations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on model analysis indicates that they may learn a lot about linguistic structure, including part of speech, syntax, word sense, and more. |
| Approach: | They introduce latent subclass learning, a modification to classifier-based probing that induces a latent categorization (or ontology) of the probe’s inputs. |
| Outcome: | The proposed model induces a latent categorization (or ontology) of the probe’s inputs without access to fine-grained gold labels. |
Pretrained Language Model Embryology: The Birth of ALBERT (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretraining of pretrained models (LMs) has been extensively studied, but what happened during pretraining is rarely studied. |
| Approach: | They propose to use a totipotent language model to study pretraining behavior . they find that linguistic knowledge and world knowledge do not generally improve as pretraining proceeds, nor do downstream tasks’ performance. |
| Outcome: | The model learns to reconstruct and predict tokens of different parts of speech (POS) in different learning speeds during pretraining. |
Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | et al., 2018a, 2018b) show that LSTMs can transfer from non-linguistic data to natural language models with different types of abstract structure. |
| Approach: | They propose to use transfer learning to analyze encoding of grammatical structure in neural language models. |
| Outcome: | The proposed method improves test performance on natural language despite no overlap in surface form or vocabulary. |
What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on pre-trained Transformers has focused on learning the meaning of positions . Embedding the position information in the self-attention mechanism is also an indispensable factor in NLP . |
| Approach: | They propose to use feature-level analysis to examine pre-trained Transformers' position embeddings . they also use empirical experiments to determine the appropriate positional encoding function . |
| Outcome: | The results of the empirical study can guide future work to choose the appropriate positional encoding function for specific tasks. |
“You are grounded!”: Latent Name Artifacts in Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models perpetuate biases originating in their training corpus to downstream models. |
| Approach: | They focus on the representations of given names in pre-trained language models and show that name perturbation can have an effect on downstream tasks. |
| Outcome: | The proposed model can be used to model the representation of given names in pre-trained language models on reading comprehension probes where name perturbation changes the model answers. |
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models possess certain commonsense and factual knowledge. |
| Approach: | They propose to use pre-trained language models to predict masked words . they introduce a probing task with 13.6k m-word-prediction probes . |
| Outcome: | The proposed model performs poorly on the diagnostic dataset prior to any fine-tuning and fine-testing with distant supervision. |
Grounded Adaptation for Zero-shot Executable Semantic Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing semantic parsers are usually engineered for each application environment, but they struggle when deployed to a new database. |
| Approach: | They propose a method to adapt existing semantic parsers to new environments . they propose combining a forward semantic parsed with a backward utterance generator to synthesize data in the new environment and select cycle-consistent examples to adapt the parser. |
| Outcome: | The proposed procedure outperforms data-augmentation and improves execution accuracy on the Spider, Sparc, and CoSQL zero-shot semantic parsing tasks. |
An Imitation Game for Learning Semantic Parsers from User Interaction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for learning semantic parsers are expensive and tedious . despite the widespread applications, bootstrapping and fine-tuning is tedious a task . |
| Approach: | They propose an alternative method for learning semantic parsers directly from users . they propose an annotation-efficient imitation learning algorithm that iteratively collects new datasets . |
| Outcome: | The proposed method is cost-effective and shows promising performance on the text-to-SQL problem. |
IGSQL: Database Schema Interaction Graph Based Neural Model for Context-Dependent Text-to-SQL Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models on context-dependent text-to-SQL task focus on utilizing historic user inputs. |
| Approach: | They propose a database schema interaction graph encoder to utilize historic user inputs. |
| Outcome: | The proposed model outperforms previous state-of-the-art models on two datasets . it also outperformed existing models on the benchmark SParC and CoSQL datasets. |
“What Do You Mean by That?” A Parser-Independent Interactive Approach for Enhancing Text-to-SQL (2020.emnlp-main)
Copied to clipboard
| Challenge: | In Natural Language Interfaces to Databases systems, text-to-SQL parsers allow users to query databases by using natural language questions. |
| Approach: | They propose a parser-independent interactive approach that interacts with users using multi-choice questions and can easily work with arbitrary parsers. |
| Outcome: | The proposed approach improves performance with limited interaction turns by using simulation and human evaluation on two cross-domain datasets with five state-of-the-art parsers. |
DuSQL: A Large-Scale and Pragmatic Chinese Text-to-SQL Dataset (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL parsing methods mainly focus on English, but there is no labeled data available for the language . a larges-scale and pragmatic Chinese dataset is used for cross-domain text- to-Sql task . |
| Approach: | They propose a larges-scale Chinese dataset for a cross-domain text-to-SQL task . they analyze questions from several representative applications and use an effective data construction framework . |
| Outcome: | The proposed dataset contains 200 databases, 813 tables, and 23,797 question/SQL pairs. |
Mention Extraction and Linking for SQL Query Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL systems take a slot-filling approach, but they are limited in capturing inter-dependencies among SQL clauses. |
| Approach: | They propose an extraction-linking approach where a unified extractor recognizes all types of slot mentions appearing in the question sentence before a linker maps the recognized columns to the table schema to generate executable SQL queries. |
| Outcome: | The proposed method achieves the first place on the WikiSQL benchmark. |
Re-examining the Role of Schema Linking in Text-to-SQL (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL models treat schema linking as a minor component . Existing solutions treat schema as merely a string component based on string matching . |
| Approach: | They build a schema linking corpus based on a Spider text-to-SQL dataset . they find schema linking is the crux for the current text- to-Sql task . |
| Outcome: | The proposed model performs well on the Spider text-to-SQL dataset despite its simplicity. |
A Multi-Task Incremental Learning Framework with Category Name Embedding for Aspect-Category Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Sentiment analysis is an increasingly popular natural language processing task in academia and industry. |
| Approach: | They propose to use category name encoding network to weaken catastrophic forgetting problem . they set both encoder and decoder shared among all categories to weaker the catastrophic forgetting problem a . |
| Outcome: | The proposed model achieves state-of-the-art on two (T)ACSA benchmark datasets. |
Train No Evil: Selective Masking for Task-Guided Pre-Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models can't capture domain-specific and task-specific patterns because of the task-agnostic pre-training stage. |
| Approach: | They propose a task-guided pre-training stage with selective masking between general pre-train and fine-tuning to learn domain-specific patterns. |
| Outcome: | The proposed method can achieve comparable or even better performance with less than 50% of computation cost. |
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models neglect to consider linguistic knowledge of texts . existing models neglect linguistic information, which is important for sentiment analysis . |
| Approach: | They propose a model that introduces word-level linguistic knowledge into pre-trained models to enhance sentiment analysis by querying SentiWordNet to acquire sentiment polarity. |
| Outcome: | The proposed model obtains state-of-the-art performance on a variety of sentiment analysis tasks. |
Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for aspect-based sentiment analysis of review text use only a few keywords describing each aspect/sentiment without using any labeled examples. |
| Approach: | They propose a weakly-supervised approach for aspect-based sentiment analysis which uses only a few keywords describing each aspect/sentiment without using any labeled examples. |
| Outcome: | The proposed method generates quality joint topics and outperforms baselines significantly on benchmark datasets. |
APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Argument mining is an important research field that attracts growing attention in recent years. |
| Approach: | They propose a new task to extract argument pairs from peer review and rebuttal . they use an open review platform to analyze the contents, structure and connections . |
| Outcome: | The proposed task is based on a dataset of 4,764 fully annotated review-rebuttal passage pairs . it is able to detect argumentative propositions and extract argument pairs from the corpus . |
Diversified Multiple Instance Learning for Document-Level Multi-Aspect Sentiment Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that D-MILN outperforms recent weakly-supervised baselines . document-level multi-aspect sentiment classification requires a lot of manual aspect-level annotations - which is time-consuming and laborious . |
| Approach: | They propose a novel Diversified Multiple Instance Learning Network to achieve DMSC with only document-level weak supervision. |
| Outcome: | The proposed method outperforms weakly-supervised baselines on TripAdvisor and BeerAdvocate datasets. |
Identifying Exaggerated Language (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on metaphor and metonymy have focused on hyperbole, but it is a relatively understudied phenomenon in the figurative language processing community. |
| Approach: | They propose to use hyperbole detection to determine whether a sentence is hyperbolic . they also perform statistical and manual analyses of the corpus and address the automatic hyperbola detection task. |
| Outcome: | The proposed dataset consists of 709 hyperbolic sentences with a non-hyperbolic version created by paraphrasing its hyperbolical counterpart. |
Unified Feature and Instance Based Domain Adaptation for Aspect-Based Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to aspect-based sentiment analysis rely on labeled data, but they lack the fine-grained labeles needed for the ABSA task. |
| Approach: | They propose a framework to perform feature adaptation and instance adaptation for the ABSA task . they learn domain-invariant feature representations by using part-of-speech features . |
| Outcome: | The proposed method improves on the state-of-the-art in two aspects of the ABSA task. |
Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge transfer models do not exploit compositionality of language, often relying on superficial features. |
| Approach: | They propose to use a knowledge distillation technique to fine tune RoBERTa, BERT and DistilBERT models to improve their performance. |
| Outcome: | The proposed models improve on the CoQA task with linguistic knowledge and are able to represent compositional and lexical information. |
Attention is Not Only a Weight: Analyzing Transformers with Vector Norms (2020.emnlp-main)
Copied to clipboard
| Challenge: | Attention is a key component of Transformers, which have achieved considerable success in natural language processing. |
| Approach: | They propose to integrate attention weights and the norm of transformed input vectors into a norm-based analysis that incorporates the norm. |
| Outcome: | The proposed analysis shows that attention weights alone determine the output of attention and that reasonable word alignment can be extracted from attention mechanisms of Transformers. |
F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models and evaluation settings have shortcomings regarding the coupling of answer and explanation which might cause serious issues in user experience. |
| Approach: | They propose a hierarchical model and a new regularization term to strengthen the coupling of answer and explanation and two evaluation scores to quantify the couple. |
| Outcome: | The proposed model strengthens the answer-explanation coupling and provides evaluation scores that align with user experience. |
On the Ability and Limitations of Transformers to Recognize Formal Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on LSTMs have not revealed their ability to model syntactic properties. |
| Approach: | They propose to build a Transformers model for a subclass of counter languages and find that their learning mechanism strongly correlates with their construction. |
| Outcome: | The proposed model generalizes well on counter languages and its learned mechanism correlates with it. |
An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graphs (KGs) vary greatly from one domain to another, resulting in a lack of domain-specific parallel graph-text data. |
| Approach: | They propose an unsupervised approach to graph-to-text generation and text-to graph knowledge extraction using WebNLG v2.1 and a new benchmark leveraging scene graphs from Visual Genome. |
| Outcome: | The proposed approach outperforms baselines on WebNLG v2.1 and a new benchmark leveraging scene graphs from Visual Genome. |
DGST: a Dual-Generator Network for Text Style Transfer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on text style transfer focus on altering sentiment words to preserve attribute-independent information. |
| Approach: | They propose a Dual-Generator network architecture for text Style Transfer using two generators. |
| Outcome: | The proposed model performs better than existing models on Yelp and IMDb datasets. |
A Knowledge-Aware Sequence-to-Tree Network for Math Word Problem Solving (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for solving math word problems ignore background common-sense knowledge . a novel knowledge-aware sequence-to-tree (KA-S2T) network incorporates external knowledge and global expression information. |
| Approach: | They propose a knowledge-aware sequence-to-tree network that incorporates external knowledge and global expression information into the problem. |
| Outcome: | The proposed model can achieve better performance than previous models on a Math23K dataset. |
Generating Fact Checking Briefs (2020.emnlp-main)
Copied to clipboard
Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes, Sebastian Riedel
| Challenge: | Existing work has framed fact checking as classification, often supported by a claim as input. |
| Approach: | They propose to use natural language briefs to increase the accuracy of fact checking . they show that QABriefer increases the accuracy by 10% while QABries reduce time . |
| Outcome: | The proposed model increases the accuracy of crowdworkers by 10% while reducing the time required by 20%. |
Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve the efficiency of GEC are not efficient enough for GEC. |
| Approach: | They propose a language-independent approach to improve the efficiency of GEC by dividing the task into two subtasks: ESD and ESC. |
| Outcome: | The proposed approach performs comparably to conventional seq2seq approaches in English and Chinese GEC benchmarks with less than 50% time cost for inference. |
Coreferential Reasoning Learning for Language Representation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing language representation models cannot explicitly handle coreference, which is essential to the coherent understanding of the whole discourse. |
| Approach: | They propose a language representation model that captures coreferential relations in context. |
| Outcome: | The proposed model can achieve significant improvements on downstream NLP tasks while maintaining comparable performance to baseline models on other common NLP task. |
Is Graph Structure Necessary for Multi-hop Question Answering? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on multi-hop question answering across multiple documents or paragraphs. |
| Approach: | They propose a graph neural network to deal with graph structure in textual multi-hop reasoning . they propose 'self-attention' and propose removing entire graph structure may not hurt the final results . |
| Outcome: | The proposed model shows that graph-attention or the entire graph structure can be replaced by self-attention . hotpotQA is a widely used benchmark for multi-hop question answering . |
XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for assessing distinct meanings of words are tied to sense inventories, restricting their usage to knowledge-based representation techniques. |
| Approach: | They propose a multilingual benchmark that models distinct meanings of words in English . they use a binary disambiguation task with gold standards in 12 new languages . |
| Outcome: | The proposed model can model distinct meanings of words in English even when no tagged instances are available for a target language. |
Generationary or “How We Went beyond Word Sense Inventories and Learned to Gloss” (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to Word Sense Disambiguation use discrete word senses . however, many language users have different understandings of words . |
| Approach: | They propose a unified computational lexical semantics model that can produce contextually appropriate definitions. |
| Outcome: | The proposed model outperforms existing models in lexical semantics and discriminative tasks. |
Probing Pretrained Language Models for Lexical Semantics (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on morphosyntactic, semantic, and world knowledge, but it remains unclear to what extent LMs derive lexical type-level knowledge from words in context. |
| Approach: | They propose to use multilingual and monolingual LMs to extract lexical type-level knowledge from words in context. |
| Outcome: | The proposed models perform well across six typologically diverse languages and five lexical tasks. |
Cross-lingual Spoken Language Understanding with Regularized Representation Alignment (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite promising results, current cross-lingual models suffer from imperfect cross-linguistic representation alignments between the source and target languages, which makes the performance sub-optimal. |
| Approach: | They propose a regularization approach to align word-level and sentence-level representations across languages without external resources. |
| Outcome: | The proposed model outperforms state-of-the-art models in few-shot and zero-shot scenarios and achieves comparable performance to supervised training with all training data. |
SLURP: A Spoken Language Understanding Resource Package (2020.emnlp-main)
Copied to clipboard
| Challenge: | Publicly available datasets for Spoken Language Understanding (SLU) are limited. |
| Approach: | They propose a publicly available SLU resource package that includes a multi-domain dataset in English spanning 18 domains. |
| Outcome: | The proposed dataset is bigger and more diverse than existing datasets. |
Neural Conversational QA: Learning to Reason vs Exploiting Patterns (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural Conversational QA tasks such as ShARC require systems to answer questions based on the contents of a given passage. |
| Approach: | They propose to modify a data-set with fewer spurious patterns to exploit them . they also propose to build a heuristic-based program to exploit spurious clues . |
| Outcome: | The proposed program exploits spurious patterns in the ShARC dataset, compared to neural models. |
Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using labeled data, named entity recognition is labor-intensive, time-consuming and expensive. |
| Approach: | They propose a method which decomposes named entity into two parts: entity and context. |
| Outcome: | The proposed method improves the generalization ability of models under limited observational examples. |
Understanding Procedural Text using Interactive Entity Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent efforts to track multiple entities in a procedural text treat each entity separately . e.g., scientific articles, instruction books, recipes, often contain multiple entities involved . |
| Approach: | They propose a recurrent network with memory equipped cells for state tracking . they maintain different attention matrices through specific memories to model different types of entity interactions . |
| Outcome: | The proposed model outperforms state-of-the-art models on a benchmark dataset. |
A Rigorous Study on Named Entity Recognition: Can Fine-tuning Pretrained Model Lead to the Promised Land? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental task of information extraction. |
| Approach: | They propose to perform randomization tests on standard NER benchmarks to examine name regularity, mention coverage and context diversity. |
| Outcome: | The proposed model performs better on standard NER benchmarks than other models on open datasets. |
DyERNIE: Dynamic Evolution of Riemannian Manifold Embeddings for Temporal Knowledge Graph Completion (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing embedding approaches for temporal knowledge graphs typically learn entity representations and their dynamic evolution in the Euclidean space. |
| Approach: | They propose a non-Euclidean embedding approach that learns evolving entity representations in a product of Riemannian manifolds. |
| Outcome: | The proposed model improves on three real-world datasets showing that the embeddings on Riemannian manifolds can capture the evolution of temporal KGs. |
Embedding Words in Non-Vector Space with Unsupervised Graph Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | GraphGlove is an unsupervised graph word representations that are learned end-to-end. |
| Approach: | They propose a method to learn weighted graph word representations end-to-end using a weighteable weighte . they adopt a hierarchical graph representation method and modify the GloVe training algorithm to learn graph representations. |
| Outcome: | The proposed method outperforms vector-based methods on word similarity and analogy tasks. |
Debiasing knowledge graph embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train knowledge graph embeddings to be neutral to sensitive attributes such as gender have been shown to increase training time by a factor of eight or more. |
| Approach: | They propose a method where all embeddings are trained to be neutral to sensitive attributes such as gender by default using an adversarial loss. |
| Outcome: | The proposed method reduces training time by eightfold and improves accuracy. |
Message Passing for Hyper-Relational Knowledge Graphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to link prediction over knowledge graphs (KGs) are designed to work over triple-based models, where facts are represented as binary relations between entities. |
| Approach: | They propose a message passing based graph encoder - StarE capable of modeling hyper-relational knowledge graphs (KGs) they propose to encode an arbitrary number of additional information along with the main triple while keeping the semantic roles of qualifiers and triples intact. |
| Outcome: | The proposed model outperforms existing models across multiple benchmarks and shows that leveraging qualifiers is vital for link prediction. |
Relation-aware Graph Attention Networks with Relational Position Encodings for Emotion Recognition in Conversations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent research on emotion recognition in conversations (ERC) does not take self-dependency or inter-speaker dependency into account. |
| Approach: | They propose a relational graph attention network (RGAT) model that takes speaker dependency and sequential information into account by encoding the relational Graph structure. |
| Outcome: | The proposed model outperforms the state-of-the-art on four ERC datasets. |
BERT Knows Punta Cana is not just beautiful, it’s gorgeous: Ranking Scalar Adjectives with Contextualised Representations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Adjectives describe positive properties of nouns but with different intensity. |
| Approach: | They propose a BERT-based approach to intensity detection for scalar adjectives by generating vectors directly from contextualised representations. |
| Outcome: | The proposed model outperforms static embeddings and previous models with dedicated resources on an Indirect Question Answering task. |
Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Adapting pre-trained language models (PrLMs) to new domains has gained much attention . Adaptation of PrLMs to newdomains is important, but requires fine-tuning . |
| Approach: | They propose to use PrLMs to adapt to new domains without fine-tuning . they use class-aware feature self-distillation to learn discriminative features . |
| Outcome: | The proposed model can learn discriminative features from pre-trained language models without fine-tuning. |
Textual Data Augmentation for Efficient Active Learning on Tiny Datasets (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing active learning approaches for textual data are limited due to the complexity of language. |
| Approach: | They propose an approach where guided outputs of a language generation model can be enhanced through an active learning process. |
| Outcome: | The proposed approach achieves performance increases of 3% and 5% on TREC-6 and SST-2 datasets compared with NGDG, which does not optimize for a reward function. |
“I’d rather just go to bed”: Understanding Indirect Answers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Humans produce and interpret complex utterances even in simple scenarios. |
| Approach: | They present a large-scale English language corpus with 34,268 (polar question, indirect answer) pairs to enable progress on this task. |
| Outcome: | The proposed corpus contains 34,268 (polar question, indirect answer) pairs, and reaches 82-88% accuracy for a 4-class distinction, and 64-85% for 6 classes. |
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Unconscious biases continue to be prevalent in modern text and media, calling for algorithms that can assist writers with bias correction. |
| Approach: | They propose a new revision task that debiases text through the lens of connotation frames to correct implicit biases in character portrayals. |
| Outcome: | The proposed approach outperforms existing methods and ablations in the literature. |
MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing discourse treebanks are limited in the application of data-driven approaches to discourse parsing. |
| Approach: | They propose a method to automatically generate discourse treebanks using distant supervision from sentiment annotated datasets by heuristic beam-search strategy extended with a stochastic component. |
| Outcome: | The proposed method generates discourse trees incorporating structure and nuclearity for documents of arbitrary length using an efficient beam-search strategy, extended with a stochastic component. |
Centering-based Neural Coherence Modeling with Hierarchical Discourse Segments (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior studies of coherence focused on identifying semantic relations between adjacent sentences. |
| Approach: | They propose a coherence model which takes discourse structural information into account without relying on human annotations. |
| Outcome: | The proposed model performs state-of-the-art on automated essay scoring and assessing writing quality tasks. |
Keeping Up Appearances: Computational Modeling of Face Acts in Persuasion Oriented Discussions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Politeness principles play a central role in shaping human interaction. |
| Approach: | They propose a generalized framework for modeling face acts in persuasion conversations using an annotated corpus and computational models. |
| Outcome: | The proposed framework reveals differences in face act utilization between asymmetric roles in persuasion conversations and predicts key conversational outcome. |
HABERTOR: An Efficient and Effective Deep Hatespeech Detector (2020.emnlp-main)
Copied to clipboard
| Challenge: | HABERTOR model is a highly efficient and effective alternative to BERT for the hatespeech classification task. |
| Approach: | They propose to modify BERT's HABERTOR model to generate its own vocabularies and pre-trained it using the largest scale hatespeech dataset. |
| Outcome: | The proposed model is faster, more efficient and more robust than existing methods for hatespeech classification. |
An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels (2020.emnlp-main)
Copied to clipboard
Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos
| Challenge: | Large-scale Multi-label Text Classification (LMTC) is a type of classification that assigns labels to a large set of labels. |
| Approach: | They propose to use probabilistic label trees to improve frequent, few and zero-shot learning . they propose to combine a new state-of-the-art method with pre-trained Transformers . |
| Outcome: | The proposed models outperform existing models on frequent, few and zero-shot learning on three datasets from different domains. |
Which *BERT? A Survey Organizing Contextualized Encoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | a survey on language representation learning aims to highlight common themes . we focus on the areas of progress, compared to other fields, and discuss how each area is evaluated. |
| Approach: | They present a survey on language representation learning to highlight common themes . they compare contributions in contextualized text encoders to ideas from other fields . |
| Outcome: | The proposed survey aims to highlight common themes in the field of language representation learning. |
Fact or Fiction: Verifying Scientific Claims (2020.emnlp-main)
Copied to clipboard
David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi
| Challenge: | SciFact is a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. |
| Approach: | They construct a dataset of 1.4K scientific claims paired with evidence-containing abstracts annotated with labels and rationales to test their system. |
| Outcome: | The proposed system can verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus. |
Semantic Role Labeling as Syntactic Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using propBank-style semantic role labeling, we reduce the task to syntactic dependency parsing. |
| Approach: | They propose to convert SRL annotations into dependency tree representations through joint labels that permit highly accurate recovery back to the original format. |
| Outcome: | The proposed scheme reduces the task of (span-based) PropBank-style semantic role labeling to syntactic dependency parsing. |
PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Paraphrase identification requires specialized domain knowledge to perform . state-of-the-art neural models and non-expert human annotators have poor performance on PARADE . |
| Approach: | They propose a benchmark dataset called PARADE for paraphrase identification that requires specialized domain knowledge. |
| Outcome: | The proposed dataset shows state-of-the-art models and non-expert human annotators have poor performance on PARADE. |
Causal Inference of Script Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on script induction relied on correlation between instances of events in corpus . instead, we propose an approach based on causal effects between events . |
| Approach: | They propose to use causal effects to induce scripts from text . they propose to compute a function that matches the intuition of what a script represents . |
| Outcome: | The proposed method matches the intuition of what a script represents, the authors show . |
Towards Debiasing NLU Models from Unknown Biases (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent proposed debiasing methods rely on the assumption that the types of bias should be known a-priori, which limits their application to many NLU tasks and datasets. |
| Approach: | They propose a framework that prevents models from mainly utilizing biases without knowing them in advance. |
| Outcome: | The proposed framework allows existing methods to retain performance improvement on challenge datasets without specifically targeting biases. |
On the Role of Supervision in Unsupervised Constituency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on unsupervised constituency parsing uses labeled examples for tuning . a few-shot parser with labeles can outperform other approaches by a significant margin . |
| Approach: | They propose to use as few labeled examples as possible for model development . they propose to train existing models on the same labeles they access . |
| Outcome: | The proposed model outperforms other models on the WSJ development set by a significant margin . the proposed model can be further improved by augmentation and self-training . |
Language Model Prior for Low-Resource Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation is based on large parallel corpora and requires expensive training and training. |
| Approach: | They propose to incorporate a LM as prior in a neural translation model (TM) they add a regularization term which pushes the output distributions to be probable under the LM prior . |
| Outcome: | The proposed approach does not compromise decoding speed, because the LM is used only at training time, unlike previous work that requires it during inference. |
Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that neural machine translation models rely heavily on source sentence information when resolving lexical ambiguity. |
| Approach: | They propose a method for the prediction of disambiguation errors based on statistical data properties and propose 'a simple adversarial attack strategy' that minimally perturbs sentences to elicit disambiguations errors. |
| Outcome: | The proposed method shows that disambiguation robustness varies substantially between domains and different models trained on the same data are vulnerable to different attacks. |
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current deep pretrained models lack capacity to represent all languages . limited capacity is an issue even for high-resource languages where models are not included in training data at all. |
| Approach: | They propose an adapter-based framework that enables high portability and parameter-efficient transfer to arbitrary tasks and languages by learning modular language and task representations. |
| Outcome: | The proposed framework outperforms state-of-the-art models on cross-lingual transfer across languages and typologically diverse models. |
Translation Artifacts in Cross-lingual Transfer Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing cross-lingual transfer learning techniques involve human and machine translations. |
| Approach: | They propose to use machine translation to translate test set or training set to introduce subtle artifacts that have a notable impact in existing cross-lingual models. |
| Outcome: | The proposed translation process reduces the lexical overlap between the premise and hypothesis by 4.3 and 2.8 points . the proposed translation-test and zero-shot approaches improve on previous work . |
A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Suicide ideation is often linked to a history of mental depression. |
| Approach: | They propose a time-aware transformer based model for preliminary screening of suicidal risk on social media that augments linguistic models with historical context. |
| Outcome: | The proposed model outperforms competing models and shows that it is time-aware and contextually useful for suicide risk assessment. |
Weakly Supervised Learning of Nuanced Frames for Analyzing Polarization in News Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study suggests a minimally supervised approach for identifying nuanced political frames in news articles on politically divisive topics. |
| Approach: | They propose a minimally supervised approach for identifying nuanced policy frames in news coverage of politically divisive topics. |
| Outcome: | The proposed subframes can capture differences in political ideology better . the proposed frameworks were tested on immigration, gun control and abortion topics . |
Where Are the Facts? Searching for Fact-checked Information to Alleviate the Spread of Fake News (2020.emnlp-main)
Copied to clipboard
| Challenge: | fabricated stories and hoaxes are still pervading our cyberspace. |
| Approach: | They propose a framework to search for fact-checking articles that address the content of an original tweet that may contain misinformation posted by online users. |
| Outcome: | The proposed framework can detect and disseminate fake news on real-world datasets and warn fake news posters and online users about misinformation. |
Fortifying Toxic Speech Detectors Against Veiled Toxicity (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern toxic speech detectors are incompetent in recognizing disguised offensive language, such as adversarial attacks that deliberately avoid known toxic lexicons. |
| Approach: | They propose a framework that fortifies existing toxic speech detectors without a large labeled corpus of veiled toxicity. |
| Outcome: | The proposed framework is aimed at fortifying existing toxic speech detectors without a large labeled corpus of disguised offensive language. |
Explainable Automated Fact-Checking for Public Health Claims (2020.emnlp-main)
Copied to clipboard
| Challenge: | a few blind spots exist in the state-of-the-art in fact-checking for political claims. |
| Approach: | They propose to use a dataset of 11.8K claims to explain fact-check labels for claims . they define and evaluate three coherence properties of explanation quality with humans . |
| Outcome: | The proposed model can be trained on in-domain data and evaluates its coherence properties with humans and computationally. |
Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Interactive Fiction (IF) games with real human-written natural language texts provide a new natural evaluation for language understanding techniques. |
| Approach: | They propose to re-formulate IF game solving as Multi-Passage Reading Comprehension tasks using context-query attention mechanisms and structured prediction to efficiently generate and evaluate action outputs. |
| Outcome: | The proposed methods achieve high winning rates and low data requirements on the recent IF benchmark (Jericho) |
DORB: Dynamically Optimizing Multiple Rewards with Bandits (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in end-to-end neural networks-based approaches have shown wide success in sequence generation tasks. |
| Approach: | They propose to optimize multiple metric rewards simultaneously using a multi-armed bandit approach . they empirically show the effectiveness of their approaches via various automatic metrics and human evaluation . |
| Outcome: | The proposed approach improves on question generation and data-to-text generation using a bandit approach. |
MedFilter: Improving Extraction of Task-relevant Utterances through Integration of Discourse Structure and Ontological Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Identifying task-relevant utterances improves performance at downstream medical processing. |
| Approach: | They propose a novel approach that uses task-oriented conversations to improve utterance classification over SOTA models. |
| Outcome: | The proposed model improves on a corpus of 7,000 doctor-patient conversations on 7,000 patient conversations. |
Hierarchical Evidence Set Modeling for Automated Fact Extraction and Verification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact extraction and verification combine all evidence sentences to produce redundant information. |
| Approach: | They propose a framework to extract evidence sets and verify a claim to be supported, refuted or not enough info . they propose to encode and attend the claim and evidence sets at different levels of hierarchy . |
| Outcome: | The proposed framework outperforms 7 state-of-the-art methods for fact extraction and verification. |
Program Enhanced Fact Verification with Verbalization and Graph Attention Network (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification based on structured data are challenging and require further study. |
| Approach: | They propose a program-enhanced verbalization and a graph attention network to integrate programs and execution into textual inference models. |
| Outcome: | The proposed framework achieves a new state-of-the-art accuracy on a benchmark dataset . it is compared with existing frameworks on symbolic and informal inference models . |
Constrained Fact Verification for FEVER (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification rely on extracted evidence, but there is little work on understanding the reasoning process. |
| Approach: | They propose a method that enforces a closed-world reliance on extracted evidence to verify a claim's factuality. |
| Outcome: | The proposed model outperforms existing models on the FEVER shared task and shows that it is more accurate than previous models. |
Entity Linking in 100 Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multilingual entity linking are cross-lingual, with a focus on zero-shot evaluation. |
| Approach: | They propose a new formulation for multilingual entity linking where language-specific mentions resolve to a language-agnostic Knowledge Base. |
| Outcome: | The proposed model outperforms state-of-the-art models on a large multilingual dataset and shows that frequency-based analysis provided key insights for the model and training enhancements. |
PatchBERT: Just-in-Time, Out-of-Vocabulary Patching (2020.emnlp-main)
Copied to clipboard
| Challenge: | a pre-trained language model with low OOV can improve performance for transfer learning . a vocabulary surrogate can provide performance boosts with no additional computation cost . |
| Approach: | They propose multiple methods to mitigate OOV during downstream task fine-tuning . they demonstrate that vocabulary surrogates can provide performance boosts with no additional computation cost . |
| Outcome: | The proposed methods improve performance with the same parameter count when combined with fine-tuning. |
On the importance of pre-training data volume for compact language models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in language modeling have led to computationally intensive and resource-demanding state-of-the-art models. |
| Approach: | They investigate the impact of pre-training data volume on compact language models . they use a French question answering task to train models with as little as 100 MB of text . |
| Outcome: | The results show that pre-training data volume can improve models with as little as 100 MB of text . the results suggest that the model performance is poorer with less data than with larger datasets . |
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing (2020.emnlp-main)
Copied to clipboard
| Challenge: | a novel approach to compress neural networks by progressive module replacement is proposed . a number of techniques have been proposed to compress pretraining and fine-tuning models . |
| Approach: | They propose a model compression approach that divides BERT into modules and builds their compact substitutes. |
| Outcome: | The proposed approach outperforms existing knowledge distillation approaches on GLUE benchmark . it is based on a model that divides the original BERT into several modules and builds their substitutes . |
Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to fine-tune deep pretrained language models face catastrophic forgetting problems. |
| Approach: | They propose a recall and learn mechanism which integrates pretraining and downstream tasks into a single mechanism. |
| Outcome: | The proposed method achieves state-of-the-art performance on the GLUE benchmark and better average performance than directly fine-tuning of BERT-large. |
Exploring and Predicting Transferability across NLP Tasks (2020.emnlp-main)
Copied to clipboard
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
| Challenge: | Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. |
| Approach: | They conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems. |
| Outcome: | The proposed model can improve performance even with low-data source tasks that differ substantially from the target task. |
To BERT or Not to BERT: Comparing Task-specific and Task-agnostic Semi-Supervised Approaches for Sequence Tagging (2020.emnlp-main)
Copied to clipboard
Kasturi Bhattacharjee, Miguel Ballesteros, Rishita Anubhai, Smaranda Muresan, Jie Ma, Faisal Ladhak, Yaser Al-Onaizan
| Challenge: | Using large amounts of unlabeled data to improve performance has become the foundation for many natural language processing tasks. |
| Approach: | They propose a task-specific semi-supervised approach that uses unlabeled data in a more task-agnostic manner. |
| Outcome: | The proposed approach achieves similar performance to BERT on a set of sequence tagging tasks with less financial and environmental impact. |
Cold-start Active Learning through Self-supervised Language Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Labeling data is a fundamental bottleneck in machine learning due to annotation cost and time. |
| Approach: | They propose a strategy that uses the pre-training loss to find examples that surprise the model and minimize labeling costs. |
| Outcome: | The proposed approach reduces labeling costs and costs by using pre-trained language models. |
Active Learning for BERT: An Empirical Study (2020.emnlp-main)
Copied to clipboard
Liat Ein-Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, Noam Slonim
| Challenge: | Existing approaches to deal with data scarcity are active learning (AL) and pre-trained models are not being considered. |
| Approach: | They propose to use active learning techniques to cope with data scarcity in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
| Outcome: | The proposed methods improve BERT performance in binary text classification scenarios where the annotation budget is very small and the data is often skewed. |
Transformer Based Multi-Source Domain Adaptation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve machine learning performance are mixed experts and domain adversarial training. |
| Approach: | They investigate the problem of unsupervised multi-source domain adaptation . they combine predictions of multiple domain experts and combine them to induce a domain agnostic representation space . |
| Outcome: | The proposed methods improve models' performance while limiting learning time. |
Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve accuracy of neural networks are slow due to computational complexity. |
| Approach: | They propose a vector-vector-matrix architecture which greatly reduces latency at inference time for NLP applications by a factor of four. |
| Outcome: | The proposed framework reduces the latency of sequence-to-sequence and Transformer models used for NMT by a factor of four. |
The importance of fillers for text representations of speech transcripts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Fillers are a type of disfluency that can be a sound ("um" or "uh") filling a pause in an utterance or conversation. |
| Approach: | They propose to represent fillers with deep contextualised embeddings to improve modelling of spoken language and two downstream tasks . |
| Outcome: | The proposed representations improve modelling of spoken language and two downstream tasks, predicting a speaker’s stance and expressed confidence. |
The role of context in neural pitch accent detection in English (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prosody is a rich information source in natural language, serving as a marker for phenomena such as contrast. |
| Approach: | They propose a model that uses full utterances as input and adds an LSTM layer to detect prosodic events in speech. |
| Outcome: | The proposed model improves on the American English speech in the Boston University Radio News Corpus. |
VolTAGE: Volatility Forecasting via Text Audio Fusion with Graph Convolution Networks for Earnings Calls (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to stock volatility forecasting ignore correlations between stocks. |
| Approach: | They propose to combine vocal cues with verbal and financial cue data to create a multimodal stock volatility prediction model that accounts for stock interdependence via graph convolutions. |
| Outcome: | The proposed model outperforms existing methods showing that it can predict volatility using multimodal learning. |
Effectively pretraining a speech translation decoder with Machine Translation data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the performance of AST systems are based on pretraining the encoder parameters using an ASR model, but using a pretrained MT decoder is not beneficial or improves the results. |
| Approach: | They propose to use an adversarial regularizer to bring the encoder representations of the ASR and NMT tasks closer even though they are in different modalities. |
| Outcome: | The proposed model can be pre-trained using the Automatic Speech Recognition (ASR) task even in different languages and improves in low resource settings. |
A Preliminary Exploration of GANs for Keyphrase Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on extractive keyphrases have shown promising results, but the results suggest that there is room for improvement. |
| Approach: | They propose a new keyphrase generation approach using Generative Adversarial Networks (GANs) their model produces a sequence of keyphrases and a discriminator distinguishes between human-curated and machine-generated keyphrase. |
| Outcome: | The proposed model outperforms the state-of-the-art generative models on benchmark datasets and is comparable to the best performing extractive models. |
TESA: A Task in Entity Semantic Aggregation for Abstractive Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization systems focus on paraphrasing and simplifying the source content, to the exclusion of such semantic abstraction capabilities. |
| Approach: | They propose a dataset and task to fine tune an abstractive summarization model to generate aggregations of 5.3K entities from a crowd-sourced dataset. |
| Outcome: | The proposed task and dataset show that the proposed model can generate aggregations at a semantic level, but that it is too complex to use. |
MLSUM: The Multilingual Summarization Corpus (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing biases in multi-lingual datasets are limiting the use of multilingual data in document summarization tasks. |
| Approach: | They present MLSUM, the first large-scale MultiLingual SUMmarization dataset. |
| Outcome: | The proposed dataset contains 1.5M+ article/summary pairs in five different languages. |
Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles (2020.emnlp-main)
Copied to clipboard
| Challenge: | Multi-XScience is a dataset construction protocol that favours abstractive modeling approaches. |
| Approach: | They propose a large-scale multi-document summarization dataset that is based on articles and lexical databases and WordNet synonymy information to generate related-work sections of a paper. |
| Outcome: | The proposed method is based on lexical databases and WordNet synonymy information to write related work sections of a paper based upon their abstract and the articles they reference. |
Intrinsic Evaluation of Summarization Datasets (2020.emnlp-main)
Copied to clipboard
| Challenge: | Almost all popular summarization datasets do not come with inherent quality assurance guarantees. |
| Approach: | They propose to use 5 metrics to evaluate quality of summarization datasets . they find that data usage in recent summarizing research is inconsistent with the properties of the data. |
| Outcome: | The proposed metrics can be inexpensive heuristics for detecting generically low quality examples. |
Iterative Feature Mining for Constraint-Based Data Collection to Increase Data Diversity and Model Robustness (2020.emnlp-main)
Copied to clipboard
Stefan Larson, Anthony Zheng, Anish Mahendran, Rishi Tekriwal, Adrian Cheung, Eric Guldan, Kevin Leach, Jonathan K. Kummerfeld
| Challenge: | Recent work on dialog has found that crowdsourced data can have limited diversity as workers tend to write simple variations from prompts. |
| Approach: | They propose a general approach for guiding workers to write more diverse text by iteratively constraining their writing. |
| Outcome: | The proposed approach improves performance on dialog tasks and improves on existing datasets. |
Conversational Semantic Parsing for Dialog State Tracking (2020.emnlp-main)
Copied to clipboard
Jianpeng Cheng, Devang Agrawal, Héctor Martínez Alonso, Shruti Bhargava, Joris Driesen, Federico Flego, Dain Kaplan, Dimitri Kartsaklis, Lin Li, Dhivya Piraviperumal, Jason D. Williams, Hong Yu, Diarmuid Ó Séaghdha, Anders Johannsen
| Challenge: | Language understanding for task-based dialog systems is often termed "dialog state tracking" (DST) whereas semantic parsing is the task of converting a single-turn utterance to a graphstructured meaning representation, DST is more complex. |
| Approach: | They propose a framework for dialog state tracking that incorporates semantic compositionality, cross-domain knowledge sharing and co-reference. |
| Outcome: | The proposed framework improves on state-of-the-art approaches for dialog state tracking (DST) it incorporates semantic compositionality, cross-domain knowledge sharing and co-reference. |
doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset (2020.emnlp-main)
Copied to clipboard
| Challenge: | doc2dial dataset is a goal-oriented document-grounded dialogue model . it is based on how the authors compose documents for guiding end users . |
| Approach: | They propose a dataset of goal-oriented dialogues grounded in documents . they use annotated conversations with an average of 14 turns to generate conversational utterances . |
| Outcome: | The proposed dataset includes over 4500 annotated conversations with an average of 14 turns grounded in over 450 documents from four domains. |
Interview: Large-scale Modeling of Media Dialog with Discourse Patterns and Knowledge Grounding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Discourse analysis has been limited to small news corpora, but this study is expanding to tens of thousands of interviews. |
| Approach: | They propose a large-scale analysis of discourse in media dialog and its impact on dialog modeling with a focus on interrogative patterns and use of external knowledge. |
| Outcome: | The proposed model outperforms strong discourse-agnostic baselines for dialog modeling, generating more specific and topical responses in interview-style conversations. |
INSPIRED: Toward Sociable Recommendation Dialog Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on recommendation dialog systems lack a study on communication strategies used by human speakers for making successful and persuasive recommendations. |
| Approach: | They propose to annotate a dataset of human-human movie recommendation dialogs with sociable recommendation strategies. |
| Outcome: | The proposed model outperforms the baseline model in automatic and human evaluation. |
Information Seeking in the Spirit of Learning: A Dataset for Conversational Curiosity (2020.emnlp-main)
Copied to clipboard
| Challenge: | Open-ended human learning and information-seeking systems often ignore the user’s pre-existing knowledge. |
| Approach: | They propose to use pre-existing user knowledge to build a model that reproduces human assistant policies and improves over a bert content model by 13 mean reciprocal rank points. |
| Outcome: | The proposed model reproduces human assistant policies and improves over a bert content model by 13 mean reciprocal rank points. |
Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Social biases present in data are often directly reflected in the predictions of models trained on that data. |
| Approach: | They analyze gender bias in dialogue data and propose techniques to mitigate it . they use counterfactual data augmentation, targeted data collection, and bias controlled training . |
| Outcome: | The proposed techniques mitigate gender bias by balancing genderedness of generated dialogue utterances. |
Discriminatively-Tuned Generative Classifiers for Robust Natural Language Inference (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has shown advantages of generative classifiers in terms of data efficiency and robustness. |
| Approach: | They propose a generative classifier for natural language inference (NLI) they compare it to discriminative models and large-scale pretrained models like BERT . |
| Outcome: | The proposed classifier outperforms discriminative and pretrained baselines across several challenging NLI experimental settings, including small training sets, imbalanced label distributions, and label noise. |
New Protocols and Negative Results for Textual Entailment Data Collection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Natural language inference data has proven useful in benchmarking and as pretraining data for tasks requiring language understanding. |
| Approach: | They propose four alternative protocols to improve annotation quality and diversity . they use 8.5k-example training sets to compare different protocols . |
| Outcome: | The proposed protocols improve the ease of training and quality of the examples. |
The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural network models have significantly pushed forward performance on natural language processing benchmarks with the development of largescale language model pre-training. |
| Approach: | They find that models on natural language inference and reading comprehension are unstable . they propose to use a model-selection routine to analyze the model's instability . |
| Outcome: | The proposed models can perform poorly on two language-related tasks, the authors show . they also show that the model selection routine is unstable, and that it is not reliable . |
Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start (2020.emnlp-main)
Copied to clipboard
| Challenge: | a current approach to solving NLP problems is to build a problem-specific dataset . current approaches do not allow for transforming tasks into textual entailment . |
| Approach: | They propose a pretrained textual entailment system that can generalize across domains . they argue that when is it worth transforming an NLP task into textual detailment? |
| Outcome: | The proposed model can generalize across domains with few examples, the authors argue . they show that it can be used for several downstream NLP tasks with limited annotations . |
ConjNLI: Natural Language Inference Over Conjunctive Sentences (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing stress tests do not consider non-boolean usages of conjunctions and use templates . large-scale pre-trained models do not understand conjunctive semantics well, we find . |
| Approach: | They propose a stress-test for natural language inference over conjunctive sentences where the premise differs from the hypothesis by conjunctions removed, added, or replaced. |
| Outcome: | The proposed stress-test for natural language inference over conjunctive sentences is challenging . it finds that pre-trained models do not understand conjunction semantics well . |
Data and Representation for Turkish Natural Language Inference (2020.emnlp-main)
Copied to clipboard
| Challenge: | Large annotated datasets in NLP are overwhelmingly in English . obtaining new annotation resources for each task in each language would be prohibitively expensive . |
| Approach: | They propose to use machine translation to translate large annotated datasets into Turkish . they find that in-language embeddings are essential and morphological parsing can be avoided . |
| Outcome: | The proposed model trains on human-translated evaluation sets. |
Multitask Learning for Cross-Lingual Transfer of Broad-coverage Semantic Dependencies (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for developing broad-coverage semantic dependency parsers for languages without semantically annotated data are limited to English, Czech and Chinese. |
| Approach: | They propose a multitask learning framework coupled with annotation projection to build broad-coverage semantic dependency parsers for languages without annotated resources. |
| Outcome: | The proposed model improves labeled F1 score on multitask tasks from English to Czech compared to baseline models . |
Precise Task Formalization Matters in Winograd Schema Evaluations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent results report a surge in performance to nearhuman levels on the Winograd Schema Challenge (WSC) however, variations in task formulation across papers and evaluations makes it hard to understand the true degree of recent progress. |
| Approach: | They propose to use a model with multiple choice to frame the task as multiple choice and reuse a pretrained language modeling head to mitigate the model's extreme sensitivity to hyperparameters. |
| Outcome: | The proposed frameworks improve the model's reasoning ability by framing the task as multiple choice and reuse of a pretrained language modeling head. |
Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural models pick up on annotation artefacts and spurious correlations, resulting in learning sentences that suffer from the same biases. |
| Approach: | They propose to tackle this problem by using adversarial training to reduce the bias in sentence representations by using an ensemble of adversaries. |
| Outcome: | The proposed approach produces more robust models outperforming previous de-biasing efforts when generalised to 12 other NLI datasets. |
SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery (2020.emnlp-main)
Copied to clipboard
| Challenge: | Entity set expansion and synonym discovery are two critical NLP tasks that are often performed separately, without exploring their interdependencies. |
| Approach: | They propose a framework that enables two tasks to mutually enhance each other by including popular entities’ infrequent synonyms into the set, which boosts set expansion recall. |
| Outcome: | The proposed framework can be used to enhance two NLP tasks by including popular entities’ infrequent synonyms into the set, which boosts set expansion recall. |
Evaluating the Calibration of Knowledge Graph Embeddings for Trustworthy Link Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing calibration techniques are less effective under the standard closed-world assumption (CWA) and the more realistic open-world hypothesis (OWA) Existing methods are not effective under OWA and provide explanations for this discrepancy. |
| Approach: | They conduct an evaluation under the standard closed-world assumption (CWA) and introduce the more realistic but challenging open-world assume (OWA) . they find existing calibration techniques are much less effective under the OWA than the CWA . |
| Outcome: | The proposed calibration techniques are much less effective under the open-world assumption (OWA) and explain the discrepancy. |
Text Graph Transformer for Document Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text classification are not scalable to large corpus and ignore heterogeneity of text graph. |
| Approach: | They propose a Transformer-based heterogeneous graph neural network that captures structure and heterogenity from the text graph. |
| Outcome: | The proposed model outperforms state-of-the-art methods on large-sized corpus datasets and significantly reduces computing and memory costs. |
CoDEx: A Comprehensive Knowledge Graph Completion Benchmark (2020.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graph completion benchmarks for knowledge graphs are often incomplete . however, the field has remained static over the past decade . |
| Approach: | They propose to use Wikidata and Wikipedia to improve on existing benchmarks . they analyze logical relation patterns, then perform baseline link prediction and triple classification . |
| Outcome: | The proposed datasets improve upon existing benchmarks in scope and difficulty. |
META: Metadata-Empowered Weak Supervision for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for weakly supervised text classification use text data alone to generate pseudo-labels . strong label indicators exist in metadata and it has been long overlooked due to challenges . |
| Approach: | They propose a framework that leverages metadata as an additional source of weak supervision by combining text data and metadata into a text-rich network. |
| Outcome: | The proposed framework exploits metadata as an additional source of weak supervision. |
Towards More Accurate Uncertainty Estimation In Text Classification (2020.emnlp-main)
Copied to clipboard
Jianfeng He, Xuchao Zhang, Shuo Lei, Zhiqian Chen, Fanglan Chen, Abdulaziz Alhamadani, Bei Xiao, ChangTien Lu
| Challenge: | Existing models of uncertainty score depend on winning score, which is the maximum probability in a semantic vector. |
| Approach: | They propose to generate accurate uncertainty score by improving the confidence of winning scores by reducing the effect of overconfidence of winning score and considering the impact of different categories simultaneously. |
| Outcome: | The proposed model reduces the effect of overconfidence of winning score and considers impact of different categories of uncertainty simultaneously. |
Chapter Captor: Text Segmentation in Novels (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using a hybrid approach, we identify chapter boundaries in novels . chapter boundaries are typically denoted by formatting conventions such as page breaks, white-space, chapter numbers, and titles. |
| Approach: | They build a project Gutenberg data set of 9,126 English novels to analyze chapter boundaries . they use neural inference and rule matching to recognize chapter title headers . |
| Outcome: | The proposed method achieves an F1 score of 0.77 on the segmentation task . the annotated data reveal interesting historical trends in the chapter structure of novels . |
Authorship Attribution for Neural Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have enabled the generation of realistic artifacts . however, the qualities of texts generated by these models are better, often confusing classifiers if they are not real. |
| Approach: | They propose to use neural network-based language models to generate realistic texts . they investigate the authorship attribution problem in three versions of a text . |
| Outcome: | The proposed models generate texts that are difficult to distinguish from human-written ones . the results show that most generators still generate texts significantly different from human ones compared to other models . |
NwQM: A neural quality assessment framework for Wikipedia (2020.emnlp-main)
Copied to clipboard
| Challenge: | Wikipedia articles are classified into several quality classes, which indicate their reliability as encyclopedic content. |
| Approach: | They propose a deep learning model which accumulates signals from key information sources to obtain improved Wikipedia article representation. |
| Outcome: | The proposed model improves Wikipedia article representation by 8% over state-of-the-art approaches with detailed ablation studies. |
Towards Modeling Revision Requirements in wikiHow Instructions (2020.emnlp-main)
Copied to clipboard
| Challenge: | wikiHow is a collaboratively edited platform of how-to guides . authors extend existing textual edits with 4 million sentences that remain unedited . |
| Approach: | They extend existing textual edits with a set of 4 million sentences that remain unedited over time. |
| Outcome: | The proposed model can predict the need for edits in wikiHow guides . the authors extend an existing resource of textual edits with a complementary set of 4 million sentences that remain unedited over time . |
Deep Attentive Learning for Stock Movement Prediction From Social Media Text and Company Correlations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that predict stock movements are based on time series and technical analysis, but price signals alone fail to capture market surprises and impacts of sudden unexpected events. |
| Approach: | They propose a model that integrates chaotic temporal signals from financial data and social media to create hierarchical temporal networks. |
| Outcome: | The proposed model can be used to forecast stock movements on real-world S&P 500 index data and English tweets. |
Natural Language Processing for Achieving Sustainable Development: the Case of Neural Labelling to Enhance Community Profiling (2020.emnlp-main)
Copied to clipboard
| Challenge: | In recent years, there has been an increasing interest in the application of Artificial Intelligence (AI) to the field of Sustainable Development (SD). |
| Approach: | They propose a new extreme multi-class multi-label Automatic UserPerceived Value classification task that uses a complex corpus of interviews to investigate the problem. |
| Outcome: | The proposed task solves a cost- and time-barrier in constructing qualitative data that prevents its widespread use and associated benefits. |
To Schedule or not to Schedule: Extracting Task Specific Temporal Entities and Associated Negation Constraints (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for date-time entity extraction from text are task agnostic, resulting in insufficient results for task specific date-timing extraction. |
| Approach: | They propose a model for extracting subset of date-time entities from text and their negation constraints. |
| Outcome: | The proposed model achieves an absolute gain of 19% f-score points compared to baseline methods in detecting date-time entities relevant to scheduling meetings and a 4% improvement over baseline methods for detecting negation constraints over date-timing entities. |
Competence-Level Prediction and Resume & Job Description Matching Using Context-Aware Transformer Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new method for resume classification reduces the time and labor needed to screen applications . the current method of screening applications involves reviewing individual resumes via string/regex matching . |
| Approach: | They propose to use transformer-based resume classification to reduce time and labor needed to screen applications. |
| Outcome: | The proposed models reduce time and labor needed to screen applications while improving the selection of suitable candidates. |
Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses (2020.emnlp-main)
Copied to clipboard
| Challenge: | CWEB is a new benchmark for grammatical error correction (GEC) systems . website data contains far fewer grammamatical errors than learner essays . |
| Approach: | They propose to broaden the target domain of grammatical error correction (GEC) systems . website data contains far fewer grammamatical errors than learner essays . |
| Outcome: | The proposed model can't rely on a strong internal language model in low error density domains. |
Deconstructing word embedding algorithms (2020.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are reliable feature representations of words used in many NLP tasks today. |
| Approach: | They propose to deconstruct Word2vec, GloVe and others into a common form . they propose to generalize several word embedding algorithms into . a low rank embedder framework is proposed to generalise the algorithms into one common form. |
| Outcome: | The proposed framework can be used to make word embeddings more performant. |
Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that detect semantically shifted words do not account for its evolution through time. |
| Approach: | They propose three variants of sequential models for detecting semantically shifted words . they demonstrate that temporal modelling of word representations yields a clear-cut advantage . |
| Outcome: | The proposed models account for the changes in word representations over time. |
Sparsity Makes Sense: Word Sense Disambiguation Using Sparse Contextualized Word Representations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using sparse word embeddings is highly applicable for word sense disambiguation (WSD) . |
| Approach: | They propose an overcomplete set of semantic basis vectors that allows for sparse word representations. |
| Outcome: | The proposed framework achieves an aggregated F score of 78.8 over five standard word sense disambiguating benchmark datasets. |
Exploring Semantic Capacity of Terms (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that measure semantic capacity of terms are not all considered equal . a good command of semantic capacity will give us more insight into the granularity of terms . |
| Approach: | They propose a model that evaluates semantic capacity of terms if text corpus can provide enough co-occurrence information of terms. |
| Outcome: | The proposed model can evaluate semantic capacity of terms if the corpus can provide enough co-occurrence information of terms. |
Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current models for document coreference resolution have large memory requirements and quadratic runtime in document length. |
| Approach: | They propose a memory-augmented neural network that tracks only a small number of entities at a time. |
| Outcome: | The proposed model outperforms existing models on OntoNotes and LitBank in memory management and memory management. |
Revealing the Myth of Higher-Order Inference in Coreference Resolution (2020.emnlp-main)
Copied to clipboard
| Challenge: | Adapted coreference resolution models have only marginally improved performance over representation learning. |
| Approach: | They implement an end-to-end coreference system and four HOI approaches to analyze the impact of higher-order inference on coreference resolution. |
| Outcome: | The proposed model shows that the impact of higher-order inference (HOI) on coreference resolution is negative to marginal, providing a new perspective on the task. |
Pre-training Mention Representations in Coreference Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve coreference resolution use labeled data. |
| Approach: | They propose two self-supervised tasks that are closely related to coreference resolution to improve mention representation. |
| Outcome: | The proposed models improve mention representations by learning them on a GAP dataset. |
Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Walk-based models have shown their advantages in knowledge graph reasoning but are limited by their representations and generalizability. |
| Approach: | They propose a walk-based model that leverages high-quality rules generated by symbolic-based methods to provide reward supervision for walk- based agents. |
| Outcome: | Experiments on benchmark datasets show that RuleGuider improves the performance of walk-based models without losing interpretability. |
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that deep contextualized language models (LMs) can extract temporal relations between events and time expressions. |
| Approach: | They propose a temporal relation extraction technique which extracts temporal relations between events and time expressions. |
| Outcome: | The proposed method significantly improves temporal dependency parsing, the authors show . their work compares the proposed method to other methods and shows where they may fail . |
Systematic Comparison of Neural Architectures and Training Approaches for Open Information Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Open information extraction (OIE) is a method for extracting facts from text in structured format . alternative formulations allow for longer tuples, but most work focuses on binary predicates only. |
| Approach: | They propose to extract facts from natural language text and represent them as structured triples . they compare different neural network architectures and training approaches . |
| Outcome: | The proposed approach improves the currently best models on the OIE16 benchmark by 0.421 F1 score and 0.420 AUC-PR . |
SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing active sequence labeling methods use the queried samples alone in each iteration, which is inefficient for leveraging human annotations. |
| Approach: | They propose a data augmentation method to augment queried samples by generating extra labeled sequences in each iteration. |
| Outcome: | The proposed method improves the standard active sequence labeling method by 2.27%–3.75% in terms of F1 scores. |
AxCell: Automatic Extraction of Results from Machine Learning Papers (2020.emnlp-main)
Copied to clipboard
Marcin Kardas, Piotr Czapla, Pontus Stenetorp, Sebastian Ruder, Sebastian Riedel, Ross Taylor, Robert Stojnic
| Challenge: | AxCell is an automatic machine learning pipeline for extracting results from papers . it uses a table segmentation subtask to learn relevant structural knowledge that aids extraction. |
| Approach: | They propose to use a table segmentation subtask to learn relevant structural knowledge that aids extraction. |
| Outcome: | The proposed approach improves state of the art for results extraction and can be used for semi-automated results extraction in production. |
Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for open attribute value extraction for emerging entities are noisy or incomplete, even missing. |
| Approach: | They propose a knowledge-guided reinforcement learning framework for open attribute value extraction for emerging entities. |
| Outcome: | The proposed framework outperforms baselines by 16.5 - 27.8%. |
DualTKB: A Dual Learning Bridge between Text and Knowledge Base (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for KB construction and sentence generation are lacking in the field of knowledge transfer. |
| Approach: | They propose a dual learning approach for unsupervised text to path and path to text transfers in Commonsense Knowledge Bases. |
| Outcome: | The proposed method compares favorably to existing baselines and is a viable step towards a more advanced system for automatic KB construction/expansion and reverse operation of sentence generation from KBs. |
Incremental Neural Coreference Resolution in Constant Memory (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on coreference resolution has focused on improving pairwise span scoring functions and methods for decoding into globally consistent clusters. |
| Approach: | They extend an incremental clustering algorithm to utilize contextualized encoders and neural components to generate a high-performing model. |
| Outcome: | The proposed model reduces memory usage to constant space with only a 0.3% relative loss in F1 on OntoNotes 5.0. |
Improving Low Compute Language Modeling with In-Domain Embedding Initialisation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train language models on in-domain data are limited. |
| Approach: | They propose to initialise and freeze in-domain embeddings to provide a useful representation of rare words in English . they find that the standard configuration is not optimal when rare words are present . |
| Outcome: | The proposed approach improves language modeling by providing a useful representation of rare words in English. |
KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for data-to-text generation rely on labeled data, which is costly to acquire and limits their application to new tasks and domains. |
| Approach: | They propose to leverage pre-training and transfer learning to address this problem by leveraging a general knowledge-grounded generation model and a knowledge-based model. |
| Outcome: | The proposed model can generate knowledge-enriched text on a knowledge-grounded text corpus crawled from the web in three settings. |
POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models cannot be directly employed to generate text under specified lexical constraints. |
| Approach: | They propose a method for insertion-based text generation that inserts tokens between existing tokens in a parallel manner. |
| Outcome: | The proposed method is intuitive and interpretable on Wikipedia and Yelp datasets. |
Unsupervised Text Style Transfer with Padded Masked Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for style transfer are difficult to obtain and require substantial amounts of parallel training examples to work well. |
| Approach: | They propose an unsupervised method for style transfer that uses masked language models to find the text spans where the two models disagree the most in terms of likelihood. |
| Outcome: | The proposed method performs competitively in a fully unsupervised setting and improves accuracy in low-resource settings by over 10 percentage points when pre-training on silver training data generated by Masker. |
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing techniques for natural language understanding and generation use autoencoding and/or autoregressive objectives to train models. |
| Approach: | They propose a self-supervised pre-training scheme that pre-trains an autoencoding and autoregressive language model on a large unlabeled corpus for generating new text conditioned on context. |
| Outcome: | The proposed scheme achieves state-of-the-art results on a variety of language generation benchmarks covering generative question answering, abstractive summarization and conversational response generation. |
Gradient-guided Unsupervised Lexically Constrained Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for lexically constrained generation fail when the search space is too large . a novel method to solve the problem is based on gradient-guided optimization . |
| Approach: | They propose a method to solve lexically-constrained generation as an unsupervised gradient-guided optimization problem. |
| Outcome: | The proposed method achieves state-of-the-art compared to previous methods . it is free of parallel data training, flexible to be used in the inference stage of any pre-trained generation model. |
TeaForN: Teacher-Forcing with N-grams (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to address exposure bias and lack of differentiability in sequence generation models with teacherforcing have failed to address these issues. |
| Approach: | They propose a method that uses a stack of N decoders to decode along a secondary time axis and allows model-parameter updates based on N prediction steps. |
| Outcome: | Empirically, teaForN boosts generation quality on one Machine Translation benchmark, WMT 2014 English-French, and two News Summarization benchmarks, CNN/Dailymail and Gigaword. |
Experience Grounds Language (2020.emnlp-main)
Copied to clipboard
Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, Joseph Turian
| Challenge: | aaron carroll: language understanding research is held back by a failure to relate language to the physical world it describes and to social interactions it facilitates. carroll says successful linguistic communication relies on a shared experience of the world. |
| Approach: | They propose to use a broader physical and social context to address communication problems . they argue that the current success of representation learning approaches is limited . |
| Outcome: | a new study suggests that the current success of representation learning requires a parallel tradition of research on the broader physical and social context of language to address the deeper questions of communication. |
Keep CALM and Explore: Language Models for Action Generation in Text-based Games (2020.emnlp-main)
Copied to clipboard
| Challenge: | Text-based games present a unique challenge for autonomous agents to operate in natural language and handle enormous action spaces. |
| Approach: | They propose a Contextual Action Language Model (CALM) to generate a compact set of action candidates at each game state. |
| Outcome: | The proposed model achieves a 69% improvement in average game score on unsupervised games . the proposed model is competitive with or better than other models that have access to ground truth admissible actions on half of the games tested . |
CapWAP: Image Captioning with a Purpose (2020.emnlp-main)
Copied to clipboard
| Challenge: | a traditional image captioning task uses generic reference captions to provide textual information about images. |
| Approach: | They propose a task that uses question-answer pairs to provide visual information instead of generic reference captions. |
| Outcome: | The proposed captioning with a purpose task can be tailored to meet user needs . question-answer pairs are used as a source of supervision for learning visual information needs a new task is proposed . |
What is More Likely to Happen Next? Video-and-Language Future Event Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models cannot make multimodal commonsense predictions of future events based on video and dialogue . |
| Approach: | They propose a task to predict which event is more likely to happen in a video clip . they use a dataset with 28,726 future event prediction examples from 10,234 videos . |
| Outcome: | The proposed model provides a good starting point but leaves room for future work. |
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has adapted vision-and-language models to generative tasks like image captioning. |
| Approach: | They propose an extension to LXMERT with training refinements to generate images from text. |
| Outcome: | The proposed model can generate images from pieces of text while still being comparable to existing models. |
Towards Understanding Sample Variance in Visually Grounded Language Generation: Evaluations and Observations (2020.emnlp-main)
Copied to clipboard
| Challenge: | A major challenge in visually grounded language generation is to build robust benchmark datasets and models that can generalize well in real-world settings. |
| Approach: | They propose to use visual attention to build robust benchmark datasets and models that can generalize well in real-world settings. |
| Outcome: | The proposed models show that human-generated references vary drastically in different datasets/tasks, revealing the nature of each task. |
Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube (2020.emnlp-main)
Copied to clipboard
| Challenge: | a representative pretraining model is fit to a diverse YouTube8M dataset . a priori, this domain is relatively easy for instructional videos . |
| Approach: | They fit a representative pretraining model to a YouTube8M dataset and examine its success and failure cases. |
| Outcome: | The proposed model can be trained on more diverse video corpora and achieve high performance on many video understanding tasks. |
Hierarchical Graph Network for Multi-hop Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing multi-hop question answering models focus on multi-level reasoning across multiple documents or paragraphs. |
| Approach: | They propose a hierarchical graph network that aggregates clues from scattered texts . they use a set of contextual encoders to initialize nodes on different levels of granularity . |
| Outcome: | The proposed model outperforms existing multi-hop QA approaches on the HotpotQA benchmark. |
A Simple Yet Strong Pipeline for HotpotQA (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for multi-hop question answering have been proposed with varying complexities. |
| Approach: | They propose to use BERT to identify potentially relevant sentences independently of each other . they feed selected sentences into a standard BERT span prediction model to choose an answer . |
| Outcome: | The proposed pipeline outperforms existing models on hotpotQA and support identification. |
Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models exploit dataset artifacts to produce correct answers without connecting information across multiple facts. |
| Approach: | They formalize disconnected reasoning across subsets of supporting facts to reduce disconnected reasoning . they propose an automatic transformation of existing datasets that reduces disconnected reasoning. |
| Outcome: | The proposed model-agnostic probe reduces disconnected reasoning in a reading comprehension setting. |
Unsupervised Question Decomposition for Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing QA systems struggle to answer complex questions because information is scattered in different places. |
| Approach: | They propose an unsupervised algorithm that decomposes hard questions into simpler sub-questions . they propose an algorithm that can be used to generate a final answer from millions of questions . |
| Outcome: | The proposed algorithm decomposes hard questions into simpler sub-questions that existing QA systems can answer. |
SRLGRN: Semantic Role Labeling Graph Reasoning Network (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that use context and type-matching heuristics do not provide realistic evaluation of reasoning capabilities. |
| Approach: | They propose a graph reasoning network based on the semantic structure of the sentences to learn cross paragraph reasoning paths and find supporting facts and the answer jointly. |
| Outcome: | The proposed network shows competitive performance on the HotpotQA distractor setting benchmark compared to the state-of-the-art models. |
CancerEmo: A Dataset for Fine-Grained Emotion Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | a lack of large annotated datasets hinders emotion detection in the health domain . a recent study shows that online sharing of emotions is beneficial to a patient's progress . |
| Approach: | They propose an emotion dataset annotated with eight fine-grained emotions from an online health community. |
| Outcome: | The proposed model achieves an average F1 of 71% on the cancerEmo dataset . the best model achieve a higher F1 than the previous model, which was improved using domain-specific pre-training. |
Exploring the Role of Argument Structure in Online Debate Persuasion (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work in NLP has shown that linguistic features extracted from debate text and features encoding the characteristics of the audience are both critical in persuasion studies. |
| Approach: | They propose to incorporate argument structure features into an LSTM-based model to assess the persuasiveness of debates. |
| Outcome: | The proposed model incorporates argument structure features to predict debaters that make the most convincing arguments on online debate forums. |
Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for stance detection are topic-specific and cross-target stance. |
| Approach: | They propose a new dataset for zero-shot stance detection that captures a wider range of topics and lexical variation than in previous datasets. |
| Outcome: | The proposed model improves performance on a number of challenging linguistic phenomena. |
Sentiment Analysis of Tweets using Heterogeneous Multi-layer Network Representation and Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to handle sentiment analysis of tweets are inadequate due to various characteristics such as under-specificity, noise, and multilingual content. |
| Approach: | They propose a multi-layer network-based representation of tweets to generate multiple representations of a tweet and classify them using a neural-based early fusion approach. |
| Outcome: | The proposed method can address the problem of under-specificity, noisy text, and multilingual content present in a tweet and provides better representations than the text-based counterparts. |
Introducing Syntactic Structures into Target Opinion Word Extraction with Deep Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current deep learning models fail to exploit syntactic information of sentences . proposed model incorporates syntax-based opinion possibility scores and syntaktic connections between the words . |
| Approach: | They propose to incorporate syntactic information of sentences into deep learning models for TOWE . they propose a novel regularization technique to improve the performance of the models . |
| Outcome: | The proposed model achieves state-of-the-art on four benchmark datasets. |
EmoTag1200: Understanding the Association between Emojis and Emotions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Emojis are increasingly used to convey affect, but their use is not trivial. |
| Approach: | They propose to use human-solicited association ratings to explore the connection between emojis and emotions to conduct experiments. |
| Outcome: | The proposed method can be inferred from word-level information when high-quality information is available. |
MIME: MIMicking Emotions for Empathetic Response Generation (2020.emnlp-main)
Copied to clipboard
Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, Soujanya Poria
| Challenge: | Empathy is a fundamental human trait that reflects our ability to understand and reflect the thoughts and feelings of the people we interact with. |
| Approach: | They propose to use polarity-based emotion clusters to generate empathetic responses . they also introduce stochasticity into the emotion mixture that yields emotionally more varied responses compared to the previous work . |
| Outcome: | The proposed methods improve empathy and contextual relevance of the response, and introduce stochasticity into the emotion mixture that yields emotionally more varied responses than the previous work. |
Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for integrating knowledge graphs into pre-trained language models have been poorly implemented. |
| Approach: | They propose a self-supervised entity masking scheme that exploits relational knowledge underlying the text. |
| Outcome: | The proposed model achieves improved performance on five benchmarks, including question answering and knowledge base completion. |
Named Entity Recognition Only from Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing named entity recognition systems require large amounts of human annotated training data. |
| Approach: | They propose a fully unsupervised named entity recognition model which takes clues from pre-trained word embeddings. |
| Outcome: | The proposed model can be trained on two CoNLL benchmark datasets without annotating lexicon or corpus. |
Text Classification Using Label Names Only: A Language Model Self-Training Approach (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current text classification methods require a large number of labeled documents as training data. |
| Approach: | They propose a model that uses only the label name of each class to train classification models on unlabeled data without using any labeled examples. |
| Outcome: | The proposed model achieves 90% accuracy on four benchmark datasets using label names as the only supervision . |
Neural Topic Modeling with Cycle-Consistent Adversarial Training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances on deep generative models have attracted significant interest in neural topic modeling. |
| Approach: | They propose an adversarial-neural topic model which uses Dirichlet prior to capture the semantic patterns in latent topics. |
| Outcome: | The proposed models outperform competing models on unsupervised/supervised topic modeling and text classification. |
Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for data augmentation produce low readability or semantic consistency. |
| Approach: | They propose a framework which augments data through reinforcement learning guided conditional generation. |
| Outcome: | The proposed framework improves F1 performance on three different classification tasks by 8.7% on average when given only 10% of the whole data for training. |
A State-independent and Time-evolving Network for Early Rumor Detection in Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to rumor detection ignored dynamical evolution of an event and failed to capture its unique features in different states. |
| Approach: | They propose a state-independent and time-evolving Network (STN) for rumor detection based on fine-grained event state detection and segmentation. |
| Outcome: | The proposed framework can significantly improve the rumor detection accuracy in comparison with some strong baseline systems. |
PyMT5: multi-mode translation of natural language and Python code with transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using Python method text-to-text transfer transformers, developers can easily model source code and natural language. |
| Approach: | They propose a Python method text-to-text transfer transformer that can translate between all pairs of Python method feature combinations. |
| Outcome: | The proposed model outperforms similar-sized auto-regressive language models on a large-scale parallel corpus of 26 million methods and 7.7 million method-docstring pairs on the CodeSearchNet test set. |
PathQG: Neural Question Generation from Facts (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research for question generation encodes text as a sequence of tokens without explicitly modeling fact information. |
| Approach: | They propose to incorporate facts in the input text for question generation in a comprehensive way. |
| Outcome: | The proposed model outperforms state-of-the-art models and human evaluation shows it generates relevant and informative questions. |
What time is it? Temporal Analysis of Novels (2020.emnlp-main)
Copied to clipboard
| Challenge: | a novel based on the flow of time provides a framework for understanding the text . a computational approach to annotate a book's lines with wall clock times is needed to understand the flow through time. |
| Approach: | They propose to annotate each line of a book with wall clock times . they use a data set of hourly time phrases from 52,183 fictional books . |
| Outcome: | The proposed method improves upon baselines by over two hours and can partition a book into segments that correspond to a particular time-of-day. |
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Natural language is characterized by compositionality: meaning of complex expressions is constructed from the meanings of its constituent parts. |
| Approach: | They propose a semantic parsing dataset based on a fragment of English to assess compositional generalization abilities. |
| Outcome: | The proposed model can generalize meanings in a given sentence in 96–99% of the tests, but generalization accuracy is lower and the generalization sensitivity is higher. |
An Analysis of Natural Language Inference Benchmarks through the Lens of Negation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for natural language inference ignore negations and can make inferences that are difficult to make. |
| Approach: | They propose a new benchmark for natural language inference in which negation plays a critical role. |
| Outcome: | The proposed benchmarks show that negation plays a critical role in inference judgments. |
On the Sentence Embeddings from Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained contextual representations like BERT have been widely used for NLP tasks. |
| Approach: | They propose to transform anisotropic sentence embedding distribution to smooth and isotropic Gaussian distribution by normalizing flows that are learned with an unsupervised objective. |
| Outcome: | The proposed method achieves significant performance gains over state-of-the-art embeddings on a variety of semantic textual similarity tasks. |
What Can We Learn from Collective Human Opinions on Natural Language Inference Data? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite the subjective nature of many NLU evaluations, little attention has been paid to the distribution of human opinions. |
| Approach: | They use a dataset with 464,500 annotations to study Collective HumAn OpinionS . they argue that models lack the ability to recover the distribution over human labels . |
| Outcome: | The proposed dataset examines the distribution of human opinions in NLU evaluation datasets. |
Improving Text Generation with Student-Forcing Optimal Transport (2020.emnlp-main)
Copied to clipboard
Jianqiao Li, Chunyuan Li, Guoyin Wang, Hao Fu, Yuhchen Lin, Liqun Chen, Yizhe Zhang, Chenyang Tao, Ruiyi Zhang, Wenlin Wang, Dinghan Shen, Qian Yang, Lawrence Carin
| Challenge: | Maximum likelihood estimation (MLE) is used to train models, but during testing, the model is conditioned on previously generated tokens, resulting in exposure bias. |
| Approach: | They propose to use optimal transport to match the sequences generated in MLE and test modes to reduce exposure bias. |
| Outcome: | The proposed method is validated on machine translation, text summarization, and text generation tasks. |
UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for open-ended text generation are poorly correlated with human judgments . despite the success of existing metrics, there are few plausible outputs for the same input . |
| Approach: | They propose a UNreferenced measure for evaluating open-ended story generation . it is built on top of BERT and is trained to distinguish human-written stories from negative samples . |
| Outcome: | The proposed measure is more generalizable than state-of-the-art metrics and correlates better with human judgments. |
Fˆ2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmax (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text generation do not fully reflect the rich diversity of human language. |
| Approach: | They propose to use F2-Softmax and MefMax to train a balanced frequency distribution using a frequency class-based method. |
| Outcome: | The proposed methods improve the diversity and quality of generated texts. |
Partially-Aligned Data-to-Text Generation with Distant Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using partially-aligned data is an alternative way of solving the dataset scarcity problem. |
| Approach: | They propose a task to generate human-readable text for describing some given structured data enabling more interpretability. |
| Outcome: | The proposed framework outperforms baseline models and validates the feasibility of using partially-aligned data. |
Like hiking? You probably enjoy nature: Persona-grounded Dialog with Commonsense Expansions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing persona-grounded dialog models fail to capture simple implications of given persona descriptions. |
| Approach: | They propose to expand available persona sentences using existing commonsense knowledge bases and paraphrasing resources to imbue dialog models with access to expanded and richer set of persona descriptions. |
| Outcome: | The proposed model outperforms baselines on the Persona-Chat dataset in terms of dialog quality and diversity while achieving persona-consistent and controllable dialog generation. |
A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Structured belief states are crucial for goal tracking and database query in task-oriented dialog systems. |
| Approach: | They propose a probabilistic dialog model where belief states are represented as discrete latent variables and jointly modeled with system responses given user inputs. |
| Outcome: | The proposed model outperforms supervised-only and semi-supervised baselines on three benchmark datasets. |
The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learning-to-rank response selection are suboptimal due to ignorance of diversity of response quality. |
| Approach: | They propose to use off-the-shelf response retrieval models as automatic grayscale data generators to train response selection models. |
| Outcome: | The proposed approach can be automated without human effort on grayscale data. |
GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating Open-Domain Dialogue Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics only consider surface features or utterance-level semantics, without explicitly considering the fine-grained topic transition dynamics of dialogue flows. |
| Approach: | They propose a graph-enhanced evaluation metric GRADE to evaluate dialogue coherence . GRADE incorporates utterance-level contextualized representations and fine-grained topic-level graph representations to improve communication logic. |
| Outcome: | The proposed evaluation metric outperforms state-of-the-art metrics on measuring diverse dialogue models in terms of Pearson and Spearman correlations with human judgments. |
MedDialog: Large-scale Medical Dialogue Datasets (2020.emnlp-main)
Copied to clipboard
Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, Pengtao Xie
| Challenge: | telemedicine is a medical practice that provides patient care remotely using video conferencing tools. |
| Approach: | They build large-scale medical dialogue datasets to facilitate research . they pretrain several models on the Chinese MedDialog dataset and compare their performance . |
| Outcome: | The proposed datasets show that models trained on MedDialog can generate doctor-like medical dialogues. |
An information theoretic view on selecting linguistic probes (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP tasks require a question of how much linguistic knowledge is encoded in neural networks. |
| Approach: | They propose to use diagnostic classifiers to perform supervised classification from internal representations. |
| Outcome: | Empirically, the two proposed criteria lead to results that agree with each other. |
With Little Power Comes Great Responsibility (2020.emnlp-main)
Copied to clipboard
| Challenge: | Underpowered experiments make it more difficult to discern the difference between statistical noise and meaningful model improvements and increase the chances of exaggerated findings. |
| Approach: | They characterize typical statistical power for a variety of settings and characterize it by a set of existing NLP papers and datasets. |
| Outcome: | The authors characterize typical power for a variety of settings and find it common in the literature. |
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics (2020.emnlp-main)
Copied to clipboard
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, Yejin Choi
| Challenge: | Large datasets have become commonplace in NLP research, but the emphasis on quantity has made it challenging to assess the quality of data. |
| Approach: | They propose a model-based tool to characterize and diagnose large datasets . they leverage the behavior of the model on individual instances during training . |
| Outcome: | Experiments on four datasets show that the tool can characterize and diagnose datasets with a model-based tool. |
Evaluating and Characterizing Human Rationales (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines how human rationales perform on automatic metrics . human-generated rationale evaluation is difficult because of its ambiguity . |
| Approach: | They propose to use model-dependent baseline performance to evaluate rationale quality . they propose to also use "fidelity curves" to reveal properties such as irrelevance and redundancy . |
| Outcome: | The proposed methods characterize rationale quality based on model retraining and using "fidelity curves" the proposed methods lead to actionable suggestions for evaluating and characterizing rationales . |
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | We present a method to produce abstractive summaries of documents that exceed several thousand words . we compare transformer based methods to extractive methods, but extractive models score higher . |
| Approach: | They propose a method to generate abstractive summaries of documents that exceed several thousand words via neural abstractive summary. |
| Outcome: | The proposed method produces abstractive summaries of documents that exceed several thousand words . it is compared with baseline methods, state-of-the-art models and variants of the proposed method . |
Multi-Fact Correction in Abstractive Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing abstractive summarization systems generate incorrect facts with respect to the source text. |
| Approach: | They propose a suite of two factual correction models that leverages question-answering knowledge to make corrections in system-generated summaries via span selection. |
| Outcome: | The proposed model improves factuality of news summarization without sacrificing summary quality. |
Evaluating the Factual Consistency of Abstractive Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | a weakly-supervised approach is needed to verify factual consistency . auxiliary span extraction tasks are useful for verifying factual consistent summaries . |
| Approach: | They propose a weakly-supervised approach for verifying factual consistency . they transfer the model to summaries generated by several neural models . |
| Outcome: | The proposed approach outperforms models trained with strong supervision on source documents and human evaluations. |
Re-evaluating Evaluation in Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Automated evaluation metrics are an essential part of the development of text-generation tasks such as summarization. |
| Approach: | They propose to use top-scoring system outputs to assess the reliability of automatic evaluation metrics for text summarization. |
| Outcome: | The proposed evaluation method is based on human judgments from 25 top-scoring neural summarization systems. |
VMSMO: Learning to Generate Multimodal Summary for Video-based News Articles (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that multimodal news can significantly improve users' sense of satisfaction for informativeness. |
| Approach: | They propose a task of Video-based Multimodal Summarization with Multimodal Output to solve this problem. |
| Outcome: | The proposed method can generate multimodal summaries with a single input . it can model the temporal dependency of video with semantic meaning of article . |