Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Probabilistic FastText for Multi-Sense Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Probabilistic FastText model for word embeddings captures word senses, sub-word structure, and uncertainty information. |
| Approach: | They propose a model for word embeddings that captures multiple word senses . they represent each word with a Gaussian mixture density, with each vector representing an n-gram . |
| Outcome: | The proposed model outperforms dictionary-level probabilistic embeddings on word-similarity benchmarks. |
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing word2vec-based methods for learning rare or unseen words have been criticized for degrading performance in small corpus settings. |
| Approach: | They propose a la carte embedding method that relies on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. |
| Outcome: | The proposed method is based on a new dataset showing that it can be used when a word is encountered even if only a single usage example is available. |
Unsupervised Learning of Distributional Relation Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing word embedding models rely on co-occurrence statistics to learn vector representations of word meaning. |
| Approach: | They propose a method which directly learns relation vectors from co-occurrence statistics. |
| Outcome: | The proposed method is based on a variant of GloVe, which has an explicit connection between word vectors and PMI weighted co-occurrence vectors. |
Explicit Retrofitting of Distributional Word Vectors (P18-1)
Copied to clipboard
| Challenge: | Existing models for word vector specialization focus on word co-occurrences from large text corpora, resulting in a tendency to fuse semantic similarity with other types of semantic relatedness. |
| Approach: | They propose to transform external lexico-semantic relations into training examples and learn an explicit retrofitting model. |
| Outcome: | The proposed model can specialize vector spaces of new languages and translate them to other languages. |
Unsupervised Neural Machine Translation with Weight Sharing (P18-1)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (NMT) is a new approach for machine translation . the model uses only one shared encoder to map pairs of sentences from different languages to a shared-latent space . |
| Approach: | They propose an unsupervised approach which trains the model without labeling data . they propose two independent encoders but share some partial weights to extract high-level representations of input sentences. |
| Outcome: | The proposed approach achieves significant improvements on English-German, English-French and Chinese-to-English translation tasks. |
Triangular Architecture for Rare Language Translation (P18-1)
Copied to clipboard
| Challenge: | Empirical results show that Neural Machine Translation (NMT) performs poor on low-resource pairs especially when Z is a rare language. |
| Approach: | They propose a triangular triangulation technique to leverage bilingual data to optimize the translation performance of low-resource pairs. |
| Outcome: | Empirical results show that the proposed architecture significantly improves translation quality of rare languages on MultiUN and IWSLT2012 datasets and even better when combining back-translation methods. |
Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (P18-1)
Copied to clipboard
| Challenge: | Subword units are an effective way to alleviate the open vocabulary problems in neural machine translation. |
| Approach: | They propose a method to regularize subword segmentations probabilistically by sampling subwords . they also propose 'unigram' language model to be used for better subword sampling . |
| Outcome: | The proposed method improves on low resource and out-of-domain settings with multiple corpora. |
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation (P18-1)
Copied to clipboard
Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, Macduff Hughes
| Challenge: | In recent years, the emergence of seq2seq models has revolutionized the field of machine translation by replacing traditional phrase-based approaches with neural machine translation (NMT) systems based on the encoder-decoder paradigm. |
| Approach: | They propose to use a convolutional seq2seq model to combine the strengths of the two approaches. |
| Outcome: | The proposed architectures outperform the existing models on the WMT’14 benchmark dataset. |
Ultra-Fine Entity Typing (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that a model that can predict ultra-fine types can be crowd-sourced . head words indicate the type of the noun phrases they appear in, and are important for context-sensitive tasks . |
| Approach: | They propose a task where sentences are given with an entity mention . they introduce a new type of distant supervision: head words, which indicate the type of noun phrases they appear in. |
| Outcome: | The proposed model can predict ultra-fine types at varying granularity and performs well on a fine-grained entity typing benchmark. |
Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking (P18-1)
Copied to clipboard
| Challenge: | Existing methods to incorporate hierarchical information into knowledge bases have yielded little benefit. |
| Approach: | They propose methods to integrate hierarchical information using real bilinear mappings . they also propose two new datasets containing wide and deep hierarchies . |
| Outcome: | The proposed methods improve on flat predictions and fine-grained entity typing on FIGER dataset. |
Improving Knowledge Graph Embedding Using Simple Constraints (P18-1)
Copied to clipboard
| Challenge: | Recent efforts focused on designing more complicated models or incorporating extra information beyond triples. |
| Approach: | They propose to use non-negativity constraints on entity representations and approximate entailment constraints on relation representations to improve KG embedding. |
| Outcome: | The proposed model outperforms baseline models on WordNet, Freebase, and DBpedia. |
Towards Understanding the Geometry of Knowledge Graph Embeddings (P18-1)
Copied to clipboard
| Challenge: | Knowledge Graph (KG) embedding has emerged as a very active area of research over the last few years, resulting in the development of several embeddable methods. |
| Approach: | They propose to use KG embedding methods to represent entities and relations as vectors in a high-dimensional space. |
| Outcome: | The proposed methods represent entities and relations in KGs as vectors in a high-dimensional space. |
A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss (P18-1)
Copied to clipboard
| Challenge: | extractive models can obtain sentence-level attention with high ROUGE scores but less readable. abstractive models generate novel words and phrases not copied from the source text. |
| Approach: | They propose to combine extractive and abstractive models to achieve a unified model that generates readable paragraphs with word-level attention. |
| Outcome: | The proposed model achieves state-of-the-art ROUGE scores while being the most informative and readable summarization on the CNN/Daily Mail dataset in a solid human evaluation. |
Extractive Summarization with SWAP-NET: Sentences and Words from Alternating Pointer Networks (P18-1)
Copied to clipboard
| Challenge: | Abstractive summarization methods use factual and grammatical errors to generate summaries. |
| Approach: | They propose a neural sequence-to-sequence model for extractive summarization called SWAP-NET . it identifies salient sentences and key words in an input document and combines them to form an extractive summary. |
| Outcome: | The proposed model outperforms state-of-the-art extractive summarization methods on large scale corpora. |
Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization (P18-1)
Copied to clipboard
| Challenge: | Existing summarization systems rely on the source text to generate summaries, which tends to work unstably. |
| Approach: | They propose to use existing summaries as soft templates to guide the seq2seq model . they use a popular IR platform to Retrieve proper summary as candidate templates . |
| Outcome: | The proposed model outperforms state-of-the-art models in terms of informativeness and readability. |
Simple and Effective Text Simplification Using Semantic and Neural Methods (P18-1)
Copied to clipboard
| Challenge: | Sentence splitting is a major simplification operation. |
| Approach: | They propose a simple and efficient splitting algorithm based on an automatic semantic parser. |
| Outcome: | The proposed method compares favorably to the state-of-the-art in combined lexical and structural simplification. |
Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words (P18-1)
Copied to clipboard
| Challenge: | Words play a central role in language and thought. |
| Approach: | They propose a Lexicon with ratings of valence, arousal, and dominance for 20,000 words . they use Best–Worst Scaling to obtain fine-grained scores . |
| Outcome: | The proposed Lexicon has human ratings of valence, arousal, and dominance for 20,000 words . the ratings are more reliable than those in existing lexicons, the authors show . |
Comprehensive Supersense Disambiguation of English Prepositions and Possessives (P18-1)
Copied to clipboard
Nathan Schneider, Jena D. Hwang, Vivek Srikumar, Jakob Prange, Austin Blodgett, Sarah R. Moeller, Aviram Stern, Adi Bitan, Omri Abend
| Challenge: | Frequent prepositions like for are maddeningly polysemous, their interpretation depends especially on the object of the preposition. |
| Approach: | They propose a new annotation scheme, corpus, and task for the disambiguation of prepositions and possessives in English. |
| Outcome: | The proposed annotations are comprehensive with respect to types and tokens of these markers and use broadly applicable supersense classes rather than fine-grained dictionary definitions. |
A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature (P18-1)
Copied to clipboard
| Challenge: | In 2015 alone, about 100 manuscripts describing randomized controlled trials for medical interventions were published every day. |
| Approach: | They propose a corpus of 5,000 medical articles annotated with demarcations of text spans that describe the Patient population enrolled, the Interventions studied and to what they were Compared, and the Outcomes measured. |
| Outcome: | The proposed corpus includes 5,000 medical articles describing clinical randomized controlled trials. |
Efficient Online Scalar Annotation with Bounded Support (P18-1)
Copied to clipboard
| Challenge: | Existing methods for efficiently eliciting scalar annotations for dataset construction and system quality estimation by human judgments are not shown. |
| Approach: | They propose a method for efficiently eliciting scalar annotations by human judgments. |
| Outcome: | The proposed method leads to increased correlation with ground truth, suggesting it is an improved mechanism for dataset creation and manual system evaluation. |
Neural Argument Generation Augmented with Externally Retrieved Evidence (P18-1)
Copied to clipboard
| Challenge: | Existing methods for generating arguments are limited to retrieval-based methods. |
| Approach: | They propose an encoder-decoder-based argument generation model enriched with externally retrieved evidence from Wikipedia. |
| Outcome: | The proposed model generates arguments with more topic-relevant content than current models based on automatic evaluation and human assessments on a large-scale dataset from reddit. |
A Stylometric Inquiry into Hyperpartisan and Fake News (P18-1)
Copied to clipboard
| Challenge: | a style analysis of hyperpartisan news and fake news can distinguish them from mainstream news . left-wing and right-wing news share significantly more stylistic similarities than mainstream news does . |
| Approach: | a comparative style analysis of hyperpartisan news and fake news is carried out . authors show that left-wing and right-wing news share significantly more stylistic similarities . |
| Outcome: | a style analysis can distinguish hyperpartisan news from mainstream, satire from both . left-wing and right-wing news share significantly more stylistic similarities than mainstream . |
Retrieval of the Best Counterargument without Prior Topic Knowledge (P18-1)
Copied to clipboard
| Challenge: | ad-hominem attacks are the most common form of argumentation in real life . |
| Approach: | They hypothesize that the best counterargument invokes the same aspects as the argument while having the opposite stance. |
| Outcome: | The proposed model is independent from the topic at hand, i.e., it applies to arbitrary arguments. |
LinkNBed: Multi-Graph Representation Learning with Entity Linkage (P18-1)
Copied to clipboard
| Challenge: | Knowledge graphs have emerged as an important model for studying complex multi-relational data. |
| Approach: | They propose a deep relational learning framework that learns entity and relationship representations across multiple graphs. |
| Outcome: | The proposed framework improves on the state-of-the-art relational learning approaches and identifies entity linkage across graphs. |
Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures (P18-1)
Copied to clipboard
| Challenge: | Structured embeddings based on regions, densities, and orderings have gained popularity for their inductive bias towards the essential asymmetries inherent in problems such as image captioning. |
| Approach: | They propose a box lattice and accompanying probability measure to capture negative correlations over arbitrary concepts. |
| Outcome: | The proposed model can capture anti-correlation and even disjoint concepts while learning from and predicting calibrated uncertainty. |
Graph-to-Sequence Learning using Gated Graph Neural Networks (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to graph-to-sequence learning ignore the full graph structure, discarding key information. |
| Approach: | They propose a graph-to-sequence learning model that encodes the full graph structure and an input transformation that allows nodes and edges to have their own hidden representations. |
| Outcome: | The proposed model outperforms baselines in generation from AMR graphs and syntax-based neural machine translation while retaining the full graph structure. |
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context (P18-1)
Copied to clipboard
| Challenge: | Recent studies have shed light on the information encoded by long-term memory networks. |
| Approach: | They propose to use a neural caching model to model the role of context in an LSTM LM . they analyze the increase in perplexity when prior context words are shuffled, replaced, or dropped . |
| Outcome: | The proposed model is highly sensitive to the order of words within the most recent sentence, but ignores word order in the long-range context, suggesting the distant past is modeled only as a rough semantic field or topic. |
Bridging CNNs, RNNs, and Weighted Finite-State Machines (P18-1)
Copied to clipboard
| Challenge: | recurrent and convolutional neural networks are useful for encoding natural language utterances. |
| Approach: | They propose a model that combines neural representation learning with weighted finite-state automatas to learn a soft version of traditional surface patterns. |
| Outcome: | The proposed model is comparable or better than a BiLSTM baseline and a CNN baseline on three text classification tasks. |
Zero-shot Learning of Classifiers from Natural Language Quantification (P18-1)
Copied to clipboard
| Challenge: | Existing methods to learn concepts from natural language are limited or no labeled examples. |
| Approach: | They propose a framework through which a set of explanations of a concept can be used to learn a classifier without access to any labeled examples. |
| Outcome: | The proposed framework outperforms previous approaches for learning with limited data and is comparable with fully supervised classifiers trained from a small number of labeled examples. |
Sentence-State LSTM for Text Representation (P18-1)
Copied to clipboard
| Challenge: | LSTMs have been shown to suffer from various limitations due to their sequential nature. |
| Approach: | They propose to model hidden states of all words simultaneously at each recurrent step rather than one word at a time. |
| Outcome: | The proposed model has strong representation power, giving competitive performances compared to stacked BiLSTM models with similar parameter numbers. |
Universal Language Model Fine-tuning for Text Classification (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to computer vision require task-specific modifications and training from scratch. |
| Approach: | They propose a method that can be applied to any task in NLP and propose to open-source it. |
| Outcome: | The proposed method outperforms the state-of-the-art on six text classification tasks, reducing error by 18-24% on majority of datasets. |
Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement (P18-1)
Copied to clipboard
| Challenge: | a number of post hoc explanation methods for deep neural networks have been proposed . due to the complexity of the DNNs they explain, these methods are necessarily approximations and come with their own sources of error. |
| Approach: | They propose two evaluation paradigms that cover two important classes of NLP problems . they propose LIMSSE, LRP and DeepLIFT as the most effective explanation methods . |
| Outcome: | The proposed methods are most effective for explaining deep neural networks in NLP . the proposed methods can explain complex models without manual annotation . |
Improving Text-to-SQL Evaluation Methodology (P18-1)
Copied to clipboard
Catherine Finegan-Dollak, Jonathan K. Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, Dragomir Radev
| Challenge: | Current evaluations of text-to-SQL systems are limited by the way they divide data into training and test sets. |
| Approach: | They propose to standardize and improve existing and new text-to-SQL datasets . they propose a template-based slot-filling baseline that cannot generalize to new queries . |
| Outcome: | The proposed system is competitive with prior work on multiple datasets and can be used on training and test sets. |
Semantic Parsing with Syntax- and Table-Aware SQL Generation (P18-1)
Copied to clipboard
Yibo Sun, Duyu Tang, Nan Duan, Jianshu Ji, Guihong Cao, Xiaocheng Feng, Bing Qin, Ting Liu, Ming Zhou
| Challenge: | Existing approaches generate a SQL query word-by-word but results are incorrect or not executable due to mismatch between question words and table contents. |
| Approach: | They propose a generative model to map natural language questions into SQL queries. |
| Outcome: | The proposed model significantly improves state-of-the-art execution accuracy from 69.0% to 74.4% on a large question- SQL dataset. |
Multitask Parsing Across Semantic Representations (P18-1)
Copied to clipboard
| Challenge: | UCCA parsing is a test case for multitask learning, with auxiliary tasks AMR, SDP and Universal Dependencies (UD) . Semantic parsers have arguably yet to reach their full potential due to the limited amount of semantically annotated training data. |
| Approach: | They propose a general transition-based parser that can parse UCCA, AMR, SDP and Universal Dependencies (UD) they use a transition-driven learning architecture and a uniform transition-basic learning architecture to train the parsers. |
| Outcome: | The proposed parser improves UCCA, AMR, SDP and Universal Dependencies (UD) parsing over training in English, German and French. |
Character-Level Models versus Morphology in Semantic Role Labeling (P18-1)
Copied to clipboard
| Challenge: | Character-level models are used for high-level semantic analysis tasks such as semantic role labeling. |
| Approach: | They train character-level models that use word, character and morphology level information . they analyze how performance of characters compare to words and a variety of morphological typologies . |
| Outcome: | The results shed light on important characteristics of character-level models and their semantic capability. |
AMR Parsing as Graph Prediction with Latent Alignment (P18-1)
Copied to clipboard
| Challenge: | Abstract meaning representations (AMRs) are sentence-level semantic representations . lack of explicit alignments between nodes in graphs and words in sentences is a challenge . |
| Approach: | They propose a neural parser which treats alignments as latent variables within a joint probabilistic model of concepts, relations and alignments. |
| Outcome: | The proposed parser achieves the best reported results on the standard benchmark (74.4% on LDC2016E25). |
Accurate SHRG-Based Semantic Parsing (P18-1)
Copied to clipboard
| Challenge: | Graph-structured semantic representations can encode rich semantic information of natural language sentences. |
| Approach: | They propose a SHRG-based parser that relates synchronous production rules to syntacto-semantic composition processes. |
| Outcome: | The proposed model improves on the best existing model by 4.87 points . it relates synchronous production rules to syntacto-semantic composition process . |
Using Intermediate Representations to Solve Math Word Problems (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to solving math word problems do not include higher-order operations that cannot be explicitly represented in equations. |
| Approach: | They propose an iterative labeling framework that generates intermediate forms and executes them to obtain the final answers. |
| Outcome: | The proposed model outperforms existing models in solving math word problems. |
Discourse Representation Structure Parsing (P18-1)
Copied to clipboard
| Challenge: | Existing semantic parsers are data-driven using annotated examples consisting of utterances and their meaning representations. |
| Approach: | They propose a method which transforms Discourse Representation Structures (DRSs) to trees and develop a structure-aware model which decomposes the decoding process into three stages. |
| Outcome: | The proposed model outperforms baseline models on the Groningen Meaning Bank (GMB) by a wide margin. |
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms (P18-1)
Copied to clipboard
Dinghan Shen, Guoyin Wang, Wenlin Wang, Martin Renqiang Min, Qinliang Su, Yizhe Zhang, Chunyuan Li, Ricardo Henao, Lawrence Carin
| Challenge: | Existing deep learning architectures to model compositionality in text sequences require a large number of parameters and expensive computations. |
| Approach: | They propose two additional pooling strategies over word embeddings for improved interpretability and hierarchical pooling for spatial (n-gram) information within text sequences. |
| Outcome: | The proposed pooling strategies improve interpretability and preserve spatial (n-gram) information within text sequences. |
ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (P18-1)
Copied to clipboard
| Challenge: | Using neural machine translation, we generate more than 50 million sentential paraphrase pairs from a large parallel corpus. |
| Approach: | They use a dataset of more than 50 million English-English sentential paraphrase pairs to generate them automatically using neural machine translation. |
| Outcome: | The proposed dataset outperforms all supervised systems on every SemEval semantic textual similarity competition and shows how it can be used for paraphrase generation. |
Event2Mind: Commonsense Inference on Events, Intents, and Reactions (P18-1)
Copied to clipboard
| Challenge: | Using a crowdsourced corpus of 25,000 event phrases, we construct a new task that uses commonsense reasoning to reason about the likely intents and reactions of the event participants. |
| Approach: | They construct a crowdsourced corpus of 25,000 event phrases and use them to construct 'commonsense inference' they demonstrate that neural encoder-decoder models can compose embedding representations of previously unseen events and reason about the likely intents and reactions of the event participants. |
| Outcome: | The proposed task can be used to uncover implicit gender inequality in movie scripts. |
Neural Adversarial Training for Semi-supervised Japanese Predicate-argument Structure Analysis (P18-1)
Copied to clipboard
| Challenge: | Japanese predicate-argument structure analysis involves zero anaphora resolution . state-of-the-art models for PAS analysis achieve an accuracy of around 50% for zero pronouns . |
| Approach: | They propose a Japanese PAS analysis model based on semi-supervised adversarial training with a raw corpus. |
| Outcome: | The proposed model outperforms existing models for Japanese PAS analysis . the model is based on semi-supervised adversarial training with a raw corpus . |
Improving Event Coreference Resolution by Modeling Correlations between Event Coreference Chains and Document Topic Structures (P18-1)
Copied to clipboard
| Challenge: | a novel approach for event coreference resolution models correlations between event chains and document topical structures. |
| Approach: | They propose a novel approach that models correlations between event coreference chains and document topical structures through an Integer Linear Programming formulation. |
| Outcome: | The proposed approach improves performance across a dataset of document topics . it shows that the models can identify and link event mentions that refer to the same event . |
DSGAN: Generative Adversarial Training for Distant Supervision Relation Extraction (P18-1)
Copied to clipboard
| Challenge: | Distant supervision can effectively label data for relation extraction, but suffers from the noise labeling problem. |
| Approach: | They propose a sentence-level true-positive generator to learn a true-negative generator from a fuzzy sentence bag. |
| Outcome: | The proposed method significantly improves the performance of distant supervision relation extraction compared to state-of-the-art systems. |
Extracting Relational Facts by an End-to-End Neural Model with Copy Mechanism (P18-1)
Copied to clipboard
| Challenge: | Existing methods focus on normal class and fail to extract relational triplets precisely. |
| Approach: | They propose an end-to-end model which can jointly extract relational triplets from sentences . they employ two different strategies in decoding process: employing only one united decoder or applying multiple separated decodeurs. |
| Outcome: | The proposed model outperforms the baseline method significantly in two datasets. |
Self-regulation: Employing a Generative Adversarial Network to Improve Event Detection (P18-1)
Copied to clipboard
| Challenge: | Recent studies show that neural networks can be used for event detection but can be contaminated by spurious features. |
| Approach: | They propose a self-regulated learning approach by utilizing a generative adversarial network to generate spurious features. |
| Outcome: | The proposed method is highly effective and adaptable on the ACE 2005 and TAC-KBP 2015 corpora. |
Context-Aware Neural Model for Temporal Information Extraction (P18-1)
Copied to clipboard
| Challenge: | Existing temporal information extraction systems rely on statistical learning with feature-engineered task-specific models. |
| Approach: | They propose a context-aware neural network model for temporal information extraction using a global context layer. |
| Outcome: | The proposed model outperforms existing models in terms of performance and performance . it is the first model to use NTM-like architecture to process the information from global context in discourse-scale natural text processing. |
Temporal Event Knowledge Acquisition via Identifying Narratives (P18-1)
Copied to clipboard
| Challenge: | Existing knowledge of narrative examples is lacking and difficult to obtain. |
| Approach: | They propose a weakly supervised approach for acquiring rich temporal event knowledge across sentences in narrative stories. |
| Outcome: | The proposed approach outperforms neural network models on the narrative cloze task. |
Textual Deconvolution Saliency (TDS) : a deep tool box for linguistic analysis (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to text analysis make no assumptions about linguistic structure and focus on stastically frequent patterns. |
| Approach: | They propose a new strategy to visualize linguistic information detected by a CNN for text classification. |
| Outcome: | The proposed strategy automatically encodes complex linguistic patterns on three different languages for each dataset. |
Coherence Modeling of Asynchronous Conversations: A Neural Entity Grid Approach (P18-1)
Copied to clipboard
| Challenge: | Existing coherence models are not able to distinguish coherent discourses from incoherent ones. |
| Approach: | They propose a novel coherence model for written asynchronous conversations . they propose to lexicalize the model's entity transitions and extend it to asynchron conversations based on conversational structure . |
| Outcome: | The proposed model outperforms existing models on coherence assessment and thread reconstruction tasks. |
Deep Reinforcement Learning for Chinese Zero Pronoun Resolution (P18-1)
Copied to clipboard
| Challenge: | Recent models for zero pronoun resolution in Chinese are short-sighted and do not capture semantic information for zeros and candidate antecedents. |
| Approach: | They propose to integrate a deep reinforcement learning approach to Chinese zero pronoun resolution. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods in three experimental settings. |
Entity-Centric Joint Modeling of Japanese Coreference Resolution and Predicate Argument Structure Analysis (P18-1)
Copied to clipboard
| Challenge: | Existing methods for predicate argument structure analysis are difficult and difficult . a Japanese model can detect a zero pronoun and identify a referent of the zero pronominator . |
| Approach: | They propose a model that performs coreference resolution and predicate argument structure analysis simultaneously. |
| Outcome: | The proposed model can improve the performance of the inter-sentential zero anaphora resolution drastically. |
Constraining MGbank: Agreement, L-Selection and Supertagging in Minimalist Grammars (P18-1)
Copied to clipboard
| Challenge: | a deep grammatical formalism that has not been applied to NLP tasks is the Minimalist Grammar (MG) formalism. |
| Approach: | They propose to extend the Minimalist Grammar (MG) formalism with a mechanism for enforcing fine-grained selectional restrictions and agreements. |
| Outcome: | The proposed system is compatible with Markovian supertaggers and enables efficient parsing on key dependency types. |
Not that much power: Linguistic alignment is influenced more by low-level linguistic features rather than social power (P18-1)
Copied to clipboard
| Challenge: | linguistic alignment between interlocutors of higher power is attributed to their relative social power, but studies on low-level linguistic features do not account for these factors. |
| Approach: | They characterize the effect of power on alignment with logistic regression models in two datasets and find it vanishes after controlling for low-level features such as utterance length. |
| Outcome: | The proposed model shows that the effect vanishes or is reversed after controlling for low-level features such as utterance length. |
TutorialBank: A Manually-Collected Corpus for Prerequisite Chains, Survey Extraction and Resource Recommendation (P18-1)
Copied to clipboard
Alexander Fabbri, Irene Li, Prawat Trairatvorakul, Yijiao He, Weitai Ting, Robert Tung, Caitlin Westerfield, Dragomir Radev
| Challenge: | TutorialBank is a publicly available dataset that aims to facilitate NLP education and research . a google search of "Natural Language Processing" returns over 100 million hits with papers, tutorials, 1 http://aan.how blog posts, codebases and other related online resources. |
| Approach: | They have manually collected and categorized over 5,600 resources on NLP . they have created a search engine and command-line tool to search the corpus . |
| Outcome: | The tutorial bank dataset is the largest manually-picked corpus of resources intended for NLP education . it includes lists of research topics, relevant resources for each topic, prerequisite relations among topics . |
Give Me More Feedback: Annotating Argument Persuasiveness and Related Attributes in Student Essays (P18-1)
Copied to clipboard
| Challenge: | Existing work on automated essay scoring has focused on holistic scoring, which summarizes the quality of an essay with a single score. |
| Approach: | They present a corpus of essays simultaneously annotated with argument components, argument persuasiveness scores, and attributes of argument components that impact an argument’s persuasiveness. |
| Outcome: | The proposed corpus could trigger the development of novel computational models that provide useful feedback to students on why their arguments are (un)persuasive . |
Inherent Biases in Reference-based Evaluation for Grammatical Error Correction (P18-1)
Copied to clipboard
| Challenge: | Existing evaluation systems obtain comparable or superior performance compared to humans by making few but targeted changes to the input. |
| Approach: | They propose to re-scale M 2 by the inter-annotator agreement and increase the number of references in any feasible range to overcome low coverage bias in GEC evaluation. |
| Outcome: | The proposed measure overcomes low coverage bias in GEC evaluation by re-scaling or increasing the number of references in any feasible range. |
The price of debiasing automatic metrics in natural language evalaution (P18-1)
Copied to clipboard
| Challenge: | Existing methods to evaluate natural language systems are expensive and expensive. |
| Approach: | They propose to combine automatic metrics with human judgment to obtain an unbiased estimator at lower cost than human evaluation alone. |
| Outcome: | The proposed estimator reduces the cost of evaluating summarization and open-response questions by 7-13%. |
Neural Document Summarization by Jointly Learning to Score and Select Sentences (P18-1)
Copied to clipboard
| Challenge: | Sentence scoring and sentence selection are two main steps in extractive document summarization systems. |
| Approach: | They propose an end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences. |
| Outcome: | The proposed framework outperforms the state-of-the-art summarization models on the CNN/Daily Mail dataset. |
Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization (P18-1)
Copied to clipboard
Guokan Shang, Wensi Ding, Zekun Zhang, Antoine Tixier, Polykarpos Meladianos, Michalis Vazirgiannis, Jean-Pierre Lorré
| Challenge: | a novel graph-based framework for abstractive meeting speech summarization is developed . instead of grammatical, well-segmented sentences, the input is made of often ill-formed and ungrammatically ungrammatized text fragments called utterances. |
| Approach: | They propose a graph-based framework for abstractive meeting speech summarization that is fully unsupervised and does not rely on annotations. |
| Outcome: | The proposed framework improves on the state-of-the-art on the AMI and ICSI corpus. |
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (P18-1)
Copied to clipboard
| Challenge: | Empirically, we achieve the new state-of-the-art on all metrics (including human evaluation) on the CNN/Daily Mail dataset, as well as significantly higher abstractiveness scores. |
| Approach: | They propose a sentence-level policy gradient method that bridges computation between two neural networks in a hierarchical way while maintaining language fluency. |
| Outcome: | The proposed model achieves state-of-the-art on all metrics and higher abstractiveness scores on the CNN/Daily Mail dataset and faster training convergence than previous models. |
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation (P18-1)
Copied to clipboard
| Challenge: | Recent advances on abstractive summarization have allowed substantial improvements in the quality of the model, but there is still scope for improvement. |
| Approach: | They propose novel multi-task architectures with high-level layer-specific sharing across multiple encoder and decoder layers of the three tasks and soft-sharing mechanisms. |
| Outcome: | The proposed model improves on the CNN/DailyMail and Gigaword datasets and on the DUC-2002 transfer setup. |
Modeling and Prediction of Online Product Review Helpfulness: A Survey (P18-1)
Copied to clipboard
| Challenge: | review helpfulness modeling is a task that studies the mechanisms that affect review helpfuliness and attempts to accurately predict it. |
| Approach: | This paper provides an overview of the most relevant work in helpfulness prediction . it discusses the insights gained from said work and provides guidelines for future research . |
| Outcome: | This paper summarizes the most relevant work in helpfulness prediction and understanding in the past decade . it outlines the insights gained from the results and provides guidelines for future research . |
Mining Cross-Cultural Differences and Similarities in Social Media (P18-1)
Copied to clipboard
| Challenge: | a new paper examines the problem of computing cross-cultural differences and similarities in natural language understanding . cross-culture differences are important for cross-lingual research, especially in social media . |
| Approach: | They propose a framework for computing cross-cultural differences and similarities from social media . they propose to use a social media platform to find similar terms for slang across languages . |
| Outcome: | The proposed framework outperforms baseline methods on two novel tasks. |
Classification of Moral Foundations in Microblog Political Discourse (P18-1)
Copied to clipboard
| Challenge: | a recent study shows correlation between political ideologies and moral foundations expressed in text . a moral foundation theory suggests that there are five basic moral values which underlie human moral perspectives . |
| Approach: | They propose to model the moral foundations of tweets by using an annotation framework . they propose to use policy frames to predict the morality of political tweets . |
| Outcome: | The proposed model can predict moral foundations of political tweets, the authors show . their model can be used to predict political slogans and political ideologies, they say . |
Coarse-to-Fine Decoding for Neural Semantic Parsing (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that semantic parsing is more efficient than using simple decoders. |
| Approach: | They propose a structure-aware neural architecture which decomposes the semantic parsing process into two stages. |
| Outcome: | The proposed architecture consistently improves performance on four datasets characteristic of different domains and meaning representations. |
Confidence Modeling for Neural Semantic Parsing (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that neural semantic parsers are difficult to interpret due to their complexity. |
| Approach: | They propose to use confidence models to estimate predictions for neural semantic parsers . they outline three major causes of uncertainty and use metrics to quantify them . |
| Outcome: | The proposed model outperforms a widely used method that relies on posterior probability and improves interpretation quality. |
StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsing (P18-1)
Copied to clipboard
| Challenge: | Semantic parsing is the task of transducing natural language (NL) utterances into formal meaning representations (MRs), commonly represented as tree structures. |
| Approach: | They propose a variational auto-encoding model for semi-supervised semantic parsing which learns from limited amounts of parallel data and readily-available unlabeled NL utterances. |
| Outcome: | Experiments on ATIS domain and Python show that with extra unlabeled data, StructVAE outperforms strong supervised models. |
Sequence-to-Action: End-to-End Semantic Graph Generation for Semantic Parsing (P18-1)
Copied to clipboard
| Challenge: | Existing methods for semantic parsing are difficult to design and learn, especially in wideopen domains. |
| Approach: | They propose a neural semantic parsing approach which models semantic par- sing as an end-to-end semantic graph generation process. |
| Outcome: | The proposed model achieves state-of-the-art performance on Overnight dataset and gets competitive performance on Geo and Atis datasets. |
On the Limitations of Unsupervised Bilingual Dictionary Induction (P18-1)
Copied to clipboard
| Challenge: | Unsupervised machine translation does not require cross-lingual supervision, whether a dictionary, translations, or comparable corpora. |
| Approach: | They propose an adversarial, unsupervised cross-lingual word embedding technique for bilingual dictionary induction that exploits a weak supervision signal from identical words. |
| Outcome: | The proposed model relies heavily on an adversarial, unsupervised cross-lingual word embedding technique for bilingual dictionary induction. |
A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing methods to learn cross-lingual word embeddings have failed in more realistic scenarios . a fully unsupervised initialization and a robust self-learning algorithm are needed to improve the existing methods. |
| Approach: | They propose an unsupervised initialization method that exploits structural similarity of embeddings and a robust self-learning algorithm that iteratively improves it. |
| Outcome: | The proposed method achieves the best published results in standard datasets even surpassing previous supervised systems. |
A Multi-lingual Multi-task Architecture for Low-resource Sequence Labeling (P18-1)
Copied to clipboard
| Challenge: | Existing studies have shown that multi-task learning can boost the performance of related tasks such as MT and abstractive text summarization. |
| Approach: | They propose a multi-lingual multi-task architecture to develop supervised models with a minimal amount of labeled data for sequence labeling. |
| Outcome: | The proposed architecture achieves 4.3%-50.5% absolute gains compared to mono-lingual model . the proposed model is particularly effective in low-resource settings . |
Two Methods for Domain Adaptation of Bilingual Tasks: Delightfully Simple and Broadly Applicable (P18-1)
Copied to clipboard
| Challenge: | Previously, domain adaptation approaches to bilingual tasks were proposed . we show that simple adaptation process involving only unlabeled text is highly effective . |
| Approach: | They propose a method for domain adaptation of bilingual word embeddings using unlabeled data . they then tailor a semi-supervised classification method from computer vision to these tasks . |
| Outcome: | The proposed method improves on two bilingual tasks using unlabeled data. |
Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge (P18-1)
Copied to clipboard
| Challenge: | a new model for reading comprehension integrates external commonsense knowledge . cloze-style reading comprehension is a language understanding task similar to question answering . |
| Approach: | They propose a reading comprehension model that integrates external commonsense knowledge in a cloze-style setting. |
| Outcome: | The proposed model improves results over a very strong baseline on a hard Common Nouns dataset, making it a strong competitor of more complex models. |
Multi-Relational Question Answering from Narratives: Machine Reading and Reasoning in Simulated Worlds (P18-1)
Copied to clipboard
| Challenge: | Question Answering (QA) has primarily focused on knowledge bases or free text as a source of knowledge. |
| Approach: | They propose a task of multi-relational QA over personal narrative using text worlds . they generate and release a lightweight Python-based framework for easily generating additional worlds and narrative . |
| Outcome: | The proposed framework combines elements of structured QA over knowledge bases and unstructured QA . it generates and analyzes five diverse datasets with dynamic narrative . the framework is lightweight and easy to use . |
Simple and Effective Multi-Paragraph Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Existing question answering models cannot scale beyond short paragraphs, so adapting a model to document-level input is difficult. |
| Approach: | They propose a method of adapting neural paragraph-level question answering models to document input. |
| Outcome: | The proposed method achieves state-of-the-art on TriviaQA and SQuAD and a 10 point gain on SQuADA. |
Semantically Equivalent Adversarial Rules for Debugging NLP models (P18-1)
Copied to clipboard
| Challenge: | Complex machine learning models are often brittle, making different predictions for input instances that are extremely similar semantically. |
| Approach: | They propose to generalize semantically equivalent adversarial rules that induce adversaries on many instances to detect brittle models. |
| Outcome: | The proposed rules generate high-quality local adversaries for more instances than humans and induce four times as many mistakes as human experts. |
Style Transfer Through Back-Translation (P18-1)
Copied to clipboard
| Challenge: | a new method for automatic style transfer is proposed to preserve the meaning of the text while reducing stylistic properties. |
| Approach: | They propose a method for automatic style transfer that uses latent representations of the input sentence to preserve meaning while reducing stylistic properties. |
| Outcome: | The proposed method improves on sentiment, gender and political slant styles on three different styles. |
Generating Fine-Grained Open Vocabulary Entity Type Descriptions (P18-1)
Copied to clipboard
| Challenge: | Fig. 1 shows an example of a concise entity description presented to a user. |
| Approach: | They propose a dynamic memory-based network that generates a short open vocabulary description of an entity by leveraging induced fact embeddings and dynamic context. |
| Outcome: | The proposed network generates a short open vocabulary description of an entity . it can discern relevant information for more accurate generation of type description . |
Hierarchical Neural Story Generation (P18-1)
Copied to clipboard
| Challenge: | a hierarchical model that generates a premise and then conditions on it creates fluent text . a novel form of model fusion improves the relevance of the story to the prompt . |
| Approach: | They use a hierarchical model that first generates a premise, then transforms it into a text . they use fusion to improve relevance of the story to the prompt and add a gated mechanism to model context . |
| Outcome: | The proposed model improves on strong baselines on automated and human evaluations. |
No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling (P18-1)
Copied to clipboard
| Challenge: | Visual captioning is aimed at depicting the concrete content of images, but its capability of performing human-like understanding is still restrictive. |
| Approach: | They propose an Adversarial REward Learning framework to learn an implicit reward function from human demonstrations and optimize policy search with the learned reward function. |
| Outcome: | The proposed framework improves performance over state-of-the-art (SOTA) methods in cloning expert behaviors, but human evaluation shows that it achieves significant improvement in generating more human-like stories than SOTA systems. |
Bridging Languages through Images with Deep Partial Canonical Correlation Analysis (P18-1)
Copied to clipboard
| Challenge: | a deep neural network can be used to improve bilingual text embeddings . a novel approach is proposed to optimize text embed-ings on shared visual information . |
| Approach: | They propose a deep neural network that leverages images to improve bilingual text embeddings. |
| Outcome: | The proposed model outperforms previous methods on word similarity and cross-lingual image description retrieval. |
Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search (P18-1)
Copied to clipboard
| Challenge: | a large-scale lookup operation to ground language via ‘snapshots’ of our physical world accessed through image search is currently used to learn word representations. |
| Approach: | They propose a large-scale lookup operation to ground language via ‘snapshots’ of our physical world accessed through image search. |
| Outcome: | The proposed model is based on a large-scale lookup operation to ground language using image search. |
What Action Causes This? Towards Naive Physical Action-Effect Prediction (P18-1)
Copied to clipboard
| Challenge: | a new task on naive physical action-effect prediction addresses the relationship between concrete actions and their effects on the state of the physical world as depicted by images. |
| Approach: | They propose a task that harnesses web image data to facilitate action-effect prediction. |
| Outcome: | The proposed approach harnesses web image data through distant supervision to facilitate learning for action-effect prediction. |
Transformation Networks for Target-Oriented Sentiment Classification (P18-1)
Copied to clipboard
| Challenge: | a new model for sentiment classification uses attention instead of attention to classify sentiment polarities over individual opinion targets. |
| Approach: | They propose a model that uses a CNN layer to extract salient features from transformed word representations from a bi-directional RNN layer. |
| Outcome: | The proposed model achieves state-of-the-art on a few benchmarks. |
Target-Sensitive Memory Networks for Aspect Sentiment Classification (P18-1)
Copied to clipboard
| Challenge: | Aspect sentiment classification (ASC) is a fundamental task in sentiment analysis. |
| Approach: | They propose to use memory networks to deal with ASC using aspect and sentence terms and use them to classify the sentiment polarity. |
| Outcome: | The proposed techniques can be implemented in a variety of contexts and their effectiveness is evaluated. |
Identifying Transferable Information Across Domains for Cross-domain Sentiment Classification (P18-1)
Copied to clipboard
| Challenge: | Cross-domain sentiment classification is challenging due to polarity orientation and significance differences . supervised learning algorithms have to be re-trained on every new domain . |
| Approach: | They propose that words that do not change their polarity and significance represent transferable information across domains for cross-domain sentiment classification. |
| Outcome: | The proposed method improves cross-domain sentiment classification performance by identifying polarity-preserving significant words across domains. |
Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach (P18-1)
Copied to clipboard
| Challenge: | Existing studies for sentiment-to-sentiment "translation" only change the underlying sentiment and fail to keep the semantic content. |
| Approach: | They propose a cycled reinforcement learning method that combines neutralization module and emotionalization module. |
| Outcome: | The proposed method outperforms state-of-the-art systems on Yelp and Amazon review datasets. |
Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to natural language inference focus on interaction architectures of sentences . but, we propose to transfer knowledge from discourse markers to augment the model . |
| Approach: | They propose to transfer knowledge from discourse markers to augment the quality of the NLI model. |
| Outcome: | The proposed method achieves state-of-the-art performance on large-scale datasets. |
Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Module (P18-1)
Copied to clipboard
| Challenge: | Recent advances in deep neural networks have enabled complex reasoning tasks. |
| Approach: | They propose a MemNN architecture with a working memory storage and reasoning module that retains relational reasoning abilities of relation networks while reducing computational complexity. |
| Outcome: | The proposed model retains the relational reasoning abilities of the RN while reducing its computational complexity from quadratic to linear. |
Reasoning with Sarcasm by Reading In-Between (P18-1)
Copied to clipboard
| Challenge: | Sarcasm is a figurative speech act which manifests on social networks such as Twitter and Reddit. |
| Approach: | They propose a model that looks in-between rather than across to explicitly model contrast and incongruity. |
| Outcome: | The proposed model achieves state-of-the-art performance on all datasets and improves interpretability. |
Adversarial Contrastive Estimation (P18-1)
Copied to clipboard
| Challenge: | Noise contrastive estimation (NCE) is a general strategy used in word embeddings and translations for knowledge graphs. |
| Approach: | They propose to augment negative sampler into mixture distribution with adversarially learned sampler and to combine it with noise contrastive estimation (NCE) they observe faster convergence and improved results on multiple metrics. |
| Outcome: | The proposed model performs better on word embeddings, order embedds and knowledge graph embeddments and faster convergence and improved results on multiple metrics. |
Adaptive Scaling for Sparse Detection in Information Extraction (P18-1)
Copied to clipboard
| Challenge: | Detection problems involving positive instances are often deficient in information extraction tasks . a number of researches have employed neural network models to solve detection problems . |
| Approach: | They propose an algorithm which can handle positive sparsity problem and directly optimize over F-measure . they borrow the idea of marginal utility from economics and propose a theoretical framework for instance importance measuring . |
| Outcome: | The proposed algorithm improves on positive sparsity problem and over F-measure . it leads to more effective and stable training of neural network based detection models. |
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (P18-1)
Copied to clipboard
| Challenge: | Existing neural models for learning under domain shifts only evaluate on a single task, on proprietary datasets, or compare to weak baselines. |
| Approach: | They propose a multi-task tri-training method that reduces time and space complexity of classic bootstrapping approaches. |
| Outcome: | The proposed method outperforms the state-of-the-art for sentiment analysis on two benchmarks. |
Fluency Boost Learning and Inference for Neural Grammatical Error Correction (P18-1)
Copied to clipboard
| Challenge: | Seq2seq models for grammatical error correction (GEC) have two limitations: (1) a seq2q model may not be well generalized with only limited error-corrected data; (2) a model may fail to completely correct a sentence with multiple errors through normal seq1sequeq inference. |
| Approach: | They propose a fluency boost learning and inference mechanism to improve the performance of seq2seq models for grammatical error correction (GEC) by generating fluency-boost sentence pairs during training. |
| Outcome: | Experiments show that the proposed model improves on both CoNLL-2014 and JFLEG benchmark datasets. |
A Neural Architecture for Automated ICD Coding (P18-1)
Copied to clipboard
| Challenge: | Medical coding is time-consuming, expensive, and error prone. |
| Approach: | They propose to use diagnosis descriptions (DDs) of a patient as inputs to select the most relevant ICD codes. |
| Outcome: | The proposed algorithms perform on a clinical dataset with 59K patient visits. |
Domain Adaptation with Adversarial Training and Graph Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing models for deep neural networks can handle data distributions between source and target domains, but they must deal with data distribution drifts. |
| Approach: | They propose a model that leverages unlabeled and labeled data from a related domain to deal with distribution drifts. |
| Outcome: | The proposed model improves over baselines on two real-world disaster datasets. |
TDNN: A Two-stage Deep Neural Network for Prompt-independent Automated Essay Scoring (P18-1)
Copied to clipboard
| Challenge: | Existing automated essay scoring (AES) models rely on rated essays for the target prompt as training data. |
| Approach: | They propose a shallow deep neural network to learn a prompt-dependent rating model using rated essays for non-target prompts as training data. |
| Outcome: | The proposed model improves on the standard ASAP dataset. |
Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation (P18-1)
Copied to clipboard
| Challenge: | Existing encoder-decoder dialog models cannot output interpretable actions as in traditional systems. |
| Approach: | They propose an unsupervised discrete sentence representation learning method that integrates with existing encoder-decoder dialog models for interpretable response generation. |
| Outcome: | The proposed model can be integrated with existing encoder-decoder dialog models and discover interpretable semantics via either auto encoding or context predicting. |
Learning to Control the Specificity in Neural Response Generation (P18-1)
Copied to clipboard
| Challenge: | Existing generative conversational models tend to favor general and trivial responses which appear frequently. |
| Approach: | They propose a controlled response generation mechanism to handle different utterance-response relationships in terms of specificity. |
| Outcome: | The proposed model outperforms state-of-the-art models under automatic and human evaluations. |
Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network (P18-1)
Copied to clipboard
| Challenge: | Existing models for matching dialogue responses rely on semantic and functional dependencies . a recent study only uses the last utterance in context for matching a reply . |
| Approach: | They propose a model that matches a response with its multi-turn context using attention. |
| Outcome: | The proposed model outperforms the state-of-the-art models on two large-scale multi-turn response selection tasks. |
MojiTalk: Generating Emotional Responses at Scale (P18-1)
Copied to clipboard
| Challenge: | Existing studies on emotion-generating systems focus on small sets of labeled datasets. |
| Approach: | They propose to leverage Twitter data that are naturally labeled with emojis to generate emotional responses. |
| Outcome: | The proposed models can generate high-quality conversation responses in accordance with designated emotions. |
Taylor’s law for Human Linguistic Sequences (P18-1)
Copied to clipboard
| Challenge: | Taylor's law characterizes how the variance of the number of events for a given time and space grows with respect to the mean, forming a power law. |
| Approach: | They propose a method to quantify Taylor's law in natural language and conduct Taylor analysis of over 1100 texts across 14 languages. |
| Outcome: | The proposed method is able to quantify the complexity of linguistic time series and evaluate language models. |
A Framework for Representing Language Acquisition in a Population Setting (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to model language acquisition and social structure are ineffective because nobody can travel back in time or fit entire natural environments into a lab. |
| Approach: | They propose a new analytic framework which combines previous network models' ability to capture realistic social structure with more elegant computational properties. |
| Outcome: | The proposed framework is able to capture real social structure and integrate with existing models while being modular and extensible. |
Prefix Lexicalization of Synchronous CFGs using Synchronous TAG (P18-1)
Copied to clipboard
| Challenge: | epsilon-free, chain-free synchronous context-free grammars can be converted into weakly equivalent synchronous tree-adjoining grammars (STAGs) this transformation doubles the grammar’s rank and cubes its size, but in practice the size increase is only quadratic. |
| Approach: | They extend Greibach normal form from CFGs to SCFGs and prove new formal properties about SCFG, a formalism with many applications in natural language processing. |
| Outcome: | The proposed grammars achieve asymptotic and empirical speed improvements on a machine translation task. |
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance (P18-1)
Copied to clipboard
| Challenge: | Compared to traditional shift-reduce parsing schemes, our approach is free from the potentially disastrous compounding error. |
| Approach: | They propose a model that predicts a scalar for each split position in a sentence and then determines the topology of grammar tree based on syntactic distances. |
| Outcome: | The proposed model achieves the state-of-the-art single model F1 score of 92.1 on PTB and 86.4 on CTB dataset, surpassing the previous single model results by a large margin. |
Gaussian Mixture Latent Vector Grammars (P18-1)
Copied to clipboard
| Challenge: | Existing models of latent variable grammars are not observable in treebanks, so latent variables are learned using expectation-maximization. |
| Approach: | They propose a new framework that extends latent variable grammars such that each nonterminal symbol is associated with a continuous vector space representing the set of (infinitely many) subtypes of the nonterminals. |
| Outcome: | The proposed framework can achieve competitive accuracies in part-of-speech tagging and constituency parsing. |
Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples (P18-1)
Copied to clipboard
| Challenge: | Statistical parsers are often criticized for their performance outside of the domain they were trained on . we show that word representations reduce the need for domain adaptation when the target domain is syntactically similar to the source domain. |
| Approach: | They propose a way to adapt a parser to a syntactically similar target domain using partial annotations. |
| Outcome: | The proposed model increases the accuracy of a parser on the Wall Street Journal by 1.7% over the previous state-of-the-art model. |
Paraphrase to Explicate: Revealing Implicit Noun-Compound Relations (P18-1)
Copied to clipboard
| Challenge: | Existing methods for paraphrasing nouncompounds lack the ability to generalize and have a hard time interpreting infrequent or new noun-compound. |
| Approach: | They propose a neural model that generalizes better by representing paraphrases in a continuous space, generalizing for both unseen noun-compounds and rare paraphrase. |
| Outcome: | The proposed model generalizes better by representing paraphrases in a continuous space, generalizing for unseen noun-compounds and rare paraphrase. |
Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing word embedding methods for natural language processing are limited in their ability to produce dense word embeds. |
| Approach: | They propose a word embedding SentiVec which is infused with sentiment information from a lexical resource and outperforms baselines on subjectivity-sensitive tasks. |
| Outcome: | The proposed word embedding SentiVec outperforms baselines on subjectivity-sensitive tasks. |
Word Embedding and WordNet Based Metaphor Identification and Interpretation (P18-1)
Copied to clipboard
| Challenge: | Existing models cannot identify exact metaphorical words within a sentence . current models do not rely on hand-crafted knowledge for training . |
| Approach: | They propose an unsupervised learning method that identifies and interprets metaphors at word-level without preprocessing. |
| Outcome: | The proposed method outperforms baseline models in two translation systems for English to Chinese showing that it paraphrases metaphors into their literal counterparts. |
Incorporating Latent Meanings of Morphological Compositions to Enhance Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing word embedding methods learn semantic information at word level while neglecting meaningful inner structures of words like morphemes. |
| Approach: | They propose to use latent meanings of morphological compositions of words to train word embeddings. |
| Outcome: | The proposed models outperform baseline models on word similarity, syntactic analogy and text classification tasks. |
A Stochastic Decoder for Neural Machine Translation (P18-1)
Copied to clipboard
| Challenge: | Neural machine translation models do not account for local lexical and syntactic variation in parallel corpora. |
| Approach: | They propose a deep generative model of machine translation which incorporates a chain of latent variables to account for local lexical and syntactic variation in parallel corpora. |
| Outcome: | The proposed model consistently improves over strong baselines on several different language pairs. |
Forest-Based Neural Machine Translation (P18-1)
Copied to clipboard
| Challenge: | Compared with string-to-string systems, tree-based NMT methods use more syntactic information and can incorporate prior knowledge. |
| Approach: | They propose a tree-based neural machine translation method that translates a linearized packed forest under a simple sequence-to-sequence framework. |
| Outcome: | The proposed method outperforms tree-based approaches in the BLEU score of the proposed model. |
Context-Aware Neural Machine Translation Learns Anaphora Resolution (P18-1)
Copied to clipboard
| Challenge: | Standard machine translation systems process sentences in isolation and ignore extra-sentential information. |
| Approach: | They propose a context-aware neural machine translation model that controls flow of information from extended context to the translation model. |
| Outcome: | The proposed model improves on an English-Russian subtitles dataset over its context-agnostic version (+0.7) and over simple concatenation of context and source sentences (+0.6). |
Document Context Neural Machine Translation with Memory Networks (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that our model exploits both source and target document context. |
| Approach: | They propose a document-level neural machine translation model which takes both source and target document context into account using memory networks. |
| Outcome: | The proposed model outperforms previous work in terms of BLEU and METEOR in English translations. |
Which Melbourne? Augmenting Geocoding with Maps (P18-1)
Copied to clipboard
| Challenge: | Existing methods to associate geographic information in text with coordinates are limited by lexical features and cartesian coordinates. |
| Approach: | They propose a geocoder that exploits implicit lexical clues to associate coordinates with text . they propose encoding of geographic metadata to generate two distinct views of the same text. |
| Outcome: | The proposed method improves state-of-the-art results on three datasets and an open-source dataset for disease outbreaks and epidemics. |
Learning Prototypical Goal Activities for Locations (P18-1)
Copied to clipboard
| Challenge: | a goal-act is an activity that represents a common reason people go to a location . recognizing goals is essential for narrative text understanding and story comprehension . |
| Approach: | They use a text corpus and semi-supervised learning to learn goal-acts for specific locations . they extract activities and locations that co-occur in goal-oriented syntactic patterns . |
| Outcome: | The proposed method outperforms baseline methods when judged against goal-acts identified by human annotators. |
Guess Me if You Can: Acronym Disambiguation for Enterprises (P18-1)
Copied to clipboard
| Challenge: | Acronyms are abbreviations formed from the initial components of words or phrases . acronyms can be difficult to understand for people who are not familiar with the subject matter . |
| Approach: | They propose a framework to automatically resolve the true meanings of acronyms in a given context . they use the enterprise corpus as input and a high-quality acronym disambiguation system as output . |
| Outcome: | The proposed framework can be deployed to any enterprise to support acronym disambiguation. |
A Multi-Axis Annotation Scheme for Event Temporal Relations (P18-1)
Copied to clipboard
| Challenge: | Existing temporal relation (TempRel) annotation schemes have low inter-annotator agreements even between experts, suggesting that the current annotation task needs a better definition. |
| Approach: | They propose to annotate temporal relation (TempRel) annotation schemes based on event start-points instead of a conventional 60’s-80’s model. |
| Outcome: | The proposed model improves IAA from the conventional 60’s to 80’s and can be used by crowdsourcing to alleviate labor intensity. |
Exemplar Encoder-Decoder for Neural Conversation Generation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to generate conversational systems suffer from lack of diversity in responses and generation of short, repetitive and uninteresting responses. |
| Approach: | They propose a novel conversation model that uses similar examples from training data to generate responses. |
| Outcome: | The proposed model outperforms state-of-the-art sequence to sequence learning on several evaluation metrics on two large data sets. |
DialSQL: Dialogue Based Structured Query Generation (P18-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning and semantic parsing have improved the translation accuracy of natural language questions to structured queries. |
| Approach: | They propose a dialogue-based structured query generation framework that leverages human intelligence to boost performance of existing algorithms via user interaction. |
| Outcome: | The proposed framework improves on a WikiSQL dataset from 61.3% to 69.0% using only 2.4 validation questions per dialogue. |
Conversations Gone Awry: Detecting Early Signs of Conversational Failure (P18-1)
Copied to clipboard
Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, Nithum Thain
| Challenge: | Prior work focused on characterizing and detecting content exhibiting antisocial online behavior. |
| Approach: | They propose a task of predicting from the very start of a conversation whether it will get out of hand. |
| Outcome: | The proposed framework can detect early warning signs of antisocial behavior in online conversations. |
Are BLEU and Meaning Representation in Opposition? (P18-1)
Copied to clipboard
| Challenge: | Empirical evaluation suggests that the better the translation quality, the worse the learned sentence representations serve in a wide range of classification and similarity tasks. |
| Approach: | They propose several variations of the attentive NMT architecture to bring this meeting point back . they propose to use a structured fixed-size representation of the input to produce static representations of input sentences. |
| Outcome: | The proposed architecture improves translation quality and performance in a range of tasks. |
Automatic Metric Validation for Grammatical Error Correction (P18-1)
Copied to clipboard
| Challenge: | Existing methods for metric validation in GEC suffer from low inter-rater agreement. |
| Approach: | They propose an automatic method for GEC metric validation that overcomes many of the difficulties in the existing method. |
| Outcome: | The proposed method sheds new light on metric quality and shows valid edits are penalized by existing metrics. |
The Hitchhiker’s Guide to Testing Statistical Significance in Natural Language Processing (P18-1)
Copied to clipboard
| Challenge: | Statistical significance testing is a standard statistical tool designed to ensure that experimental results are not coincidental. |
| Approach: | They propose a protocol for statistical significance test selection in NLP setups . they propose he proposes a survey of the most relevant tests to help guide the protocol . |
| Outcome: | The proposed protocol includes a survey of the most relevant tests. |
Distilling Knowledge for Search-based Structured Prediction (P18-1)
Copied to clipboard
| Challenge: | Existing studies have focused on the performance of structured prediction models, but they are often limited by the ambiguities of the reference policy. |
| Approach: | They propose to distill an ensemble of multiple models trained with different initializations into a single model and use it to explore the search space. |
| Outcome: | The proposed model outperforms the greedy models on two typical search-based structured prediction tasks and achieves 1.32 in LAS and 2.65 in BLEU over strong baselines. |
Stack-Pointer Networks for Dependency Parsing (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to dependency parsing are local and greedy transitionbased . StackPtr parsers use the information of whole sentences and previously derived subtree structures . |
| Approach: | They propose a stack-pointer network-based dependency parser that reads whole sentence and builds dependency tree top-down in a depth-first fashion. |
| Outcome: | The proposed model reads and encodes whole sentence, then builds dependency tree top-down (from root-to-leaf) in a depth-first fashion. |
Twitter Universal Dependency Parsing for African-American and Mainstream American English (P18-1)
Copied to clipboard
| Challenge: | We analyze the performance disparities between AAE and Mainstream American English (MAE) because of Twitter-specific conventions and dialectal language. |
| Approach: | They develop a dataset of 500 tweets, 250 of which are in AAE, within the Universal Dependencies 2.0 framework and annotate it. |
| Outcome: | The proposed model improves performance for AAE tweets with no or very little in-domain labeled data and assesses its lexical and syntactic features. |
LSTMs Can Learn Syntax-Sensitive Dependencies Well, But Modeling Structure Makes Them Better (P18-1)
Copied to clipboard
| Challenge: | a recent study found that language models fail to learn long-range syntax sensitive dependencies. |
| Approach: | They propose to use a subject-verb agreement diagnostic to determine whether language models can learn long-range syntax sensitive dependencies. |
| Outcome: | The proposed model outperforms left-corner and bottom-up variants in learning non-local dependencies. |
Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures (P18-1)
Copied to clipboard
| Challenge: | Existing solutions to task-oriented dialogue systems follow pipeline designs which introduces complexity and fragility. |
| Approach: | They propose a novel sequence-to-sequence (seq2sequ) model which tracks dialogue believes and a two stage copynet instantiation which emonstrates good scalability. |
| Outcome: | The proposed framework outperforms state-of-the-art pipeline-based methods on large datasets and retains satisfactory entity match rate on out-of vocabulary (OOV) cases where pipeline-designed competitors totally fail. |
An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking (P18-1)
Copied to clipboard
| Challenge: | a dialogue state tracker is a core component in most of today's spoken dialogue systems . slot-filling dialogues are composed of a predefined set of slots that need to be filled through the conversation . |
| Approach: | They propose an E2E architecture that extracts unknown slot values while still achieving state-of-the-art accuracy on the standard DSTC2 benchmark. |
| Outcome: | The proposed architecture achieves state-of-the-art accuracy on the DSTC2 benchmark while retaining predefined slot values. |
Global-Locally Self-Attentive Encoder for Dialogue State Tracking (P18-1)
Copied to clipboard
| Challenge: | a global-local self-attentive dialogue state tracker estimates user goals and requests given the dialogue context . GLAD significantly improves tracking of rare states, compared to prior work . task-oriented dialogue systems can significantly reduce operating costs . |
| Approach: | They propose a global-local self-attentive dialogue state tracker which shares global-level modules with global-specific estimators for different types of dialogue states. |
| Outcome: | The proposed model outperforms previous models on the WoZ state tracking task by 3.9% and 4.8%. |
Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems (P18-1)
Copied to clipboard
| Challenge: | End-to-end task-oriented dialog systems often suffer from the challenge of incorporating knowledge bases. |
| Approach: | They propose a novel yet simple end-to-end differentiable model called memory-tosequence to address this issue. |
| Outcome: | The proposed model can be trained faster and achieve state-of-the-art performance on three different task-oriented dialog datasets. |
Tailored Sequence to Sequence Models to Different Conversation Scenarios (P18-1)
Copied to clipboard
| Challenge: | Sequence to sequence (Seq2Sequeq) models fail to meet the diverse requirements for different conversation scenarios, such as customer service and chatbot. |
| Approach: | They propose two optimized criteria for Sequence to sequence (Seq2Sequeq) to meet different conversation scenarios, i.e., maximum generated likelihood for specific-requirement scenario, and conditional value-at-risk for diverse-requrement scenarios. |
| Outcome: | The proposed models satisfies diverse requirements for different conversation scenarios and yields better performances than existing models. |
Knowledge Diffusion for Neural Dialogue Generation (P18-1)
Copied to clipboard
| Challenge: | End-to-end neural dialogue generation does not employ knowledge to guide the generation. |
| Approach: | They propose a neural knowledge diffusion model to introduce knowledge into dialogue generation. |
| Outcome: | The proposed model outperforms baseline models on a real-world dataset. |
Generating Informative Responses with Controlled Sentence Function (P18-1)
Copied to clipboard
| Challenge: | Sentence function is a significant factor to achieve the purpose of the speaker, but has not been touched in large-scale conversation generation. |
| Approach: | They propose a model to generate informative responses with controlled sentence function using a latent variable and a type controller to deal with compatibility. |
| Outcome: | The proposed model outperforms state-of-the-art models and generates responses with controlled sentence function and informative content. |
Sentiment Adaptive End-to-End Dialog Systems (P18-1)
Copied to clipboard
| Challenge: | Existing methods to train dialog systems only consider semantic inputs and under-utilize other user information. |
| Approach: | They propose to include user sentiment in the end-to-end learning framework to make dialog systems more user-adaptive and effective. |
| Outcome: | The proposed system improves on a bus information search task with sentiment information. |
Embedding Learning Through Multilingual Concept Induction (P18-1)
Copied to clipboard
| Challenge: | Existing methods for learning vector space representations of words are based on word-context information. |
| Approach: | They propose a method for estimating vector space representations of words by concept induction. |
| Outcome: | The proposed method performs better on crosslingual word similarity and sentiment analysis on a parallel corpus. |
Isomorphic Transfer of Syntactic Structures in Cross-Lingual NLP (P18-1)
Copied to clipboard
| Challenge: | Using frameworks such as Universal Dependencies (UD) to transfer knowledge between languages can be challenging because of variation in syntactic structures. |
| Approach: | They propose a typologically driven method which reduces anisomorphism in UD treebanks by considering both morphological and structural properties. |
| Outcome: | The proposed method is effective for machine translation and cross-lingual sentence similarity. |
Language Modeling for Code-Mixing: The Role of Linguistic Theory based Synthetic Data (P18-1)
Copied to clipboard
| Challenge: | Code-mixed (CM) language training is a difficult problem because of lack of data and the increased confusability due to the presence of more than one language. |
| Approach: | They propose a computational technique for creating grammatically valid artificial CM data based on the Equivalence Constraint Theory. |
| Outcome: | The proposed method reduces the perplexity of the model and does not reduce the perceptibility of the models. |
Chinese NER Using Lattice LSTM (P18-1)
Copied to clipboard
| Challenge: | Chinese named entity recognition (NER) is a fundamental task in information extraction. |
| Approach: | They propose a lattice-structured LSTM model for Chinese named entity recognition (NER) model leverages word and word sequence information to encode a sequence of input characters and all potential words that match a lexicon. |
| Outcome: | The proposed model outperforms word-based and character-based models on Chinese NER datasets. |
Nugget Proposal Networks for Chinese Event Detection (P18-1)
Copied to clipboard
| Challenge: | Nugget Proposal Networks (NPNs) can solve word-trigger mismatch problem . word-wise event detection models suffer from word-tree mismatch because of multiple triggers . |
| Approach: | They propose a novel way to detect event triggers in a character-wise paradigm . they propose entire trigger nuggets centered at each character regardless of word boundaries . |
| Outcome: | The proposed model outperforms the state-of-the-art methods on two datasets. |
Higher-order Relation Schema Induction using Tensor Factorization with Back-off and Aggregation (P18-1)
Copied to clipboard
| Challenge: | Relation Schema Induction (RSI) is a problem of identifying type signatures of arguments from unlabeled text. |
| Approach: | They propose a framework for inducing higher-order relation schemata from unlabeled text. |
| Outcome: | The proposed framework helps in dealing with sparsity and induces higher-order relation schemata. |
Discovering Implicit Knowledge with Unary Relations (P18-1)
Copied to clipboard
| Challenge: | State-of-the-art relation extraction methods only recognize relationships between mentions of entity arguments stated explicitly in the text. |
| Approach: | They propose a method to identify relations between two entities using unary relations and a common deep learning based representation. |
| Outcome: | The proposed method outperforms state-of-the-art relation extraction technology on a web scale knowledge base population benchmark. |
Improving Entity Linking by Modeling Latent Relations between Mentions (P18-1)
Copied to clipboard
| Challenge: | Entity linking systems often exploit relations between textual mentions to decide if the linking decisions are compatible. |
| Approach: | They treat relations as latent variables while optimizing the neural entity-linking model without supervision. |
| Outcome: | The proposed model outperforms its relation-agnostic version and significantly outperformed its relational version. |
Dating Documents using Graph Convolution Networks (P18-1)
Copied to clipboard
| Challenge: | Existing approaches for document dating assume accurate knowledge of document date, but this is not always available for arbitrary documents from the Web. |
| Approach: | They propose a Graph Convolutional Network (GCN) based document dating approach which exploits syntactic and temporal graph structures of document in a principled way. |
| Outcome: | The proposed approach outperforms state-of-the-art models on real-world datasets by 19% absolute accuracy points. |
A Graph-to-Sequence Model for AMR-to-Text Generation (P18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic formalism that encodes the meaning of a sentence as a rooted, directed graph. |
| Approach: | They propose a neural graph-to-sequence model that leverages LSTM to encode a linearized AMR structure. |
| Outcome: | The proposed model outperforms existing methods on a benchmark. |
GTR-LSTM: A Triple Encoder for Sentence Generation from RDF Data (P18-1)
Copied to clipboard
| Challenge: | Knowledge bases are becoming an enabling resource for many applications including Q&A systems, recommender systems, and summarization tools. |
| Approach: | They propose a system to translate RDF triples into natural sentences using an encoder-decoder framework. |
| Outcome: | The proposed model outperforms the baseline model by 17.6%, 6.0%, and 16.4% in terms of BLEU, METEOR, and TER scores. |
Learning to Write with Cooperative Discriminators (P18-1)
Copied to clipboard
| Challenge: | Despite their local fluency, long-form text generated from RNNs is often generic, repetitive, and even self-contradictory. |
| Approach: | They propose a unified learning framework that can guide a base RNN generator towards more globally coherent generations by combining discriminators with a composite decoding objective. |
| Outcome: | The proposed framework can guide a base RNN generator towards more globally coherent generations by combining discriminators with the base RRN generator through a composite decoding objective. |
A Neural Approach to Pun Generation (P18-1)
Copied to clipboard
| Challenge: | generating puns with artificial intelligence techniques requires manual training and templates. |
| Approach: | They propose neural network models for homographic pun generation that can generate puns without requiring any pun data for training. |
| Outcome: | The proposed models generate homographic puns of good readability and quality without training. |
Learning to Generate Move-by-Move Commentary for Chess Games from Large-Scale Social Forum Data (P18-1)
Copied to clipboard
| Challenge: | Using a large-scale chess commentary dataset, we generate a set of comments for individual moves in a game. |
| Approach: | They propose a large-scale chess commentary dataset and a method to generate commentary for individual moves in a chessian game. |
| Outcome: | The proposed method is rated similar to ground truth commentary texts in terms of correctness and fluency. |
From Credit Assignment to Entropy Regularization: Two New Algorithms for Neural Sequence Prediction (P18-1)
Copied to clipboard
| Challenge: | equivalence between credit assignment problem and entropy regularized reinforcement learning is established . a wide range of successful sequence prediction algorithms have been developed . |
| Approach: | They propose to extend credit assignment in reward augmented maximum likelihood learning by credit assignment and entropy regularization. |
| Outcome: | The proposed algorithms outperform RAML and Actor-Critic on two benchmark datasets. |
DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | DuoRC contains 186,089 unique question-answer pairs created from 7680 movie plots . |
| Approach: | They propose a novel dataset for Reading Comprehension that motivates new challenges for neural approaches in language understanding beyond those offered by existing RC datasets. |
| Outcome: | The proposed dataset motivates several new challenges for neural approaches in language understanding beyond those offered by existing RC datasets. |
Stochastic Answer Networks for Machine Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Several recent MRC models employ multi-step reasoning . we show that the use of a stochastic prediction dropout improves robustness . |
| Approach: | They propose a stochastic answer network that simulates multi-step reasoning in machine reading comprehension. |
| Outcome: | The proposed model improves robustness and results competitive with state-of-the-art models on the Stanford Question Answering Dataset and Microsoft MAchine Reading COmprehension Dataset. |
Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to read comprehension style question answering are limited by the volume of annotated datasets. |
| Approach: | They propose a hierarchical attention network for reading comprehension style question answering . they first encode the question and paragraph with fine-grained language embeddings . then propose fusion approach to fuse information from both global and attended representations based on the hierarchic attention network . |
| Outcome: | The proposed method achieves state-of-the-art on the SQuAD and TriviaQA Wiki leaderboards and two adversarial SQu AD datasets. |
Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | Various advanced neural models have been proposed for reading comprehension, but most models ignore its relations with other answer candidates. |
| Approach: | They propose to model reading comprehension as an extract-then-select two-stage procedure . they first extract answer candidates from passages, then select the final answer by combining information from all candidates. |
| Outcome: | The proposed approach improves state-of-the-art performance on open-domain reading comprehension datasets. |
Efficient and Robust Question Answering from Minimal Context over Documents (P18-1)
Copied to clipboard
| Challenge: | Recent work shows that neural QA models are sensitive to adversarial inputs. |
| Approach: | They propose a sentence selector to select the minimal set of sentences to feed into a QA model. |
| Outcome: | The proposed system reduces training time and inference time by up to 13 times . it is comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQu AD-Open . |
Denoising Distantly Supervised Open-Domain Question Answering (P18-1)
Copied to clipboard
| Challenge: | Existing DS-QA models ignore rich information contained in other paragraphs and are noisy . Existing systems rely on pre-identified relevant texts, which do not always exist in real-world QA scenarios. |
| Approach: | They propose a model which uses a paragraph selector to filter out noisy paragraphs and a reader to extract the correct answer from denoised paragraphs. |
| Outcome: | The proposed model can capture useful information from noisy data and achieve significant improvements on open domain question answering. |
Question Condensing Networks for Answer Selection in Community Question Answering (P18-1)
Copied to clipboard
| Challenge: | Community question answering (CQA) is a subtask of community question answering . previous researches ignored the difference between the two parts and concatenated them as the question representation . |
| Approach: | They propose a question condensing network that makes use of the subject-body relationship of community questions. |
| Outcome: | The proposed model outperforms existing models on two CQA datasets. |
Towards Robust Neural Machine Translation (P18-1)
Copied to clipboard
| Challenge: | Small perturbations in the input can severely distort intermediate representations and thus impact translation quality of neural machine translation models. |
| Approach: | They propose adversarial stability training to make encoder and decoder robust to perturbations by enabling them to behave similarly for the original input and its perturbed counterpart. |
| Outcome: | The proposed approach improves translation quality and robustness over strong models on Chinese-English, English-German and English-French translation tasks. |
Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings (P18-1)
Copied to clipboard
| Challenge: | Neural machine translation uses source and target word embeddings to improve translation quality . source and targeted word embeds are at the two ends of a long information processing procedure . |
| Approach: | They propose a method to shorten the distance between source and target words in neural machine translation by bridging source and targeting word embeddings. |
| Outcome: | The proposed method shortens the distance between source and target words in neural machine translation and strengthens their association. |
Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning (P18-1)
Copied to clipboard
| Challenge: | Recent work has shown that reinforcement learning (RL) can be scaled to games with large state-action spaces, achieving human-level performance or even superhuman performance. |
| Approach: | They propose to use bandit feedback to improve sequence-to-sequence learning by simulating reward signals by evaluation metrics such as BLEU, F1-score, or ROUGE. |
| Outcome: | The proposed methods improve performance even from small amounts of human feedback, pointing to a great potential for applications at larger scale. |
Accelerating Neural Transformer via an Average Attention Network (P18-1)
Copied to clipboard
| Challenge: | Using parallelizable attention networks, the neural Transformer is slow to train due to auto-regressive architecture and self-attention in the decoder. |
| Approach: | They propose an average attention network to replace the original self-attention model in the decoder of the neural Transformer. |
| Outcome: | The proposed network can decode sentences over four times faster than the original version with almost no loss in training time and translation performance. |
How Much Attention Do You Need? A Granular Analysis of Neural Machine Translation Architectures (P18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) has been replaced by convolutional or self-attentional approaches. |
| Approach: | They propose an architecture definition language that allows for a flexible combination of common building blocks. |
| Outcome: | The proposed architectures can bring recurrent and convolutional models close to the Transformer architecture, but not using self-attention. |
Weakly Supervised Semantic Parsing with Abstract Examples (P18-1)
Copied to clipboard
| Challenge: | training semantic parsers from weak supervision complicates training in two ways . spurious programs that accidentally lead to a correct denotation add noise to training . |
| Approach: | They propose to use tokens in both language utterance and program to map denotations to executable programs. |
| Outcome: | The proposed method improves performance and reaches 82.5% accuracy compared to the best reported accuracy so far. |
Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback (P18-1)
Copied to clipboard
| Challenge: | a recent study shows that counterfactual learning from human bandit feedback can improve neural semantic parsers . cost and difficulty of manually preparing large amounts of parses is a bottleneck for supervised learning . |
| Approach: | They propose to use human bandit feedback to apply counterfactual learning to neural parsing . they devise an easy-to-use interface to collect human feedback on semantic parses . |
| Outcome: | The proposed framework improves semantic parsers by reducing the cost of manual parsing . the proposed framework is based on human bandit feedback collected by the user . |
AMR dependency parsing with a typed semantic algebra (P18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representations (AMRs) are graphs which describe the predicate-argument structure of a sentence. |
| Approach: | They propose a semantic parser which parses strings into tree representations of the compositional structure of an AMR graph. |
| Outcome: | The proposed parser outperforms baselines and standard neural techniques for supertagging and dependency tree parsing. |
Sequence-to-sequence Models for Cache Transition Systems (P18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic formalism where the meaning of a sentence is encoded as a rooted, directed graph. |
| Approach: | They propose a sequence-to-sequence based approach for mapping natural language sentences to AMR semantic graphs using a special transition system called a cache transition system. |
| Outcome: | The proposed model outperforms other sequence-to-sequence approaches and achieves competitive results in comparison with the best-performing models. |
Batch IS NOT Heavy: Learning Word Representations From All Samples (P18-1)
Copied to clipboard
| Challenge: | Stochastic Gradient Descent with negative sampling is the most prevalent approach to learn word representations. |
| Approach: | They propose a method that uses batch gradient learning to generate word representations from all training samples. |
| Outcome: | The proposed method outperforms sampling-based methods on several benchmark tasks. |
Backpropagating through Structured Argmax using a SPIGOT (P18-1)
Copied to clipboard
| Challenge: | Structured projection of intermediate gradients (SPIGOT) is a new method for backpropagating through neural networks . structure-based learning methods for natural language processing are increasingly dominated by end-to-end differentiable functions . |
| Approach: | They propose a structured projection of intermediate gradients method for backpropagating through neural networks that includes hard-decision structured predictions in intermediate layers. |
| Outcome: | The proposed method improves on two structured NLP pipelines: syntactic-then-semantic dependency parsing and semantic parser followed by sentiment classification. |
Learning How to Actively Learn: A Deep Imitation Learning Approach (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that heuristic-based active learning methods are limited when the data distribution of the underlying learning problems vary. |
| Approach: | They propose a method that learns an AL "policy" using "imitation learning" they use an efficient "algorithmic expert" which provides the policy learner with good actions in the encountered AL situations. |
| Outcome: | The proposed method is more effective than previous methods on two tasks . labeled data is rare while unlabelled data is abundant . |
Training Classifiers with Natural Language Explanations (P18-1)
Copied to clipboard
| Challenge: | a semantic parser converts explanations into programmatic labeling functions . a standard protocol for obtaining a labeled dataset provides only one bit of information per example . |
| Approach: | They propose a framework where an annotator provides an explanation for each labeling decision . they use a semantic parser to convert these explanations into programmatic labeling functions . |
| Outcome: | The proposed framework trains classifiers faster by providing explanations instead of labels . the proposed framework is based on a rule-based semantic parser . |
Did the Model Understand the Question? (P18-1)
Copied to clipboard
| Challenge: | Using the notion of “attribution,” deep learning models often ignore important question terms. |
| Approach: | They propose techniques to analyze the sensitivity of a deep learning model to question words . they use attribution to generate adversarial questions using visual and tabular questions . |
| Outcome: | The proposed techniques reduce the accuracy of a visual question answering model by 61.1% and that of 'tabular' question answering models by 3.3%. |
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia (P18-1)
Copied to clipboard
| Challenge: | Existing models that only take into account sentence-level information do not generate question-answer pairs. |
| Approach: | They propose a neural network approach that incorporates coreference knowledge via a novel gating mechanism for paragraphlevel question generation. |
| Outcome: | The proposed model outperforms existing models on a Wikipedia article question-answer generation task. |
Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification (P18-1)
Copied to clipboard
| Challenge: | Recent years have seen rapid growth in the MRC community . MRC is believed to be a crucial step in building a general intelligent agent . |
| Approach: | They propose an end-to-end neural model that enables multiple passages to verify each other based on their content representations. |
| Outcome: | The proposed model outperforms the baseline on the English MS-MARCO dataset and the Chinese DuReader dataset, and achieves state-of-the-art performance on both datasets. |
Language Generation via DAG Transduction (P18-1)
Copied to clipboard
| Challenge: | Existing formal frameworks for graph manipulation are underexploited. |
| Approach: | They propose a DAG transducer to perform graph-to-program transformation using a declarative programming language. |
| Outcome: | The proposed transducer achieves a BLEU-4 score of 68.07 for natural language generation from type-logical semantic graphs. |
A Distributional and Orthographic Aggregation Model for English Derivational Morphology (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to derived word generation model derivational morphology to generate words with particular semantics are not effective. |
| Approach: | They propose a novel aggregation model that learns derivational transformations as orthographic functions and as functions in distributional word embedding space. |
| Outcome: | The proposed model learns to choose between the hypothesis of each system and the hypothesis from the model. |
Deep-speare: A joint neural model of poetic language, meter and rhyme (P18-1)
Copied to clipboard
| Challenge: | a recent surge of interest in deep learning has led to creative applications for poetry generation . a novel joint architecture captures language, rhyme and meter for sonnet modelling . |
| Approach: | They propose a joint architecture that captures language, rhyme and meter for sonnet modelling. |
| Outcome: | The proposed architecture captures language, rhyme and meter for sonnet modelling. |
NeuralREG: An end-to-end approach to referring expression generation (P18-1)
Copied to clipboard
| Challenge: | Referring Expression Generation models typically rely on features such as salience and grammatical function to make decisions about form and content. |
| Approach: | They propose a new approach that makes decisions about form and content in one go . they use a delexicalized version of the WebNLG corpus to test the approach . |
| Outcome: | The proposed approach significantly improves over two strong baselines. |
Stock Movement Prediction from Tweets and Historical Prices (P18-1)
Copied to clipboard
| Challenge: | a novel deep generative model exploits text and price signals to make stochastic stock movement predictions. |
| Approach: | They propose a deep generative model exploiting text and price signals to solve this problem. |
| Outcome: | The proposed model exploits text and price signals to make temporally-dependent predictions from chaotic data. |
Rumor Detection on Twitter with Tree-structured Recursive Neural Networks (P18-1)
Copied to clipboard
| Challenge: | Existing methods for detecting rumors are difficult to implement and require a lot of effort. |
| Approach: | They propose two recursive neural models that follow tweets' propagation layouts to learn discriminative features from tweets and generate more powerful representations for rumors detection. |
| Outcome: | The proposed models perform better than state-of-the-art approaches on two public Twitter datasets and show superior performance on detecting rumors at very early stage. |
Visual Attention Model for Name Tagging in Multimodal Social Media (P18-1)
Copied to clipboard
| Challenge: | Name tagging is a key task for language understanding, but is often limited by the short textual components. |
| Approach: | They propose a novel model architecture based on visual attention that outperforms other methods . they use multimodal datasets to analyze the name tagging task on social media . |
| Outcome: | The proposed model outperforms existing methods and significantly outperformed existing methods. |
Multimodal Named Entity Disambiguation for Noisy Social Media Posts (P18-1)
Copied to clipboard
| Challenge: | Social media posts often contain unstructured text or images, making opinion mining challenging. |
| Approach: | They propose a new task for multimodal social media captions with named entities annotated and linked to external knowledge bases. |
| Outcome: | The proposed model outperforms state-of-the-art text-only NED models . it predicts correct entities in knowledge graph embeddings space, showing its efficacy and potentials . |
Semi-supervised User Geolocation via Graph Convolutional Networks (P18-1)
Copied to clipboard
| Challenge: | Social media user geolocation is vital to many applications such as event detection. |
| Approach: | They propose a multiview geolocation model that uses both text and network context. |
| Outcome: | The proposed model outperforms baseline models and the state-of-the-art models under minimal supervision. |
Document Modeling with External Attention for Sentence Extraction (P18-1)
Copied to clipboard
Shashi Narayan, Ronald Cardenas, Nikos Papasarantopoulos, Shay B. Cohen, Mirella Lapata, Jiangsheng Yu, Yi Chang
| Challenge: | Document modeling is essential to a variety of natural language understanding tasks. |
| Approach: | They propose to use external information to improve document modeling for sentence extraction problems. |
| Outcome: | The proposed model outperforms baseline models on document summarization and answer selection tasks and achieves state-of-the-art results on WikiQA and NewsQA. |
Neural Models for Documents with Metadata (P18-1)
Copied to clipboard
| Challenge: | specialized models are often used to model text corpora without metadata . specialized algorithms are not widely used in the digital humanities and political science fields . |
| Approach: | They propose a general neural framework based on topic models to enable customization of metadata. |
| Outcome: | The proposed framework achieves strong performance with a manageable tradeoff between perplexity, coherence, and sparsity. |
NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing (P18-1)
Copied to clipboard
Dinghan Shen, Qinliang Su, Paidamoyo Chapfuwa, Wenlin Wang, Guoyin Wang, Ricardo Henao, Lawrence Carin
| Challenge: | Existing approaches to fast similarity search require two-stage training and the binary constraints are handled ad-hoc. |
| Approach: | They propose an end-to-end neural architecture for semantic hashing where binary hash codes are treated as Bernoulli latent variables. |
| Outcome: | The proposed approach outperforms state-of-the-art models on unsupervised and supervised scenarios on three public datasets. |
Large-Scale QA-SRL Parsing (P18-1)
Copied to clipboard
| Challenge: | a crowd-sourced approach to learning semantic parsers to predict predicateargument structures is open to many researchers. |
| Approach: | They propose a large-scale corpus of Question-Answer driven Semantic Role Labeling annotations . they also propose QA-SRL Bank 2.0, a crowd-sourcing scheme that can be used to train high quality parsers . |
| Outcome: | The proposed QA-SRL parser can generate high-quality questions at low cost and is intuitive to non-experts. |
Syntax for Semantic Role Labeling, To Be, Or Not To Be (P18-1)
Copied to clipboard
| Challenge: | Existing neural SRL models lack syntactic backbone for performance, limiting its use in deep learning. |
| Approach: | They propose an enhanced argument labeling model with extended korder argument pruning algorithm for effectively exploiting syntactic information. |
| Outcome: | The proposed model achieves state-of-the-art on the CoNLL-2008 and 2009 benchmarks in English and Chinese. |
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to map context-dependent sequential instructions to actions are based on discourse and state dependencies . we evaluate on SCONE domains and show absolute accuracy improvements of 9.8%-25.3% . |
| Approach: | They propose a model that considers previous utterances and the state of the world to map sequential instructions to actions. |
| Outcome: | The proposed model improves on the SCONE domains and on the target domains. |
Marrying Up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding (P18-1)
Copied to clipboard
| Challenge: | Experimental results show that the combination of regular expressions and NNs improves learning effectiveness when a small number of training examples are available. |
| Approach: | They propose to combine a neural network (NN) with regular expressions (RE) to improve supervised learning for NLP by exploiting the rich expressiveness of REs at different levels within a NN. |
| Outcome: | The proposed approach significantly improves learning effectiveness when a small number of training examples are available. |
Token-level and sequence-level loss smoothing for RNN language models (P18-1)
Copied to clipboard
| Challenge: | Maximum likelihood estimation treats all sentences that do not match the ground truth as equally poor, ignoring the structure of the output space. |
| Approach: | They propose to extend the reward augmented maximum likelihood approach to token-level loss smoothing by using token-based approaches to improve the model's performance. |
| Outcome: | The proposed model improves on image captioning and machine translation tasks and treats all sentences that do not match the ground truth as poor . |
Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numbers (P18-1)
Copied to clipboard
| Challenge: | Numeracy is the ability to understand and work with numbers. |
| Approach: | They propose a neural architecture that uses a continuous probability density function to model numerals from an open vocabulary using hierarchical models. |
| Outcome: | The proposed model reduces errors by 18% and 54% on clinical and scientific datasets compared to the second best model for each dataset . |
To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness (P18-1)
Copied to clipboard
| Challenge: | Recent success of Recurrent Neural Networks (RNNs) in Machine Translation (MT) has prompted attention mechanisms to be used in machine translation. |
| Approach: | They propose a tree-structured attention model on Tree Long Short-Term Memory Networks . they also experiment with three LSTM variants: bidirectional-LSTMs, Constituency Tree-LSTS, and Dependency Tree LSTS. |
| Outcome: | The proposed model is based on tree-LSTMs, constituency trees, and dependencies trees. |
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (P18-1)
Copied to clipboard
| Challenge: | a lack of understanding of the properties of sentence embeddings is limiting the use of the techniques. |
| Approach: | They propose 10 probing tasks designed to capture simple linguistic features of sentences . they use three different encoders to train embeddings in eight different ways . |
| Outcome: | The proposed tasks capture key linguistic features of sentences, but they are difficult to infer from them. |
Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning (P18-1)
Copied to clipboard
| Challenge: | Distant supervision is an efficient method for relation extraction, but it is noisy. |
| Approach: | They propose a deep reinforcement learning strategy to generate false-positive indicators . they redistribute false positives into negative examples to reduce false positive problem . |
| Outcome: | The proposed method significantly improves the performance of distant supervision compared to state-of-the-art systems. |
Interpretable and Compositional Relation Learning by Joint Training with an Autoencoder (P18-1)
Copied to clipboard
| Challenge: | Embedding models for entities and relations are useful for recovering missing facts in knowledge bases. |
| Approach: | They propose a dimension reduction technique by training relations jointly with an autoencoder to capture compositional constraints. |
| Outcome: | The proposed model improves on Knowledge Base Completion tasks with a significantly higher mean rank and better compositional training. |
Zero-Shot Transfer Learning for Event Extraction (P18-1)
Copied to clipboard
| Challenge: | Existing supervised event extraction methods rely on manual annotations and features specific to each event type. |
| Approach: | They propose a framework that maps event mentions to a specific type in an event ontology . they use existing annotations to extract event types from unstructured text data . |
| Outcome: | The proposed framework can be applied to new unseen event types without manual annotations. |
Recursive Neural Structural Correspondence Network for Cross-domain Aspect and Opinion Co-Extraction (P18-1)
Copied to clipboard
| Challenge: | supervised learning methods have been used for fine-grained opinion analysis but lack of labeled data hinders learning . authors develop a recursive neural network that could reduce domain shift in word level . a recent paper shows that unsupervised methods fail to adapt well across domains . |
| Approach: | They propose a supervised neural network that reduces domain shift effectively in word level . they treat these relations as invariant "pivot information" across domains to build structural correspondences . |
| Outcome: | The proposed model reduces domain shift effectively in word level through syntactic relations . it can be used to predict the relation between two adjacent words in the dependency tree . |
Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning (P18-1)
Copied to clipboard
| Challenge: | Training a task-completion dialogue agent via reinforcement learning (RL) is costly because it requires many interactions with real users. |
| Approach: | They propose a framework that integrates planning for task-completion dialogue policy learning into a dialogue agent using a world model to mimic real user response and generate simulated experience. |
| Outcome: | The proposed framework integrates planning for task-completion dialogue policy learning with real user interaction and simulated user behavior. |
Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders (P18-1)
Copied to clipboard
| Challenge: | Extensive experiments show that typed decoders outperform state-of-the-art baselines and can generate more meaningful questions. |
| Approach: | They devised two typed decoders that generate questions with different types of interrogatives, topic words, and ordinary words. |
| Outcome: | Extensive experiments show that the typed decoders outperform state-of-the-art baselines and can generate more meaningful questions. |
Personalizing Dialogue Agents: I have a dog, do you have pets too? (P18-1)
Copied to clipboard
| Challenge: | chit-chat models lack specificity, do not display a consistent personality and are often not very captivating. |
| Approach: | They propose to train chit-chat models to condition on profile information and profile information about the interlocutors. |
| Outcome: | The proposed model can predict profile information about the interlocutors based on the data . the proposed model is able to generate meaningful responses in a chit-chat setting . |
Efficient Large-Scale Neural Domain Classification with Personalized Attention (P18-1)
Copied to clipboard
| Challenge: | Using a scalable neural model, we show that personalization improves domain classification accuracy in a setting with thousands of overlapping domains. |
| Approach: | They propose a scalable neural model architecture with a shared encoder that incorporates personalization information and domain-specific classifiers that solves the problem efficiently. |
| Outcome: | The proposed architecture achieves two orders of magnitude faster than full model retraining. |
Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to classify human affect and subjective information from multiple data sources are limited by the lack of high-level feature associations. |
| Approach: | They propose a hierarchical multimodal architecture with attention and word-level fusion to classify utterance-level sentiment and emotion from text and audio data. |
| Outcome: | The proposed model outperforms state-of-the-art approaches on published datasets and visualizes and interprets synchronized attention over modalities. |
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph (P18-1)
Copied to clipboard
| Challenge: | Analyzing human multimodal language is emerging area of research in NLP. |
| Approach: | They propose a multimodal fusion technique to exploit how modalities interact in multimodal language. |
| Outcome: | The proposed technique exploits how modalities interact with each other in human multimodal language. |
Efficient Low-rank Multimodal Fusion With Modality-Specific Factors (P18-1)
Copied to clipboard
Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan, Paul Pu Liang, AmirAli Bagher Zadeh, Louis-Philippe Morency
| Challenge: | Multimodal research is a growing field of artificial intelligence, and fusion is one of the main research problems. |
| Approach: | They propose a low-rank multimodal fusion method which integrates multiple unimodal representations into one compact multimodal representation. |
| Outcome: | The proposed method achieves competitive results on multimodal sentiment analysis, speaker trait analysis, and emotion recognition tasks while reducing computational complexity. |
Discourse Coherence: Concurrent Explicit and Implicit Relations (P18-1)
Copied to clipboard
| Challenge: | Existing studies on discourse coherence show that multiple discourse relations can be operative between two segments for reasons not predicted by the literature. |
| Approach: | They show that people endorse seemingly divergent conjunctions to express the link they see between two segments in a crowdsourced conjunctioninsertion experiment. |
| Outcome: | The proposed results can inform future work on discourse coherence and lead to higher levels of performance in discourse parsing. |
A Spatial Model for Extracting and Visualizing Latent Discourse Structure in Text (P18-1)
Copied to clipboard
| Challenge: | Using sequences of sentences, we show that learning long-range latent discourse structure from large corpora can be useful for machine learning. |
| Approach: | They propose a probabilistic model of documents as sequences of sentences with a 2- or 3-D spatial grid and embed sentences into a grid. |
| Outcome: | The proposed model outperforms or is competitive with state-of-the-art generative approaches on tasks such as predicting the outcome of a story, and sentence ordering. |
Joint Reasoning for Temporal and Causal Relations (P18-1)
Copied to clipboard
| Challenge: | a cause must occur earlier than its effect, temporal and causal relations are closely related . a joint inference framework is developed for studying temporal, causal relations . |
| Approach: | They propose a joint inference framework for temporal and causal relations . they use constraints inherent in time and causality to enforce constraints . |
| Outcome: | The proposed framework improves extraction of temporal and causal relations from text. |
Modeling Naive Psychology of Characters in Simple Commonsense Stories (P18-1)
Copied to clipboard
| Challenge: | Understanding a narrative requires reasoning about the causal links between the events in the story and the mental states of the characters, even when those relationships are not explicitly stated. |
| Approach: | They propose a new annotation framework to explain naive psychology of story characters as fully-specified chains of mental states with respect to motivations and emotional reactions. |
| Outcome: | The proposed framework provides a baseline performance on several new tasks suggesting avenues for future research. |
A Deep Relevance Model for Zero-Shot Document Filtering (P18-1)
Copied to clipboard
| Challenge: | Existing methods for document classification do not consider document filtering . existing methods do not include document filter. |
| Approach: | They propose a deep relevance model for zero-shot document filtering called DAZER . they use word embeddings to extract the relevance signals from word embeds . |
| Outcome: | The proposed model outperforms existing models on two document collections . it estimates the relevance between a document and a category by using seed words . |
Disconnected Recurrent Neural Networks for Text Categorization (P18-1)
Copied to clipboard
| Challenge: | Recurrent neural network (RNN) can model the entire sequence and capture long-term dependencies, but it does not do well in extracting key patterns. |
| Approach: | They propose a novel model which incorporates position-invariance into RNN and restricts the hidden state at each time step to represent words near the current position. |
| Outcome: | The proposed model improves on several benchmark datasets and achieves the best performance on several datasets. |
Joint Embedding of Words and Labels for Text Classification (P18-1)
Copied to clipboard
Guoyin Wang, Chunyuan Li, Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin
| Challenge: | Existing approaches to text classification use word embeddings to capture semantic regularities between words. |
| Approach: | They propose to view text classification as a label-word joint embedding problem . they use a framework that measures compatibility between text sequences and labels . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on large text datasets. |
Neural Sparse Topical Coding (P18-1)
Copied to clipboard
| Challenge: | Topic models with sparsity enhancement are effective at learning discriminative and coherent latent topics of short texts. |
| Approach: | They propose a novel sparsity-enhanced topic model with back propagation that replaces the inference process with the back propagations, making it easy to explore extensions. |
| Outcome: | The proposed model outperforms existing methods on Web Snippet and 20Newsgroups datasets. |
Document Similarity for Texts of Varying Lengths via Hidden Topics (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to measure document similarity are inadequate for document pairs with non-comparable lengths, such as a long document and its summary. |
| Approach: | They propose a document matching approach to bridge the gap between long documents and their abstract information in a common space of hidden topics. |
| Outcome: | The proposed approach outperforms strong baselines on two matching tasks and incorporates domain knowledge to gain further performance improvement. |
Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour (P18-1)
Copied to clipboard
Sandeep Mathias, Diptesh Kanojia, Kevin Patel, Samarth Agrawal, Abhijit Mishra, Pushpak Bhattacharyya
| Challenge: | Existing methods to predict text quality include estimating subjective aspects of text, like structure, clarity, etc. |
| Approach: | They propose to capture gaze behaviour to help predict text quality by reporting improvements obtained by adding gaze features to traditional textual features for score prediction. |
| Outcome: | The proposed model shows that capturing gaze behaviour improves the accuracy of score prediction when the reader has fully understood the text. |
Multi-Input Attention for Unsupervised OCR Correction (P18-1)
Copied to clipboard
| Challenge: | Existing methods for OCR correction are mostly supervised methods that correct recognition errors in a single output. |
| Approach: | They propose a sequence-to-sequence model with attention and a decoder with attention averaging to search for consensus among multiple sequences. |
| Outcome: | The proposed methods cut the character and word error rates nearly in half on single inputs and can rival supervised methods. |
Building Language Models for Text with Named Entities (P18-1)
Copied to clipboard
| Challenge: | Existing language models fail to predict the entity names due to their wide variations. |
| Approach: | They propose a language model which can learn the entity names by leveraging their entity type information. |
| Outcome: | The proposed model achieves 52.2% better perplexity in recipe generation and 22.06% on code generation than state-of-the-art language models. |
hyperdoc2vec: Distributed Representations of Hypertext Documents (P18-1)
Copied to clipboard
| Challenge: | Conventional text embedding methods suffer from information loss if directly adapted to hyper-documents. |
| Approach: | They propose an embedding approach for hyper-documents that incorporates four criteria to preserve necessary information for embeddable models. |
| Outcome: | The proposed model outperforms several existing models on two tasks in the academic domain. |
Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval (P18-1)
Copied to clipboard
| Challenge: | Entity-oriented search and neural-IR push the boundary of search engines from two different aspects. |
| Approach: | They propose an Entity-Duet Neural Ranking Model which integrates knowledge graphs into neural search systems. |
| Outcome: | The proposed model improves generalization ability of neural ranking models on a commercial search log. |
Neural Natural Language Inference Models Enhanced with External Knowledge (P18-1)
Copied to clipboard
| Challenge: | Existing datasets that allow for complex models to be trained are limited . if data is not available, can machines learn all knowledge needed to perform natural language inference? |
| Approach: | They propose to enrich neural natural language inference models with external knowledge . they propose to use this knowledge to build NLI models to leverage it . |
| Outcome: | The proposed models improve on the SNLI and MultiNLI datasets. |
AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examples (P18-1)
Copied to clipboard
| Challenge: | Recent deep learning entailment systems have achieved close to human level performance on large datasets, but the problem is far from solved. |
| Approach: | They propose a knowledge-guided adversarial example generator for incorporating large lexical resources into entailment models via only a handful of rule templates and a natural language example generator that iteratively adjusts to the discriminator’s weaknesses. |
| Outcome: | The proposed methods increase accuracy by 4.7% on SciTail and 2.8% on a 1% sub-sample of SNLI. |
Subword-level Word Vector Representations for Korean (P18-1)
Copied to clipboard
| Challenge: | Existing research on word vectors for English focuses on decomposing words into subword units and using subwords to improve performance. |
| Approach: | They propose to decompose Korean words into the jamo-level, beyond the character-level . they develop Korean test sets for word similarity and analogy and make them publicly available . |
| Outcome: | The proposed method outperforms word2vec and character-level skip-grams on similarity and analogy tasks and contributes positively toward downstream NLP tasks such as sentiment analysis. |
Incorporating Chinese Characters of Words for Lexical Sememe Prediction (P18-1)
Copied to clipboard
| Challenge: | Existing methods of lexical sememe prediction rely on external context information of words to represent meaning. |
| Approach: | They propose a character-enhanced sememe prediction framework for Chinese language that takes advantage of internal character information and external context information. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on a Chinese sememe knowledge base and maintains robust performance even for low-frequency words. |
SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentiment (P18-1)
Copied to clipboard
| Challenge: | SemAxis characterizes word semantics using many semantic axes in word-vector spaces beyond sentiment . lexicon-based text analysis assumes that meaning of words does not change across contexts . but, recent advances in vector-space representations can tackle this challenge . |
| Approach: | They propose a framework to characterize word semantics using many semantic axes beyond sentiment . they demonstrate that SemAxis can capture nuanced semantic representations in multiple online communities . |
| Outcome: | The proposed framework outperforms state-of-the-art approaches in building domain-specific sentiment lexicons. |
End-to-End Reinforcement Learning for Automatic Taxonomy Induction (P18-1)
Copied to clipboard
| Challenge: | Existing methods for automating taxonomy induction often divide the problem into two subtasks . a novel end-to-end reinforcement learning approach is proposed to improve the accuracy of such methods. |
| Approach: | They propose an end-to-end reinforcement learning approach to automatic taxonomy induction from a set of terms. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on two public datasets of different domains. |
Incorporating Glosses into Neural Word Sense Disambiguation (P18-1)
Copied to clipboard
| Challenge: | Existing neural networks for Word Sense Disambiguation rely on labeled data and lexical knowledge. |
| Approach: | They propose a gloss-augmented WSD neural network which integrates context and glosses of the target word into a unified framework. |
| Outcome: | The proposed model outperforms the state-of-the-art systems on several English all-words WSD datasets. |
Bilingual Sentiment Embeddings: Joint Projection of Sentiment Across Languages (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to sentiment analysis in low-resource languages lack annotated corpora or do not capture sentiment information. |
| Approach: | They propose a model that represents sentiment in a source and target language without annotated corpus. |
| Outcome: | The proposed model outperforms state-of-the-art methods on four out of six setups and captures complementary information to machine translation. |
Learning Domain-Sensitive and Sentiment-Aware Word Embeddings (P18-1)
Copied to clipboard
| Challenge: | Existing word embeddings cannot produce domain-sensitive embeddables due to domain-specific nature of words. |
| Approach: | They propose a method for learning domain-sensitive and sentiment-aware embeddings that captures sentiment semantics and domain sensitivity of individual words. |
| Outcome: | The proposed method can produce domain-common embeddings and domain-specific embedds. |
Cross-Domain Sentiment Classification with Target Domain Specific Information (P18-1)
Copied to clipboard
| Challenge: | Existing methods for sentiment classification focus on learning domain-invariant representations . few of them pay attention to domain-specific information, which should also be informative. |
| Approach: | They propose a method to extract domain specific and invariant representations and train a classifier on each of them. |
| Outcome: | The proposed model can achieve better performance than state-of-the-art methods. |
Aspect Based Sentiment Analysis with Gated Convolutional Networks (P18-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis can provide more detailed information than general sentiment analysis. |
| Approach: | They propose a model based on convolutional neural networks and gating mechanisms which can selectively output the sentiment features according to the given aspect or entity. |
| Outcome: | The proposed model can selectively output sentiment features according to the given aspect or entity. |
A Helping Hand: Transfer Learning for Deep Sentiment Analysis (P18-1)
Copied to clipboard
| Challenge: | Existing deep neural models for sentiment polarity classification require large amounts of training data. |
| Approach: | They propose to feed generic cues into the training process of deep convolutional neural networks for sentiment analysis. |
| Outcome: | The proposed approach improves sentiment polarity classification on a range of datasets in seven languages. |
Cold-Start Aware User and Product Attention for Sentiment Classification (P18-1)
Copied to clipboard
| Challenge: | Existing models do not deal with cold-start problem typical in review websites. |
| Approach: | They propose a Hybrid Contextualized Sentiment Classifier that uses word encoder and Cold-Start Aware Attention to pool word vectors. |
| Outcome: | The proposed model performs significantly better on famous datasets despite having less complexity and can be trained much faster. |
Modeling Deliberative Argumentation Strategies on Wikipedia (P18-1)
Copied to clipboard
| Challenge: | Existing models for deliberative discussions have been built manually based on a small set of discussions, resulting in a level of abstraction that is not suitable for move recommendation. |
| Approach: | They propose to model argumentation strategies of deliberative discussions by annotating ongoing discussions with a label that can be used for move description. |
| Outcome: | The proposed model can predict arguments of participants in deliberative discussions using metadata from Wikipedia talk pages. |
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning (P18-1)
Copied to clipboard
| Challenge: | Practical applications of automatic image description systems include leveraging descriptions for image indexing or retrieval, and helping those with visual impairments by transforming visual signals into information that can be communicated via text-to-speech technology. |
| Approach: | They propose to extract and filter image caption annotations from billions of webpages and use them to train models. |
| Outcome: | The proposed model architectures perform better when trained on the Conceptual Captions dataset. |
Learning Translations via Images with a Massively Multilingual Image Dataset (P18-1)
Copied to clipboard
| Challenge: | Existing datasets for learning translations of words are limited to a few high-resource languages and unrealistically easy settings. |
| Approach: | They propose a large-scale multilingual corpus of images labeled with the word they represent to facilitate translation research. |
| Outcome: | The proposed method improves on an unsupervised technique that has been limited to a few languages and unrealistic settings. |
On the Automatic Generation of Medical Imaging Reports (P18-1)
Copied to clipboard
| Challenge: | a complete medical imaging report contains multiple heterogeneous forms of information, including findings and tags . abnormal regions in medical images are difficult to identify and the reports are typically long, containing multiple sentences. |
| Approach: | They propose a multi-task learning framework which predicts tags and generates paragraphs for abnormal regions in medical images. |
| Outcome: | The proposed framework can generate long paragraphs on two publicly available datasets. |
Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning (P18-1)
Copied to clipboard
| Challenge: | Visual language grounding is widely studied in modern neural image captioning systems . a novel algorithm for crafting adversarial examples in image captions is proposed . |
| Approach: | They propose an algorithm to craft adversarial examples in machine vision and perception . their approach provides two evaluation approaches to check if they can mislead systems . |
| Outcome: | The proposed algorithm can craft visually-similar adversarial examples with randomly targeted captions or keywords, and the results are transferable to other image captioning systems. |
Think Visually: Question Answering through Virtual Imagery (P18-1)
Copied to clipboard
| Challenge: | Existing models of geometric reasoning are based on visual representations of objects and objects, but they are not based in symbols or words. |
| Approach: | They propose a new deep network architecture that specializes in answering questions that admit latent visual representations and learns to generate and reason over such representations. |
| Outcome: | The proposed model can generate and reason over latent visual representations and is validated by two synthetic benchmarks. |
Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game (P18-1)
Copied to clipboard
| Challenge: | supervised language learning is limited by the ability of capturing mainly the statistics of training data. |
| Approach: | They propose to use conversational games to train agents to use new knowledge . they propose to mimic and reinforce conversational game and use it in one-shot fashion . |
| Outcome: | The proposed approach is able to acquire information by asking questions about novel objects and use the just-learned knowledge in subsequent conversations in a one-shot fashion. |
A Purely End-to-End System for Multi-speaker Speech Recognition (P18-1)
Copied to clipboard
| Challenge: | Existing methods for multi-speaker speech recognition require isolated source signals or senone alignments for effective learning. |
| Approach: | They propose a sequence-to-sequence framework to decode multiple label sequences from a single speech sequence by unifying source separation and speech recognition functions in an end-to end manner. |
| Outcome: | The proposed model improves on existing models by 83.1% relative to previous models with explicit separation and recognition modules. |
A Structured Variational Autoencoder for Contextual Morphological Inflection (P18-1)
Copied to clipboard
| Challenge: | morphological inflectors typically trained on fully supervised, type-level data, but how can we improve their performance? et al., 2016: a novel latent-variable model for semi-supervised learning of inflection generation. |
| Approach: | They propose a latent-variable model for semi-supervised learning of inflection generation . they use a wake-sleep algorithm to enable posterior inference over latent variables . |
| Outcome: | The proposed model improves on 23 languages and shows 10% accuracy improvement . the proposed model is based on the wake-sleep algorithm . |
Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings (P18-1)
Copied to clipboard
| Challenge: | recurrent neural networks have produced significant advances in part-of-speech tagging accuracy . a common feature of these models is the presence of rich initial word encodings . however, word or sub-word information interacts only through subsequent recursive layers . |
| Approach: | They propose to use recurrent neural networks with sentence-level context for initial character and word-based representations. |
| Outcome: | The proposed model has the highest accuracy of all participating systems in the CoNLL 2017 task. |
Neural Factor Graph Models for Cross-lingual Morphological Tagging (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to morphological tagging are limited by the assumption that tag sets overlap . a limited amount of data is available for most languages to learn these morphology taggers. |
| Approach: | They propose a method for cross-lingual morphological tagging that relaxes this assumption . they use factorial conditional random fields with neural network potentials to smooth over superficial differences in the surface forms . |
| Outcome: | The proposed model can smooth over superficial differences in the surface forms and generate unseen or rare tag sets. |
Global Transition-based Non-projective Dependency Parsing (P18-1)
Copied to clipboard
| Challenge: | Until recently, transition-based dependency parsers were limited to approximate inference due to their incompatibility with rich feature models. |
| Approach: | They propose a transition-based parser with high coverage on non-projective treebanks to support non- projective parsing. |
| Outcome: | The proposed approach is more efficient than its projective counterpart in non-projective languages. |
Constituency Parsing with a Self-Attentive Encoder (P18-1)
Copied to clipboard
| Challenge: | Recent work on LSTM encoders based on recurrent neural networks has led to improvements in constituency parsing accuracy. |
| Approach: | They propose to replace an LSTM encoder with a self-attentive architecture to improve a discriminative constituency parser. |
| Outcome: | The proposed model outperforms the previous best-published results on 8 of the 9 languages in the SPMRL dataset. |
Pre- and In-Parsing Models for Neural Empty Category Detection (P18-1)
Copied to clipboard
| Challenge: | Existing studies on empty category detection have shown positive effects on syntactic parsing . empty categories are used to indicate long-distance dependencies, discontinuous constituents, and certain dropped elements. |
| Approach: | They propose to use ECD to detect empty categories without syntactic analysis. |
| Outcome: | The proposed models outperform the prior state-of-the-art by significant margins. |
Composing Finite State Transducers on GPUs (P18-1)
Copied to clipboard
| Challenge: | Weighted finite state transducers (FSTs) are used in language processing . a GPU implementation of the composition operation is currently under development . |
| Approach: | They propose a GPU implementation of the composition operation for weighted finite state transducers. |
| Outcome: | The proposed approach achieves speedups of up to 6 times over the serial implementation and 4.5 times over OpenFST on the GPU. |
Supervised Treebank Conversion: Data and Approaches (P18-1)
Copied to clipboard
| Challenge: | Existing work on treebank conversion focuses on unsupervised treebanks . lack of manually labeled data means that sentences have two syntactic trees at the same time. |
| Approach: | They propose supervised treebank conversion using bi-tree aligned sentences . they propose two conversion approaches based on state-of-the-art deep biaffine parser . |
| Outcome: | The proposed method outperforms the state-of-the-art deep biaffine parser on the English WSJ dataset by 0.97 (93.76% -92.79%) |
Object-oriented Neural Programming (OONP) for Document Understanding (P18-1)
Copied to clipboard
| Challenge: | Object-oriented Neural Programming (OONP) is a framework for semantically parsing documents in domains. |
| Approach: | They propose a framework for semantically parsing documents in specific domains using OONP . OOPN parsers use a rich family of operations to represent the semantics of the document . |
| Outcome: | The proposed framework can learn to handle fairly complicated ontology with training data of modest sizes. |
Finding syntax in human encephalography with beam search (P18-1)
Copied to clipboard
| Challenge: | RNNGs are generative models of (tree , string ) pairs that evaluate derivational choices . a non-syntactic neural language model yields no reliable effects . |
| Approach: | They propose to combine a probabilistic generative grammar with a parsing procedure that uses it to manage syntactic derivations as it advances from one word to the next. |
| Outcome: | The proposed model derives two amplitude effects when used against human encephalography data. |
Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information (P18-1)
Copied to clipboard
| Challenge: | StackExchange users routinely ask clarifying questions to fill information gaps . a principle goal of asking questions is to fill this information gap . |
| Approach: | They build a model to rank candidates by their usefulness to a given post . they use data from StackExchange to evaluate the model against human judgments . |
| Outcome: | The proposed model outperforms baselines on 500 samples of StackExchange's clarification questions. |
Let’s do it “again”: A First Computational Approach to Detecting Adverbial Presupposition Triggers (P18-1)
Copied to clipboard
| Challenge: | a novel task of predicting adverbial presupposition triggers is useful for natural language generation . a focus is on a new attention mechanism for predicting presuposition trigger . |
| Approach: | They propose a new attention mechanism for predicting adverbial presupposition triggers . they propose to augment a baseline neural network without additional trainable parameters . |
| Outcome: | The proposed model outperforms baseline models in predicting adverbial presupposition triggers. |