Papers with LSTM
Word Acquisition in Neural Language Models (2022.tacl-1)
Copied to clipboard
| Challenge: | Language models acquire individual words during training, based on unigram token frequencies, before transitioning loosely to bigram probabilities, eventually converging on more nuanced predictions. |
| Approach: | They examine how neural language models acquire individual words during training, extracting learning curves and ages of acquisition for over 600 words on the MacArthur-Bates Communicative Development Inventory. |
| Outcome: | The models follow consistent patterns during training for both unidirectional and bidirectional models, and for both LSTM and Transformer architectures. |
Neural Mention Detection (2020.lrec-1)
Copied to clipboard
| Challenge: | Mention detection is an important preprocessing step for downstream applications such as NER and coreference resolution. |
| Approach: | They propose and compare three approaches to mention detection using ELMO embeddings and a biaffine classifier. |
| Outcome: | The proposed model outperforms state-of-the-art models on the GENIA corpora and improves on mention recall. |
Neural language models as psycholinguistic subjects: Representations of syntactic state (N19-1)
Copied to clipboard
| Challenge: | a recent study examines the extent to which neural network language models reflect incremental representations of syntactic state . we examine neural network model behavior on sentences chosen to probe specific aspects of the learned representations . |
| Approach: | They employ experimental methodologies developed in psycholinguistics to study syntactic representation in the human mind. |
| Outcome: | The proposed models are trained on large datasets and only sensitive to subtle cues . the results raise questions about the accuracy of the models and their performance . |
Automatic detection of dyslexia based on eye movements during reading in Russian (2025.acl-short)
Copied to clipboard
| Challenge: | Existing screening tests for dyslexia are time- and resource-consuming . early diagnosis is key for learning disabilities, but eye tracking technology is promising . |
| Approach: | They propose to automatically classify dyslexia based on eye movements recorded during natural reading combined with basic demographic information and linguistic features. |
| Outcome: | The proposed model outperforms the state-of-the-art model by 7 % and has an AUC of 0.93 . the focus features matter the most for classification, the authors show . |
An Interactive Web-Interface for Visualizing the Inner Workings of the Question Answering LSTM (D18-2)
Copied to clipboard
| Challenge: | Existing visualisation methods for deep learning models are limited by their low interpretability and lack a tool for interpreting them. |
| Approach: | They propose a visualisation tool which plots heatmaps of neurons’ firings and allows a user to check the dependency between neurons and manual features. |
| Outcome: | The proposed visualisation tool plots heatmaps of neurons’ firings and allows a user to check the dependency between neurons and manual features. |
Document Representation Learning for Patient History Visualization (C18-2)
Copied to clipboard
| Challenge: | In medicine, selecting relevant reports from a large database is time-consuming and could result in overlooking important information. |
| Approach: | They propose a representation learning algorithm that creates a semantic representation space for documents where the clinically related documents lie close to each other. |
| Outcome: | The proposed model can be used to generate a diagrammatic summary of a set of documents each of which pertains to loosely related topics. |
A Neural Pipeline Approach for the PharmaCoNER Shared Task using Contextual Exhaustive Models (D19-57)
Copied to clipboard
| Challenge: | NER and concept indexing perform named entity recognition and concept identifiers (CUIs) in a knowledge base. |
| Approach: | They propose a neural pipeline approach that performs named entity recognition (NER) and concept indexing (CI) they use bi-LSTM to capture the semantic information of a sequence and classify them into entities or no entities . |
| Outcome: | The proposed approach performs named entity recognition (NER) and concept indexing (CI) which links them to concept unique identifiers (CUIs) in a knowledge base. |
Siamese Neural Networks for Detecting Complementary Products (2021.eacl-srw)
Copied to clipboard
| Challenge: | e-commerce websites are often too overwhelming for users to find what they need at one place . a recommendation system can detect complementary products using only the purchase history . |
| Approach: | They propose a content-based recommender system for detecting complementary products using Siamese Neural Networks (SNN). |
| Outcome: | The proposed system detects complementary products with 85% accuracy using only the product titles. |
Neural Architectures for Fine-Grained Propaganda Detection in News (D19-50)
Copied to clipboard
| Challenge: | MIC-CIS is a fine grained propaganda detection system . previous work focused on document level, labeling articles as propaganda . |
| Approach: | They propose to use different neural architectures to jointly perform propaganda detection tasks . they also investigate different ensemble schemes such as majority-voting, relax-vote, etc. |
| Outcome: | The proposed system performs sentences and fragment level propaganda detection tasks. |
RNN Simulations of Grammaticality Judgments on Long-distance Dependencies (C18-1)
Copied to clipboard
| Challenge: | LSTM networks can detect linguistic structures which are ungrammatical due to extraction violations, but are sensitive to linguistic processing factors. |
| Approach: | They propose to use LSTM networks to detect ungrammatical sentences by detecting extra arguments and subject-relative clause island violations. |
| Outcome: | The proposed model can correctly classify (un)grammatical sentences, in certain conditions, but is sensitive to linguistic processing factors and unable to induce a more abstract notion of grammaticality. |
NCRF++: An Open-source Neural Sequence Labeling Toolkit (P18-4)
Copied to clipboard
| Challenge: | Existing statistical approaches to neural sequence labeling have been used for many tasks. |
| Approach: | They describe a toolkit for neural sequence labeling that provides a CRF inference layer for quick implementation. |
| Outcome: | The toolkit is based on PyTorch and can be run on GPUs. |
Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examples (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to detect adversarial examples are limited by the nature of these examples. |
| Approach: | They propose a frequency-guided word substitution algorithm that exploits adversarial word substitutions for the detection of adversarials. |
| Outcome: | The proposed algorithm outperforms existing detection methods by 13.0% on the SST-2 and IMDb sentiment datasets. |
Affection Driven Neural Networks for Sentiment Analysis (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing deep neural network models lack mechanisms to highlight important sentiment terms. |
| Approach: | They propose a method to incorporate affective knowledge into deep neural network models by mapping affective influence vectors to an affective impact value and integrating them into long-term memory models to highlight affective terms. |
| Outcome: | The proposed approach improves on three large datasets by 1.0% to 1.5% on the benchmark datasets. |
Feature Difference Makes Sense: A medical image captioning model exploiting feature difference and tag information (2020.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for medical image captioning are limited and lack diversity . current methods do not generalize well when applied to unfamiliar images . |
| Approach: | They propose a feature difference and tag information combined long short-term memory model for chest x-ray report generation. |
| Outcome: | The proposed model outperforms existing models in chest x-ray report generation. |
Incorporating Image Matching Into Knowledge Acquisition for Event-Oriented Relation Recognition (C18-1)
Copied to clipboard
| Challenge: | Event relation recognition is a challenging language processing task because the query events are selected from different paragraphs in a document or even different documents, so there is lack of explicit clue. |
| Approach: | They propose to use image processing to acquire similar event instances and use image matching to approximate calculation between events. |
| Outcome: | The proposed model performs comparable to CNN while slightly better than LSTM on the ACE-R2 corpus. |
On Sentence Representations for Propaganda Detection: From Handcrafted Features to Word Embeddings (D19-50)
Copied to clipboard
| Challenge: | The rise of fake and hyperpartisan news on social media and online news outlets calls for improved automatic detection of propaganda in texts. |
| Approach: | They propose to use handcrafted features and learn dense semantic representations to detect propaganda in sentence-level and with random undersampling of the majority class (non-propaganda) |
| Outcome: | The proposed system achieves a ranking of 10 among 25 participants, with 59.5 F1-score. |
Building a De-identification System for Real Swedish Clinical Text Using Pseudonymised Clinical Text (D19-62)
Copied to clipboard
| Challenge: | Several ethical and scientific issues arise regarding the balance between maintaining patient confidentiality and the need for wider application of trained models. |
| Approach: | They propose to use pseudonymised clinical text as training data to de-identify real clinical text in other hospitals. |
| Outcome: | The proposed model performed better for some PHI information than the standard model and poor performance on Location and Health Care Unit information. |
Morphology Matters: A Multilingual Language Modeling Analysis (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing studies on inflectional morphology disagree on whether or not it makes languages harder to model. |
| Approach: | They propose to use a corpus of 145 Bible translations in 92 languages to investigate whether inflectional morphology makes languages harder to model. |
| Outcome: | The proposed model trains with linguistically motivated subword segmentation strategies and reduces the impact of morphology on language modeling. |
Comparison of Grammatical Error Correction Using Back-Translation Models (2021.naacl-srw)
Copied to clipboard
| Challenge: | Currently, a mainstream approach to generate pseudo data is back-translation (BT). |
| Approach: | They propose to use back-translation to generate pseudo data that contains grammatical and ungrammatically produced sentences. |
| Outcome: | The proposed methods improve or interpolate the performance of each error type compared with a single BT model with different seeds. |
YNU-junyi in BioNLP-OST 2019: Using CNN-LSTM Model with Embeddings for SeeDev Binary Event Extraction (D19-57)
Copied to clipboard
| Challenge: | BioNLP 2019 Shared Tasks: binary relation extraction of SeeDev task . Biological information extraction (Bio-IE) is a new field of research . |
| Approach: | They propose to use convolutional neural networks and long short term memory networks to construct a binary relation extraction model. |
| Outcome: | The proposed method performed well in the binary relation extraction task. |
Incorporating EDS Graph for AMR Parsing (2021.starsem-1)
Copied to clipboard
| Challenge: | AMR is abstract and conceptual, while EDS is low level, closer to the lexical structures of the given sentences. |
| Approach: | They propose to add EDS graphs as additional semantic features to AMR parsers by adding transition-based parser to add LSTM layer and GCN layer. |
| Outcome: | The proposed parser adds EDS graphs as additional semantic features to boost performance . Currently the parsing accuracies for AMR are in low 80s, while they can be improved by adding more information from EDS. |
Bag of Experts Architectures for Model Reuse in Conversational Language Understanding (N18-3)
Copied to clipboard
| Challenge: | Slot tagging is a key component of natural language understanding systems for personal digital assistants. |
| Approach: | They propose to use a bag of experts architecture to reuse domain data for slot tagging models. |
| Outcome: | Experiments with 10 domains show that the proposed models outperform baseline models by 5.06% and 12.16% when training with only 25% of the training data. |
Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection (N18-2)
Copied to clipboard
| Challenge: | Existing methods that focus on a single tweet as input are likely to yield high false positive and negative rates. |
| Approach: | They propose a model that leverages intra-user and inter-user representation learning to improve hate speech detection on Twitter by suppressing the noise in a single Tweet. |
| Outcome: | The proposed model significantly improves the f-score of a strong bidirectional LSTM model by 10.1%. |
TeluguNER: Leveraging Multi-Domain Named Entity Recognition with Deep Transformers (2022.acl-srw)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a successful and well-researched problem in English due to the availability of resources. |
| Approach: | They propose to use two annotated NER datasets for the Telugu language . they compare the finetuned Telugus model with the existing model in NER . |
| Outcome: | The proposed models outperform existing models on a large dataset of 38,363 sentences on telugu and other languages. |
Mixed Feelings: Natural Text Generation with Variable, Coexistent Affective Categories (P18-3)
Copied to clipboard
| Challenge: | a recent study has shown that language models which can generate emotional sentences are limited to one affective category out of a few. |
| Approach: | a new research proposal proposes a language model which can produce multiple emotions simultaneously. authors propose to use a long-term memory language model to allow for variation in multiple emotions. |
| Outcome: | a new language model allows for variation in multiple emotions simultaneously . the proposed model is based on a model of long-term memory . |
NSIT@NLP4IF-2019: Propaganda Detection from News Articles using Transfer Learning (D19-50)
Copied to clipboard
| Challenge: | In this paper, we describe our approach and system description for NLP4IF 2019 Workshop: Shared Task on Fine-Grained Propaganda Detection. |
| Approach: | They propose to use document Embeddings and LSTM to detect whether a sentence contains a propagandistic agenda. |
| Outcome: | The proposed approach ranked 21st in the NLP4IF 2019 Workshop: Shared Task on Fine-Grained Propaganda Detection. |
Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | a self-attention network can be easily parallelized at sequence level, but LSTMs are slower to train . a recent study shows that LS models require a lot of computations to perform . |
| Approach: | They propose to compute LSTMs at sequence level to enable sequence-level parallelization . they use a bag-of-words representation of the preceding tokens context to approximate LStms . |
| Outcome: | The proposed model performs better than existing models while being faster to train . the model can be trained efficiently due to the highly parallelized self-attention network . |
BLiMP: The Benchmark of Linguistic Minimal Pairs for English (2020.tacl-1)
Copied to clipboard
Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, Samuel R. Bowman
| Challenge: | Recent studies have examined how linguistic knowledge of language models (LMs) varies across English phenomena. |
| Approach: | They propose a benchmark to evaluate linguistic knowledge of language models on major grammatical phenomena in English. |
| Outcome: | The proposed benchmark evaluates the linguistic knowledge of language models on major grammatical phenomena in English. |
T2NER: Transformers based Transfer Learning Framework for Named Entity Recognition (2021.eacl-demos)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is an important task in information extraction due to large variations in entity names and flexibility in how entities are mentioned. |
| Approach: | They propose a Transformers based Transfer Learning framework for Named Entity Recognition (T2NER) that integrates transformer models with the state-of-the-art in NLP and provides a unified platform for transfer learning. |
| Outcome: | The proposed framework bridges the gap between the state-of-the-art in transformer models and the state of the art in NER with deep transformer models. |
Building Accurate Low Latency ASR for Streaming Voice Search in E-commerce (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent years have witnessed the popularity of end-to-end ASR models, which have demonstrated higher accuracy compared to traditional pipelines with separate acoustic, pronunciation, and language models. |
| Approach: | They build accurate LSTM, attention and CTC based streaming ASR models for large-scale Hinglish voice search. |
| Outcome: | The proposed model achieves a word error rate (WER) of 3.69% without EOS and 4.78% with EOS, with 1300 ms (46.64%) reduction in latency. |
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context (P18-1)
Copied to clipboard
| Challenge: | Recent studies have shed light on the information encoded by long-term memory networks. |
| Approach: | They propose to use a neural caching model to model the role of context in an LSTM LM . they analyze the increase in perplexity when prior context words are shuffled, replaced, or dropped . |
| Outcome: | The proposed model is highly sensitive to the order of words within the most recent sentence, but ignores word order in the long-range context, suggesting the distant past is modeled only as a rough semantic field or topic. |
Multiple Character Embeddings for Chinese Word Segmentation (P19-2)
Copied to clipboard
| Challenge: | Chinese word segmentation is regarded as character-based sequence labeling task in most current work but it neglects important fact: Chinese characters contain both semantic and phonetic meanings. |
| Approach: | They propose a shared bi-LSTM-CRF model which fuses linguistic features efficiently by sharing the LSTM network during the training procedure. |
| Outcome: | The proposed model achieves state-of-the-art in AS and CityU corpora without external lexical resources. |
How much complexity does an RNN architecture need to learn syntax-sensitive dependencies? (2020.acl-srw)
Copied to clipboard
| Challenge: | Long-term memory (LSTM) networks are capable of encapsulating long-range dependencies . but simple recurrent networks (SRNs) have been less successful at capturing long-term dependencies and loci of grammatical errors in an unsupervised setting. |
| Approach: | They propose a new architecture that incorporates the decaying nature of neuronal activations and models the excitatory and inhibitory connections in a population of neurons. |
| Outcome: | The proposed architecture shows competitive performance relative to LSTMs on subject-verb agreement, sentence grammaticality, and language modeling tasks. |
Ad Headline Generation using Self-Critical Masked Language Model (2021.naacl-industry)
Copied to clipboard
| Challenge: | We propose a programmatic solution to generate product advertising headlines using retail content. |
| Approach: | They propose a programmatic solution to generate product advertising headlines using retail content . they use Reinforcement Learning (RL) Policy gradient methods on Transformer . |
| Outcome: | The proposed method outperforms existing methods in overlap metrics and quality audits. |
Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition (2021.tacl-1)
Copied to clipboard
| Challenge: | Name Regularity Bias is a problem in NER models that use contextual information to predict the type of an ambiguous entity. |
| Approach: | They propose a model-agnostic training method that adds learnable adversarial noise to some entity mentions to improve their accuracy. |
| Outcome: | The proposed method outperforms feature-based models on name regularity bias . it adds learnable adversarial noise to some entity mentions, leading to gains . |
How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN (2023.tacl-1)
Copied to clipboard
| Challenge: | Current language models generate high-quality text, but are they copying it or have they learned generalizable linguistic abstractions? |
| Approach: | They propose a suite of analyses for assessing the novelty of generated text . they focus on sequential structure (n-grams) and syntactic structure (syntactical structure). |
| Outcome: | The proposed model-generated text is as novel as the baseline human-generated model- generated text, but it is copied substantially, the authors show . |
Sentiment Analysis of Yelp Review Dataset: A Comparative Study of Machine Learning Methods (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for sentiment analysis are inconsistent and require manual processing. |
| Approach: | They use natural language processing and machine learning to classify Yelp reviews' sentiments. |
| Outcome: | The proposed model outperforms other models on Yelp reviews. |
Examining the Inductive Bias of Neural Language Models with Artificial Languages (2021.acl-long)
Copied to clipboard
| Challenge: | a novel method for investigating inductive biases of language models using artificial languages is proposed . we show that modern neural architectures used for language modeling are intrinsically black boxes . |
| Approach: | They propose a method to investigate inductive biases of language models using artificial languages . they use languages to create parallel corpora across languages that differ only in word order . |
| Outcome: | The proposed method shows that language models can be used to model a wide variety of languages. |
LLM-supertagger: Categorial Grammar Supertagging via Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that LLMs are underperforming in classification tasks due to their decoder-based nature. |
| Approach: | They propose a method that significantly boosts LLMs' performance in supertagging for both Combinatory Categorial Grammar (CCG) and Lambek Categorian Grammar (LCG). |
| Outcome: | The proposed method outperforms LSTM and encoder-based models and achieves state-of-the-art performance. |
Transformer versus LSTM Language Models trained on Uncertain ASR Hypotheses in Limited Data Scenarios (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies show that domain-specific LMs can only rely on limited in-domain speech data . a qualitative analysis reveals that Transformer LM can predict less frequent words . |
| Approach: | They propose a method to train Transformer LMs on ASR confusion networks . they find they are better at exploiting alternate uncertain ASR hypotheses . |
| Outcome: | The proposed method reduces perplexity by 3-6% on AMI scenarios but performs similar to LSTM LMs on Verbmobil conversational corpus. |
A Formal Hierarchy of RNN Architectures (2020.acl-main)
Copied to clipboard
| Challenge: | Existing theories of expressive power of RNNs are limited. |
| Approach: | They propose a formal hierarchy of the expressive capacity of RNN architectures based on two formal properties: space complexity and rational recurrence. |
| Outcome: | The proposed model is based on the theory of “saturated” RNNs and shows that it obeys a similar hierarchy to unsaturated RNN models. |
Context-Aware Neural Model for Temporal Information Extraction (P18-1)
Copied to clipboard
| Challenge: | Existing temporal information extraction systems rely on statistical learning with feature-engineered task-specific models. |
| Approach: | They propose a context-aware neural network model for temporal information extraction using a global context layer. |
| Outcome: | The proposed model outperforms existing models in terms of performance and performance . it is the first model to use NTM-like architecture to process the information from global context in discourse-scale natural text processing. |
Investigating grammatical abstraction in language models using few-shot learning of novel noun gender (2024.findings-eacl)
Copied to clipboard
| Challenge: | a new study shows that language models can generalise novel noun gender from one to two learning examples and apply it across agreement contexts. |
| Approach: | They conduct a noun learning experiment to assess whether a transformer and an LSTM can achieve human-like abstraction of grammatical gender in French. |
| Outcome: | The proposed models generalise gender from one to two learning examples and apply gender across agreement contexts, albeit with a bias for the masculine gender category. |
Solving Data Sparsity for Aspect Based Sentiment Analysis Using Cross-Linguality and Multi-Linguality (N18-1)
Copied to clipboard
| Challenge: | Efficient word representations play an important role in solving various problems related to NLP, data mining, text mining etc. |
| Approach: | They propose to leverage bilingual word embeddings learned through a parallel corpus to minimize the effect of data sparsity. |
| Outcome: | The proposed model is tested against state-of-the-art methods in two experimental setups. |
From Language to Language-ish: How Brain-Like is an LSTM’s Representation of Nonsensical Language Stimuli? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | LSTMs are often used to measure event related potentials, but are they able to generalize to new data in a human-like way? |
| Approach: | They asked whether an LSTM model represents a language sample with degraded semantic or syntactic information and whether it resembles the brain's reaction to the stimuli. |
| Outcome: | The results suggest that LSTMs and human brain handle nonsensical data similarly. |
Top-down Discourse Parsing via Sequence Labelling (2021.eacl-main)
Copied to clipboard
| Challenge: | Discourse analysis is a systematic way to understand how texts are segmented hierarchically into discourse units. |
| Approach: | They propose a top-down approach to discourse parsing that is conceptually simpler than its predecessors. |
| Outcome: | The proposed model eliminates the decoder and reduces the search space for splitting points. |
Does Higher Order LSTM Have Better Accuracy for Segmenting and Labeling Sequence Data? (C18-1)
Copied to clipboard
| Challenge: | Existing neural models take long distance dependencies into account when predicting the tag of the current token. |
| Approach: | They propose a method to capture long distance tag dependencies and use them for dependency analysis. |
| Outcome: | The proposed model can predict multiple tags for the current token without taking dependencies between tags into account. |
“Will You Find These Shortcuts?” A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on faithfulness evaluation is not conclusive and does not provide a clear answer as to how different methods are to be compared. |
| Approach: | They propose a protocol for faithfulness evaluation that makes use of partially synthetic data to obtain ground truth for feature importance ranking. |
| Outcome: | The proposed method is based on partially synthetic data and is compared with lexical shortcuts on a range of datasets and LSTM models. |
Provably Confidential Language Modelling (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to train language models without memorizing sensitive data are mismatched and can be difficult to screen and filter. |
| Approach: | They propose a method to train language generation models while protecting the confidential segments of training data. |
| Outcome: | The proposed method prevents unintended memorization by randomizing parts of the training process while protecting strong confidentiality. |
Augmenting Small Data to Classify Contextualized Dialogue Acts for Exploratory Visualization (2020.lrec-1)
Copied to clipboard
| Challenge: | a new corpus of conversations is being developed to support data visualization exploration . we use data augmentation to improve our methods for dialogue act classification . |
| Approach: | They propose to use a corpus of conversations to annotate contextualized dialogue acts . they highlight how thinking aloud affects interpretation of dialogue acts in the context . |
| Outcome: | The proposed AI can support visualization exploration with a small corpus of conversations . the proposed AI outperforms existing models in terms of performance and performance . |
Chinese Relation Classification using Long Short Term Memory Networks (L18-1)
Copied to clipboard
| Challenge: | Relation classification is the task to predict semantic relations between pairs of entities in a given text. |
| Approach: | They propose to extract relations between entities in Chinese text using a long-term memory network. |
| Outcome: | The proposed system achieves state-of-the-art F-measure on ACE 2005 corpus . it predicts relations between head entity e h and tail entity t from sentence . |
BanglaBook: A Large-scale Bangla Dataset for Sentiment Analysis from Book Reviews (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing literature on Bangla Sentiment Analysis (SA) has limited data and cross-domain adaptability. |
| Approach: | They present a large-scale dataset of Bangla book reviews with 158,065 samples . they employ a range of machine learning models to establish baselines including SVM, LSTM, and Bangla-BERT. |
| Outcome: | The proposed model improves performance over models that rely on manual features. |
Sense-Aware Neural Models for Pun Location in Texts (P18-2)
Copied to clipboard
| Challenge: | Puns where the two meanings share the same pronunciation are known as homographic puns. |
| Approach: | They propose a sense-aware neural model to address the task of pun location . they first obtain several WSD results for the text and then leverage a bidirectional LSTM network to model each word senses. |
| Outcome: | The proposed model is based on a SemEval 2017 benchmark dataset showing that it can predict homographic puns. |
Transition-based Parsing with Stack-Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing parsing systems use local or global models of the parser state to improve performance. |
| Approach: | They propose to modify the sequence-to-sequence Transformer to model global or local parser states in transition-based parsing. |
| Outcome: | The proposed model significantly improves performance on dependency and Abstract Meaning Representation (AMR) parsing tasks. |
Which Model Should We Use for a Real-World Conversational Dialogue System? a Cross-Language Relevance Model or a Deep Neural Net? (2020.lrec-1)
Copied to clipboard
| Challenge: | End-to-end neural network models of conversational dialogue are popular for conversational tasks, but there are still questions about how well they work for real applications and how much data is needed to achieve acceptable performance. |
| Approach: | They compare two different kinds of end-to-end dialogue models based on cross-language relevance and cross-linguistic LSTM models for corpus-based selection of dialogue responses. |
| Outcome: | The proposed models perform well on a large corpus, but are dominated by a more moderate-sized corpus. |
Exploiting Document Knowledge for Aspect-level Sentiment Classification (P18-2)
Copied to clipboard
| Challenge: | Existing public aspect-level datasets for aspect-based sentiment classification are small . existing methods for aspect level sentiment classification require annotation of all opinion targets . |
| Approach: | They propose two approaches that transfer knowledge from document-level data to improve aspect-level sentiment classification. |
| Outcome: | The proposed methods improve aspect-level sentiment classification on 4 public datasets. |
Code Summarization with Structure-induced Transformer (2021.findings-acl)
Copied to clipboard
| Challenge: | Code summarization (CS) is a promising area in recent language understanding . previous work using structurebased traversal or non-sequential models to learn structural program semantics has shown no performance gain . |
| Approach: | They propose to use a structure-based traversal model to learn structural program semantics to generate human language automatically for programming language in the format of source code. |
| Outcome: | Experiments show that the proposed method achieves state-of-the-art on benchmarks. |
Effective Attention Modeling for Aspect-Level Sentiment Classification (C18-1)
Copied to clipboard
| Challenge: | Aspect-level sentiment classification aims to determine sentiment polarity of review sentence towards opinion target . main challenge is to separate different opinion contexts for different targets . |
| Approach: | They propose a method that captures the semantic meaning of the opinion target and a model that incorporates syntactic information into the attention mechanism. |
| Outcome: | The proposed method captures the semantic meaning of the opinion target and incorporates syntactic information into the attention mechanism. |
Generating Token-Level Explanations for Natural Language Inference (N19-1)
Copied to clipboard
| Challenge: | Existing methods to generate token-level explanations for NLI on single sentences have not been tested. |
| Approach: | They propose to generate token-level explanations for NLI without explicitly annotating training data. |
| Outcome: | The proposed approach is faster and more accurate than the black-box methods. |
Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to attack text classification are limited due to the complexity of the problem. |
| Approach: | They propose a greedy algorithm to generate adversarial examples that maintain lexical correctness, grammatical correctiness and semantic similarity. |
| Outcome: | The proposed algorithm maintains lexical correctness, grammatical correctity and semantic similarity well and is hard for humans to perceive. |
A Deep Neural Network Sentence Level Classification Method with Context Information (D18-1)
Copied to clipboard
| Challenge: | Existing methods that use context for sentence classification are difficult to scale . Usually, sentences are treated as separate instances for the task . however, in many situations the sentence that is the focus of classification appears in a context that can provide additional information. |
| Approach: | They propose a method that uses potentially large contexts to classify sentences . they use an LSTM, and short-span features to classize sentences based on a stacked CNN . |
| Outcome: | The proposed method consistently improves on two different datasets. |
Pivot Based Language Modeling for Improved Neural Domain Adaptation (N18-1)
Copied to clipboard
| Challenge: | Existing work on domain adaptation does not exploit the structure of the input text . PBLM can naturally feed structure aware text classifiers such as LSTM and CNN . |
| Approach: | They propose a model that integrates pivot-based and NN modeling in a structure aware manner. |
| Outcome: | The proposed model can naturally feed structure aware text classifiers such as LSTM and CNN. |
Tensor Product Generation Networks for Deep NLP Modeling (N18-1)
Copied to clipboard
| Challenge: | Using Tensor Product Representations (TPRs) we propose a new architecture for natural language processing based on the principle that hypothesis space for learning includes network hypotheses that are independently known to be suitable for performing the target task. |
| Approach: | They propose a Tensor Product Generation Network (TPGN) which is capable of carrying out TPR computation but uses unconstrained deep learning to design its internal representations. |
| Outcome: | The proposed architecture outperforms baselines on the COCO dataset and can interpret internal representations and operations. |
Neural Sequence Segmentation as Determining the Leftmost Segments (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to segment sentences are mostly at token level, limiting their full potential to capture long-term dependencies. |
| Approach: | They propose a framework that incrementally segments natural language sentences at segment level. |
| Outcome: | The proposed framework outperforms baseline methods on syntactic chunking and Chinese part-of-speech tagging datasets. |
Examining the Ordering of Rhetorical Strategies in Persuasive Requests (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Numerous studies have been conducted to understand persuasiveness of text, from explorations of rhetoric in presidential campaigns to the impact of a communicator's likability on persuasiveness. |
| Approach: | They use a Variational Autoencoder model to disentangle content and rhetorical strategies in textual requests from a large-scale loan request corpus and visualize interplay between content and strategy through an attentional LSTM that predicts the success of textual request. |
| Outcome: | The proposed model disentangles content and rhetorical strategies in textual requests from a large-scale loan request corpus and visualizes interplay between content and strategy through attentional LSTM that predicts the success rate of textual request. |
Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum (P18-2)
Copied to clipboard
| Challenge: | LSTMs were introduced to combat vanishing gradients in simple RNNs by augmenting them with gated additive recurrent connections. |
| Approach: | They propose to decouple the LSTM’s gates from the embedded RNN and create a new class of RNNs where the recurrence computes an element-wise weighted sum of context-independent functions of the input. |
| Outcome: | The proposed model performs as well as an LSTM on a range of problems, strongly suggesting that the gates are doing much more in practice than just alleviating vanishing gradients. |
On the Practical Computational Power of Finite Precision RNNs for Language Recognition (P18-2)
Copied to clipboard
| Challenge: | Recurrent Neural Networks (RNNs) are famously known to be Turing complete, but this relies on infinite precision in the states and unbounded computation time. |
| Approach: | They propose to use LSTM and Elman-RNN with ReLU activation to study RNNs . they show that LS and ReLU-RNns can easily implement counting behavior . |
| Outcome: | The LSTM and the Elman-RNN with ReLU activation are stronger than the RNN with squashing activation and the GRU. |
JBLiMP: Japanese Benchmark of Linguistic Minimal Pairs (2023.findings-eacl)
Copied to clipboard
| Challenge: | In this paper, we compare syntactic knowledge of language models across different languages. |
| Approach: | They introduce a dataset for targeted syntactic evaluations of language models in Japanese. |
| Outcome: | The proposed dataset compares the syntactic knowledge of language models across languages. |
A Comprehensive Comparison of Neural Networks as Cognitive Models of Inflection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural networks are at the center of a debate about human behavior in inflectional morphology. |
| Approach: | They measure correlation between human judgments and neural network probabilities for unknown word inflections. |
| Outcome: | The proposed model for morphological inflections correlates best with human wug ratings, but not with humans. |
Semantic and Syntactic Enhanced Aspect Sentiment Triplet Extraction (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to extract triplets from sentences neglect the mutual information between aspects and have the problem of error propagation. |
| Approach: | They propose a Semantic and Syntactic Enhanced aspect Sentiment triplet Extraction model to exploit the syntactical and semantic relationships between the triplet elements and jointly extract them. |
| Outcome: | The proposed model outperforms existing methods on four benchmark datasets and significantly outperformed existing approaches. |
A Neural Layered Model for Nested Named Entity Recognition (N18-1)
Copied to clipboard
| Challenge: | Entity mentions embedded in longer entity mentions are referred to as nested entities due to the properties of natural language. |
| Approach: | They propose a neural model that dynamically stacks flat NER layers to identify nested entities by capturing sequential context representation with bidirectional long-term memory. |
| Outcome: | The proposed model outperforms state-of-the-art feature-based systems on nested NER, achieving 74.7% and 72.2% on GENIA and ACE2005 datasets, respectively in terms of F-score. |
Cross-Corpora Evaluation and Analysis of Grammatical Error Correction Models — Is Single-Corpus Evaluation Enough? (N19-1)
Copied to clipboard
| Challenge: | Existing studies have evaluated grammatical error correction models on a single corpus, but the evaluation is incomplete because the task difficulty varies depending on the corpus and conditions such as proficiency levels of the writers and essay topics. |
| Approach: | They evaluate the performance of several GEC models against various learner corpora and compare their rankings against the corpus. |
| Outcome: | The evaluation of several models against learner corpora shows that the models’ rankings vary depending on the corpus, indicating that single-corpus evaluation is insufficient for GEC models. |
Detecting Cybersecurity Events from Noisy Short Text (N19-1)
Copied to clipboard
Semih Yagcioglu, Mehmet Saygin Seyfioglu, Begum Citamak, Batuhan Bardak, Seren Guldamlasioglu, Azmi Yuksel, Emin Islam Tatli
| Challenge: | Using domain-specific word embeddings, we propose a method to detect cyber security events from noisy short text. |
| Approach: | They propose a method that leverages domain-specific word embeddings and task-specific features to detect cyber security events from tweets. |
| Outcome: | The proposed model outperforms both baselines and traditional models on a dataset of 2K tweets and manually annotates them. |
Chinese NER Using Lattice LSTM (P18-1)
Copied to clipboard
| Challenge: | Chinese named entity recognition (NER) is a fundamental task in information extraction. |
| Approach: | They propose a lattice-structured LSTM model for Chinese named entity recognition (NER) model leverages word and word sequence information to encode a sequence of input characters and all potential words that match a lexicon. |
| Outcome: | The proposed model outperforms word-based and character-based models on Chinese NER datasets. |
Combining Deep Learning and Topic Modeling for Review Understanding in Context-Aware Recommendation (N18-1)
Copied to clipboard
| Challenge: | Existing models for user reviews are limited by data sparsity and lack of data. |
| Approach: | They propose to integrate LSTM and Topic Modeling to extract review information for recommender systems by utilizing user reviews. |
| Outcome: | The proposed model outperforms existing models on Amazon review dataset and shows better ability on making topic clustering than traditional topic model based method. |
Reproducing and Regularizing the SCRN Model (C18-1)
Copied to clipboard
| Challenge: | Recurrent neural networks (RNNs) have demonstrated tremendous success in sequence modeling . naive dropout, variational dropout and weight tying are common techniques used to regularize the SCRN model . |
| Approach: | They propose a Structurally Constrained Recurrent Network (SCRN) model and regularize it using existing techniques. |
| Outcome: | The proposed model outperforms the LSTM model on non-English data while being much simpler. |
A Graph-to-Sequence Model for AMR-to-Text Generation (P18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic formalism that encodes the meaning of a sentence as a rooted, directed graph. |
| Approach: | They propose a neural graph-to-sequence model that leverages LSTM to encode a linearized AMR structure. |
| Outcome: | The proposed model outperforms existing methods on a benchmark. |
Targeted Syntactic Evaluation of Language Models (D18-1)
Copied to clipboard
| Challenge: | Recent advances have led to an explosion of neural network-based LM architectures. |
| Approach: | They propose to supplement perplexity with a metric that assesses whether a language model can predict the grammatical sentence more accurately than an ungrammatically-based model. |
| Outcome: | The proposed model performed poorly on many of the constructions. |
Higher-Order Syntactic Attention Network for Longer Sentence Compression (N18-1)
Copied to clipboard
| Challenge: | Existing sentence compression methods do not handle syntactic features, causing performance degradation . et al. (2015) reported that the longer the input sentences are, the worse the performance becomes. |
| Approach: | They propose a higher-order syntactic attention network that handles higher-level dependency features as an attention distribution on LSTM hidden states. |
| Outcome: | The proposed method outperforms baseline methods on a Google sentence compression dataset. |
Automated Essay Scoring System for Nonnative Japanese Learners (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing systems only provide a holistic score that summarizes the quality of an essay, which provides little feedback for a language learner. |
| Approach: | They developed an automated essay scoring system for Japanese as a second language learners using an essay dataset with annotations for a holistic score and multiple trait scores. |
| Outcome: | The proposed system achieves the highest accuracy in various natural language processing tasks. |
A Real-World Data Resource of Complex Sensitive Sentences Based on Documents from the Monsanto Trial (2020.lrec-1)
Copied to clipboard
| Challenge: | Sensitive information detection is of great importance in a number of applications where unintended leaks of sensitive information may incur severe negative consequences. |
| Approach: | They propose to use a corpus of sentences to evaluate sensitive information detection approaches . they employ human annotations and automatically infer labels from domain experts . |
| Outcome: | The proposed models are based on a monsanto trial and are evaluated on sentence level. |
Recursive Subtree Composition in LSTM-Based Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing studies show that tree structure modelling on top of sequence modelling is not feasible. |
| Approach: | They propose to recursively compose subtree representations in a biLSTM-based parser to capture subtreas. |
| Outcome: | The proposed model improves performance under ablating the backward LSTM and the forward LS. |
Overestimation of Syntactic Representation in Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Several testing methodologies have been developed to probe models’ syntactic representations. |
| Approach: | They propose a method to determine syntactic structure by training a model on strings generated according to a template and testing its ability to distinguish between similar ones with different syntax. |
| Outcome: | The proposed method reproduces positive results with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs. |
LSTMEmbed: Learning Word and Sense Representations from a Large Semantically Annotated Corpus with Long Short-Term Memories (P19-1)
Copied to clipboard
| Challenge: | Recent work has focused on vector representations which capture different meanings, i.e., senses, of words. |
| Approach: | They propose a bidirectional LSTM model which learns word senses from semantically annotated corpora by focusing on word order. |
| Outcome: | The proposed model achieves state-of-the-art on the SemEval-2014 word-to-sense similarity task and is available online at http://lcl.uniroma1.it/LSTMEmbed. |
Grouped-Attention for Content-Selection and Content-Plan Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent neural data-to-text generation models explicitly learn content-plan given a set of attributes as input. |
| Approach: | They propose a neural content-planner that captures local and global contexts . they use a token-level attention constrained within each input attribute . |
| Outcome: | The proposed model outperforms competitors by 4.92%, 4.70%, and 16.56% on real-world datasets. |
A Label-Aware Autoregressive Framework for Cross-Domain NER (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition (NER) focus on reducing discrepancy between tokens and tokens, but transfer of valuable label information is often not considered or ignored. |
| Approach: | They propose a framework that borrows entity information from the source domain to enhance NER in the target domain. |
| Outcome: | The proposed model improves over the state-of-the-art model on several datasets. |
“Laughing at you or with you”: The Role of Sarcasm in Shaping the Disagreement Space (2021.eacl-main)
Copied to clipboard
| Challenge: | Detecting arguments in online interactions is useful to understand how conflicts arise and get resolved. |
| Approach: | They propose to use a corpus annotated with argumentative moves and sarcasm to model sarcastic relationships using deep learning architectures. |
| Outcome: | The proposed setup improves the argumentative relation classification task using deep learning architectures. |
How Much Syntactic Supervision is “Good Enough”? (2023.findings-eacl)
Copied to clipboard
| Challenge: | RNNGs with syntactic supervision underperformed RNNs with some syntaktic supervision, whereas RNNS with mild supervision achieved the best performance comparable to the state-of-the-art GPT-2-XL. |
| Approach: | They propose a method where syntactic LMs are gradually ablated from full syntatic supervision to zero syntastic supervision by preserving NP, VP, PP, SBAR nonterminal symbols. |
| Outcome: | The proposed method underperforms the RNNGs with zero syntactic supervision, and the LMs with mild syntaktic supervision perform better than the state-of-the-art GPT-2-XL. |
Dissecting Contextual Word Embeddings: Architecture and Representation (D18-1)
Copied to clipboard
| Challenge: | Existing work on learning contextual representations has used LSTM-based biLMs, but there is no reason to believe this is effective. |
| Approach: | They propose to use pre-trained bidirectional language models to learn contextual word embeddings for four NLP tasks and to use them to study the effects of architecture on endtask accuracy. |
| Outcome: | The proposed models outperform word embeddings for four NLP tasks and all learn representations that vary with network depth. |
Simple Yet Powerful: An Overlooked Architecture for Nested Named Entity Recognition (2022.coling-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is an important task in Natural Language Processing that aims to identify text spans belonging to predefined categories. |
| Approach: | They propose to revisit the Multiple LSTM-CRF (MLC) model, a simple, overlooked, yet powerful approach based on training independent sequence labeling models for each entity type. |
| Outcome: | The proposed model achieves state-of-the-art results in the Chilean Waiting List corpus by including pre-trained language models. |
Challenging Reading Comprehension on Daily Conversation: Passage Completion on Multiparty Dialog (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to reading comprehension on multiparty dialogs have focused on children's stories or newswire. |
| Approach: | They propose a new corpus and a robust deep learning architecture for a task in reading comprehension on multiparty dialog. |
| Outcome: | The proposed model outperforms the state-of-the-art model on a different genre using bidirectional LSTM, showing a 13.0+% improvement for longer dialogs. |
Tweet Stance Detection Using an Attention based Neural Ensemble Model (N19-1)
Copied to clipboard
| Challenge: | Existing deep learning approaches to stance detection in twitter are inadequate to deal with the vanishing-gradient and overfitting problems. |
| Approach: | They propose a neural ensemble model that adopts strengths of two LSTM variants to learn better long-term dependencies. |
| Outcome: | The proposed model improves on the existing deep learning models on single and multi-target stance detection datasets. |
FLiText: A Faster and Lighter Semi-Supervised Text Classification with Convolution Networks (2021.emnlp-main)
Copied to clipboard
| Challenge: | obtaining large amounts of labeled data is expensive. |
| Approach: | They develop a semi-supervised learning framework called FLiText which improves text classification accuracy. |
| Outcome: | The proposed framework improves accuracy of lightweight models on IMDb, Yelp-5, and Yahoo! Answer . the framework improve accuracy by 6.59%, 3.94%, and 3.22% on the datasets of IMDa, Yep-5 and Yahoo. Answer compared with the fully supervised method on the full dataset . |
Binarized LSTM Language Model (N18-1)
Copied to clipboard
| Challenge: | Long short-term memory (LSTM) language models are widely used for automatic speech recognition and natural language processing (NLP) however, they are limited by the word embedding layer. |
| Approach: | They propose to encode words into binary vectors and use binarized LSTM parameters to achieve high memory compression. |
| Outcome: | The proposed model achieves 11.3 compression ratio without loss of performance and 31.6 compression ratio with acceptable performance degradation. |
Subword Pooling Makes a Difference (2021.eacl-main)
Copied to clipboard
| Challenge: | Contextual word-representations use subword tokenization to handle large vocabularies and unknown words. |
| Approach: | They propose to use the first subword for morphological probing, POS tagging and NER to pool multiple subwords that correspond to a single word in contextual language models. |
| Outcome: | The proposed model outperforms two multilingual models on morphological probing, POS tagging and NER tasks in 9 languages. |
To Attend or not to Attend: A Case Study on Syntactic Structures for Semantic Relatedness (P18-1)
Copied to clipboard
| Challenge: | Recent success of Recurrent Neural Networks (RNNs) in Machine Translation (MT) has prompted attention mechanisms to be used in machine translation. |
| Approach: | They propose a tree-structured attention model on Tree Long Short-Term Memory Networks . they also experiment with three LSTM variants: bidirectional-LSTMs, Constituency Tree-LSTS, and Dependency Tree LSTS. |
| Outcome: | The proposed model is based on tree-LSTMs, constituency trees, and dependencies trees. |
BengaliLCP: A Dataset for Lexical Complexity Prediction in the Bengali Texts (2024.lrec-main)
Copied to clipboard
| Challenge: | Lexical Complexity Prediction (LCP) is a task for predicting the complexity score of a word or phrase based on its context. |
| Approach: | They propose a deep neural approach to predict lexical complexity of Bengali tokens using an annotated dataset. |
| Outcome: | The proposed neural approach outperforms state-of-the-art models for Bengali language. |
CEASE, a Corpus of Emotion Annotated Suicide notes in English (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on suicide notes have not explored the topic of emotion detection. |
| Approach: | They develop a fine-grained emotion annotated corpus of suicide notes in English and use it to perform emotion detection on a curated dataset. |
| Outcome: | The proposed model performs emotion detection on a curated dataset of 205 suicide notes in English. |
Cycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised text style transfer lack parallel data and difficulties in content preservation. |
| Approach: | They propose a neural approach to unsupervised text style transfer using non-parallel data. |
| Outcome: | The proposed approach can be trained end-to-end on two widely-used public datasets. |
A LSTM Approach with Sub-Word Embeddings for Mongolian Phrase Break Prediction (C18-1)
Copied to clipboard
| Challenge: | Existing word embedding methods for Mongolian PB prediction are expensive and time-consuming. |
| Approach: | They propose to use Mongolian word embedding to build a robust Mongolian PB prediction model . they encode sub-word units and feed it to LSTM to decode the best corresponding PB label . |
| Outcome: | The proposed model outperforms traditional model using manual features and achieves 7.49% gain. |
Synchronous Dual Network with Cross-Type Attention for Joint Entity and Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on joint entity and relation extraction fail to fully utilize the interdependence between entity types and relation types. |
| Approach: | They propose a synchronous dual network with cross-type attention via separately and interactively considering the entity types and relation types. |
| Outcome: | The proposed model achieves state-of-the-art on NYT and WebNLG datasets. |
Mapping Text to Knowledge Graph Entities using Multi-Sense LSTMs (D18-1)
Copied to clipboard
| Challenge: | a paper addresses the problem of mapping natural language text to knowledge base entities. |
| Approach: | They propose a model for mapping natural language text to knowledge base entities using a multi-dimensional entity space obtained from a knowledge graph. |
| Outcome: | The proposed model is applied to large-scale text-to-entity mapping and entity classification tasks with state-of-the-art results. |
LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent trends in NLP research have raised an interest in linguistic code-switching . however, many of these approaches are limited to a few language pairs and a specific domain . |
| Approach: | They propose a centralized benchmark for Linguistic Code-switching Evaluation that combines eleven corpora covering four different code-switch languages and four tasks. |
| Outcome: | The proposed benchmark provides a centralized benchmark and compares with other benchmarks in real-time. |
Aspect Based Sentiment Analysis with Gated Convolutional Networks (P18-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis can provide more detailed information than general sentiment analysis. |
| Approach: | They propose a model based on convolutional neural networks and gating mechanisms which can selectively output the sentiment features according to the given aspect or entity. |
| Outcome: | The proposed model can selectively output sentiment features according to the given aspect or entity. |
Modeling Human Sentence Processing with Left-Corner Recurrent Neural Network Grammars (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature is agnostic about a parsing strategy of hierarchical models . a recent study showed that hierarchically model hierarchic structures capture grammatical dependencies much better than RNNs in targeted syntactic evaluations. |
| Approach: | They evaluated three LMs with head-final left-branching structures and Recurrent Neural Network Grammars with top-down and left-corner parsing strategies as hierarchical models. |
| Outcome: | The proposed model outperforms top-down and left-corner models against human reading times in Japanese. |
Multi-Modal Sarcasm Detection in Twitter with Hierarchical Fusion Model (P19-1)
Copied to clipboard
| Challenge: | Existing methods to detect sarcasm focus on text, but they are insufficient for multi-modal messages. |
| Approach: | They propose a multi-modal hierarchical sarcasm detection model for tweets consisting of texts and images in Twitter. |
| Outcome: | The proposed model is able to detect sarcasm on twitter using three modalities . the proposed model can be used in customer service, opinion mining and harassment detection . |
CAMAL: A Novel Dataset for Multi-label Conversational Argument Move Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models that combine CNN and LSTM structures with speaker ID graphs improve the F1-score of our baseline models to detect speakers’ intents by a large margin. |
| Approach: | They propose a conversational multi-label corpus of teaching transcripts for Conversational Argument Move AnaLysis (CAMAL) the dataset includes 165 discussion transcripts facilitated by pre-service teachers and students . |
| Outcome: | The proposed model improves the F1-score of the baseline model to detect speakers’ intents by a large margin. |
On the Use of Bert for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representation (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained models have not been used to outperform other deep learning models such as CNN in Automated Essay Scoring (AES). |
| Approach: | They propose a novel multi-scale essay representation for BERT that can be jointly learned . they employ multiple losses and transfer learning from out-of-domain essays to further improve performance . |
| Outcome: | The proposed model outperforms existing models in the area of automated essay scoring . the proposed model generalizes well to the CommonLit Readability Prize data set . |
LSTMs Compose—and Learn—Bottom-Up (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in NLP shows that LSTMs capture compositional structure in language data. |
| Approach: | They propose to measure the decompositional interdependence between word meanings in an LSTM based on their gate interactions. |
| Outcome: | The proposed model can model syntactic relationships rather than learning the longer-range relations independently. |
FENAS: Flexible and Expressive Neural Architecture Search (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent approaches to architecture search have shown good improvements in terms of performance with reasonable training speed. |
| Approach: | They propose an algorithm with more activation functions, input edges, and atomic operations to search for architectures that are optimal for given task. |
| Outcome: | The proposed algorithm reproduces well-known LSTM and GRU architectures and initializes with them for finding architectures more efficiently. |
Incorporating Graph Attention Mechanism into Knowledge Graph Reasoning Based on Deep Reinforcement Learning (D19-1)
Copied to clipboard
| Challenge: | Existing methods for learning knowledge Graphs are incomplete and therefore need well-pretraining. |
| Approach: | They propose a deep reinforcement learning based model which incorporates LSTM and Graph Attention Mechanism as the memory components. |
| Outcome: | The proposed model can get rid of the pretraining process and achieve state-of-the-art performance compared with the other models. |
Hallucination Detection for Grounded Instruction Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for generating instructions for navigation generate references to objects or actions that are inconsistent with what a human follower would perform or encounter along the path. |
| Approach: | They propose a weakly supervised approach that detects hallucinated references by using a pre-trained vision-language model. |
| Outcome: | The proposed model outperforms baseline models and supervised models on generating navigation instructions. |
XNLI: Evaluating Cross-lingual Sentence Representations (D18-1)
Copied to clipboard
Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, Veselin Stoyanov
| Challenge: | State-of-the-art natural language processing systems rely on annotated data to learn competent models. |
| Approach: | They extend the development and test sets of the Multi-Genre Natural Language Inference Corpus to 14 languages, including Swahili and Urdu. |
| Outcome: | The proposed evaluation set extends the development and test sets of the Multi-Genre Natural Language Inference Corpus (MultiNLI) to 14 languages including low-resource languages such as Swahili and Urdu. |
Better Feature Integration for Named Entity Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition (NER) focus on stacking the LSTM and graph neural networks (GCNs) however, the exact interaction mechanism between the two types of features is not clear and the performance gain is not significant. |
| Approach: | They propose a model that incorporates both types of features with a Synergized-LSTM which captures how the two types of feature interact. |
| Outcome: | The proposed model achieves better performance than previous approaches while requiring fewer parameters. |
Attending Sentences to detect Satirical Fake News (C18-1)
Copied to clipboard
| Challenge: | Existing approaches to capture news satire do not explore sentence and document difference . |
| Approach: | They propose a hierarchical deep neural network approach for satire detection . it is able to capture satirical news both at the sentence level and document level . |
| Outcome: | The proposed approach can capture satire at sentence and document levels. |
C-Journal: A Journaling Application for Detecting and Classifying Cognitive Distortions Using Deep-Learning Based on a Crowd-sourced Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Cognitive distortions are negatively biased thinking patterns and erroneous self-statements resulting from and leading to logical errors in one’s internal reasoning. |
| Approach: | They propose to provide an open-source labeled dataset of 14 cognitive distortions consisting of 34370 entries collected via crowd-sourcing, user questionnaires, and re-purposing emotions dataset from social media. |
| Outcome: | The proposed model was built into a free journaling and mood-tracking mobile application that pinpoints cognitive distortions to the users. |
Emotion Classification by Jointly Learning to Lexiconize and Classify (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to identify emotions in short text are limited and lack coverage and inaccuracies when applied to informal short text. |
| Approach: | They propose a novel emotional network to jointly learn sentence emotions and construct emotion lexicons which are dynamically adapted to a given context. |
| Outcome: | The proposed model outperforms several approaches proposed in previous studies and achieves new state-of-the-art on the benchmark Twitter dataset. |
Session-level Language Modeling for Conversational Speech (D18-1)
Copied to clipboard
| Challenge: | Xiong et al., 2017) generalizes language models for conversational speech recognition . recurrent neural networks (RNNs) read a list of words sequentially and predict the next word at each position. |
| Approach: | They propose to generalize language models for conversational speech recognition to capture conversation-level phenomena such as adjacency pairs, lexical entrainment, and topical coherence. |
| Outcome: | The proposed model reduces perplexity and improves word error rate over standard models in the conversational telephone speech domain. |
A Richer-but-Smarter Shortest Dependency Path with Attentive Augmentation for Relation Extraction (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to extract relationship between entities in sentences suffer from missing or redundant information. |
| Approach: | They propose a deep neural model that combines the advantages of the two approaches to extract the relationship between two entities in a sentence. |
| Outcome: | The proposed model outperforms baseline models on the SemEval-2010 dataset. |
TAN-NTM: Topic Attention Networks for Neural Topic Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Topic models have been widely used to learn text representations and gain insight into document corpora. |
| Approach: | They propose a framework which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. |
| Outcome: | The proposed model improves on two downstream tasks: document classification and topic guided keyphrase generation. |
Bridge Video and Text with Cascade Syntactic Structure (C18-1)
Copied to clipboard
| Challenge: | Using LSTM-CSS, we construct basic syntactic structure by completing syntastic structure. |
| Approach: | They propose a video captioning approach that progressively completes syntactic structure by a conditional random field to construct basic syntaktic structure. |
| Outcome: | The proposed method produces natural sentences with 42.3% and 28.5% accuracy compared to state-of-the-art methods. |
Learning Robust Latent Representations for Controllable Speech Synthesis (2021.findings-acl)
Copied to clipboard
| Challenge: | Variational Auto-Encoders (VAEs) for learning disentangled latent representations in speech fail to learn latent clusters of speaker attributes when trained on limited or noisy datasets. |
| Approach: | They propose a Variational Auto-Encoder (VAE) that minimizes mutual information between latent variables and learns controllable latent representations in speech data. |
| Outcome: | The proposed model reduces the cluster overlap of speaker attributes by 30% over LSTM-VAE. |
Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering (C18-1)
Copied to clipboard
| Challenge: | Sentence pair modeling is a fundamental technique underlying many NLP tasks. |
| Approach: | They analyze several neural network designs for sentence pair modeling and compare their performance extensively across eight datasets. |
| Outcome: | The proposed models perform well across eight datasets including paraphrase identification, semantic textual similarity, natural language inference, and question answering tasks. |
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies (N19-1)
Copied to clipboard
| Challenge: | State-of-the-art LSTM language models learn sequential contingencies with some success . LS models fail to learn other non-local grammatical dependencies, however . |
| Approach: | They compare LSTM language models with RNNGs to examine grammatical dependencies . they find that hierarchical supervision improves learning of non-local dependencies. |
| Outcome: | The proposed model outperforms the existing model on non-local dependencies and learns many of the Island Constraints on the filler-gap dependency. |
Scalable Syntax-Aware Language Models Using Knowledge Distillation (P19-1)
Copied to clipboard
| Challenge: | Prior work has shown that syntactic neural language models learn from small amounts of training data more effectively than sequential models. |
| Approach: | They propose a knowledge distillation technique that transfers knowledge from a syntactic language model trained on a small corpus to an LSTM language model and enables it to develop a more structurally sensitive representation of the larger training data. |
| Outcome: | The proposed method improves on baseline syntactic evaluations on LSTMs with a higher level of accuracy than previous methods. |
Transferable Neural Projection Representations (N19-1)
Copied to clipboard
| Challenge: | Neural word embeddings require lookup and a large memory footprint making it hard to deploy on-device. |
| Approach: | They propose a skip-gram based architecture coupled with Locality-Sensitive Hashing projections to learn efficient dynamically computable representations. |
| Outcome: | The proposed model performs better than previous models on multiple NLP tasks. |
Investigating Dynamic Routing in Tree-Structured LSTM for Sentiment Analysis (D19-1)
Copied to clipboard
| Challenge: | Existing deep neural network models such as LSTM and tree-LSTM have a bias problem where the words in the tail of a sentence are more heavily emphasized than those in the header. |
| Approach: | They propose a capsule tree-LSTM model that uses dynamic routing to build sentence representations by assigning different weights to nodes according to their contributions to prediction. |
| Outcome: | The proposed model improves on the Stanford Sentiment Treebank and EmoBank datasets. |
How May I Help You? Using Neural Text Simplification to Improve Downstream NLP Tasks (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have focused on rule-based and neural sequence-to-sequence (seq2sequ) TS is a technique that reduces text complexity for human consumption. |
| Approach: | They evaluate two possible uses of neural TS: simplifying input texts at prediction time and augmenting training data to provide machines with additional information during training. |
| Outcome: | The proposed approach improves performance on two datasets. |
Domain adaptation for part-of-speech tagging of noisy user-generated text (N19-1)
Copied to clipboard
| Challenge: | Existing POS taggers for canonical German text achieve good results around 97% accuracy, but when applying these trained models to out-of-domain data the performance decreases drastically. |
| Approach: | They propose a neural network that trains an out-of-domain model on a large newswire corpus and transfers those weights by using them as a prior for a model trained on the target domain. |
| Outcome: | The proposed model achieves a tagging accuracy of slightly over 90%, improving on the previous state of the art for this task. |
Irony Detection in Persian Language: A Transfer Learning Approach Using Emoji Prediction (2020.lrec-1)
Copied to clipboard
Preni Golazizian, Behnam Sabeti, Seyed Arad Ashrafi Asli, Zahra Majdabadi, Omid Momenzadeh, Reza Fahmi
| Challenge: | Existing methods for emotion extraction and sentiment analysis produce invalid results due to the use of irony. |
| Approach: | They propose to use emoji prediction to fine tune a model using hand labeled tweets with irony tags. |
| Outcome: | The proposed method outperforms the state-of-the-art method on Persian dataset with an accuracy of 83.1% and offers strong baseline for further research in Persian language. |
Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for improving model interpretability require prior information or human annotations as additional inputs. |
| Approach: | They propose a variational word mask method to automatically learn task-specific important words and reduce irrelevant information on classification, which ultimately improves model interpretability. |
| Outcome: | The proposed method improves model prediction accuracy and interpretability on seven datasets. |
Predicting in-hospital mortality by combining clinical notes with time-series data (2021.findings-acl)
Copied to clipboard
| Challenge: | In intensive care units, patient health is monitored through vital signals and clinical notes . previous work focused on predicting patient health using time-series data gathered from medical devices . |
| Approach: | They propose a model that combines clinical notes and vital data to make accurate mortality predictions. |
| Outcome: | The proposed model achieves an AUC score of 0.9, compared to the previous 0.87 . it can be used to make accurate in-hospital mortality predictions . |
Sequence Models for Computational Etymology of Borrowings (2021.findings-acl)
Copied to clipboard
| Challenge: | a computational model of word borrowing can be useful for lexicon expansion and language preservation. |
| Approach: | They propose to use neural sequence models to model word borrowings from a donor word to an incorporated word. |
| Outcome: | The proposed model beats baseline models in both directions, with the quantity of data strongly influencing performance. |
Literary Event Detection (P19-1)
Copied to clipboard
| Challenge: | a new dataset of literary events is presented to examine the nature of narratives . literature presents a number of challenges for existing systems, including complex narration . |
| Approach: | They propose a dataset of literary events that are depicted as taking place within the imagined space of a novel. |
| Outcome: | The proposed model achieves an F1 score of 73.9 for prestige and popularity . the best performing model achieve a score of 79.9 for prestige compared to the previous model . |
How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text (2020.coling-main)
Copied to clipboard
| Challenge: | LSTMs are widely used to capture informative long-term syntactic dependencies, but how they are reflected in their internal vectors for natural text has not been adequately investigated. |
| Approach: | They analyze how syntactic dependencies are reflected in LSTM's internal gates by learning a language model where syntaktic structures are implicitly given. |
| Outcome: | The proposed model can predict whether a word is inside a phrase structure or not from a small number of components of the context-update vector. |
Correcting Pronoun Homophones with Subtle Semantics in Chinese Speech Recognition (2024.lrec-main)
Copied to clipboard
| Challenge: | Chinese speech recognition is becoming prevalent due to the similar semantic context of the entities and the overlap of Chinese pronunciation. |
| Approach: | They propose three models to address common confusion issues in Chinese speech recognition . they implement a language model, a LSTM model with semantic features and a rule-based assisted Ngram model . |
| Outcome: | The proposed models achieve highest recognition rate for “T” correction with improvements from 70% in the popular voice input methods up to 90%. |
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it. |
| Approach: | They develop controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training. |
| Outcome: | The proposed models can make syntactic generalizations for tokens seen as few as two times during training and transfer them to transformed contexts. |
Modeling Multi-Granularity Hierarchical Features for Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work on relation extraction focuses on constructing explicit structured features using knowledge graph and dependency tree. |
| Approach: | They propose a method to extract multi-granularity features based solely on the original input sentences. |
| Outcome: | The proposed method outperforms state-of-the-art models that even use external knowledge on three public benchmarks: SemEval 2010 Task 8, Tacred, and Tacred Revisited. |
Probing for Referential Information in Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Neural network-based language models (LMs) have been shown to learn relevant properties of language without being explicitly trained for them. |
| Approach: | They extend their previous work to analyze whether language models capture anaphoric relations and pronoun-antecedent relations in English. |
| Outcome: | The Transformer outperforms the LSTM in all analyses. |
Towards Transparent and Explainable Attention Models (2020.acl-main)
Copied to clipboard
Akash Kumar Mohankumar, Preksha Nema, Sharan Narasimhan, Mitesh M. Khapra, Balaji Vasan Srinivasan, Balaraman Ravindran
| Challenge: | Recent studies on interpretability of attention distributions have led to notions of faithful and plausible explanations for a model’s predictions. |
| Approach: | They propose to modify LSTM cells to ensure that the hidden representations learned at different time steps are diverse. |
| Outcome: | The proposed model can provide a faithful explanation if a higher attention weight implies a greater impact on the model’s prediction. |
Incorporating Inner-word and Out-word Features for Mongolian Morphological Segmentation (2020.coling-main)
Copied to clipboard
| Challenge: | Mongolian morphological segmentation is a crucial preprocessing step in many Mongolian related NLP applications. |
| Approach: | They propose a neural network incorporating inner-word and out-word features for Mongolian morphological segmentation. |
| Outcome: | The proposed network is compared with baselines and evaluates its performance. |
Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | LSTMs can capture syntactic rules in artificial languages, but it is unclear whether they are as capable in natural languages. |
| Approach: | They propose a causal account of structural properties as carried by paths across gates and neurons of a recurrent neural network that localizes and segments the concept into a set of gate or neuron-level paths. |
| Outcome: | The proposed model improves on a widely-studied multi-layer LSTM language model showing that it can learn subject-verb number agreement in English. |
Embedding Time Expressions for Deep Temporal Ordering Models (P19-1)
Copied to clipboard
| Challenge: | Existing data-driven models fail to capture explicit temporal signals, such as dates and time windows. |
| Approach: | They propose a framework to infuse temporal awareness into data-driven models by learning a pre-trained model to embed timexes. |
| Outcome: | The proposed framework infuses temporal awareness into data-driven models by learning a pre-trained model to embed timexes. |
DisSent: Learning Sentence Representations from Explicit Discourse Relations (P19-1)
Copied to clipboard
| Challenge: | Existing models train on vast amounts of text or require costly, manually curated datasets. |
| Approach: | They propose to leverage the discourse relations between sentences to curate a high quality sentence relation task by leveraging explicit discourse relations. |
| Outcome: | The proposed model can be used to learn the meaning of two sentences in a bidirectional LSTM sentence encoder. |
Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks (D18-1)
Copied to clipboard
| Challenge: | Existing gated recurrent networks have a vanishing gradient, allowing for more matrix transformations and less transparent functions. |
| Approach: | They propose an additionsubtraction twin-gated recurrent network (ATR) to simplify neural machine translation. |
| Outcome: | The proposed system is more transparent than LSTM/GRU due to the simplification. |
Interpreting Character Embeddings With Perceptual Representations: The Case of Shape, Sound, and Color (2022.acl-long)
Copied to clipboard
| Challenge: | Character-level information is included in many NLP models, but evaluating the information encoded in character embeddings is an open issue. |
| Approach: | They propose to use shape, sound, and color embeddings to evaluate the information encoded in character representations in five languages to perform cross-lingual analysis. |
| Outcome: | The proposed classifiers evaluate phonological information encoded in character embeddings and LSTM models. |
Is it Time to Swish? Comparing Deep Learning Activation Functions Across NLP tasks (D18-1)
Copied to clipboard
| Challenge: | Activation functions are nonlinearities which have been attributed to the success story of deep learning. |
| Approach: | They propose to use a penalized tanh function to replace the sigmoid and tansh gates in LSTM cells and to improve the performance of the activation function. |
| Outcome: | The proposed activation function performs best on all tasks and can replace the sigmoid and tanh gates in LSTM cells. |
Speed Reading: Learning to Read ForBackward via Shuttle (D18-1)
Copied to clipboard
| Challenge: | LSTM-Shuttle uses human speed reading techniques to perform natural language processing tasks. |
| Approach: | They propose a model which uses human speed reading techniques to perform natural language processing tasks for accurate and efficient comprehension. |
| Outcome: | The proposed model predicts on IMDB, Rotten Tomatoes, AG, and Children’s Book Test datasets and goes backwards. |
Simple Recurrent Units for Highly Parallelizable Recurrence (D18-1)
Copied to clipboard
| Challenge: | recurrent neural networks scale poorly due to the intrinsic difficulty in parallelizing their state computations. |
| Approach: | They propose a simple recurrent unit that provides expressive recurrence and allows highly parallel implementation. |
| Outcome: | The proposed model achieves 5—9x speed-up over cuDNN-optimized LSTM on classification and question answering datasets and delivers stronger results than LS and convolutional models. |
Dual Encoder: Exploiting the Potential of Syntactic and Semantic for Aspect Sentiment Triplet Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Aspect Sentiment Triple Extraction (ASTE) is an advanced natural language processing task. |
| Approach: | They propose a Dual Encoder: Exploiting the potential of Syntactic and Semantic model which maximizes syntactical and semantic relationships among words. |
| Outcome: | The proposed model surpasses the current state-of-the-art on public benchmarks and shows that it is highly efficient. |
Cross-Linguistic Syntactic Evaluation of Word Prediction Models (2020.acl-main)
Copied to clipboard
| Challenge: | A range of studies have concluded that neural word prediction models can distinguish grammatical from ungrammatically sentences with high accuracy. |
| Approach: | They propose to use CLAMS to evaluate LSTM and multilingual BERT models. |
| Outcome: | The proposed model can learn syntax on English, French, German, Hebrew and Russian, and LSTM language models on multilingual and multilingual models. |
Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for generating explanations for neural networks ignore feature interactions between words and phrases. |
| Approach: | They propose to build hierarchical explanations by detecting feature interactions by combining words and phrases at different levels of the hierarchy. |
| Outcome: | The proposed method is evaluated on two benchmark datasets, via automatic and human evaluations. |
Sound Signal Processing with Seq2Tree Network (L18-1)
Copied to clipboard
| Challenge: | Recent LSTM models have been used to model sequential data processing tasks because of their ability to preserve previous information weighted on distance. |
| Approach: | They propose to use a tree-structured tree-based neural network architecture to solve the problem of unbalanced connections between data units inside and outside semantic groups. |
| Outcome: | The proposed model outperforms the state-of-the-art Bidirectional LSTM model on a signal and noise separation task. |
Like a Baby: Visually Situated Neural Language Acquisition (P19-1)
Copied to clipboard
| Challenge: | A multi-modal neural architecture outperforms its equivalent trained on language alone with a 2% decrease in perplexity . |
| Approach: | They propose to use visual context to train neural language models to perform next-word prediction. |
| Outcome: | The proposed model outperforms its equivalent trained on language with 2% decrease in perplexity even when no visual context is available at test. |
PRADO: Projection Attention Networks for Document Classification On-Device (D19-1)
Copied to clipboard
| Challenge: | Recent advances in deep learning have improved the performance of on-device neural networks for long text classification. |
| Approach: | They propose a projection attention neural network PRADO that combines trainable projections with attention and convolutions to train tiny neural networks that achieve high performance on multiple long document classification tasks. |
| Outcome: | The proposed model achieves high performance on multiple long document classification tasks while maintaining compact size. |
Learning Word Representations with Cross-Sentence Dependency for End-to-End Co-reference Resolution (D18-1)
Copied to clipboard
| Challenge: | Existing word embedding models generate word representations by running long short-term memory recurrent neural networks on each sentence of an input article or conversation separately. |
| Approach: | They propose a word embedding model that learns cross-sentence dependency . they use linear sentence linking and attentional sentence linking to learn cross-entry dependency based on context sentences . |
| Outcome: | The proposed model improves end-to-end co-reference resolution by taking knowledge from context sentences and the entire document. |
Adversarial Reprogramming of Text Classification Neural Networks (D19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can cause a machine learning model to misclassify a sample from the classifier's input domain. |
| Approach: | They propose a context-based vocabulary remapping method that performs a computationally inexpensive input transformation to reprogram a victim classification model for a new set of sequences. |
| Outcome: | The proposed method performs a cost-effective input transformation to reprogram a model for a new set of sequences without altering the network architecture or parameters. |
Neural Architectures for Nested NER through Linearization (P19-1)
Copied to clipboard
| Challenge: | a nested named entity recognition (NER) is a set of entities that can overlap and be labeled with more than one label. |
| Approach: | They propose two neural network architectures for nested named entity recognition . they propose to model nesting entities as multilabels and predict a sequence-to-sequence problem . |
| Outcome: | The proposed methods outperform the state-of-the-art on four corpora . the proposed models also improve on the recently published contextual embeddings . |
State-of-the-art Chinese Word Segmentation with Bi-LSTMs (D18-1)
Copied to clipboard
| Challenge: | A wide variety of neural-network architectures have been proposed for the task of Chinese word segmentation. |
| Approach: | They propose a bidirectional LSTM model with standard deep learning techniques and best practices for the task of Chinese word segmentation. |
| Outcome: | The proposed model outperforms models based on standard deep learning techniques and best practices on Chinese word segmentation datasets. |
The Learnability of Model-Theoretic Interpretation Functions in Artificial Neural Networks (2026.findings-acl)
Copied to clipboard
| Challenge: | Entity vectors improve scores on basic event, while gated architectures benefit most. |
| Approach: | They extend entity-level semantic representations, modern architectures, principled competing event generation, extended systematicity tests and a two-dimensional difficulty analysis disaggregating results by modifier complexity. |
| Outcome: | The proposed model-theoretic interpretation functions generalize systematically to out-of-training-sample sentences. |
Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference (D18-1)
Copied to clipboard
| Challenge: | In this paper, we examine the behavior of deep learning models in their intermediate layers . saliency determines what is critical for the final decision of a deep model . |
| Approach: | They propose to interpret the intermediate layers of deep models by visualizing the saliency of attention and LSTM gating signals. |
| Outcome: | The proposed methods reveal interesting insights and identify critical information contributing to the model decisions. |
Pointing to Select: A Fast Pointer-LSTM for Long Text Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to skip irrelevant words in text processing are slow and vanishing gradients can cause slow inference and a loss of coherence. |
| Approach: | They propose a pointer network-based LSTM framework which can change skip rates during inference. |
| Outcome: | The proposed model is 1.1x3.5x faster than the standard LSTM framework and more accurate than Leap-LSTM at high skip rates. |
MODE-LSTM: A Parameter-efficient Recurrent Network with Multi-Scale for Sentence Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for sentence classification use linear convolution, which may not be sufficient to model the non-consecutive dependency of the phrase and may overfit the sequential information. |
| Approach: | They propose a model that extracts multi-scale n-gram features for understanding the semantic meaning of sentences by some key-phrases located at different positions. |
| Outcome: | The proposed model outperforms existing models on eight benchmark datasets and is competitive against state-of-the-art models. |
Distilling Structured Knowledge for Text-Based Relational Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-based relational reasoning models lack a symbolic representation of text . performance gap between NLP models and structured models remains . |
| Approach: | They first pre-train a GNN on a reasoning task using structured inputs and then incorporate its knowledge into an NLP model. |
| Outcome: | The proposed model improves on two state-of-the-art NLP models on 13 different inductive reasoning datasets from the CLUTRR benchmark. |
A Neural Network Based Model for Loanword Identification in Uyghur (L18-1)
Copied to clipboard
| Challenge: | Lexical borrowing happens in almost all languages, and we propose a new method to identify loanwords in Uyghur. |
| Approach: | They propose a neural network based loanword identification model for Uyghur that captures past and future information and learns both word level and character level features automatically. |
| Outcome: | The proposed model outperforms baseline models on Chinese, Arabic and Russian loanword detection in Uyghur. |
Contrastive Deterministic Autoencoders For Language Modeling (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Variational autoencoders (VAEs) are a popular family of generative models with wide applicability. |
| Approach: | They propose to modify a deterministic model designed for images to avoid posterior collapse by controlling the entropy of the aggregate posterior to make it Gaussian. |
| Outcome: | The proposed models outperform a broad range of VAE models on text generation and downstream tasks from representations while avoiding reparametrization steps. |
Bipartite Flat-Graph Network for Nested Named Entity Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models only consider the unidirectional delivery of information from innermost layers to outer ones, but instead focus on nested entities. |
| Approach: | They propose a bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER) the bipartites are bidirectional LSTM and graph convolutional network (GCN) they first use the entities recognized by the flat NER module to construct an entity graph . |
| Outcome: | The proposed model outperforms existing models on three standard nested NER datasets. |
Hint-Based Training for Non-Autoregressive Machine Translation (D19-1)
Copied to clipboard
| Challenge: | AutoRegressive Translation models have to generate tokens sequentially during decoding and thus suffer from high inference latency. |
| Approach: | They propose to use hidden states and word alignments to help train NART models. |
| Outcome: | The proposed model improves on the WMT14 En-De and De-En datasets but is faster in inference than the current models. |
Multi-lingual Mathematical Word Problem Generation using Long Short Term Memory Networks with Enhanced Input Features (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for multi-lingual MWP generation are incapable of identifying language specific constraints, especially in morphologically rich yet low resource languages such as Sinhala and Tamil. |
| Approach: | They propose to use a long-term memory network to generate elementary level MWPs by adding character embeddings, word embedds and Part of Speech (POS) tag embeddements to the network. |
| Outcome: | The proposed model generates elementary level MWPs while satisfying language specific constraints while providing attention for numerical values and units. |
Adversarial Learning of Privacy-Preserving Text Representations for De-Identification of Medical Records (P19-1)
Copied to clipboard
| Challenge: | De-identification is the task of detecting protected health information (PHI) in medical text. |
| Approach: | They propose to create shareable representations of medical text that contain no PHI and can be shared between organizations to create unified datasets for training de-identification models. |
| Outcome: | The proposed representation allows training a simple LSTM-CRF model to an F1 score of 97.4%. |
Creation of Corpus and analysis in Code-Mixed Kannada-English Twitter data for Emotion Prediction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on emotion prediction for resource-rich languages has focused on code-mixed social media corpus but not on Kannada-English code-mixed Twitter data. |
| Approach: | They analyze Kannada-English code-mixed Twitter corpus annotated with their respective ‘Emotion’ for each tweet. |
| Outcome: | The proposed model based on Kannada-English code-mixed Twitter corpus yielded an accuracy of 30% and 32% respectively. |
What Part of the Neural Network Does This? Understanding LSTMs by Measuring and Dissecting Neurons (D19-1)
Copied to clipboard
| Challenge: | Biological neural systems consist of a huge number of neurons, and can react to the environment in complicated ways. |
| Approach: | They propose a metric to quantify the sensitivity of neurons to each label and conduct experiments to prove it. |
| Outcome: | The proposed metric is based on a set of experiments that show that dropping an arbitrary neuron significantly degrades the accuracy of the model. |
Improving NMT Quality Using Terminology Injection (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study has explored the use of vetted terminology in neural machine translation . a number of organizations use domain- or organization-specific words and phrases . |
| Approach: | They propose a method for injecting terminology and for evaluating terminology injection. |
| Outcome: | The proposed method is based on the long-term memory (LSTM) attention mechanism prevalent in state-of-the-art systems . it also introduces a new translation metric more sensitive to approved terminological content in MT output. |
Multi-domain Tweet Corpora for Sentiment Analysis: Resource Creation and Evaluation (2020.lrec-1)
Copied to clipboard
| Challenge: | a huge amount of content is being generated every day due to the pervasiveness of social media. |
| Approach: | They firstly create a multi-domain tweet sentiment corpora and then establish a deep neural network based baseline framework to address the above mentioned issues. |
| Outcome: | The proposed dataset achieves 84.65% accuracy for sentiment analysis using a neural network, long short term memory, and gated recurrent unit (GRU). |
Aggregating Bidirectional Encoder Representations Using MatchLSTM for Sequence Matching (D19-1)
Copied to clipboard
| Challenge: | Recent work on text sequence matching tasks uses task specific supervised datasets, which are always limited to the amount due to the cost of annotation. |
| Approach: | They propose an aggregation method to combine Bidirectional Encoder Representations from Transformer (BERT) with a MatchLSTM layer for Sequence Matching. |
| Outcome: | The proposed model improves on two publicly available datasets, WikiQA and SNLI. |
Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approach (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can alter models' predicted sentiment due to their sensitivity to specific word choices. |
| Approach: | They propose a regularization technique to improve NLP model robustness by reducing the impact of input perturbations on model outputs. |
| Outcome: | The proposed method outperforms state-of-the-art methods in adversarial defense. |
Automatic Labeling of Problem-Solving Dialogues for Computational Microgenetic Learning Analytics (L18-1)
Copied to clipboard
| Challenge: | This paper presents a recurrent neural network model to automate the analysis of students' computational thinking in problem-solving dialogue. |
| Approach: | They propose a recurrent neural network model to automate the analysis of students' computational thinking in problem-solving dialogue. |
| Outcome: | The proposed model outperforms the baseline model and outperformed the nave model by a large margin. |
Parsing as Tagging (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing treat parse as tagging, but they are not perfect. |
| Approach: | They propose a simple yet accurate method that treats parsing as tagging . they use a sequence model with a bidirectional LSTM over BERT embeddings . |
| Outcome: | The proposed method outperforms the state-of-the-art method on universal dependency (UD) by 1.76% unlabeled attachment score (UAS) for English, 1.98% UAS for French, and 1.16% UAS in German. |
Multi-Task Learning for Chemical Named Entity Recognition with Chemical Compound Paraphrasing (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is one of the important basic technologies for Natural Language Processing (NLP) . |
| Approach: | They propose to use long short-term memory (LSTM) of NER model to capture chemical com- pound paraphrases by sharing parameters of LSTM and character embeddings be- tween the two models. |
| Outcome: | The proposed method improves chemi- cal NER and achieves state-of-the-art performance on the BioCreative IV’s CHEMDNER task. |
Lattice Transformer for Speech Translation (P19-1)
Copied to clipboard
| Challenge: | Recent advances in sequence modeling have highlighted the strengths of the transformer architecture. |
| Approach: | They propose a general lattice transformer for speech translation where the input is the output of the automatic speech recognition (ASR) they propose 'controllable' lattica attention mechanism to consume latent representations. |
| Outcome: | The proposed model outperforms baseline and lattice LSTM on the Chinese-English translation task. |
Comparing Methods for Measuring Dialect Similarity in Norwegian (2020.lrec-1)
Copied to clipboard
| Challenge: | a coarse-grained transcription of speech is sufficient to replicate dialectal boundaries, but it can be generalised over by an automatic method. |
| Approach: | They propose to use two different methods to measure dialect similarity in Norwegian . they use the Levenshtein method and the neural long short term memory algorithm . the paper shows that coarse-grained transcriptions of speech can generate dialect maps . |
| Outcome: | The proposed method can generalise over coarse-grained transcriptions, but it needs a large dataset . the proposed method is compared with canonical maps found in the dialect literature . |
Tree Transformer’s Disambiguation Ability of Prepositional Phrase Attachment and Garden Path Effects (2024.acl-long)
Copied to clipboard
| Challenge: | Prepositional phrase attachment ambiguity is structural in nature, while garden path constructions are incremental in nature. |
| Approach: | They pretrain and evaluate an unsupervised Transformer model that induces tree representations internally and compare it to a pretrained supervised BiLSTM model. |
| Outcome: | The Tree Transformer model induces tree representations internally, but its parsing ability is inferior to the supervised BiLSTM model, and it is not as sensitive to lexical cues as other large LSTM models. |
Logical Natural Language Generation from Open-Domain Tables (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on neural natural language generation focus on surface-level realizations with limited emphasis on logical inference. |
| Approach: | They propose a task where a model is tasked with generating natural language statements that can be logically entailed by facts in an open-domain semi-structured table. |
| Outcome: | The proposed task is based on the existing TabFact dataset with a wide range of logical/symbolic inferences. |
RP-DNN: A Tweet Level Propagation Context Based Deep Neural Networks for Early Rumor Detection in Social Media (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for early rumor detection on social media platforms are limited, incomplete and noisy. |
| Approach: | They propose a novel hybrid neural network architecture which combines a task-specific character-based bidirectional language model and stacked Long Short-Term Memory (LSTM) networks to represent textual contents and social-temporal contexts of input source tweets. |
| Outcome: | The proposed model achieves state-of-the-art for detecting unseen rumors on large augmented data which covers more than 12 events and 2,967 rumors. |
Burmese Speech Corpus, Finite-State Text Normalization and Pronunciation Grammars with an Application to Text-to-Speech (2020.lrec-1)
Copied to clipboard
Yin May Oo, Theeraphol Wattanavekin, Chenfang Li, Pasindu De Silva, Supheakmungkol Sarin, Knot Pipatsrisawat, Martin Jansche, Oddur Kjartansson, Alexander Gutkin
| Challenge: | Using crowd-sourced speech corpus and finite-state transducer grammars, we build a text-to-speech system for Burmese, a tonal Southeast Asian language from the Sino-Tibetan family. |
| Approach: | They propose an open-source crowd-sourced multi-speaker speech corpus and finite-state grammars for performing grapheme-to-phoneme conversion for Burmese. |
| Outcome: | The proposed system performs well for Burmese in a low-resource setting. |
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios. |
| Approach: | They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset. |
| Outcome: | The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation. |
OOVs in the Spotlight: How to Inflect Them? (2024.lrec-main)
Copied to clipboard
| Challenge: | Inflection is a process of word formation in which a base word form (lemma) is modified to express grammatical categories. |
| Approach: | They develop a retrograde model and two sequence-to-sequence models based on LSTM and Transformer. |
| Outcome: | The proposed systems outperform the existing systems on 9 out of 16 languages in the OOV evaluation. |
Do Transformers Grok Succinct Algorithms? Mechanistic Evidence for Counting Circuits (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies suggest that Transformers are inherently succinct, capable of representing recursive algorithms like binary counting over exponential state spaces. |
| Approach: | They propose to bridge this gap by testing the Succinctness Hypothesis using mechanistic interpretability on a large-scale computation task. |
| Outcome: | The proposed model can represent recursive algorithms over exponential state spaces . the proposed model is able to generalize perfectly, whereas massive LSTM baselines fail completely. |
Targeted Syntactic Evaluation on the Chomsky Hierarchy (2024.lrec-main)
Copied to clipboard
| Challenge: | a novel evaluation paradigm for targeted syntactic evaluations is proposed . we create formal languages that abstract four syntaktic phenomena in natural languages . |
| Approach: | They propose a new evaluation paradigm for Targeted Syntactic Evaluations . they create formal languages that abstract syntactical phenomena in natural languages . |
| Outcome: | The proposed evaluation paradigm evaluates language models on language modeling tasks . it shows that they can capture the structural patterns of the (Adj)n NP type formal language . |