Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Catchphrase: Automatic Detection of Cultural References (2021.acl-short)
Copied to clipboard
| Challenge: | a snowclone is a customizable phrasal template that can be realized in multiple, instantly recognized variants. |
| Approach: | They propose to use pop-culture quotes to train algorithms to detect cultural references in text. |
| Outcome: | The proposed algorithm can detect cultural references in pop-culture quotes and train on them. |
On Training Instance Selection for Few-Shot Neural Text Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Pretraining large neural networks with a language modeling objective has led to dramatic improvements in text generation. |
| Approach: | They propose a selection strategy to select few-shot training instances based on unlabeled data to identify the most worthwhile data points that should be annotated under some budget of labeling cost. |
| Outcome: | The proposed strategy outperforms random sampling on three text generation tasks. |
Coreference Resolution without Span Representations (2021.acl-short)
Copied to clipboard
| Challenge: | Pretraining has reduced many complex task-specific NLP models to simple lightweight layers. |
| Approach: | They propose a lightweight end-to-end coreference model that removes the dependency on span representations, handcrafted features, pruning heuristics, and more. |
| Outcome: | The proposed model performs competitively with the current standard model, while being simpler and more efficient. |
Enhancing Entity Boundary Detection for Better Chinese Named Entity Recognition (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to Chinese Named Entity Recognition (NER) lack explicit word boundary and tenses information. |
| Approach: | They propose a boundary enhanced approach for Chinese Named Entity Recognition . they add an additional Graph Attention Network(GAT) layer to capture internal dependency of phrases . |
| Outcome: | The proposed approach improves Chinese Named Entity Recognition (NER) on OntoNotes and Weibo corpora. |
Difficulty-Aware Machine Translation Evaluation (2021.acl-short)
Copied to clipboard
| Challenge: | Current MT evaluation measures pay the same attention to each sentence component . in real-world examinations, the questions vary in difficulty and weightings . |
| Approach: | They propose a difficulty-aware MT evaluation metric that takes translation difficulty into account . they propose to use this metric to evaluate machine translation (MT) results . |
| Outcome: | The proposed method outperforms most MT evaluation metrics in terms of human correlation. |
Uncertainty and Surprisal Jointly Deliver the Punchline: Exploiting Incongruity-Based Features for Humor Recognition (2021.acl-short)
Copied to clipboard
| Challenge: | Existing work on humor recognition does not examine the actual joke mechanism . a recent study focused on humor-specific stylistic features, but few have tried to establish a connection between them and humor theories. |
| Approach: | They propose to model the set-up and punchline as part developing semantic uncertainty and disrupt audience expectations. |
| Outcome: | The proposed features can tell jokes from non-jokes, compared with baselines. |
Counterfactuals to Control Latent Disentangled Text Representations for Style Transfer (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for unsupervised text style transfer focus on transferring a specific attribute, but this technique has never been explored in natural language generation tasks. |
| Approach: | They propose a counterfactual-based method to modify latent representations by posing a ‘what-if’ scenario. |
| Outcome: | The proposed method is tested on multiple attribute transfer tasks like Sentiment, Formality and Excitement to support the hypothesis. |
Attention Flows are Shapley Value Explanations (2021.acl-short)
Copied to clipboard
| Challenge: | Shapley Values are a popular type of explanation in machine learning, but leave-one-out and attention-based explanations still predominate in NLP. |
| Approach: | They propose to use attention flow to explain the importance of features, embeddings, and even neurons to explain credit assignment problems in cooperative game theory. |
| Outcome: | The proposed explanations can explain the importance of features, embeddings, and even neurons, but in NLP, leave-one-out and attention-based explanations still predominate. |
Video Paragraph Captioning as a Text Summarization Task (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for video paragraph captioning use ground-truth event segments. |
| Approach: | They propose a video paragraph captioning task that generates coherent paragraphs without ground-truth event segments. |
| Outcome: | The proposed framework outperforms existing methods on two popular datasets. |
Are VQA Systems RAD? Measuring Robustness to Augmented Data with Focused Interventions (2021.acl-short)
Copied to clipboard
| Challenge: | Existing systems for visual question answering are overfitted to training data and are sensitive to small perturbations. |
| Approach: | They propose a robustness measure to augment visual question answering datasets to measure generalization capabilities. |
| Outcome: | The proposed model can quantify failure cases which reveal that current systems are still brittle. |
How Helpful is Inverse Reinforcement Learning for Table-to-Text Generation? (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to Table-to-Text generation suffer from issues such as missing information, repetition and repetition. |
| Approach: | They propose to use Inverse Reinforcement Learning (IRL) to solve the Table-to-Text task . they use multiple interpretable unsupervised reward components that are combined linearly to form a composite reward function. |
| Outcome: | The proposed task outperforms strong RL baselines marginally in the Table-to-Text task. |
Automatic Fake News Detection: Are Models Learning to Reason? (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fake news detection rely on reasoning . existing work has not explored the predictive power of isolated evidence . |
| Approach: | They investigate the relationship and importance of both claim and evidence in fact checking models. |
| Outcome: | The proposed model performs better on political fact checking datasets using both the claim and evidence. |
Saying No is An Art: Contextualized Fallback Responses for Unanswerable Dialogue Queries (2021.acl-short)
Copied to clipboard
| Challenge: | despite advances in task-oriented and chit-chat based dialogue systems, many systems rely on static and unnatural responses. |
| Approach: | They propose a neural approach which generates contextually aware responses to user queries . they perform automatic and manual evaluations to demonstrate the efficacy of the system . |
| Outcome: | The proposed approach generates responses which are contextually aware with the user query and say no to the user. |
N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypotheses (2021.acl-short)
Copied to clipboard
| Challenge: | Spoken Language Understanding systems parse spoken utterances into semantic structures like dialog acts and slots. |
| Approach: | They propose to use concatenated N-best ASR alternatives to represent utterances . they propose to employ a simpler utteration representation with no special delimiter . |
| Outcome: | The proposed model outperforms the prior state-of-the-art model on DSTC2 dataset. |
Gender bias amplification during Speed-Quality optimization in Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | et al., 2002) show that gendered noun translation performance degrades faster than BLEU. |
| Approach: | They propose to use greedy search, quantization, AANs and shallow decoders to speed up decoding . they find minimal degradation of BLEU, but gendered noun translation degrades faster . |
| Outcome: | The proposed model degrades gendered noun translation performance faster than other models. |
Machine Translation into Low-resource Language Varieties (2021.acl-short)
Copied to clipboard
| Challenge: | Current machine translation systems generate a "standard" target language, but many languages have multiple varieties that are different from the standard language. |
| Approach: | They propose a framework to rapidly adapt machine translation systems to generate different target varieties . they propose to use no parallel data to generate languages close to, but different from, the standard target language . |
| Outcome: | The proposed model improves on a system that generates Ukrainian and Belarusian in two languages with no parallel data. |
Is Sparse Attention more Interpretable? (2021.acl-short)
Copied to clipboard
| Challenge: | Sparse attention has been claimed to increase model interpretability . however, the attention distribution is typically over representations internal to the model rather than the inputs themselves . |
| Approach: | They conduct experiments to understand how sparsity affects our ability to use attention as an explainability tool. |
| Outcome: | The proposed model does not map to a sparse set of influential inputs, but rather to fewer inputs. |
The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches for positional dependencies do not satisfy all criteria for optimal position encoding. |
| Approach: | They propose a translation-invariant self-attention approach that accounts for relative position between tokens in an interpretable fashion without conventional embeddings. |
| Outcome: | The proposed model improves on regular ALBERT on GLUE tasks while adding orders of magnitude less positional parameters. |
Relative Importance in Sentence Processing (2021.acl-short)
Copied to clipboard
| Challenge: | In natural language processing, the relative importance of words is usually interpreted with respect to a specific task. |
| Approach: | They compare the relative importance of words in English language processing by humans and neural language models by using saliency methods. |
| Outcome: | The proposed method could be used to interpret neural language models. |
Doing Good or Doing Right? Exploring the Weakness of Commonsense Causal Reasoning Models (2021.acl-short)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) achieve surprising performance on the Choice of Plausible Alternatives (COPA) task. |
| Approach: | They propose to add a regularization loss to the existing COPA models to mitigate the problem of semantic similarity bias by adding a normalization loss. |
| Outcome: | The proposed model improves generalization ability and performs better on a challenging dataset, BCOPA-CE, which has unbiased token distribution and is more difficult for models to distinguish cause and effect. |
AND does not mean OR: Using Formal Languages to Study Language Models’ Representations (2021.acl-short)
Copied to clipboard
| Challenge: | A current open question in natural language processing is to what extent language models are able to capture the meaning of language. |
| Approach: | They propose to simulate a distributional language model’s ability to differentiate logical symbols using motivated constraints and motivated constraints. |
| Outcome: | The results show that the proposed models are unable to differentiate meaningfully different symbols, suggesting a limitation to the types of semantic signals that current models are capable of exploiting. |
Enforcing Consistency in Weakly Supervised Semantic Parsing (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for training semantic parsers from only (utterance, denotation) supervision are challenging. |
| Approach: | They propose to use consistency between output programs for related inputs to reduce the impact of spurious programs. |
| Outcome: | The proposed formalisms improve model performance even without consistency-based training. |
An Improved Model for Voicing Silent Speech (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models for voicing silent speech use hand-designed features instead of EMG signals. |
| Approach: | They propose to use facial electromyography signals as input instead of hand-designed features to give the model greater flexibility to learn its own features. |
| Outcome: | The proposed model improves state-of-the-art on an open vocabulary intelligibility evaluation by 25.8%. |
What’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus (2021.acl-short)
Copied to clipboard
| Challenge: | Recent advances in NLP have been driven by Transformer-based language models. |
| Approach: | They analyze the Common Crawl, a web corpus extensively used for training language models. |
| Outcome: | The Common Crawl contains hate speech and sexually explicit content even after filtering procedures. |
Continual Quality Estimation with Online Bayesian Meta-Learning (2021.acl-short)
Copied to clipboard
| Challenge: | Existing quality estimation models for machine translation are trained and evaluated in a static setting . however, in real-life settings, test data may differ from training data . |
| Approach: | They propose an online Bayesian meta-learning framework for continuous training of QE models that adapts to the needs of different users while being robust to distributional shifts in training and test data. |
| Outcome: | The proposed framework adapts to the needs of different users while being robust to distributional shifts in training and test data. |
A Span-based Dynamic Local Attention Model for Sequential Sentence Classification (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for sentence classification ignore latent segment structure of document, in which contiguous sentences have coherent semantics. |
| Approach: | They propose a span-based dynamic local attention model that captures structural information by supervised dynamic local focus. |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets. |
How effective is BERT without word ordering? Implications for language understanding and data privacy (2021.acl-short)
Copied to clipboard
| Challenge: | Ordered word sequences contain the rich structures that define language. |
| Approach: | They show that token representations and self-attention activations within BERT are resilient to shuffling the order of input tokens. |
| Outcome: | The proposed model is able to handle shuffled token representations and self-attention activations . the model can handle GLUE language understanding tasks with bag-of-words data . |
WikiSum: Coherent Summarization Dataset for Efficient Human-Evaluation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing summarization datasets are limited in their ability to evaluate output . a human evaluation is necessary to understand and improve summarizing systems . |
| Approach: | They propose a dataset based on how-to articles and coherent paragraph summaries written in plain language. |
| Outcome: | The proposed dataset makes human evaluation easier and more effective . the authors compare the proposed dataset to existing ones on PubMed and the literature. |
UMIC: An Unreferenced Metric for Image Captioning via Contrastive Learning (2021.acl-short)
Copied to clipboard
| Challenge: | BERTScore and other text generation metrics do not use reference captions to evaluate image captions. |
| Approach: | They propose a new metric which does not require reference captions to evaluate image captions . they train UMIC to discriminate negative captions via contrastive learning . |
| Outcome: | The proposed metric has higher correlation than previous metrics that require multiple references. |
Anchor-based Bilingual Word Embeddings for Low-Resource Languages (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to build monolingual word embeddings rely on a cheap bilingual signal and monolingual data. |
| Approach: | They propose a method where the vector space of the high resource source language is used as a starting point for training an embedding space for the low resource target language. |
| Outcome: | The proposed approach improves bilingual lexicon induction performance and target language MWE quality. |
Multilingual Agreement for Multilingual Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models that only use auxiliary languages to encourage multilingual agreement ignore the relationships between different language pairs. |
| Approach: | They propose a multilingual agreement-based method which explicitly models the agreement between different translation directions by randomly substituting some fragments of the source language with their counterpart translations of auxiliary languages. |
| Outcome: | The proposed method improves on the multilingual translation task of 10 language pairs. |
Higher-order Derivatives of Weighted Finite-state Machines (2021.acl-short)
Copied to clipboard
| Challenge: | Weighted finite-state machines (WFSMs) have a storied role in NLP . e.g., conditional random fields for part-of-speech tagging are considered special cases of WFSM. |
| Approach: | They propose an algorithm for evaluating derivatives of all orders that is faster than prior algorithms . they also find that it is faster for computing second-order expectations . |
| Outcome: | The proposed algorithm is faster than prior algorithms and can compute second-order expectations. |
Reinforcement Learning for Abstractive Question Summarization with Question-aware Semantic Rewards (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for summarizing long questions are difficult due to the lack of training data and the complexity of the related subtasks. |
| Approach: | They propose a reinforcement learning-based framework for abstractive question summarization that rewards question-type identification and question-focus recognition for regularizing the question generation model. |
| Outcome: | The proposed method achieves higher performance over state-of-the-art models on two benchmark datasets. |
A Semantics-aware Transformer Model of Relation Linking for Knowledge Base Question Answering (2021.acl-short)
Copied to clipboard
Tahira Naseem, Srinivas Ravishankar, Nandana Mihindukulasooriya, Ibrahim Abdelaziz, Young-Suk Lee, Pavan Kapanipathi, Salim Roukos, Alfio Gliozzo, Alexander Gray
| Challenge: | Existing knowledge base question answering systems do not leverage the explicit semantic parse of the question text. |
| Approach: | They propose a transformer-based neural model that leverages the AMR semantic parse of a sentence. |
| Outcome: | The proposed model outperforms the state-of-the-art on 4 popular benchmark datasets. |
Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation (2021.acl-short)
Copied to clipboard
| Challenge: | Early fusion models with cross-attention have shown better-than-human performance on some question answer benchmarks, while it is a poor fit for retrieval since it prevents pre-computation of the answer representations. |
| Approach: | They propose a supervised data mining method to train an efficient late fusion retrieval model by using cross-attention models with cross-references. |
| Outcome: | The proposed model outperforms retrieval models trained with gold annotations on Precision at N (P@N) and Mean Reciprocal Rank (MRR). |
Enhancing Descriptive Image Captioning with Natural Language Inference (2021.acl-short)
Copied to clipboard
| Challenge: | Existing captioning models tend to generate generic captions, but generating descriptive captions is important. |
| Approach: | They propose a novel approach to encourage captioning models to produce more detailed captions using natural language inference. |
| Outcome: | The proposed method outperforms baseline models on MSCOCO metrics on descriptiveness and descriptiveness. |
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to entity linking represent each entity with a single vector, but instead use a contextualized mention-encoder that learns to place similar mentions of the same entity closer in vector space than mentions from different entities. |
| Approach: | They propose an instance-based nearest neighbor approach to entity linking that allows for a contextualized mention-encoder to learn to place similar mentions of the same entity closer in vector space than mentions from different entities. |
| Outcome: | The proposed approach outperforms all other systems on two multilingual benchmarks and is simpler to train and interpretable. |
eMLM: A New Pre-training Objective for Emotion Related Tasks (2021.acl-short)
Copied to clipboard
| Challenge: | Emotion Masked Language Modelling improves the performance of a pretraining language model for emotion detection and sentiment analysis tasks. |
| Approach: | They propose a BERT-based version of Masked Language Modelling that induces emotion into the model. |
| Outcome: | The proposed model improves on emotion detection and sentiment analysis tasks by 1.2% F-1 . the proposed model also shows increased robustness in the test. |
On Positivity Bias in Negative Reviews (2021.acl-short)
Copied to clipboard
| Challenge: | Existing studies have shown positive words are more frequently used in negative reviews . however, it remains unclear whether the Pollyanna hypothesis holds in negative review . |
| Approach: | They validate the Pollyanna hypothesis that positive words occur more frequently than negative words in human expressions . they use a variety of review datasets to examine the use of positive and negative words . |
| Outcome: | The results confirm the pollyanna hypothesis that positive words occur more frequently than negative words in human expressions. |
PRAL: A Tailored Pre-Training Model for Task-Oriented Dialog Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to building task-oriented dialog systems require a substantial amount of annotations and thus are labor-intensive. |
| Approach: | They propose a Pre-trainedRole Alternating Language model (PRAL) that is explicitly designed for task-oriented dialog tasks. |
| Outcome: | The proposed model outperforms or is on par with state-of-the-art models on task-oriented dialog tasks. |
ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction (2021.acl-short)
Copied to clipboard
Chen-Yu Lee, Chun-Liang Li, Chu Wang, Renshen Wang, Yasuhisa Fujii, Siyang Qin, Ashok Popat, Tomas Pfister
| Challenge: | Graph Convolutional Networks (GCNs) have limited ability to capture reading orders of given word-level node representations in a graph. |
| Approach: | They propose a new positional encoding technique to capture word-level nodes in a graph. |
| Outcome: | The proposed method improves existing GCNs with an 8.4% F1 score on two datasets and a large-scale payment dataset. |
Zero-shot Event Extraction via Transfer Learning: Challenges and Insights (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for event extraction require expensive annotation and are not extensible to new event ontologies. |
| Approach: | They propose to use textual entailment and/or question answering queries to extract a zero-shot event from a set of TE and/ or QA queries. |
| Outcome: | The proposed method achieves acceptable results on ACE-2005 and ERE, but there is still a large gap from supervised approaches. |
Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models (2021.acl-short)
Copied to clipboard
| Challenge: | Pre-trained language models have achieved human-level performance on many Machine Reading Comprehension (MRC) tasks, but it remains unclear whether these models truly understand language or answer questions by exploiting statistical biases in datasets. |
| Approach: | They propose a method to attack MRC models by exposing statistical biases in a RACE dataset and propose an augmented training method that can greatly reduce models’ statistical bias. |
| Outcome: | The proposed method can reduce models’ statistical biases from human-level performance to chance-level. |
Quantifying and Avoiding Unfair Qualification Labour in Crowdsourcing (2021.acl-short)
Copied to clipboard
| Challenge: | Existing research suggests that crowd workers need to complete a substantial amount of poorly paid work to earn a fair wage. |
| Approach: | They propose to use a qualification that requires workers to have completed a certain number of tasks to earn a fair wage. |
| Outcome: | The proposed qualification reduces the burden on workers while still collecting high quality data. |
Men Are Elected, Women Are Married: Events Gender Bias on Wikipedia (2021.acl-short)
Copied to clipboard
| Challenge: | Disproportional event distributions can manifest and amplify social stereotypes . researchers have been using NLP tools to analyze corpora for various tasks on online platforms. |
| Approach: | They propose to scrape a corpus of career and personal life descriptions with demographic information from 10,412 celebrities to facilitate the study. |
| Outcome: | The proposed model detects gender biases in a corpus of career and personal life descriptions and calibrates the results using strategically generated templates. |
Modeling Task-Aware MIMO Cardinality for Efficient Multilingual Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing work has increased the modeling capacity of multilingual NMT by deepening or widening the Transformer. |
| Approach: | They propose to increase the model capacity by deepening the Transformer . they propose to use a multi-input-multi-output architecture to combine multiple inputs . |
| Outcome: | The proposed model surpasses previous work and is 1.31 times faster than existing models. |
Adaptive Nearest Neighbor Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | kNN-MT uses pre-trained NMT model with token-level k-nearest-neighbor retrieval to improve translation accuracy. |
| Approach: | They propose a method that combines a pre-trained NMT model with token-level k-nearest-neighbor retrieval to improve translation accuracy. |
| Outcome: | The proposed method outperforms the existing model on four benchmark datasets and is open-source. |
On Orthogonality Constraints for Transformers (2021.acl-short)
Copied to clipboard
Aston Zhang, Alvin Chan, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang, Huajie Shao, Shuochao Yao, Roy Ka-Wei Lee
| Challenge: | a dedicated study on orthogonality constraints for transformers has been lacking . plug-and-play constraints increase the BLEU of transformers . |
| Approach: | They propose to use plug-and-play constraints to encourage matrices to be orthogonal for numerical stability. |
| Outcome: | The proposed constraint increases the BLEU on the large-scale WMT’16 EnDe benchmark by a factor of 28.4 to 29.6. |
Measuring and Improving BERT’s Mathematical Abilities by Predicting the Order of Reasoning. (2021.acl-short)
Copied to clipboard
| Challenge: | a common language model for word math problems lacks mathematical abilities . a data-driven approach to solving word problems is lacking in many areas . |
| Approach: | They propose to train a language model with mathematical abilities to teach word maths . they propose to use semi-formal steps to explain how math results are derived . |
| Outcome: | The proposed model achieves better outcomes than baseline models and on-par with more tailored models. |
Happy Dance, Slow Clap: Using Reaction GIFs to Predict Induced Affect on Twitter (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for labeling emotions in text are limited, but they can be useful for many tasks. |
| Approach: | They propose a method to collect texts with induced emotion and induced sentiment labels. |
| Outcome: | The proposed method can augment the data with induced emotion and induced sentiment labels. |
Exploring Listwise Evidence Reasoning with T5 for Fact Verification (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification use pretrained sequence-to-sequence transformers for sentence selection and label prediction. |
| Approach: | They propose a framework for fact verification that leverages pretrained sequence-to-sequence transformer models for sentence selection and label prediction. |
| Outcome: | The proposed framework scores higher than the second place approach on the blind test set . the proposed framework can be useful for a broader range of NLP tasks, the authors say . |
DefSent: Sentence Embeddings using Definition Sentences (2021.acl-short)
Copied to clipboard
| Challenge: | Sentence embedding methods using natural language inference datasets are limited for limited languages due to large datasets. |
| Approach: | They propose a sentence embedding method that uses definition sentences from a word dictionary. |
| Outcome: | The proposed method performs comparably on unsupervised semantics textual similarity tasks and slightly better on SentEval tasks than methods using large NLI datasets. |
Discrete Cosine Transform as Universal Sentence Encoder (2021.acl-short)
Copied to clipboard
| Challenge: | Modern sentence encoders capture underlying linguistic characteristics of words . Discrete Cosine Transform (DCT) is an efficient alternative to averaging . |
| Approach: | They propose to use a Discrete Cosine Transform to generate universal sentence representations in different languages. |
| Outcome: | The proposed model captures the underlying syntactic characteristics of a given text without compromising practical efficiency. |
AligNarr: Aligning Narratives on Movies (2021.acl-short)
Copied to clipboard
| Challenge: | Experimental results show the viability of an unsupervised approach to align movie scripts with plot summaries. |
| Approach: | They propose an unsupervised method to align movie scripts with plot summaries using a global optimization model. |
| Outcome: | The proposed method outperforms a baseline alignment model on ten movies with 76% F1 score. |
An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers (2021.acl-short)
Copied to clipboard
| Challenge: | Existing word-level quality estimation models require labelled data for each language pair and expensive maintenance. |
| Approach: | They propose to use multilingual QE models to generalise across languages . they propose to train models on other language pairs to predict word-level quality . |
| Outcome: | The proposed models generalise well across languages, making them more useful in real-world scenarios. |
Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models (2021.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that a sequence-to-sequence learning framework with neural networks can be effective for Chinese Spelling Correction (CSC) |
| Approach: | They propose a sequence-to-sequence learning framework with neural networks that generates more valuable training instances and adds task-specific examples to enhance the model. |
| Outcome: | The proposed method improves generalization and robustness of multiple CSC models across three datasets. |
Training Adaptive Computation for Open-Domain Question Answering with Computational Constraints (2021.acl-short)
Copied to clipboard
| Challenge: | Adaptive Computation (AC) has been shown to be effective in improving the efficiency of Open-Domain Question Answering systems. |
| Approach: | They propose an AC method that can be applied to an existing ODQA model and can be trained efficiently on a single GPU. |
| Outcome: | The proposed method improves upon a state-of-the-art model on two datasets and is more accurate than previous AC methods due to the stronger base ODQA model. |
An Empirical Study on Adversarial Attack on NMT: Languages and Positions Matter (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to generating NMT adversarial examples inject perturbations into source sentences or target translations to improve the robustness of NMT models. |
| Approach: | They investigate adversarial attack on NMT from two aspects: languages (the source vs. the target language) and positions (front v. rear). |
| Outcome: | The proposed approach is more effective than adversarial attacks by sampling positions randomly or according to gradients. |
OntoGUM: Evaluating Contextualized SOTA Coreference Resolution on 12 More Genres (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for coreference resolution are unable to evaluate generalizability to open domain data. |
| Approach: | They propose to make an OntoNotes-like coreference dataset publicly available and convert it into an English corpus. |
| Outcome: | The proposed dataset is the largest human-annotated coreference corpus following the OntoNotes guidelines and the first to be evaluated for consistency with the OnToNote's scheme. |
In Factuality: Efficient Integration of Relevant Facts for Visual Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Current Visual Question Answering (VQA) models are trained on labelled data that may be insufficient to learn complex knowledge representations. |
| Approach: | They propose a method to integrate external knowledge into a visual pre-trained model by integrating facts extracted from a knowledge base. |
| Outcome: | The proposed method outperforms baseline models on the KVQA dataset benchmark by 19% and shows that it is weaker than previous models. |
Zero-shot Fact Verification by Claim Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification require large datasets, which can be expensive. |
| Approach: | They propose a framework for training a robust fact verification model by using automatically generated claims that can be supported, refuted, or unverifiable from evidence from Wikipedia. |
| Outcome: | The proposed framework reduces the demand for human-annotated training data and improves a model's F1 from 50% to 77%, equivalent in performance to 2K+ manually-curated examples. |
Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer (2021.acl-short)
Copied to clipboard
| Challenge: | Formality style transfer models have limited success in preserving content due to the scarcity of parallel data. |
| Approach: | They propose to fine-tune pre-trained language and sequence-to-sequence models with rewards that target style and content to enhance content preservation. |
| Outcome: | The proposed models can be fine-tuned with rewards that target style and content, and achieve good performance even with limited amounts of parallel data. |
Deep Context- and Relation-Aware Learning for Aspect-based Sentiment Analysis (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for aspect-based sentiment analysis (ABSA) consider relationships implicitly among subtasks at the word level. |
| Approach: | They propose a deep contextualized relation-aware network that allows interactive relations among subtasks . they propose self-supervised strategies that deal with multiple aspects . |
| Outcome: | The proposed method outperforms state-of-the-art methods on three widely used benchmarks. |
Towards Generative Aspect-Based Sentiment Analysis (2021.acl-short)
Copied to clipboard
| Challenge: | Existing work on Aspect-based sentiment analysis ignores the rich label semantics of ABSA. |
| Approach: | They propose to tackle various ABSA tasks in a unified generative framework . they propose to use annotation-style and extraction-style modeling to enable training . |
| Outcome: | The proposed framework achieves state-of-the-art on four ABSA tasks across multiple benchmark datasets. |
Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to token-level adaptive training only use static word frequency information without considering the source language. |
| Approach: | They propose a bilingual mutual information based adaptive objective that assigns weights to target tokens with higher BMI . they propose to use this approach to improve token-level adaptive training . |
| Outcome: | The proposed method improves token-level adaptive training on two languages. |
Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods to learn consecutive tasks without forgetting how to perform previously trained problems are lacking. |
| Approach: | They propose a continual learning method which preserves performance on previously encountered tasks while accelerating learning progress on subsequent tasks. |
| Outcome: | The proposed method preserves performance on previously encountered tasks while accelerating learning progress on subsequent tasks. |
TIMERS: Document-level Temporal Relation Extraction (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for temporal relation extraction focus on extracting temporal relations between event pairs present in the same sentence or adjacent sentences, mostly ignoring document-level pairs. |
| Approach: | They propose a TIME, Rhetorical and Syntactic-aware model for document-level temporal relation classification in the English language that leverages rhetorical discourse features and temporal arguments from semantic role labels. |
| Outcome: | The proposed model outperforms previous methods on the TDDiscourse, TimeBank-Dense, and MATRES datasets due to its discourse-level modeling. |
Improving Arabic Diacritization with Regularized Decoding and Adversarial Training (2021.acl-short)
Copied to clipboard
| Challenge: | Existing studies regard auto-generated knowledge instances as gold references, which limits their effectiveness since they are not always accurate and inferior instances can lead to incorrect predictions. |
| Approach: | They propose to use regularized decoding and adversarial training to appropriately learn from noisy knowledge instances for Arabic diacritization. |
| Outcome: | The proposed model outperforms existing models on two benchmark datasets even with flawed auto-generated knowledge. |
When is Char Better Than Subword: A Systematic Study of Segmentation Algorithms for Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Subword segmentation algorithms can produce sub-optimal segmentation when the target language is rich in morphological changes or there is not enough data for learning compact composition rules. |
| Approach: | They compare character-based and subword-based neural machine translation systems . they find character-driven models are better at handling morphological phenomena . |
| Outcome: | The character-based models are better at handling morphological phenomena, generating rare and unknown words, and more suitable for transferring to unseen domains. |
More than Text: Multi-modal Chinese Word Segmentation (2021.acl-short)
Copied to clipboard
| Challenge: | Currently, word segmentation is performed in many languages without word delimiters. |
| Approach: | They propose to combine the multi-modality to perform Chinese word segmentation . they propose a time-dependent multi-module interactive model to integrate multi-modality information . |
| Outcome: | The proposed model integrates multi-modal information for word sequence labeling with Chinese language as target . the proposed model performs well on three training sets on Chinese and other languages without word delimiters. |
A Mixture-of-Experts Model for Antonym-Synonym Discrimination (2021.acl-short)
Copied to clipboard
| Challenge: | Anatomy-synonymy discrimination (ASD) is a crucial problem in lexical semantics and is difficult to distinguish between antonyms and synonyms. |
| Approach: | They propose a divide-and-conquer strategy where localized experts focus on their own domains to learn their specialties. |
| Outcome: | The proposed method achieves state-of-the-art performance on the Antonymy-synonymy discrimination task. |
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking (2021.acl-short)
Copied to clipboard
| Challenge: | Existing work on transferring domain-specific knowledge from a pretraining model to a resource-poor language is limited to English . a novel cross-lingual biomedical entity linking task is proposed to improve this capability. |
| Approach: | They propose a cross-lingual biomedical entity linking task and establish a new benchmark spanning 10 typologically diverse languages. |
| Outcome: | The proposed methods yield consistent gains across all target languages, sometimes up to 20 Precision@1 points, without any in-domain knowledge in the target language and without any parallel data. |
A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to address the representation degeneration problem in contextual embedding spaces require a learning process to retrain models with additional objectives. |
| Approach: | They propose a local cluster-based method to address the representation degeneration problem in contextual embedding spaces by removing local dominant directions from verb representations. |
| Outcome: | The proposed method improves CWRs performance on semantic tasks by removing dominant directions of verb representations. |
Unsupervised Enrichment of Persona-grounded Dialog with Background Stories (2021.acl-short)
Copied to clipboard
| Challenge: | Existing dialog models do not contain such narratives, so we propose a gradient-based rewriting technique to enrich dialog personas with relevant background events. |
| Approach: | They propose to use existing dialog datasets to enrich dialog responses with 'background stories' based on a gradient-based rewriting technique which encourages the generated response to be fluent with the dialog history, minimally different from the retrieved story, and consistent with the original persona. |
| Outcome: | The proposed method generates responses that are more diverse and human-like compared to outputs from existing dialog models. |
Beyond Laurel/Yanny: An Autoencoder-Enabled Search for Polyperceivable Audio (2021.acl-short)
Copied to clipboard
| Challenge: | a popular audio clip elicits dramatically different responses from different listeners . a "laurel/yanny" phenomenon is a common phenomenon in spoken language . polyperceivability is surprisingly prevalent in natural language, a new study finds . |
| Approach: | They propose a metric that correlates with polyperceivability of audio clips and validate results with humans . they find new "laurel/yanny"-type examples that are surprisingly prevalent in natural language . |
| Outcome: | The proposed metric correlates with polyperceivability of audio clips and validates it with human experiments. |
Don’t Let Discourse Confine Your Model: Sequence Perturbations for Improved Event Language Models (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to train event language models on text constrain them to follow discourse order of events. |
| Approach: | They propose a method to perturb event sequences so that they can relax model dependence on text order. |
| Outcome: | The proposed technique improves performance on applications and out-of-domain events data. |
The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes (2021.acl-short)
Copied to clipboard
| Challenge: | Existing studies have shown that dense representations outperform sparse representations with large index sizes. |
| Approach: | They propose to use dense low-dimensional representations to retrieve relevant documents . they show performance decreases quicker for increasing index sizes than for sparse representations . |
| Outcome: | The proposed representations outperform sparse representations with large index sizes. |
Cross-lingual Text Classification with Heterogeneous Graph Neural Network (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual text classification only consider factors beyond semantic similarity, causing performance degradation between some language pairs. |
| Approach: | They propose a method to incorporate heterogeneous information within and across languages for cross-lingual text classification using graph convolutional networks. |
| Outcome: | The proposed method significantly outperforms state-of-the-art models on all tasks and achieves consistent performance gain over baselines in low-resource settings. |
Towards more equitable question answering systems: How much more data do you need? (2021.acl-short)
Copied to clipboard
| Challenge: | Question answering datasets in English are relatively new, but lack of linguistic diversity in the field is a challenge. |
| Approach: | They propose to use translation and cross-lingual transfer to produce QA systems in multiple languages to improve their performance. |
| Outcome: | The proposed approaches take advantage of existing resources to produce QA systems in multiple languages. |
Embedding Time Differences in Context-sensitive Neural Networks for Learning Time to Event (2021.acl-short)
Copied to clipboard
| Challenge: | Current approaches focus on news articles and expect at least one temporal expressions in each input data to predict TTE. |
| Approach: | They propose a context-sensitive neural model for time to event prediction task . they enrich the model with time difference embeddings to improve accuracy . |
| Outcome: | The proposed model is 1.4 and 3.3 hours more accurate than the current state-of-the-art model on English and Dutch tweets respectively. |
Improving Compositional Generalization in Classification Tasks via Structure Annotations (2021.acl-short)
Copied to clipboard
| Challenge: | Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. |
| Approach: | They propose to convert a natural language sequence-to-sequence dataset into a classification dataset that requires compositional generalization. |
| Outcome: | The proposed model can generalize compositionally by providing hints on the structure of the input. |
Learning to Generate Task-Specific Adapters from Task Description (2021.acl-short)
Copied to clipboard
| Challenge: | Pre-trained text-to-text transformers have achieved impressive performance across a range of NLP tasks, such as question answering and commonsense reasoning. |
| Approach: | They propose a framework that improves text-to-text transformer’s generalization ability to unseen tasks by training a hypernetwork to generate task-specific adapters from task descriptions. |
| Outcome: | Experiments on ZEST and a synthetic SQuAD dataset show that Hypter improves upon fine-tuning baselines. |
QA-Driven Zero-shot Slot Filling with Weak Supervision Pretraining (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods to predict slots and their values do not encode enough semantic information, limiting the models’ zero-shot capability. |
| Approach: | They propose a QA-driven slot filling model which extracts slot-filler spans from utterances with a span-based QA model. |
| Outcome: | The proposed model outperforms baselines by over 5% on the SNIPS benchmark. |
Domain-Adaptive Pretraining Methods for Dialogue Understanding (2021.acl-short)
Copied to clipboard
| Challenge: | Recent advances in pretraining methods have achieved promising results on NLP tasks . however, it is unclear which pretraining objective is the most effective for each downstream task . |
| Approach: | They evaluate the effectiveness of domain-adaptive pretraining objectives on downstream tasks . they use open-domain data to pretrain language models like BERT and SpanBERT . |
| Outcome: | The proposed model improves on two dialogue understanding tasks with domain-adaptive pretraining objectives. |
Targeting the Benchmark: On Methodology in Current Natural Language Processing Research (2021.acl-short)
Copied to clipboard
| Challenge: | a language benchmark is a task devised that is restricted enough to be managable with current methods, but is deemed challenging enough to serve as a benchmark. |
| Approach: | They propose to use a language task as a benchmark and a baseline model to argue it is challenging enough to be a good one. |
| Outcome: | The proposed language benchmarks are based on a dataset and a language task . the proposed benchmarks can be used to measure progress towards the goal of the research . |
X-Fact: A New Benchmark Dataset for Multilingual Fact Checking (2021.acl-short)
Copied to clipboard
| Challenge: | Several fact-checking initiatives, such as PolitiFact, expend manual labor to investigate and determine the truthfulness of viral statements. |
| Approach: | They propose a multilingual dataset for factual verification of naturally existing claims . they use a benchmark to evaluate the multilingual models . |
| Outcome: | The proposed model achieves an F-score of around 40%, suggesting it is a challenging benchmark for multilingual fact-checking models. |
nmT5 - Is parallel data still relevant for pre-training massively multilingual language models? (2021.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that cross-lingual transfer learning in pre-trained multilingual models could be improved further by incorporating parallel data. |
| Approach: | They propose to integrate parallel data into mT5 pre-training to improve results on downstream multilingual and cross-lingual tasks. |
| Outcome: | The proposed model improves cross-lingual transfer significantly in small fine-tuning datasets and small model sizes. |
Question Generation for Adaptive Education (2021.acl-short)
Copied to clipboard
| Challenge: | Existing systems depend on a pool of hand-made questions, limiting how fine-grained and open-ended they can be in adapting to individual students. |
| Approach: | They propose to fine-tune pre-trained language models for deep knowledge tracing to generate reversetranslation questions conditioned on the student and target difficulty. |
| Outcome: | The proposed model can generate well-calibrated language translation questions for second language learners from a real online education platform. |
A Simple Recipe for Multilingual Grammatical Error Correction (2021.acl-short)
Copied to clipboard
| Challenge: | Modern approaches view the task of Grammatical Error Correction (GEC) as monolingual text-to-text rewriting and employ encoderdecoder neural architectures. |
| Approach: | They propose a language-agnostic method to generate a large number of synthetic examples and use large-scale multilingual language models to train state-of-the-art GEC models. |
| Outcome: | The proposed method surpasses state-of-the-art results on GEC benchmarks in English, Czech, German and Russian. |
Towards Visual Question Answering on Pathology Images (2021.acl-short)
Copied to clipboard
| Challenge: | Pathology imaging is used for identifying the causes and effects of diseases or injuries. |
| Approach: | They propose a pathological visual question answering framework to analyze pathology images and answer medical questions related to these images. |
| Outcome: | The proposed framework performs self-supervised pretraining and finetuning end-to-end to learn powerful visual and textual representations jointly and automatically identifies and excludes noisy self-controlled examples from pretraining. |
Efficient Text-based Reinforcement Learning by Jointly Leveraging State and Commonsense Graph Representations (2021.acl-short)
Copied to clipboard
Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Kartik Talamadupula, Mrinmaya Sachan, Murray Campbell
| Challenge: | Text-based games (TBGs) are useful benchmarks for evaluating progress in grounded language understanding and reinforcement learning (RL). |
| Approach: | They propose an agent that induces a graph representation of the game state and jointly grounds it with a commonsense knowledge from ConceptNet. |
| Outcome: | The proposed agent outperforms baseline agents in the proposed game . |
mTVR: Multilingual Moment Retrieval in Videos (2021.acl-short)
Copied to clipboard
| Challenge: | mTVR is a multilingual video moment retrieval dataset with 218K queries in English and Chinese . Various datasets have been proposed or adapted for the task, but they are all created for a single language (English). |
| Approach: | They propose a multilingual video moment retrieval dataset with 218K queries from 21.8K TV show video clips. |
| Outcome: | The proposed model outperforms strong monolingual baselines while using fewer parameters. |
Explicitly Capturing Relations between Entity Mentions via Graph Neural Networks for Domain-specific Named Entity Recognition (2021.acl-short)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is well studied for the general domain, but the performance is still moderate for specialized domains. |
| Approach: | They propose to explicitly connect entity mentions based on global coreference relations and local dependency relations to build better entity mention representations. |
| Outcome: | The proposed system improves the NER performance even with a tiny amount of labeled data. |
Improving Lexically Constrained Neural Machine Translation with Source-Conditioned Masked Span Prediction (2021.acl-short)
Copied to clipboard
| Challenge: | Accurate terminology translation is crucial for ensuring the practicality and reliability of neural machine translation systems. |
| Approach: | They propose a method to preserve terminology in translations as lexical constraints with or without a term dictionary at test time. |
| Outcome: | The proposed setup achieves consistent improvements on terminology and sentence-level translation for three domain-specific corpora in two language pairs. |
Quotation Recommendation and Interpretation Based on Transformation from Queries to Quotations (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models of quotation recommendation ignore the relationship between quotations and queries. |
| Approach: | They propose a transformation matrix that directly maps quotations to quotation representations. |
| Outcome: | The proposed model outperforms state-of-the-art models on two datasets in English and Chinese. |
Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence (2021.acl-short)
Copied to clipboard
| Challenge: | Recent neural topic models extract words from documents, but they are not coherent . coherence is crucial for topic models, but many use bag-of-words document representations as input . pre-trained language models are becoming ubiquitous in natural language processing . |
| Approach: | They combine contextualized representations with neural topic models to produce more coherent topics . they say that future improvements in language models will translate into better topic models . |
| Outcome: | The proposed approach produces more meaningful and coherent topics than bag-of-words models and recent neural models. |
Input Representations for Parsing Discourse Representation Structures: Comparing English with Chinese (2021.acl-short)
Copied to clipboard
| Challenge: | Neural semantic parsers have obtained acceptable results in parsing DRSs . previous studies have focused on parse of DRS in English, but have focused only on a few languages . |
| Approach: | They propose to use character sequences as input to map meaning representations to string format. |
| Outcome: | The proposed models learn the meaning of a series of semantic phenomena by taking sentences as input and outputting the corresponding DRSs, without the aid of any extra linguistic information. |
Code Generation from Natural Language with Less Prior Knowledge and More Monolingual Data (2021.acl-short)
Copied to clipboard
| Challenge: | a generic transformer-based model can achieve competitive performance with minimal code-generation-specific inductive bias design. |
| Approach: | They investigate whether a generic transformer-based seq2seq model can achieve competitive performance with minimal code-generation-specific inductive bias design. |
| Outcome: | The proposed model achieves 81.03% exact match accuracy on Django and 32.57 BLEU score on CoNaLa. |
Issues with Entailment-based Zero-shot Text Classification (2021.acl-short)
Copied to clipboard
| Challenge: | Pre-trained BERT models with no fine-tuning can yield competitive performance against BERT fine- tuned for NLI. |
| Approach: | They propose to use any target label into a sentence of hypothesis and verify whether it could be entailed by the input. |
| Outcome: | The proposed models perform better than models fine-tuned for BERT, but the results are in general negative. |
Neural-Symbolic Commonsense Reasoner with Relation Predictors (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models for commonsense reasoning are limited by their limited set of facts, rendering them unfit for reasoning over new unseen situations and events. |
| Approach: | They propose a neural-symbolic reasoner which can combine commonsense facts with large-scale dynamic CKGs to draw conclusions about ordinary situations. |
| Outcome: | The proposed model outperforms the state-of-the-art models on the task of link prediction on CKGs. |
What Motivates You? Benchmarking Automatic Detection of Basic Needs from Short Posts (2021.acl-short)
Copied to clipboard
| Challenge: | According to the self-determination theory, the levels of satisfaction of three basic needs (competence, autonomy and relatedness) have implications on people’s everyday life and career. |
| Approach: | They propose to model a task that automatically detects three basic needs on short posts in English and then apply them to a binary task. |
| Outcome: | The proposed model achieves similar performance as a trained human annotator in the real-world. |
Semantic Frame Induction using Masked Word Embeddings and Two-Step Clustering (2021.acl-short)
Copied to clipboard
| Challenge: | Recent studies show that clustering-based methods focus too much on the surface information of frame-evoking verbs and divide instances of the same verb into too many different frame clusters. |
| Approach: | They propose a semantic frame induction method using masked word embeddings and two-step clustering to overcome these drawbacks. |
| Outcome: | The proposed method reduces the number of instances of the same verb into too many clusters . it uses masked word embeddings and two-step clustering to avoid drawbacks compared with other methods . |
Lightweight Adapter Tuning for Multilingual Speech Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Adapter tuning is an efficient alternative to fine-tuning in NLP . a multilingual model could be outperformed by its bilingual counterparts . |
| Approach: | They propose to use adapter tuning to optimize for multilingual speech translation . they use pre-trained models to freeze pre-train parameters and inject lightweight modules . |
| Outcome: | The proposed adapters can specialize to specific language pairs with low extra cost . the proposed models outperform bilingual models on high-resource language pairs . |
Parameter Selection: Why We Should Pay More Attention to It (2021.acl-short)
Copied to clipboard
| Challenge: | supervised learning is a challenging process due to the huge number of parameter combinations. |
| Approach: | They present an example of parameter selection in supervised learning . authors use a set of frequently occurring labels without a parameter tuning . they say this illustrates the seriousness of parameter tuning in a supervised field . |
| Outcome: | The proposed study shows that without adequate attention, the research progress can be uncertain or even illusive. |
Distinct Label Representations for Few-Shot Text Classification (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for few-shot text classification ignore the semantic relevance of labels and are difficult to train because of the lack of training examples. |
| Approach: | They propose a method that generates distinct label representations that embed information specific to each label. |
| Outcome: | The proposed method significantly improves few-shot text classification across models and datasets. |
Learning to Solve NLP Tasks in an Incremental Number of Languages (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models that support multiple languages are expensive and time-consuming to retrain . retraining only with the new language data will result in Catastrophic Forgetting of previously acquired knowledge. |
| Approach: | They propose a Continual Learning strategy that updates a model to support new languages over time while maintaining consistent results on previously learned languages. |
| Outcome: | The proposed model is based on an existing model that "teaches" to a student while the student is also trained on a new language. |
Hi-Transformer: Hierarchical Interactive Transformer for Efficient and Effective Long Document Modeling (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to model long documents are difficult due to the quadratic complexity of text length. |
| Approach: | They propose a hierarchical interactive Transformer for efficient long document modeling. |
| Outcome: | Extensive experiments on three benchmark datasets validate the efficiency and effectiveness of Hi-Transformer in long document modeling. |
Robust Transfer Learning with Pretrained Language Models through Adapters (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to transfer learning with pretrained transformer-based language models are not robust and can be adversarial. |
| Approach: | They propose a simple yet effective adapter-based approach to fine-tune language models on downstream tasks. |
| Outcome: | The proposed approach improves stability and adversarial robustness in transfer learning to various downstream tasks. |
Embracing Ambiguity: Shifting the Training Target of NLI Models (2021.acl-short)
Copied to clipboard
| Challenge: | Previously, it was common to disregard ambiguity as noise or as a sign of poor quality data. |
| Approach: | They propose to train on the estimated label distribution of annotators in a NLI task . they use a learning loss based on this ambiguity distribution instead of gold-labels . |
| Outcome: | The proposed training method reduces divergence scores on a trial dataset . the study shows that targeting the ambiguity distribution can improve performance . |
Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods of OOD detection only focus on whether a sample is correctly classified . lack of real OOD examples leads to poor prior knowledge about these unknown intents . |
| Approach: | They propose a supervised contrastive learning objective to minimize intra-class variance . they employ an adversarial augmentation mechanism to obtain pseudo diverse views . |
| Outcome: | The proposed method minimizes intra-class variance by pulling together in-domain intents belonging to the same class and maximizes inter-class variation by pushing apart samples from different classes. |
Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking (2021.acl-short)
Copied to clipboard
| Challenge: | Existing dialog state tracking models neglect rich structural information in a dataset. |
| Approach: | They propose to use curriculum learning to leverage dialog state tracking data . they propose a model-agnostic framework that pre-trains a DST model with schema information . |
| Outcome: | The proposed framework improves performance over a transformer-based and RNN-based model on WOZ2.0 and MultiWOZ2.1. |
On the Generation of Medical Dialogs for COVID-19 (2021.acl-short)
Copied to clipboard
Meng Zhou, Zechen Li, Bowen Tan, Guangtao Zeng, Wenmian Yang, Xuehai He, Zeqian Ju, Subrato Chakravorty, Shu Chen, Xingyi Yang, Yichen Zhang, Qingyang Wu, Zhou Yu, Kun Xu, Eric Xing, Pengtao Xie
| Challenge: | under the pandemic of COVID-19, people experiencing COVI D19-related symptoms have a pressing need to consult doctors. |
| Approach: | They develop a medical dialog system that can provide COVID19-related consultations . they use two dialog datasets containing conversations between doctors and patients . |
| Outcome: | The proposed system can provide COVID19-related consultations, but is too small compared with general-domain dialog datasets. |
Constructing Multi-Modal Dialogue Dataset by Replacing Text with Semantically Relevant Images (2021.acl-short)
Copied to clipboard
| Challenge: | Existing training methods for multi-modal dialogue systems rely on image captioning or visual question answering datasets that are irrelevant to the dialogue context. |
| Approach: | They propose to create a 45k multi-modal dialogue dataset with minimal human intervention . they use text dialogue datasets, image-mixed dialogues and contextual-similarity filtering . |
| Outcome: | The proposed dataset can be used as training data for multi-modal dialogue systems . human evaluations show that the model can be effectively used . |
Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection (2021.acl-short)
Copied to clipboard
| Challenge: | a lack of labeled, non-English resources for hate speech detection limits research on hate speech . a recent study shows that zero-shot, cross-lingual learning models cannot be used as they are . lack of consistency limits research, and lack of models for non-english languages limits learning . |
| Approach: | They propose a zero-shot, cross-lingual transfer learning framework for hate speech detection . they use benchmark data sets in English, Italian, and Spanish to detect hate speech . |
| Outcome: | The proposed framework can't be used as it is, but needs to be carefully designed, the authors say . they find that non-hateful, language-specific taboo interjections are misinterpreted as signals of hate speech . |
BERTTune: Fine-Tuning Neural Machine Translation with BERTScore (2021.acl-short)
Copied to clipboard
| Challenge: | Neural machine translation models are biased toward limited translation references . BERTScore is a scoring function based on contextual embeddings that overcomes the limitations of n-gram-based metrics. |
| Approach: | They propose to fine-tune models with a new evaluation metric based on contextual embeddings to overcome the limitations of n-gram-based metrics. |
| Outcome: | The proposed training objective improves translations that are different from the translations but close in the contextual embedding space. |
Entity Enhancement for Implicit Discourse Relation Classification in the Biomedical Domain (2021.acl-short)
Copied to clipboard
| Challenge: | Discourse relation classification is a challenging task when the text domain is different from the standard Penn Discourse Treebank (PDTB) training corpus domain. |
| Approach: | They propose to use the Biomedical Discourse Relation Bank to improve discourse relational argument representation by linking explicit instances of similar relations with a voting pipeline. |
| Outcome: | The proposed model outperforms the pre-trained BioBERT model by 2% points. |
Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction (2021.acl-short)
Copied to clipboard
| Challenge: | Masked Noun-Phrase Prediction (MNPP) is a pre-training strategy to tackle pronoun resolution in a fully unsupervised setting. |
| Approach: | They propose a pre-training strategy to tackle pronoun resolution in an unsupervised setting by fine-tuning a large pre-trained model on a human-labeled dataset and then transferring to a smaller dataset such as Winograd Schema Challenge (WSC). |
| Outcome: | The proposed method outperforms all previous unsupervised methods on all datasets by large margins. |
Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021.acl-short)
Copied to clipboard
| Challenge: | Recent work focuses on question answering based on machine reading comprehension . current approaches treat QA as extracting a consecutive piece of text to a given question. |
| Approach: | They propose a generative QA model that incorporates an extractive mechanism into a model. |
| Outcome: | The proposed model improves quality and semantic accuracy over baseline models. |
Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning (2021.acl-short)
Copied to clipboard
| Challenge: | In news articles the lead bias dominates the learning signals for neural extractive summarizations, severely limiting their performance on data with different or even no bias. |
| Approach: | They propose a method to demote the lead bias in news and make the model focus more on the content semantics. |
| Outcome: | The proposed method can demote the model’s learned lead bias and improve its generality on out-of-distribution data with little to no performance loss on in-difference data. |
DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications (2021.acl-short)
Copied to clipboard
| Challenge: | In order to comprehensively verify the robustness and generalization of MRC models, we construct a real-world Chinese dataset - DuReader_robust . |
| Approach: | They introduce a real-world Chinese dataset to evaluate the robustness and generalization of MRC models from three aspects: over-sensitivity, over-stability and generalisation. |
| Outcome: | The proposed model fails to perform well on the challenge test set and may provide suggestions for future model development. |
Sequence to General Tree: Knowledge-Guided Geometry Word Problem Solving (2021.acl-short)
Copied to clipboard
| Challenge: | Existing neural solvers only generate binary expression trees that contain basic arithmetic operators and do not explicitly use the math formulas. |
| Approach: | They propose a sequence-to-general tree that generates interpretable and executable operation trees where nodes can be formulas with an arbitrary number of arguments. |
| Outcome: | The proposed tree generates interpretable and executable operation trees with formulas with an arbitrary number of arguments. |
Multi-Scale Progressive Attention Network for Video Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Experimental evaluations on three benchmarks: TGIF-QA, MSVD-QA and MSRVTT-QA show our method has achieved state-of-the-art performance. |
| Approach: | They propose a multi-scale progressive attention network to fuse visual and text information. |
| Outcome: | The proposed method achieves state-of-the-art on three benchmarks: TGIF-QA, MSVD-QA and MSRVTT-QA. |
Efficient Passage Retrieval with Hashing for Open-domain Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Open-domain question answering systems often require large memory to run because of the massive size of their passage index. |
| Approach: | They propose a memory-efficient neural retrieval model that integrates a learning-to-hash technique into the state-of-the-art Dense Passage Retriever to represent the passage index using compact binary codes. |
| Outcome: | The proposed model significantly reduces memory cost from 65GB to 2GB without loss of accuracy on two open-domain question answering benchmarks. |
Entity Concept-enhanced Few-shot Relation Extraction (2021.acl-short)
Copied to clipboard
| Challenge: | Existing FSRE methods fail to classify relations based on information of sentences and entity pairs due to limited samples and lack of knowledge. |
| Approach: | They propose a concept-sentence attention module to select the most appropriate concept from multiple concepts of each entity by calculating the semantic similarity between sentences and concepts. |
| Outcome: | The proposed scheme outperforms existing methods on a few-shot relation extraction dataset. |
Improving Model Generalization: A Chinese Named Entity Recognition Case Study (2021.acl-short)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental building block for various downstream natural language processing tasks due to the ambiguous word boundaries and complex composition. |
| Approach: | They propose to resample entities within the same category to encourage a model to leverage both name and context knowledge in the training process. |
| Outcome: | The proposed method significantly improves a model’s ability to detect unseen entities, especially for company, organization and position categories. |
Three Sentences Are All You Need: Local Path Enhanced Document Relation Extraction (2021.acl-short)
Copied to clipboard
| Challenge: | Document-level relation extraction (RE) is more challenging than sentence RE as it often requires reasoning over multiple sentences. |
| Approach: | They propose a method to heuristically select evidence sentences for document-level relation extraction. |
| Outcome: | The proposed method can be easily combined with BiLSTM to achieve good performance on benchmark datasets even better than fancy graph neural network based methods. |
Unsupervised Cross-Domain Prerequisite Chain Learning using Variational Graph Autoencoders (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods to learn prerequisite relations between concepts require annotated concept pairs during training. |
| Approach: | They propose to use an optimized variational graph autoencoder to learn prerequisite chains in unsupervised manner using an information-rich domain and an information poor domain. |
| Outcome: | The proposed model learns to transfer concept prerequisite relations from an information-rich domain (source domain) to an information poor domain (target domain) the annotated data and resources as well as the code will be made publicly available. |
Attentive Multiview Text Representation for Differential Diagnosis (2021.acl-short)
Copied to clipboard
| Challenge: | Using data from the Undiagnosed Diseases Network, we find that NLP algorithms can reproduce the performance of clinical experts in the task of differential diagnosis. |
| Approach: | They propose a text representation approach that can combine different views of the same input to improve ranking. |
| Outcome: | The proposed model outperforms several ranking approaches by effectively prioritizing and combining representations obtained from traditional and recent text representation techniques. |
MedNLI Is Not Immune: Natural Language Inference Artifacts in the Clinical Domain (2021.acl-short)
Copied to clipboard
| Challenge: | a large number of crowdworker-constructed datasets have been used to conduct natural language inference (NLI) on unstructured, domainspecific texts such as patient notes, pathology reports, and scientific papers. |
| Approach: | They investigate whether MedNLI contains lexical and syntactic annotation artifacts associated with annotation process that allow hypothesis-only classifiers to achieve better-than-random performance. |
| Outcome: | The proposed model outperforms a majority-class baseline model on a physician-annotated dataset with premises extracted from clinical notes. |
Towards a more Robust Evaluation for Conversational Question Answering (2021.acl-short)
Copied to clipboard
| Challenge: | Conversational Question Answering (CQA) is a new form of NLP . it uses conversation history to extract the answer of the current question. |
| Approach: | They propose to use conversation history to evaluate models which can access the ground truth answers of previous turns at each turn of the conversation. |
| Outcome: | The proposed evaluation protocol severely limits the effectiveness of the proposed models in fully autonomous chatbots and leads to unsuspected biases in their behavior. |
VAULT: VAriable Unified Long Text Representation for Machine Reading Comprehension (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models on Machine Reading Comprehension (MRC) require complex model architecture for effectively modeling long texts with paragraph representation and classification, making inference computationally inefficient for production use. |
| Approach: | They propose a novel Gaussian distribution-based paragraph representation for Machine Reading Comprehension (MRC) that is light-weight and parallel-efficient. |
| Outcome: | The proposed model can achieve comparable performance on Wikipedia-based (NQ) and TechNotes (TechQA) with a state-of-the-art (SOTA) complex document modeling approach while being 16 times faster, demonstrating the efficiency of the proposed model. |
Avoiding Overlap in Data Augmentation for AMR-to-Text Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Using unlabeled data to boost model performance is common practice in machine learning and natural language processing. |
| Approach: | They propose methods for excluding parts of Gigaword to remove overlap . they propose to use the AMR dataset for AMR-to-text generation . |
| Outcome: | The proposed approach leads to a more realistic evaluation of the task of AMR-to-text generation. |
Weakly-Supervised Methods for Suicide Risk Assessment: Role of Related Domains (2021.acl-short)
Copied to clipboard
| Challenge: | Among social media platforms, Reddit has emerged as the most promising one due to its anonymity and its focus on topic-based communities (subreddits) . a challenge for previous work on suicide risk assessment has been the small amount of labeled data. |
| Approach: | They propose to use social media to collect user data from r/SuicideWatch subreddit and annotate it with user-level suicide risk: no-risk, low-risk and high-risk. |
| Outcome: | The proposed model improves by using pseudo-labeling based on related issues around mental health (e.g., anxiety, depression) |
Can Transformer Models Measure Coherence In Text: Re-Thinking the Shuffle Test (2021.acl-short)
Copied to clipboard
| Challenge: | Recent work shows that modern NLP models can detect shuffled text without supervision. |
| Approach: | They propose to use Shuffle Test to evaluate whether NLP models can measure coherence in text . they argue that this is unlikely to lead to a good model of text coherency . |
| Outcome: | The Shuffle Test is the most common task to evaluate whether NLP models can measure coherence in text. |
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization (2021.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that SimCLS can improve existing top-performing models by a large margin. |
| Approach: | They propose a framework for abstractive summarization that is conceptually simple and empirically powerful. |
| Outcome: | The proposed framework improves the performance of top-performing models by a large margin against existing top-scoring systems. |
SaRoCo: Detecting Satire in a Novel Romanian Corpus of News Articles (2021.acl-short)
Copied to clipboard
| Challenge: | a corpus for satire detection in Romanian news is based on satirical reporting . the goal is to ridicule public figures, politics or contemporary events . |
| Approach: | They propose a corpus for satire detection in Romanian news . they gather 55,608 public news articles from multiple real and satirical sources . |
| Outcome: | The proposed corpus is one of the largest corpora for satire detection regardless of language . it is the only one for the Romanian language, and the results show that it is low on the machine level compared to human level . |
Bringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents (2021.acl-short)
Copied to clipboard
| Challenge: | Faceted summarization provides briefings of a document from different perspectives. |
| Approach: | They propose a faceted summarization benchmark built on Emerald journal articles . they propose faceted models that bring structure into faceted documents . |
| Outcome: | The proposed benchmark is based on Emerald journal articles and covers a diverse range of domains. |
Replicating and Extending “Because Their Treebanks Leak”: Graph Isomorphism, Covariants, and Parser Performance (2021.acl-short)
Copied to clipboard
| Challenge: | a small sample size and unreliable results suggest a correlation between parser performance and graph isomorphism is not observed in the wild. |
| Approach: | They propose to replicate a study which found graph isomorphism is a non-trivial variable . they also bin sentences by length and find correlation between parser performance and isopathism disappears . |
| Outcome: | The results show that the original analysis was unreliable and had methodological issues . the study also bin sentences by length and shows that the correlation between parser performance and graph isomorphism disappears when controlling for covariants. |
Don’t Rule Out Monolingual Speakers: A Method For Crowdsourcing Machine Translation Data (2021.acl-short)
Copied to clipboard
| Challenge: | High-performing machine translation systems require large amounts of training data in the form of parallel sentences, and translators are difficult to find and expensive. |
| Approach: | They propose a data collection strategy which uses graphics interchange formats (GIFs) as a pivot to collect parallel sentences from monolingual annotators. |
| Outcome: | The proposed method collects parallel sentences from monolingual annotators in Hindi, Tamil and English. |