Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Investigating label suggestions for opinion mining in German Covid-19 social media (2021.acl-long)
Copied to clipboard
| Challenge: | Existing difficulties in data annotation are due to prolonged data gathering processes or opinion surveys being subject to reactivity. |
| Approach: | They propose to use label suggestions to improve annotation efficiency in german Covid-19 data by providing annotators with pre-recorded annotations. |
| Outcome: | The proposed model improves inter-annotator agreement and annotation quality in a controlled study with social science students. |
How Did This Get Funded?! Automatically Identifying Quirky Scientific Achievements (2021.acl-long)
Copied to clipboard
| Challenge: | Humor is an important social phenomenon, serving complex social and psychological functions. |
| Approach: | They propose a novel algorithm for automatically detecting funny scientific papers . they use a dataset containing thousands of funny papers to learn classifiers . |
| Outcome: | The proposed task is based on a dataset containing thousands of funny scientific papers . it is a novel task that can be automated and improves on existing methods . |
Engage the Public: Poll Question Generation for Social Media Posts (2021.acl-long)
Copied to clipboard
| Challenge: | a novel application to generate poll questions for social media posts offers an easy way to hear the public's voice . for the silent majority, they tend to read others' messages instead of voicing their opinions with words . |
| Approach: | They propose to encode user comments and discover latent topics therein as contexts to generate poll questions for social media posts. |
| Outcome: | The proposed model outperforms popular models without exploiting topics from comments . human evaluations show it can generate high-quality polls useful to draw user engagements . |
HateCheck: Functional Tests for Hate Speech Detection Models (2021.acl-long)
Copied to clipboard
| Challenge: | Hate speech detection models are evaluated by measuring their performance on held-out test data using metrics such as accuracy and F1 score. |
| Approach: | They propose a suite of functional tests for hate speech detection models that measure model performance on held-out test data and then craft test cases to validate their quality. |
| Outcome: | The proposed tests show that the proposed models perform poorly on a small set of widely-used hate speech datasets. |
Unified Dual-view Cognitive Model for Interpretable Claim Verification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies constructing direct interactions between the claim and each single user response to capture evidence have shown remarkable success in interpretable claim verification. |
| Approach: | They propose a Dual-view model based on the views of Collective and Individual Cognition (CICD) that captures word-level semantics based . on individual cognition, they adjust the proportion between them to generate global evidence. |
| Outcome: | The proposed model is based on the views of collective and individual cognition and achieves state-of-the-art performance on three benchmark datasets. |
DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing systems for rap generation focus on rhyming lyrics but ignore rhythmic beats . rap lyrics need to be semantically meaningful and fashionable to convey interesting stories . |
| Approach: | They develop a Transformer-based rap generation system that can model both rhymes and rhythms. |
| Outcome: | The proposed system generates high-quality raps with rhymes and rhythms . it is based on a Transformer-based language model . |
PENS: A Dataset and Generic Framework for Personalized News Headline Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Using a dataset of Microsoft News, we propose a generic framework to personalize a text generator and establish personalized headlines. |
| Approach: | They propose a generic framework to personalize a news headline generator and establish personalized headlines by leveraging user behavioral data. |
| Outcome: | The proposed framework is based on user preference data and user preference injections to personalize a text generator and establish personalized headlines. |
Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization (2021.acl-long)
Copied to clipboard
| Challenge: | Text style transfer aims to alter the style of a sentence while preserving its content. |
| Approach: | They propose to remove style information at token level and fuse it to style representations using conditional layer normalization. |
| Outcome: | The proposed model outperforms the state-of-the-art models in terms of content preservation and fluency. |
Mention Flags (MF): Constraining Transformer-based Text Generators (2021.acl-long)
Copied to clipboard
| Challenge: | Constrained decoding algorithms produce hypotheses satisfying all constraints, but they are computationally expensive and can lower the generated text quality. |
| Approach: | They propose a Mention Flag mechanism which traces whether lexical constraints are satisfied in outputs of an S2S decoder. |
| Outcome: | The proposed models maintain higher constraint satisfaction and text quality than baseline models and other constrained decoding algorithms. |
Generalising Multilingual Concept-to-Text NLG with Language Agnostic Delexicalisation (2021.acl-long)
Copied to clipboard
| Challenge: | Concept-to-text Natural Language Generation requires a delexicalisation of the input, but this often requires that the input appears verbatim in the output text. |
| Approach: | They propose a method that uses multilingual pretrained embeddings to inflect words in their correct form during relexicalisation. |
| Outcome: | The proposed method outperforms monolingual models in concept-to-text and in low resource environments. |
Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances (2021.acl-long)
Copied to clipboard
| Challenge: | Recent intelligent open-domain chatbots have made substantial progress thanks to the rapid development of large-scale pre-training approaches. |
| Approach: | They propose a dynamic flow mechanism to model the context flow and a model to capture the information dynamics across dialogue utterances. |
| Outcome: | The proposed model outperforms the DialoGPT on the dialogue generation task. |
Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to predict dialogue state from scratch are inefficient and lead to errors . empirical results show that our method achieves 56.93%, 60.73%, and 58.04% joint accuracy on multi-domain conversations . |
| Approach: | They propose a dual-stage dialogue state tracking method that uses a slot selector and a Slot Value generator to predict the current dialogue state. |
| Outcome: | The proposed method achieves 56.93%, 60.73%, and 58.04% joint accuracy on multi-domain conversations. |
Transferable Dialogue Systems and User Simulators (2021.acl-long)
Copied to clipboard
| Challenge: | a lack of training data is limiting the development of dialogue systems . we develop a framework for creating dialogue data through self-play between agents . |
| Approach: | They propose a framework that can incorporate new dialogue scenarios through self-play between two agents. |
| Outcome: | The proposed framework is highly effective in bootstrapping the performance of two agents in transfer learning. |
BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing persona-based dialogue models use crowd-sourced data, such as the PersonaChat . however, the cost of such datasets is limited, and the model is not robust. |
| Approach: | They propose to disentangle persona-based dialogue generation into two sub-tasks by using a BERT-over-BERT model. |
| Outcome: | The proposed model outperforms baselines in response quality and persona consistency under different limited data settings. |
GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing joint models for multi-intent SLU only consider intent detection while ignoring slot filling task. |
| Approach: | They propose a non-autoregressive model for joint multiple intent detection and slot filling . their framework is 11.5 times faster than existing joint models . |
| Outcome: | The proposed model is 11.5 times faster than existing models and is faster than current models. |
Accelerating BERT Inference for Sequence Labeling via Early-Exit (2021.acl-long)
Copied to clipboard
| Challenge: | Existing early-exit mechanisms are designed for sequence-level tasks, rather than sequence labeling. |
| Approach: | They propose to extend sentence-level early-exit to accelerate inference of PTMs . they propose a token-level mechanism that allows partial tokens to exit early at different layers . |
| Outcome: | The proposed approach can save up to 66%75% inference cost with minimal performance degradation. |
Modularized Interaction Network for Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models focus on word-level information, while segment-based models focus only on word level information. |
| Approach: | They propose a Modularized Interaction Network (MIN) model which utilizes both word-level information and segment-level dependencies. |
| Outcome: | The proposed model outperforms the current state-of-the-art models on three NER benchmark datasets. |
Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder (2021.acl-long)
Copied to clipboard
| Challenge: | Existing efforts to capture event argument interactions are limited by the argument role type information of contextual entities. |
| Approach: | They propose to capture event argument interactions as a Seq2Seq-like learning problem where a sentence with a specific event trigger is mapped to a sequence of event argument roles. |
| Outcome: | The proposed neural architecture generates argument roles by incorporating contextual entities’ argument role predictions, like a word-by-word text generation process, thereby distinguishing implicit argument distribution patterns within an event more accurately. |
UniRE: A Unified Label Space for Entity Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing joint entity relation extraction models setup two separate label spaces for the two sub-tasks . |
| Approach: | They propose to eliminate the different treatment on the two sub-tasks’ label spaces by applying a unified classifier to predict each cell’s label. |
| Outcome: | The proposed model achieves competitive accuracy with the best extractor and is faster. |
Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to extract relationships from texts depend on memory size and replay these memorized samples in subsequent tasks. |
| Approach: | They propose to use a model to extract relations between entities from texts where the samples of different relations are delivered into the model continuously. |
| Outcome: | The proposed model outperforms the state-of-the-art models and avoids catastrophic forgetting. |
Contrastive Learning for Many-to-many Multilingual Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing multilingual machine translation approaches focus on English-centric directions, while non-English directions lag behind. |
| Approach: | They propose a multilingual machine translation system with an emphasis on non-English directions. |
| Outcome: | The proposed model outperforms existing models on English-centric and non-English directions on multilingual translation benchmarks. |
Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) currently exhibits biases such as producing translations that are too short and overgenerating frequent words. |
| Approach: | They propose to use minimum bayes risk decoding instead of beam search to investigate the effects of beam decoding on unbiased samples. |
| Outcome: | The proposed method improves on a number of previously reported biases and failure cases of beam search on unbiased samples. |
Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | a self-attention network can be easily parallelized at sequence level, but LSTMs are slower to train . a recent study shows that LS models require a lot of computations to perform . |
| Approach: | They propose to compute LSTMs at sequence level to enable sequence-level parallelization . they use a bag-of-words representation of the preceding tokens context to approximate LStms . |
| Outcome: | The proposed model performs better than existing models while being faster to train . the model can be trained efficiently due to the highly parallelized self-attention network . |
A Bidirectional Transformer Based Alignment Model for Unsupervised Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning word alignment include statistical word aligners (e.g. GIZA++) Existing word alignment models employ a target-to-source attention mechanism which can provide rough word alignments but with a low accuracy. |
| Approach: | They propose a bidirectional Transformer based alignment model for unsupervised learning of the word alignment task. |
| Outcome: | The proposed model outperforms both previous neural word alignment approaches and the popular statistical word aligner GIZA++ on three word alignment tasks. |
Learning Language Specific Sub-network for Multilingual Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation models suffer from performance degradation when learning multiple languages. |
| Approach: | They propose to use LaSS to jointly train a single unified multilingual MT model. |
| Outcome: | The proposed model gains on 36 language pairs by up to 1.2 BLEU and zero-shot translation with 8.3 BLUE on 30 language pairs. |
Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve performance of deep neural models are limited by the nature of spurious patterns in the data. |
| Approach: | They propose to use augmented data to generate spurious patterns in NLP models . they propose to generate counterfactual data for data augmentation and explanation . |
| Outcome: | The proposed approach improves performance on augmented data and on human-generated data. |
Bridge-Based Active Domain Adaptation for Aspect Term Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to transfer aspect terms are limited because they require labeled pivot words or expensive computing resources. |
| Approach: | They propose a method that actively supplements transferable knowledge by recognizing syntactic roles as pivots instead of links to pivots. |
| Outcome: | The proposed method significantly outperforms existing methods. |
Multimodal Sentiment Detection Based on Multi-channel Graph Neural Networks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies only considered the representation of a single image-text post . Fig. 1 shows that multimodal sentiment expressions have global characteristics . |
| Approach: | They propose a multi-channel Graph Neural Networks with Sentiment-awareness approach for image-text sentiment detection. |
| Outcome: | The proposed approach is effective for image-text sentiment detection on three publicly available datasets. |
Aspect-Category-Opinion-Sentiment Quadruple Extraction with Implicit Aspects and Opinions (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies in aspect-based sentiment analysis ignore aspects and opinions in product reviews. |
| Approach: | They propose a task to extract aspect-category-opinion-sentiment quadruples from review sentences . they construct two new datasets that contain annotations of implicit aspects and opinions . |
| Outcome: | The proposed task provides full support for aspect-based sentiment analysis with implicit aspects and opinions. |
PASS: Perturb-and-Select Summarizer for Product Reviews (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on product reviews summarization focuses on generating concise, coherent and informative summaries, but this task is challenging. |
| Approach: | They propose a product reviews summarization task that employs a large pre-trained Transformer-based model and a method for ranking these summaries according to desired criteria. |
| Outcome: | The proposed system avoids the problem of self-contradiction by ranking the summaries according to desired criteria. |
Deep Differential Amplifier for Extractive Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to extract summary from document with a disproportionate ratio of selected and unselected sentences are far from human performance. |
| Approach: | They propose a model that rebalances sentence-level extractive summarization by amplifying the semantic difference between each sentence and all other sentences and applying the residual unit as the second item of the differential amplifier to deepen the architecture. |
| Outcome: | The proposed model performs competitively against state-of-the-art methods on two benchmark datasets. |
Multi-TimeLine Summarization (MTLS): Improving Timeline Summarization by Generating Multiple Summaries (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on Time-Line Summarization (TLS) has focused on improving the performance of summarization but its drawbacks are as follows: a homogeneous dataset makes it hard to generalize; output is usually a single timeline regardless of the size and complexity of the input dataset. |
| Approach: | They propose a task that generates a time-line for each story given a news article . they propose MTLS task that can be generalized to other news articles . |
| Outcome: | The proposed task can generate bet-ter results than Time-Line Summarization (TLS) the proposed task is based on previous evaluation methods. |
Self-Supervised Multimodal Opinion Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for opinion summarization use text data, but non-text data are less abundant. |
| Approach: | They propose a self-supervised opinion summarization framework that uses non-text data to generate a summary from multiple reviews. |
| Outcome: | The proposed framework is superior to existing methods on Yelp and Amazon datasets. |
A Training-free and Reference-free Summarization Evaluation Metric via Centrality-weighted Relevance and Self-referenced Redundancy (2021.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for text summarization systems are expensive and time-consuming. |
| Approach: | They propose a training-free and reference-free summarization evaluation metric that incorporates a centrality-weighted relevance score and a self-referenced redundancy score. |
| Outcome: | The proposed evaluation metric outperforms existing methods on multi-document and single-document summarization evaluation. |
DESCGEN: A Distantly Supervised Datasetfor Generating Entity Descriptions (2021.acl-long)
Copied to clipboard
| Challenge: | Short textual descriptions of entities provide summaries of their key attributes but generating entity descriptions can be challenging since information is scattered across multiple sources with varied content and style. |
| Approach: | They propose to generate an entity summary description from 37K entities from Wikipedia and Fandom, paired with nine evidence documents on average. |
| Outcome: | The proposed task is entity-centric, more abstractive, and covers a wide range of domains. |
Introducing Orthogonal Constraint in Structural Probes (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on interpreting pre-trained models' representations and analyzing their structures. |
| Approach: | They propose a new type of structural probing where a linear projection is decomposed into two types. |
| Outcome: | The proposed method is tested on two novel tasks and shows that lexical and syntactic information is separated in the representations. |
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for textual backdoor attacks insert additional contents into normal samples as triggers, causing detection and blocking of backdoors. |
| Approach: | They propose to use syntactic structure as trigger in textual backdoor attacks . they propose to achieve similar attack performance but have higher invisibility . |
| Outcome: | The proposed method achieves almost 100% success rate but has higher invisibility and stronger resistance to defenses than the insertion-based methods. |
Examining the Inductive Bias of Neural Language Models with Artificial Languages (2021.acl-long)
Copied to clipboard
| Challenge: | a novel method for investigating inductive biases of language models using artificial languages is proposed . we show that modern neural architectures used for language modeling are intrinsically black boxes . |
| Approach: | They propose a method to investigate inductive biases of language models using artificial languages . they use languages to create parallel corpora across languages that differ only in word order . |
| Outcome: | The proposed method shows that language models can be used to model a wide variety of languages. |
Explaining Contextualization in Language Models using Visual Analytics (2021.acl-long)
Copied to clipboard
| Challenge: | Contextualized language models (LMs) have learned highly transferable and task-agnostic properties of language, even to a degree of imitating the classical NLP pipeline. |
| Approach: | They propose to use an existing similarity-based score to measure contextualization and integrate it into a visual analytics technique that combines the model's layers simultaneously and highlighting intra-layer properties and inter-layer differences. |
| Outcome: | The proposed approach combines linguistically-informed insights with scoring and visual analytics to show that contextualization is neither driven by polysemy nor by pure context variation. |
Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that attention cannot be considered as a faithful explanation across encoders and tasks. |
| Approach: | They propose a new family of Task-Scaling mechanisms that scale attention weights across tasks and two attention mechanisms. |
| Outcome: | The proposed models improve explanation faithfulness across two attention mechanisms, five encoders and five text classification datasets without sacrificing predictive performance. |
Generating Landmark Navigation Instructions from Maps as a Graph-to-Text Problem (2021.acl-long)
Copied to clipboard
| Challenge: | Current navigation services based on turns and distances of named streets . humans use efficient mode of navigation based around visible and salient landmarks . |
| Approach: | They propose a neural model that takes OpenStreetMap representations as input and learns to generate navigation instructions that contain salient landmarks from human natural language instructions. |
| Outcome: | The proposed model can generate navigation instructions that contain salient landmarks from openStreetMap . it is based on a dataset of 7,672 instances verified by human navigation in Street View . |
E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing vision-language pre-training methods use a two-step training procedure to learn visual features from image-text pairs. |
| Approach: | They propose a vision-language pre-trained model for V+L understanding and generation using a unified Transformer framework. |
| Outcome: | The proposed model can learn visual representation and semantic alignments between image and text on visual-text pairs and on visual processing tasks. |
Learning Relation Alignment for Calibrated Cross-modal Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | despite advances in multimodal pre-training, cross-modal retrieval remains challenging . lack of relation consistency impairs contextualized representation of image-text pairs . |
| Approach: | They propose a new metric to quantify the relation consistency by measuring the semantic distance between linguistic and visual relations. |
| Outcome: | The proposed method boosts the performance of prevailing models on Flickr30k and MS COCO datasets by a considerable margin. |
KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for visual and language understanding are not capable of multimodal reasoning. |
| Approach: | They propose a Transformer-based sequence-to-sequence model capable of reasoning about commonsense knowledge from multimodal inputs of images and texts. |
| Outcome: | The proposed model performs state-of-the-art on the Visual Commonsense Generation task. |
Cascaded Head-colliding Attention (2021.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for natural language processing ignore interactions among different heads, which wastes the capacity of the model. |
| Approach: | They propose a model which explicitly models interactions between attention heads through a hierarchical variational distribution. |
| Outcome: | The proposed model outperforms the baseline model on Wikitext-103 and WMT14 EN-DE on language modeling and translation tasks. |
Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor (2021.acl-long)
Copied to clipboard
Xinyu Wang, Yong Jiang, Zhaohui Yan, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu
| Challenge: | Knowledge distillation is a technique to transfer knowledge between models, typically from a large model (the teacher) to a more fine-grained one (the student). |
| Approach: | They propose a factorized form of the knowledge distillation objective for structured prediction which is tractable for many typical choices of the teacher and student models. |
| Outcome: | The proposed model is able to transfer knowledge between teacher and student models without loss of accuracy under four different scenarios. |
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks (2021.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art parameter-efficient fine-tuning methods rely on introducing adapter modules between the layers of a pretrained language model. |
| Approach: | They propose a framework that can learn adapter parameters for all layers and tasks by generating them using shared hypernetworks. |
| Outcome: | The proposed framework improves performance on the well-known GLUE benchmark while adding only 0.29% parameters per task. |
COSY: COunterfactual SYntax for Cross-Lingual Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models suffer from a large performance gap between source and target languages . e.g., multilingual-BERT models are widely used in cross-lingual tasks . |
| Approach: | They propose a language-agnostic approach to integrate universal syntax into language models . they use SYntax-aware networks and a COunterfactual training method . |
| Outcome: | The proposed model achieves state-of-the-art performance on natural language inference and question answering without auxiliary training data. |
OoMMix: Out-of-manifold Regularization in Contextual Embedding Space for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies on neural networks with pre-trained weights focus on low-dimensional subspace where the embedding vectors computed from input words are located. |
| Approach: | They propose an approach to find and regularize the remainder of the space, referred to as out-of-manifold, which cannot be accessed through the words. |
| Outcome: | The proposed approach is able to fine-tune the out-of-manifold embedding space on text classification benchmarks. |
Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model (2021.acl-long)
Copied to clipboard
| Challenge: | Stereotypical language expresses widely-held beliefs about different social categories. |
| Approach: | They propose a computational approach to interpreting stereotypes in text through the Stereotype Content Model (SCM), a comprehensive causal theory from social psychology. |
| Outcome: | The proposed model compares favourably with survey-based studies in the psychological literature on stereotypes and shows that it is realistic and effective. |
Structurizing Misinformation Stories via Rationalizing Fact-Checks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on misinformation include a coarse concept of misinformation . key phrases in fact-check articles that identify misinformation types act as rationales . |
| Approach: | They propose to use fact-check articles to structure misinformation stories by leveraging fact-search articles. |
| Outcome: | The proposed model uses key phrases in fact-check articles to identify misinformation types and rationalize them . the results compare misinformation stories between the 2016/2020 elections and the COVID-19 pandemics . |
Modeling Language Usage and Listener Engagement in Podcasts (2021.acl-long)
Copied to clipboard
| Challenge: | linguistic factors such as vocabulary diversity, distinctiveness, emotion, and syntax are highly predictive of engagement in podcasts, but little research has been done into how they contribute to overall listener engagement. |
| Approach: | They build models with different textual representations to test popular wisdom about stylistic elements in high-engagement podcasts, corroborating some pieces of advice and adding new perspectives on others. |
| Outcome: | The proposed models show that stylistic features are highly predictive of engagement for podcasts with low absolute streams than for the most popular podcasts. |
Breaking Down the Invisible Wall of Informal Fallacies in Online Discussions (2021.acl-long)
Copied to clipboard
| Challenge: | a number of people engage in unsound argumentation techniques to prove a claim on online platforms . fallacies are weak arguments that seem convincing, but their evidence does not prove or disprove the conclusion . |
| Approach: | They propose to use user comments containing fallacy mentions as noisy labels to classify fallacies . they use the pragma-dialectical theory of argumentation to study the most common fallacias on Reddit . |
| Outcome: | The proposed dataset of fallacies on reddit shows that neural models perform better in conversational context. |
SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on identifying entities' relations from the semantics of dialogues-they utilize either the attention mechanism or a refined token graph to locate informative words. |
| Approach: | They propose a sequential structure prediction task to incrementally parse SocAoG for dynamic inference upon any incoming utterance. |
| Outcome: | Empirical results show that the proposed model infers social relations more accurately than the state-of-the-art methods. |
TicketTalk: Toward human-level performance with end-to-end, transaction-based dialog systems (2021.acl-long)
Copied to clipboard
| Challenge: | TicketTalk dataset with 23,789 annotated dialogs is a data-driven, end-to-end approach to transaction-based dialog systems that performs at near-human levels in terms of verbal response quality and factual grounding accuracy. |
| Approach: | They propose a data-driven, end-to-end approach to transaction-based dialog systems that performs at near-human levels in terms of verbal response quality and factual grounding accuracy. |
| Outcome: | The proposed model generates verbal responses and API call predictions on a movie ticketing dialog dataset with 23,789 annotated conversations. |
Improving Dialog Systems for Negotiation with Personality Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | In this paper, we introduce a framework for generating strategic dialog inspired by the idea of incorporating a theory of mind (ToM) into machines. |
| Approach: | They propose a probabilistic formulation to encapsulate the opponent's personality type during both learning and inference. |
| Outcome: | The proposed model achieves 20% higher dialog agreement rate compared to baselines on a mixed population of opponents. |
Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training (2021.acl-long)
Copied to clipboard
| Challenge: | Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. |
| Approach: | They propose an algorithm that encourages the model to be sensitive to perturbations in dialogue history and learn from perturbations. |
| Outcome: | The proposed approach can model dialogue history better and generate more diverse responses on two benchmark datasets. |
Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features (2021.acl-long)
Copied to clipboard
| Challenge: | Existing systems that strive to be informative teachers are difficult to build . knowledge grounded dialogue systems are difficult because of limited training objectives . |
| Approach: | They propose to train a generative neural dialogue model that is controlled to stay faithful to evidence . they propose to use additional inputs to generate more objective responses . |
| Outcome: | The proposed model produces responses that are perceived by humans to be objective and faithful to evidence. |
CitationIE: Leveraging the Citation Graph for Scientific Information Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on scientific information extraction (SciIE) considers extraction solely based on the content of an individual paper, without considering the paper’s place in the broader literature. |
| Approach: | They propose to automate the extraction of key information from scientific documents by leveraging a complementary source: the citation graph of referential links between citing and cited papers. |
| Outcome: | The proposed model improves on a set of English-language scientific documents. |
From Discourse to Narrative: Knowledge Projection for Event Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing event-centric knowledge graphs rely on explicit connectives to extract relations between events. |
| Approach: | They propose a knowledge projection paradigm for event relation extraction using commonalities between events. |
| Outcome: | The proposed method achieves state-of-the-art performance and extrinsic results verify the extracted event relations. |
AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER (2021.acl-long)
Copied to clipboard
| Challenge: | Named entity recognition models rely on expensive labeled data for training, which is not always available across languages. |
| Approach: | They propose an adversarial approach where an encoder learns entity domain knowledge from labeled source-language data and better shared features are captured via adversarially trained discriminators. |
| Outcome: | The proposed approach outperforms existing state-of-the-art methods on standard benchmark datasets and outperformed existing methods on the target language. |
Compare to The Knowledge: Graph Neural Fake News Detection with External Knowledge (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fake news detection rely on linguistic and semantic features from news content and do not exploit external knowledge. |
| Approach: | They propose a graph neural model which compares news to knowledge base through entities for fake news detection. |
| Outcome: | The proposed model significantly outperforms state-of-the-art methods on two benchmark datasets. |
Discontinuous Named Entity Recognition as Maximal Clique Discovery (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition break the recognition process into several sequential steps. |
| Approach: | They propose a method that breaks the recognition process into several sequential steps . they construct a segment graph for each sentence and a grid tagging scheme to learn it . |
| Outcome: | Experiments show that the proposed method outperforms the state-of-the-art model and achieves 5x speedup over the SOTA model. |
LNN-EL: A Neuro-Symbolic Approach to Short-text Entity Linking (2021.acl-long)
Copied to clipboard
Hang Jiang, Sairam Gurajada, Qiuhao Lu, Sumit Neelam, Lucian Popa, Prithviraj Sen, Yunyao Li, Alexander Gray
| Challenge: | Existing work deals with EL in the context of longer text, such as a sentence. |
| Approach: | They propose a neuro-symbolic approach that uses interpretable rules based on first-order logic to achieve better performance with black-box neural approaches. |
| Outcome: | The proposed approach achieves better performance than heuristics-based approaches on short-text EL . it can easily blend existing rule templates with multiple types of features, and even with scores resulting from previous EL methods. |
Do Context-Aware Translation Models Pay the Right Attention? (2021.acl-long)
Copied to clipboard
| Challenge: | Context-aware machine translation models fail to leverage contextual information to resolve ambiguous words and pronouns. |
| Approach: | They propose a new dataset that includes supporting context words for 14K translations that professional translators found useful for pronoun disambiguation. |
| Outcome: | The proposed model can automatically disambiguate pronouns and polysemous words when they are not in the same context. |
Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data (2021.acl-long)
Copied to clipboard
Wei-Jen Ko, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Naman Goyal, Francisco Guzmán, Pascale Fung, Philipp Koehn, Mona Diab
| Challenge: | linguistic overlap between low-resource languages and high-resourced languages is a major obstacle for training high-quality machine translation systems. |
| Approach: | They exploit linguistic overlap to facilitate translation to and from low-resource languages . they use monolingual data and parallel data in related high-resourced languages based on their method . |
| Outcome: | The proposed method significantly improves translation into low-resource language compared to baselines on 7 languages from three different language families. |
Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for bilingual lexicon induction are linear and require simplifying assumptions. |
| Approach: | They propose methods that combine unsupervised bitext mining and unsupervised word alignment to produce higher quality lexicons. |
| Outcome: | The proposed method outperforms the state-of-the-art on the BUCC 2020 task by 14 F1 points . further analysis suggests they are comparable quality . |
Multilingual Speech Translation from Efficient Finetuning of Pretrained Models (2021.acl-long)
Copied to clipboard
Xian Li, Changhan Wang, Yun Tang, Chau Tran, Yuqing Tang, Juan Pino, Alexei Baevski, Alexis Conneau, Michael Auli
| Challenge: | Recent advances in text pretraining and finetuning have improved multitasking applications significantly. |
| Approach: | They propose a minimalistic LNA finetuning approach to build multilingual speech-to-text translation using a pretrained speech encoder and text decoder. |
| Outcome: | The proposed approach surpasses the cascaded ST benchmark for 36 translation directions on the large-scale multilingual ST benchmark CoVoST 2. |
Learning Faithful Representations of Causal Graphs (2021.acl-long)
Copied to clipboard
| Challenge: | Existing text embeddings that predict direct causal links fail to capture other indirect causal links, leading to spurious correlations in downstream tasks. |
| Approach: | They define faithfulness property of contextual embeddings to capture geometric distance-based properties of directed acyclic causal graphs. |
| Outcome: | The embeddings are 31.3% more faithful to human validated graphs with 800K and 200K causal links and achieve better Precision-Recall AUC in a link prediction fine-tuning task. |
What Context Features Can Transformer Language Models Use? (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that transformer-based language models benefit from conditioning on contexts of hundreds to thousands of previous tokens. |
| Approach: | They propose to use lexical and structural information to ablate usable information in transformer language models. |
| Outcome: | The proposed model improves when conditioning on contexts of thousands of previous tokens. |
Integrated Directional Gradients: Feature Interaction Attribution for Neural NLP Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for attribution of importance to features borrowed from cooperative game theory . success of Deep Neural Networks has led to their ability to learn from complex higher order interactions from raw features. |
| Approach: | They propose a method for attributing importance scores to groups of features . they propose axioms that any intuitive feature group attribution method should satisfy . |
| Outcome: | The proposed method captures the importance of features in a linguistic model using negations and conjunctions. |
DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations (2021.acl-long)
Copied to clipboard
| Challenge: | Sentence embeddings are an important component of many natural language processing systems. |
| Approach: | They propose a self-supervised objective for learning universal sentence embeddings that does not require labelled training data. |
| Outcome: | The proposed approach closes the performance gap between unsupervised and supervised pretraining for universal sentence encoders. |
XLPT-AMR: Cross-Lingual Pre-Training via Multi-Task Learning for Zero-Shot AMR Parsing and Text Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) research is limited and challenging for languages other than English. |
| Approach: | They propose a cross-lingual pre-training approach for AMR parsing and text generation . they use an English-to-English parallel dataset and a multi-task learning approach . |
| Outcome: | The proposed approach outperforms baseline pre-training methods on English parsing and text generation tasks. |
Span-based Semantic Parsing for Compositional Generalization (2021.acl-long)
Copied to clipboard
| Challenge: | despite success of sequence-to-sequence models, they fail in compositional generalization . a span-based parser that predicts a utterance over spans improves performance . |
| Approach: | They propose a span-based parser that predicts a utterance over a given span tree . they propose to use CKY to encode how partial programs compose over spans . |
| Outcome: | The proposed model performs better on random splits than baselines that require compositional generalization. |
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to semantic parsing only evaluated on synthetic datasets that are not representative of natural language variation. |
| Approach: | They propose a semantic parsing approach that handles both natural language variation and compositional generalization. |
| Outcome: | The proposed model outperforms existing models across compositional generalization challenges on non-synthetic datasets while being competitive with the state-of-the-art on standard evaluations. |
A Targeted Assessment of Incremental Processing in Neural Language Models and Humans (2021.acl-long)
Copied to clipboard
| Challenge: | Using by-word reaction time data, we compare incremental processing in humans and neural language models across a range of structural phenomena. |
| Approach: | They propose to scale up incremental processing in humans and language models by collecting by-word reaction time data for 16 different syntactic test suites. |
| Outcome: | The proposed model outputs match human and model accuracy scores, but underpredict the difference in magnitude of incremental processing difficulty between grammatical and ungrammatically-spaced sentences. |
The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies restrict modal expressions to a closed syntactic class . modal sense labels are vastly different across different studies, lacking an accepted standard . |
| Approach: | They propose a task where modal expressions can be words of any syntactic class and sense labels are drawn from a comprehensive taxonomy which harmonizes the modal concepts contributed by the different studies. |
| Outcome: | The proposed task is based on the Georgetown Gradable Modal Expressions corpus . it detects and classifies fine-grained modal concepts and associates them with modified events . |
To POS Tag or Not to POS Tag: The Impact of POS Tags on Morphological Learning in Low-Resource Settings (2021.acl-long)
Copied to clipboard
| Challenge: | Part-of-Speech (POS) tags are routinely included in many NLP tasks. |
| Approach: | They propose to use POS tags to examine morphological learning in low-resource languages . they find that POS tagging improves joint segmentation and glossing . |
| Outcome: | The proposed task is tested on two identical datasets with the Transformer architecture. |
Prosodic segmentation for parsing spoken dialogue (2021.acl-long)
Copied to clipboard
| Challenge: | Existing parsers struggle to parse spoken dialogue because of disfluencies and unmarked boundaries between sentence-like units (SUs). |
| Approach: | They hypothesize that prosody affects a parser that receives an entire dialogue turn as input, instead of gold standard pre-segmented SUs. |
| Outcome: | The proposed model performs better than the SU-based model on the English Switchboard corpus despite performing two tasks rather than one, and pitch and intensity features are the most important for this corpus. |
VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation (2021.acl-long)
Copied to clipboard
Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, Emmanuel Dupoux
| Challenge: | VoxPopuli provides 400K hours of unlabeled speech data in 23 languages . large amounts of multilingual audio data are needed to achieve similar progress for multilingual ASR and ST. |
| Approach: | They propose a large-scale multilingual corpus that provides 400K hours of unlabeled speech data in 23 languages. |
| Outcome: | The proposed corpus provides 400K hours of unlabeled speech data in 23 languages and 1.8K hours transcribed speeches in 15 languages and their aligned oral interpretations into 15 target languages totaling 17.3K hours. |
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets (2021.acl-long)
Copied to clipboard
| Challenge: | Several recent efforts have focused on benchmark datasets consisting of pairs of contrastive sentences, which are often accompanied by metrics that aggregate an NLP system’s behavior on these pairs into measurements of harms. |
| Approach: | They apply a measurement modeling lens to inventory pitfalls that threaten benchmarks' validity as measurement models for stereotyping. |
| Outcome: | The proposed benchmarks lack clarity and assumptions that affect how they conceptualize and operationalize stereotyping. |
Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge graphs (KGs) are incomplete because of the large number of benchmark datasets that are not representative of real KGs. |
| Approach: | They develop a deep convolutional network that utilizes textual entity representations to distill the knowledge from the convolution into a student network that re-ranks promising candidate entities. |
| Outcome: | The proposed model outperforms recent methods in a realistic setting where dense connectivity is not guaranteed. |
A DQN-based Approach to Finding Precise Evidences for Fact Verification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fact verification do not target the retrieval of precise evidences. |
| Approach: | They propose a DQN-based approach to retrieval of precise evidences . they propose best thresholds for determining the true labels of computed evidences. |
| Outcome: | The proposed method improves accuracy of fact verification by reducing label bias . it can retrieve evidence consisting of the first two sentences, but it can contain unnecessary sentences . |
The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have improved the state-of-the-art results on many NLP applications. |
| Approach: | They propose a simple error regularization trick that improves confidence estimation without substantially increasing the computation budget. |
| Outcome: | The proposed regularization improves confidence estimation without increasing computation budget. |
Unsupervised Out-of-Domain Detection via Pre-trained Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | Prior work on out-of-domain detection requires in-domain task labels and is limited to supervised classification scenarios. |
| Approach: | They propose a method to construct out-of-domain detectors efficiently using pre-trained transformers. |
| Outcome: | The proposed method greatly improves out-of-domain detection ability in a more general scenario. |
MATE-KD: Masked Adversarial TExt, a Companion to Knowledge Distillation (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that the trillion parameter mark for pre-trained language models is not achievable without a change in training paradigm. |
| Approach: | They propose a text-based adversarial training algorithm which enhances the performance of knowledge distillation by maximizing the divergence between teacher and student logits. |
| Outcome: | The proposed algorithm outperforms competing adversarial learning and data augmentation baselines on the GLUE benchmark. |
Selecting Informative Contexts Improves Language Model Fine-tuning (2021.acl-long)
Copied to clipboard
| Challenge: | Language model fine-tuning is computationally expensive and time-consuming . however, the inclusion of training examples that negatively affect performance is limited . |
| Approach: | They propose a general fine-tuning method that incorporates information gain filtration . they propose to release pre-trained secondary learners on common corpora to promote efficient fine-uning. |
| Outcome: | The proposed method achieves a median perplexity of 54.0 on a books dataset compared to 57.3 for standard fine-tuning. |
Explainable Prediction of Text Complexity: The Missing Preliminaries for Text Simplification (2021.acl-long)
Copied to clipboard
| Challenge: | Text simplification reduces the language complexity of professional content for accessibility purposes. |
| Approach: | They propose that text simplification can be decomposed into a pipeline of tasks . they show that the pipeline can be used to predict whether a text needs to be simplified . |
| Outcome: | The proposed model improves the performance of out-of-sample simplification tests on a blackbox lexical model . the proposed model reduces the complexity of professional text by a large margin . |
Multi-Task Retrieval for Knowledge-Intensive Tasks (2021.acl-long)
Copied to clipboard
Jean Maillard, Vladimir Karpukhin, Fabio Petroni, Wen-tau Yih, Barlas Oguz, Veselin Stoyanov, Gargi Ghosh
| Challenge: | Knowledge-intensive tasks require large amounts of knowledge about the world . recent neural retrieval models achieve better results by learning directly from task-specific training data. |
| Approach: | They propose a multi-task trained neural retrieval model that can be universally trained on a wide variety of problems. |
| Outcome: | The proposed model outperforms specialised retrievers on a few-shot setting and matches or improves state-of-the-art on multiple benchmarks. |
When Do You Need Billions of Words of Pretraining Data? (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained language models (LMs) are dominated by models that can encode billions of words. |
| Approach: | They use classifier probing, information-theoretic probing and unsupervised relative acceptability judgments to evaluate model ability. |
| Outcome: | The proposed models require only about 10M to 100M words to learn to encode most syntactic and semantic features. |
Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models are influenced by two types of context, source and target, but none explicitly evaluates relative contribution to generation decision. |
| Approach: | They propose to adopt a variant of Layerwise Relevance Propagation which evaluates relative contributions to the generation decision by a proportion of token influence. |
| Outcome: | The proposed model can evaluate the relative contribution of source and target to the generation decision by using a variant of Layerwise Relevance Propagation (LRP) |
Comparing Test Sets with Item Response Theory (2021.acl-long)
Copied to clipboard
Clara Vania, Phu Mon Htut, William Huang, Dhara Mungra, Richard Yuanzhe Pang, Jason Phang, Haokun Liu, Kyunghyun Cho, Samuel R. Bowman
| Challenge: | Recent results from large pretrained models show that many datasets are saturated and unlikely to detect further progress. |
| Approach: | They evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples. |
| Outcome: | The proposed datasets are saturated and unlikely to detect future improvements. |
Uncovering Constraint-Based Behavior in Neural Models via Targeted Fine-Tuning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work has shown that non-linguistic biases in language models obscure linguistic knowledge. |
| Approach: | They hypothesize competing linguistic processes within a language could obscure linguistic knowledge. |
| Outcome: | The proposed model can learn linguistic constraints in a language and their relative ranking, the authors show . linguistic biases can obscure underlying linguistic knowledge, they show a single phenomenon in four languages. |
More Identifiable yet Equally Performant Transformers for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Current studies prove that attention weights are not unique and therefore unfit for interpretation. |
| Approach: | They propose a transformer encoder layer that decouples the relationship between key and value vector and provides identifiable weights up to the desired length of the input. |
| Outcome: | The proposed model is more identifiable than previously thought but still prone to be non-unique attentions that make them unfit for interpretation. |
AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation (2021.acl-long)
Copied to clipboard
| Challenge: | Large-scale conversational systems typically generate unnatural, robotic responses using template-based approaches. |
| Approach: | They propose a data augmentation approach that combines a self-trained neural retrieval model with a few-shot learned NLU model to automatically create MR-to-Text data from open-domain texts. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods on the FewshotWOZ data in both BLEU and Slot Error Rate. |
Can vectors read minds better than experts? Comparing data augmentation strategies for the automated scoring of children’s mindreading ability (2021.acl-long)
Copied to clipboard
| Challenge: | In-domain experts are recruited to reannotate augmented samples and determine to what extent each strategy preserves the original rating. |
| Approach: | They implement 7 different data augmentation strategies for the task of automatic scoring of children’s ability to understand others’ thoughts, feelings, and desires. |
| Outcome: | The data augmentation strategies outperform task-agnostic augmentations and automatic augmentation systems perform worst on the MIND-CA corpus. |
A Dataset and Baselines for Multilingual Reply Suggestion (2021.acl-long)
Copied to clipboard
| Challenge: | Reply suggestion models help users process emails and chats faster. |
| Approach: | They present a multilingual reply suggestion dataset with ten languages . they build a generation model and a retrieval model as baselines for MRS . |
| Outcome: | The proposed model complements existing benchmarks for cross-lingual generalization . the model has different strengths in the English monolingual setting and requires different strategies to generalize across languages. |
What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks? (2021.acl-long)
Copied to clipboard
| Challenge: | Despite the importance of datasets for natural language understanding, there has been little attention on crowdsourcing methods for collecting datasets. |
| Approach: | They compare the effectiveness of crowdsourcing methods for boosting NLU example difficulty with training crowdworkers instead of expert judgments. |
| Outcome: | The proposed method is ineffective for boosting NLU example difficulty, but it is not effective for training crowdworkers and qualifying workers based on expert judgments. |
Align Voting Behavior with Public Statements for Legislator Representation Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies rely on roll call data to estimate political preference of legislators. |
| Approach: | They propose to integrate voting behavior and public statements on Twitter to jointly model legislators. |
| Outcome: | The proposed model improves on the task of roll call vote prediction . it also shows that the model captures nuances in statements . |
Measure and Evaluation of Semantic Divergence across Two Languages (2021.acl-long)
Copied to clipboard
| Challenge: | Languages are dynamic systems: word usage may change over time, reflecting various societal factors. |
| Approach: | They propose to build time-varying and bilingual word embeddings using contextualised and non-contextualised embeddables. |
| Outcome: | The proposed methods perform best on newspaper corpora, but are competitive on large corpus. |
Improving Zero-Shot Translation by Disentangling Positional Information (2021.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation has shown the capability of directly translating between language pairs unseen in training, i.e. zero-shot translation. |
| Approach: | They propose to remove residual connections in an encoder layer to reduce the difficulty of generalizing to new translation directions. |
| Outcome: | The proposed model outperforms pivot-based translation in terms of quality and ease of integration of new languages. |
Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Using multilingual language models, commonsense reasoning research has been limited to English. |
| Approach: | They propose a Mickey Probe task to evaluate commonsense across languages . they propose X-CSQA and XCODAH datasets to be translated to 14 languages based on the Mickey corpus . |
| Outcome: | The proposed method significantly improves sentence representations beyond English. |
Attention Calibration for Transformer in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Attention mechanisms have been ubiquitous in neural machine translation (NMT) however, many studies doubt whether highlyattended inputs have a large impact on the model outputs. |
| Approach: | They propose to introduce a mask perturbation model that automatically evaluates each input’s contribution to the model outputs. |
| Outcome: | The proposed model is more uniform at lower layers while more concentrated on the specific inputs at higher layers. |
Diverse Pretrained Context Encodings Improve Document Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for sentence-level sequence-to-sequence translations do not use extra-sentential information. |
| Approach: | They propose a sentence-level sequence-to-sequence transformer with multiple pre-trained context signals. |
| Outcome: | The proposed model outperforms existing models on Chinese-English and English-German tasks. |
Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study (2021.acl-long)
Copied to clipboard
Yash Khemchandani, Sarvesh Mehtani, Vaidehi Patil, Abhijeet Awasthi, Partha Talukdar, Sunita Sarawagi
| Challenge: | Recent research in multilingual language models (LMs) has demonstrated their ability to effectively handle multiple languages in a single model. |
| Approach: | They propose to exploit relatedness among languages in a language family to overcome corpora limitations of LRLs. |
| Outcome: | The proposed model exploits relatedness among languages in a language family to overcome corpora limitations for low web-resource languages. |
On Finding the K-best Non-projective Dependency Trees (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on finding the one-best dependency tree has not extended this to finding the K-best tree. |
| Approach: | They propose to simplify the K-best spanning tree algorithm by decoding the K best dependency trees with a root constraint. |
| Outcome: | The proposed algorithm can be used to find the K-best dependency trees without root constraint. |
Towards Argument Mining for Social Good: A Survey (2021.acl-long)
Copied to clipboard
| Challenge: | Argument Mining is a social science-based approach to analysis and analysis of arguments. |
| Approach: | They propose a novel definition of argument quality which integrates the social science literature and the argument quality. |
| Outcome: | The proposed definition of argument quality integrates the social science literature and the argument quality debate. |
Automated Generation of Storytelling Vocabulary from Photographs for use in AAC (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating context-related vocabulary from photographs are lacking for enhancing social interaction support. |
| Approach: | They propose a method for generating context-related vocabulary from photographs . their results provide insights for system optimization and usage . |
| Outcome: | The proposed method generates vocabulary with 4.6 gain in average precision . it is compared to baseline generation of English words and 6.9 for photos with correct information. |
CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes (2021.acl-long)
Copied to clipboard
James Mullenbach, Yada Pruksachatkun, Sean Adler, Jennifer Seale, Jordan Swartz, Greg McKelvey, Hui Dai, Yi Yang, David Sontag
| Challenge: | Continuity of care is crucial to ensuring positive health outcomes for patients discharged from an inpatient hospital setting. |
| Approach: | They propose to annotate clinical action items from a dataset of medical notes annotated by physicians and extract them as multi-aspect extractive summarization. |
| Outcome: | The proposed dataset is annotated by physicians and covers 718 documents representing 100K sentences. |
Assessing Emoji Use in Modern Text Processing Tools (2021.acl-long)
Copied to clipboard
| Challenge: | Emojis are textual elements that are encoded as characters but rendered as small digital images or icons that can be used to express an idea or emotion. |
| Approach: | They propose to use a set of popular NLP tools to assess the support of emojis in tweets. |
| Outcome: | The proposed methods show that many systems still have notable shortcomings when operating on text containing emojis. |
Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention (2021.acl-long)
Copied to clipboard
| Challenge: | Generally, documents are truncated before being inputs to deep neural networks, resulting in missing keyphrases . evaluators use layer-wise coverage attention to cover all the critical points in a document . |
| Approach: | They propose a neural keyphrase generation model that identifies the salient sentences in a document and an extractor-generator that jointly extracts and generates keyphrases from the selected sentences. |
| Outcome: | The proposed model outperforms the state-of-the-art keyphrase generation methods on keyphrases generated from scientific and web documents. |
Factorising Meaning and Form for Intent-Preserving Paraphrasing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for paraphrasing questions retain original intent but use a different surface form. |
| Approach: | They propose a method for generating paraphrases of English questions that retain original intent but use a different surface form. |
| Outcome: | The proposed method generates paraphrases with better tradeoff between semantic preservation and novelty compared to previous methods. |
AggGen: Ordering and Aggregating while Generating (2021.acl-long)
Copied to clipboard
| Challenge: | AggGen is a data-to-text model which re-introduces two explicit sentence planning stages into neural data- to-text systems: input ordering and input aggregation. |
| Approach: | AggGen re-introduces two explicit sentence planning stages into neural data-to-text systems: input ordering and input aggregation. |
| Outcome: | AggGen is a data-to-text model which re-introduces two explicit sentence planning stages into neural data- to-text systems: input ordering and input aggregation. |
Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating text are unsupervised and require supervision. |
| Approach: | They propose an unsupervised method that uses two off-the-shelf pretrained LMs in opposite directions to apply them to non-sequential tasks. |
| Outcome: | The proposed method outperforms strong unsupervised baselines on paraphrasing and abductive text infilling. |
Towards Table-to-Text Generation with Numerical Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown improvement in generating descriptive text from structured data. |
| Approach: | They propose a framework for numerical table-to-text generation based on numerical reasoning . they use a pre-trained model and a copy mechanism to fine-tune the models to produce fluent text . |
| Outcome: | The proposed framework lacks fidelity to the table contents and is based on a pre-trained model and a copy mechanism. |
BACO: A Background Knowledge- and Content-Based Framework for Citing Sentence Generation (2021.acl-long)
Copied to clipboard
| Challenge: | citing sentences capture salient information in cited papers and the connection between citing and citing papers. |
| Approach: | They propose a BAckground knowledge- and COntent-based framework for citing sentence generation that integrates two types of information: background knowledge and content. |
| Outcome: | The proposed framework outperforms baselines in the citation sentence generation task. |
Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing dialogue summarization systems encode text with a number of general semantic features, but these are often not available in open-domain tools. |
| Approach: | They propose to use DialoGPT to label three types of features on two datasets . they propose to employ pre-trained and non-pre-tried models as dialogue annotators . |
| Outcome: | The proposed method improves on two dialogue summarization datasets and achieves state-of-the-art performance. |
Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | Existing pretrained language models have solved reading comprehension benchmarks, but datasets with information-seeking queries remain challenging. |
| Approach: | They analyze why answering information-seeking queries is more challenging . they manually annotate 800 unanswerable examples across six languages . |
| Outcome: | The proposed model outperforms human annotators on 800 unanswerable examples across six languages. |
A Gradually Soft Multi-Task and Data-Augmented Approach to Medical Question Understanding (2021.acl-long)
Copied to clipboard
Khalil Mrini, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, Emilia Farcas, Ndapa Nakashole
| Challenge: | Existing methods for medical question understanding often fail to provide high recall in answer retrieval. |
| Approach: | They propose a multi-task learning method with data augmentation for medical question understanding that uses just one dataset to optimize for both tasks. |
| Outcome: | The proposed method outperforms existing MTL methods across 4 datasets of medical question pairs in ROUGE scores, RQE accuracy and human evaluation. |
Leveraging Type Descriptions for Zero-shot Named Entity Recognition and Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Named entity recognition and classification (NERC) tasks require annotated data for the target classes during training. |
| Approach: | They propose a novel approach that leverages the fact that textual descriptions for many entity classes occur naturally. |
| Outcome: | The proposed approach outperforms baselines adapted from machine reading comprehension and zero-shot text classification. |
MECT: Multi-Metadata Embedding based Cross-Transformer for Chinese Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a sequence tagging task that extracts named entities from unstructured text. |
| Approach: | They propose to integrate Chinese character features with radical-level embedding to improve Chinese NER by integrating Chinese character information. |
| Outcome: | The proposed method can improve Chinese Named Entity Recognition (NER) on well-known datasets. |
Factuality Assessment as Modal Dependency Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | a critical step towards factuality assessment is to determine the factuality of events in text. |
| Approach: | They propose a modal dependency parsing task that assesses the factuality of events in text . they crowdsource a large-scale data set annotated with modal dependence structures . |
| Outcome: | The proposed model outperforms the pipeline model in factuality assessment . the proposed model is based on a crowdsourced dataset . |
Directed Acyclic Graph Network for Conversational Emotion Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Empirical evidence shows that a good representation of conversation context significantly contributes to the model performance. |
| Approach: | They propose to encode query utterances with a directed acyclic graph to better model the intrinsic structure within a conversation. |
| Outcome: | The proposed model outperforms existing models on four ERC benchmarks with state-of-the-art models employed as baselines. |
Improving Formality Style Transfer with Context-Aware Rule Injection (2021.acl-long)
Copied to clipboard
| Challenge: | Existing language models pre-trained on large-scale corpora often do not work well for user-generated data where the language styles differ significantly from the mainstream text. |
| Approach: | They propose a method for formality style transfer by injecting multiple rules into an end-to-end BERT-based encoder and decoder model. |
| Outcome: | The proposed method outperforms existing rule-based FST approaches on tweet sentiment analysis tasks. |
Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Emotion detection in dialogues requires the identification of thematic topics underlying a conversation, commonsense knowledge, and the intricate transition patterns between affective states. |
| Approach: | They propose a Topic-Driven Knowledge-Aware Transformer model that integrates topic representation and commonsense knowledge from ATOMIC for dialogue emotion detection. |
| Outcome: | The proposed model outperforms state-of-the-art models on four dialogue datasets . it can detect topics which help distinguish emotion categories, the authors show . |
Syntopical Graphs for Computational Argumentation Tasks (2021.acl-long)
Copied to clipboard
Joe Barrow, Rajiv Jain, Nedim Lipka, Franck Dernoncourt, Vlad Morariu, Varun Manjunatha, Douglas Oard, Philip Resnik, Henning Wachsmuth
| Challenge: | adler and van Doren (1940) proposed a formalized manual process for understanding a topic based on multiple viewpoints. |
| Approach: | They propose a syntopical reading process that emphasizes comparing and contrasting viewpoints to improve topic understanding. |
| Outcome: | The proposed method outperforms approaches that do not use collection-level information. |
Stance Detection in COVID-19 Tweets (2021.acl-long)
Copied to clipboard
| Challenge: | a global pandemic of COVID-19 has forced major changes in our daily lives . a new stance detection dataset is being used to track the stances of Twitter users . |
| Approach: | They use Twitter stance data to collect stances on topics related to the pandemic . they train models to take advantage of large amounts of unlabeled data . |
| Outcome: | The proposed model improves on existing stance detection datasets and unlabeled data. |
Topic-Aware Evidence Reasoning and Stance-Aware Aggregation for Fact Verification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fact verification focus on analyzing semantic interaction between claim and evidence but fail to capture their topical consistency . Existing models focus on the aggregation of multiple pieces of evidence without considering their implicit stances to the claim, thereby introducing spurious information. |
| Approach: | They propose a topic-aware evidence reasoning and stance-again aggregation model that checks topical consistency between claims and evidence. |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets. |
Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset (2021.acl-long)
Copied to clipboard
| Challenge: | Using supervised machine learning, we assess how solidarity discourses changed before and during the COVID-19 crisis. |
| Approach: | They use social scientific concept of solidarity and its contestation, anti-solidarity, as problem setting to assess how European solidarity discourses changed before and during COVID-19. |
| Outcome: | The proposed model outperforms the baseline classifier with expert annotations by 25 points, from 58% macro-F1 to almost 85%. |
Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions (2021.acl-long)
Copied to clipboard
Dorottya Demszky, Jing Liu, Zid Mancenido, Julie Cohen, Heather Hill, Dan Jurafsky, Tatsunori Hashimoto
| Challenge: | Despite extensive research showing the positive impact of uptake on student learning and achievement, there is little evidence that it is effective in teaching. |
| Approach: | They propose a framework for computationally measuring uptake by releasing a dataset of student-teacher exchanges extracted from US math classroom transcripts annotated for uptake . they formalize uptake as pointwise Jensen-Shannon Divergence (pJSD) and conduct a linguistically-motivated comparison of different unsupervised measures. |
| Outcome: | The proposed framework outperforms baseline measures in identifying uptake phenomena like question answering and reformulation. |
A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies (2021.acl-long)
Copied to clipboard
| Challenge: | linguistic and social aspects of code-switching are not discussed in the literature in linguistics. |
| Approach: | They propose to examine linguistic and social aspects of code-switching across a wide range of languages in a survey of the literature in linguistics and language technologies. |
| Outcome: | The proposed framework aims to increase the clarity and depth of computational investigations of C-S and bridge the fields so that they might be mutually reinforcing. |
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Detecting online hate speech has proven difficult and concerns raised about performance, robustness, generalisability and fairness of stateof-the-art models. |
| Approach: | They propose a human-and-model-in-the-loop process for dynamically generating datasets and training better performing hate detection models. |
| Outcome: | The proposed model improves on a dataset of 40,000 hateful entries . the model is harder for annotators to trick and better on HateCheck . |
InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection (2021.acl-long)
Copied to clipboard
Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, Avi Sil
| Challenge: | a novel approach to detect fake news is needed due to training data scarcity . current methods focus on document-level fake news detection using lexical features and semantic embeddings . |
| Approach: | They propose a novel benchmark for fake news detection at the knowledge element level . they propose synthesis method which manipulates knowledge elements to generate noisy training data . |
| Outcome: | The proposed method outperforms the state-of-the-art in detecting misinformation . it yields fine-grained explanations and outperformed the current methods . |
I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Recent advances on neural approaches to natural language processing have triggered a resurgent interest on building intelligent open-domain chatbots. |
| Approach: | They propose a dialoguE COntradiction DEtection task and a conversational dataset . they show that their best contradiction detection model correlates well with human judgments . |
| Outcome: | The proposed model is more robust and generalizes well on analysis and out-of-distribution dialogues than standard (unstructured) Transformer models that explicitly hinge on utterance structures are more robust, the study shows . |
A Sequence-to-Sequence Approach to Dialogue State Tracking (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for dialogue state tracking are still challenging, but they are improving . a new approach to dialogue state monitoring is proposed, called Seq2Seq-DU . |
| Approach: | They propose a new dialogue state tracking module that formalizes DST as a sequence-to-sequence problem. |
| Outcome: | The proposed method outperforms existing methods on benchmark datasets in different settings. |
Discovering Dialog Structure Graph for Coherent Dialog Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on dialog structure graphs from open-domain dialogs have limited number of dialog states and can be laborious and costly to annotate manually. |
| Approach: | They propose to use dialog structure graph as a model to discover hierarchical latent dialog states and their transitions from corpus to facilitate dialog management in a RL based dialog system. |
| Outcome: | The proposed model can discover meaningful dialog structure graph and significantly improve multi-turn coherence on two benchmark corpora. |
Dialogue Response Selection with Hierarchical Curriculum Learning (2021.acl-long)
Copied to clipboard
Yixuan Su, Deng Cai, Qingyu Zhou, Zibo Lin, Simon Baker, Yunbo Cao, Shuming Shi, Nigel Collier, Yan Wang
| Challenge: | Empirical studies on three benchmark datasets with three state-of-the-art matching models demonstrate that the proposed learning framework significantly improves the model performance across various evaluation metrics. |
| Approach: | They propose a hierarchical curriculum learning framework that trains matching models in an “easy-to-difficult” scheme. |
| Outcome: | The proposed framework significantly improves the model performance across evaluation metrics on three benchmark datasets with three state-of-the-art matching models. |
A Joint Model for Dropped Pronoun Recovery and Conversational Discourse Parsing in Chinese Conversational Speech (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work regards dropped pronoun recovery and conversational discourse parsing as two separate tasks and tackles them separately. |
| Approach: | They propose a neural model for dropped pronoun recovery and conversational discourse parsing in Chinese conversational speech. |
| Outcome: | The proposed model outperforms the state-of-the-art models on a new dataset . the proposed model is based on linguistic and semantic information from Chinese conversational speech . |
A Systematic Investigation of KB-Text Embedding Alignment at Scale (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge bases (KBs) and text often contain complementary knowledge. |
| Approach: | They propose a framework for aligning KB and text embeddings for joint reasoning . they also evaluate alignment methods to infuse textual information into KB embeddables . |
| Outcome: | The proposed framework can be used to predict link prediction on emerging entities and events using textual information. |
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work focuses on learning deep NER models with weak supervision without any human annotation. |
| Approach: | They propose a framework that can suppress the noise of the weak labels and fine-tune over the strongly labeled data. |
| Outcome: | The proposed framework outperforms existing methods on Named Entity Recognition tasks with weak supervision and weakly labeled data. |
Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-grained entity typing use weak labels that are automatically generated. |
| Approach: | They propose to obtain training data by using a BERT Masked Language Model (MLM) given a mention in a sentence, they construct an input for the MLM so it predicts context dependent hypernyms of the mention, which can be used as type labels. |
| Outcome: | The proposed model improves performance by using type labels generated from a BERT Masked Language Model given a mention in a sentence. |
Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work shows document-level contexts can significantly improve Named Entity Recognition models. |
| Approach: | They propose to find external contexts of a sentence by retrieving and selecting a set of semantically relevant texts through a search engine with the original sentence as the query. |
| Outcome: | The proposed approach can achieve new state-of-the-art performance on 8 NER data sets across 5 domains. |
Implicit Representations of Meaning in Neural Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Neural language models (NLMs) encode lexical relations and syntactic structure, but their effectiveness is still unclear. |
| Approach: | They propose to use text as a model to model entities and situations as they evolve throughout a discourse. |
| Outcome: | The proposed models have functional similarities to linguistic models of dynamic semantics and can be learned with only text as training data. |
Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Targeted syntactic evaluations have demonstrated the ability of language models to perform subject-verb agreement given difficult contexts. |
| Approach: | They apply causal mediation analysis to pre-trained neural language models to investigate their models' preferences for grammatical inflections and whether neurons process subject-verb agreement similarly across sentences with different syntactic structures. |
| Outcome: | The proposed model can predict correct token from grammatically minimally different continuations with high accuracy even in difficult contexts. |
Bird’s Eye: Probing for Linguistic Graph Structures with a Simple Information-Theoretic Approach (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work on analyzing contextualized text representations has focused on hand-designed probe models to understand how and to what extent do these representations encode a particular linguistic phenomenon. |
| Approach: | They propose a new information-theoretic probe, Bird’s Eye, which detects if and how representations encode the information in contextualized text representations. |
| Outcome: | The proposed method estimates the mutual information between the linguistic graph embedded in a continuous space and the contextualized word representations. |
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained masked language models can be factual knowledge bases. |
| Approach: | They conduct a rigorous study to explore the underlying predicting mechanisms of MLMs . they find that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts a . |
| Outcome: | The proposed model improves on illustrative cases and external contexts . the results question the previous findings that MLMs can be reliable factual knowledge bases . |
Poisoning Knowledge Graph Embeddings via Relation Inference Patterns (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge graph embeddings (KGE) models are increasingly deployed in domains with high stake decision making where it is critical to identify the potential security vulnerabilities that might cause failure. |
| Approach: | They propose to exploit the inductive abilities of knowledge graph embedding models by crafting adversarial additions that can improve model’s confidence on decoy facts. |
| Outcome: | The proposed attacks outperform state-of-the-art baselines on four KGE models for two publicly available datasets and generalize across all model-dataset combinations. |
Bad Seeds: Evaluating Lexical Methods for Bias Measurement (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for measuring bias use crowd-sourced seed lexicons, but there is little guidance for their selection. |
| Approach: | They use lexicons of different types of social biases and linguistic features to enumerate biased seeds from three English-language corpora. |
| Outcome: | The results show that seed lexicons can be used to measure bias in English-language corpora . the results show the seeds can be re-used in other contexts . |
A Survey of Race, Racism, and Anti-Racism in NLP (2021.acl-long)
Copied to clipboard
| Challenge: | despite inextricable ties between race and language, little work has considered race in NLP research and development. |
| Approach: | They survey 79 papers from the ACL anthology that mention race . they find race has been siloed as a niche topic and ignored in many NLP tasks . authors call for inclusion and racial justice in NLP research practices . |
| Outcome: | The findings highlight the need for inclusion and racial justice in NLP research practices. |
Intrinsic Bias Metrics Do Not Correlate with Application Bias (2021.acl-long)
Copied to clipboard
| Challenge: | a recent survey of bias in natural language processing found that a coreference system makes more errors in an anti-stereotypical coreferent than in a pro-sterereotype one. |
| Approach: | They compare intrinsic and extrinsic bias metrics across hundreds of trained models . they urge researchers to focus on extrindic measures of bias, not easy to measure . |
| Outcome: | a new intrinsic metric and an annotated test set on gender bias in hate speech are tested . authors urge researchers to focus on extrinsic measures of bias, and to make them more feasible . |
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work has focused on measuring and mitigating bias in pretrained language models. |
| Approach: | They propose a dataset that measures and mitigates bias across gender,race, religion, and queerness . they compare REDDITBIAS to a widely used conversational DialoGPT model . |
| Outcome: | The proposed framework measures and mitigates bias across gender,race, religion, and queerness dimensions. |
Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks (2021.acl-long)
Copied to clipboard
| Challenge: | Prior research has found that only a few attention heads are important in each mono-lingual NLP task and pruning the remaining heads leads to comparable or improved performance of the model. |
| Approach: | They examine the relative importance of attention heads in Transformer-based models to aid their interpretability in cross-lingual and multi-lingual tasks. |
| Outcome: | The proposed model performs better with the remaining heads pruned than with the other models, the authors show . |
Crafting Adversarial Examples for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Effective adversary generation for neural machine translation is crucial for robust systems. |
| Approach: | They propose to leverage round-trip translation technique to build valid metrics for evaluating NMT adversarial attacks. |
| Outcome: | The proposed method could break the state-of-art NMT models with small perturbations. |
UXLA: A Robust Unsupervised Data Augmentation Framework for Zero-Resource Cross-Lingual NLP (2021.acl-long)
Copied to clipboard
| Challenge: | Transfer learning has yielded state-of-the-art (SoTA) results, but annotated data for every target task in every target language is rare, especially for low-resource languages. |
| Approach: | They propose a framework for unsupervised data augmentation for zero-resource transfer learning scenarios that performs simultaneous self-training with data hausse and unsupervised sample selection. |
| Outcome: | The proposed framework outperforms baselines on three zero-resource transfer tasks and achieves SoTA results in all the tasks. |
Glancing Transformer for Non-Autoregressive Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation methods are either inferior to Transformer or require multiple decoding passes, leading to reduced speedup. |
| Approach: | They propose a Glancing Language Model (GLM) for single-pass parallel generation models and Glancing Transformer (GLAT) with only single- pass decoding, GLAT is able to generate high-quality translation with 8-15 speedup. |
| Outcome: | The proposed model outperforms all previous non-autoregressive methods on multiple language directions and is nearly comparable to Transformer. |
Hierarchical Context-aware Network for Dense Video Event Captioning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing technologies for dense video event captioning generate fine-grained captions for all events in a long untrimmed video. |
| Approach: | They propose a hierarchical context-aware network for dense video event captioning to capture context from various aspects. |
| Outcome: | The proposed model outperforms the existing model on youcook2 and activitynet . it generates coherent captions for events in a long untrimmed video . |
Control Image Captioning Spatially and Temporally (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to generate image captions with user intention are still under exploration. |
| Approach: | They propose a model that connects Contrastive constraints and Attention Guidance in a loop manner and engages explicit spatial and temporal constraints to the generating process. |
| Outcome: | The proposed model improves performance on a trace-controlled image captioning task. |
Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation (2021.acl-long)
Copied to clipboard
| Challenge: | Edited media frames are structured annotations with respect to intents, emotional reactions, attacks on individuals, and the implications of disinformation. |
| Approach: | They propose a new formalism to understand visual media manipulation as structured annotations with respect to intents, emotional reactions, attacks on individuals, and the implications of disinformation. |
| Outcome: | The proposed model obtains promising results on a dataset with 56k question-answer pairs written in rich natural language. |
PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World (2021.acl-long)
Copied to clipboard
Rowan Zellers, Ari Holtzman, Matthew Peters, Roozbeh Mottaghi, Aniruddha Kembhavi, Ali Farhadi, Yejin Choi
| Challenge: | PIGLeT model learns physical commonsense knowledge through interaction, then uses this knowledge to ground language. |
| Approach: | They propose a model that learns physical commonsense knowledge through interaction . they factorize PIGLeT into a physical dynamics model and a separate language model . |
| Outcome: | The proposed model outperforms a 100x larger, text-to-text approach in forecasting language . it can read a sentence, simulate neurally what might happen next, and communicate that result through a literal symbolic representation, or natural language. |
Modeling Fine-Grained Entity Types with Box Embeddings (2021.acl-long)
Copied to clipboard
| Challenge: | Neural entity typing models typically represent fine-grained entity types as vectors in a high-dimensional space, but such spaces are not well-suited to modeling complex interdependencies. |
| Approach: | They propose to use box embeddings to embed types into a high-dimensional hyperrectangle space and then use it to hypothesize a type representation for the mention. |
| Outcome: | The proposed model captures latent type hierarchies better than a vector-based model on several entity typing benchmarks. |
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information (2021.acl-long)
Copied to clipboard
| Challenge: | ChineseBERT model incorporates glyph and pinyin information of Chinese characters into pretraining . proposed model achieves new performance boost over baseline models with fewer training steps . |
| Approach: | They propose a ChineseBERT model that incorporates glyph and pinyin information into pretraining . the glyph embedding is obtained based on different fonts of a character, and the pinyink embeddment characterizes the pronunciation of Chinese characters. |
| Outcome: | The proposed model achieves new performance boosts over baseline models with fewer training steps. |
Weight Distillation: Transferring the Knowledge in Neural Network Parameters (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge distillation is an effective method for model acceleration and compression. |
| Approach: | They propose to use parameters to distill knowledge from large neural networks to small ones . they propose to do this by using a parameter generator to transfer the knowledge to a small neural network . |
| Outcome: | The proposed method learns a small network 1.88 2.94x faster than the large network but with competitive BLEU points. |
Optimizing Deeper Transformers on Small Datasets (2021.acl-long)
Copied to clipboard
Peng Xu, Dhruv Kumar, Wei Yang, Wenjie Zi, Keyi Tang, Chenyang Huang, Jackie Chi Kit Cheung, Simon J.D. Prince, Yanshuai Cao
| Challenge: | a common belief that training deep transformers from scratch requires large datasets is wrong . however, with proper initialization and optimization, the benefits of very deep transformer can carry over to challenging tasks with small datasets. |
| Approach: | They train 48 layers of transformers from pre-trained RoBERTa and 24 relation-aware layers from scratch. |
| Outcome: | The proposed scheme achieves state-of-the-art performance on a text-to-sql parsing benchmark . it uses 24 fine-tuned layers from pre-trained RoBERTa and 24 relation-aware layers from scratch . |
BERTAC: Enhancing Transformer-based Language Models with Adversarially Pretrained Convolutional Neural Networks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models of NLP are fading away, but new ones are needed to maintain their dominance. |
| Approach: | They propose a method to pretrain a CNN using Wikipedia data and integrate it with standard TLMs. |
| Outcome: | The proposed method outperforms the original ALBERT on GLUE tasks and achieves similar performance to SOTA on open-domain QA tasks. |
COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic (2021.acl-long)
Copied to clipboard
| Challenge: | a new method for fact-checking is needed to detect disinformation on the web . a dataset COVID-Fact contains 4,086 claims concerning the COVId-19 pandemic . |
| Approach: | They propose a FEVER-like dataset COVID-Fact of 4,086 claims concerning the COVId-19 pandemic . they automatically detect true claims and their source articles and generate counter-claims using automatic methods . |
| Outcome: | The proposed method reduces the cost of building domain-specific datasets for detecting misinformation . the proposed dataset contains 4,086 claims concerning the COVID-19 pandemic . |
Explaining Relationships Between Scientific Documents (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to explain relationships between scientific documents using natural language text can be useful for research efficiency. |
| Approach: | They propose a task of explaining relationships between scientific documents using natural language text. |
| Outcome: | The proposed models can be automated and humanely evaluated. |
IrEne: Interpretable Energy Prediction for Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. |
| Approach: | They propose an interpretable and extensible energy prediction system that predicts inference energy consumption of Transformer-based NLP models. |
| Outcome: | The proposed system predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. |
Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach (2021.acl-long)
Copied to clipboard
| Challenge: | Existing efforts to enhance the performance of session-based cyberbullying detection have overlooked unintended social biases in existing datasets. |
| Approach: | They propose a model-agnostic debiasing strategy that leverages a reinforcement learning technique to mitigate unintended biases in existing datasets. |
| Outcome: | The proposed approach can mitigate unintended biases without impairing the detection performance. |
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context (2021.acl-long)
Copied to clipboard
| Challenge: | Creating effective visualizations is a challenge given the myriad of parameters that users need to provide. |
| Approach: | They propose to synthesize visualization programs from natural language utterances and programmatic context using PlotCoder. |
| Outcome: | The proposed architecture models both the code context and the input utterance. |
Changing the World by Changing the Data (2021.acl-long)
Copied to clipboard
| Challenge: | a new paper argues that data curation is already happening, and it is changing the world . social biases and spurious patterns are attracting more attention in NLP models . |
| Approach: | They argue that data curation is already happening and will be happening . they argue that social biases and spurious patterns are the main problems . |
| Outcome: | a new paper argues that data curation is already and will be happening, and it is changing the world. |
EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets (2021.acl-long)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models require enormous computational resources and long training time. |
| Approach: | They propose an algorithm to reduce inference time and train large NLP models by slimming the self-attention and fully-connected sub-layers inside a transformer. |
| Outcome: | The proposed algorithm achieves comparable performance to standard BERT with 35 45% less training time. |
On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation (2021.acl-long)
Copied to clipboard
Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jiawei Low, Lidong Bing, Luo Si
| Challenge: | Existing studies have shown that adapter-based tuning is more parameter-efficient than fine-tuning. |
| Approach: | They propose to add adapter modules to a pretrained language model and update the parameters of adapter module when learning on a downstream task. |
| Outcome: | The proposed method outperforms fine-tuning on low-resource and cross-lingual tasks and settings. |
Data Augmentation for Text Generation Without Any Augmented Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for data augmentation need to define or choose proper data mapping functions to create augmented samples. |
| Approach: | They propose to use data mapping functions to augment text samples without using specific mapping functions. |
| Outcome: | The proposed approach can approximate or even surpass popular data augmentation methods on two text generation tasks with a convergence rate guarantee. |
Integrating Semantics and Neighborhood Information with Graph-Driven Generative Models for Document Retrieval (2021.acl-long)
Copied to clipboard
Zijing Ou, Qinliang Su, Jianxing Yu, Bang Liu, Jingwen Wang, Ruihui Zhao, Changyou Chen, Yefeng Zheng
| Challenge: | Existing methods for document hashing combine only one of semantics and neighborhood information, lacking a theoretical principle to guide the integration process. |
| Approach: | They propose to encode neighborhood information with a graph-induced Gaussian distribution and integrate it with generative models. |
| Outcome: | The proposed model can be trained as efficiently as state-of-the-art methods on benchmark datasets. |
SMURF: SeMantic and linguistic UndeRstanding Fusion for Caption Evaluation via Typicality Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Visual captioning is an open-ended area for evaluation, requiring specialized training to improve human-correlation. |
| Approach: | They propose a new evaluation framework rooted in information theory . they propose metric SPURTS and metric SMURF to measure fluency . |
| Outcome: | The proposed metrics achieve state-of-the-art correlation with human judgment compared with other evaluation metrics. |
KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers (2021.acl-long)
Copied to clipboard
| Challenge: | Recent large-scale datasets such as Spider and WikiSQL facilitated novel modeling techniques for text-to-SQl parsing. |
| Approach: | They propose a new cross-domain evaluation dataset of real Web databases . they examine the choice of evaluation tasks for text-to-SQL parsers . |
| Outcome: | The proposed model improves accuracy by 13.2% over state-of-the-art parsers in real-life environments. |
QASR: QCRI Aljazeera Speech Resource A Large Scale Annotated Arabic Speech Corpus (2021.acl-long)
Copied to clipboard
| Challenge: | QASR is the largest transcribed Arabic speech corpus in the broadcast domain. |
| Approach: | They introduce the largest transcribed Arabic speech corpus, QASR, collected from the broadcast domain. |
| Outcome: | The proposed dataset contains 2,000 hours of speech sampled at 16kHz crawled from Aljazeera news channel. |
An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | In the recent years, pre-trained language models have achieved great success in the NLP community. |
| Approach: | They propose two general strategies and an experimental procedure to troubleshoot HPO’s failure cases. |
| Outcome: | The proposed methods outperform grid search on two state-of-the-art language models using the same time budget and overfitting. |
Better than Average: Paired Evaluation of NLP systems (2021.acl-long)
Copied to clipboard
| Challenge: | Evaluation in NLP is usually done by comparing the scores of competing systems . averaging scores independently and declaring the best system is difficult . |
| Approach: | They examine the use of averages to aggregate evaluation scores into a final number . they argue that the average ignores the pairing arising from the fact that systems are evaluated on the same test instances. |
| Outcome: | The proposed method ignores the pairing arising from the fact that systems are evaluated on the same test instances. |
Chase: A Large-Scale and Pragmatic Chinese Dataset for Cross-Database Context-Dependent Text-to-SQL (2021.acl-long)
Copied to clipboard
| Challenge: | XDTS is a cross-database context-dependent text-to-sql problem with wide range of applications. |
| Approach: | They present a large-scale Chinese dataset for cross-database context-dependent Text-to-SQL . they find that only 35% of questions are context-independent and 28% of SQL queries are easy . |
| Outcome: | The proposed approach achieves an exact match accuracy of 40% over all questions and 16% over all question sequences. |
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models are vulnerable to simple perturbations, causing poor robustness . recent studies show that adversarial training is useless or harmful for the model to detect these semantic changes. |
| Approach: | They propose to use adversarial training to improve the robustness of pre-trained models . they propose to construct negative examples with similar and opposite semantics . |
| Outcome: | Empirical results show that the proposed approach improves on sentiment analysis, reasoning, and reading comprehension tasks. |
Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for topic modeling learn topics with a flat structure . however, such methods have data scalability issues . |
| Approach: | They propose to use nonparametric neural variational inference to extract a tree-structured topic model with reasonable structure, low redundancy, and adaptable widths. |
| Outcome: | The proposed model extracts a tree-structured topic hierarchy with reasonable structure, low redundancy, and adaptable widths. |
ExCAR: Event Graph Knowledge Enhanced Explainable Causal Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work infers the causation between events based on knowledge from annotated causal event pairs, but additional evidence information is unexploited. |
| Approach: | They propose an Event graph knowledge enhanced explainable CAusal Reasoning framework that acquires additional evidence information from a large-scale causal event graph as logical rules for causal reasoning. |
| Outcome: | The proposed framework outperforms state-of-the-art methods in human evaluation and in animal models. |
Distributed Representations of Emotion Categories in Emotion Space (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on emotion detection focus on how to improve performance of models . however, emotion relations are ignored in one-hot representations . |
| Approach: | They propose a framework to learn distributed representations for emotion categories in emotion space from a given emotion classification dataset. |
| Outcome: | The proposed representations can express emotion relations much better than word vectors in semantic space. |
Style is NOT a single variable: Case Studies for Cross-Stylistic Language Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | a benchmark corpus of text in 15 different styles is used to study stylistic language . a similar benchmark is used for cross-style language understanding . |
| Approach: | They propose a benchmark corpus that combines existing datasets and collects a new one for cross-style language understanding. |
| Outcome: | The proposed benchmark corpus contains 15 different styles under four theoretical groupings: figurative, personal, affective, and interpersonal groups. |
DynaSent: A Dynamic Benchmark for Sentiment Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Sentiment analysis is an early success story for NLP, in both a technical and an industrial sense. |
| Approach: | They propose to combine naturally occurring sentences with sentences created using the open-source Dynabench Platform, which facilities human-and-model-in-the-loop dataset creation. |
| Outcome: | The proposed model is more coherent than comparable models and motivates training models from scratch over successive fine-tuning. |
A Hierarchical VAE for Calibrating Attributes while Generating Text using Normalizing Flow (2021.acl-long)
Copied to clipboard
| Challenge: | Existing text-style transfer techniques focus on flipping the style attribute polarity instead of fine-grained regulation of attributes to generate multiple variants of a sentence. |
| Approach: | They propose a hierarchical architecture for finer control over the attribute, preserving content using attribute dis- entanglement. |
| Outcome: | The proposed framework generates natural looking sentences with finer control of intensity of a given attribute. |
A Unified Generative Framework for Aspect-based Sentiment Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Existing complicated ABSA models focus on subtasks, which leads to complicated solutions . et al., j. c. d. r., and j dr. s. v. present a unified approach to solve seven subtask tasks in one framework. |
| Approach: | They redefine every subtask target as a sequence mixed by pointer indexes and sentiment class indexe . they exploit the pre-training sequence-to-sequence model BART to solve all ABSA subtasks in an end-to end framework. |
| Outcome: | The proposed framework achieves substantial performance gain and provides a real unified solution for the whole ABSA subtasks. |
Discovering Dialogue Slots with Weak Supervision (2021.acl-long)
Copied to clipboard
| Challenge: | Task-oriented dialogue systems typically require manual annotation of dialogue slots in training data. |
| Approach: | They propose a method that uses weak supervision to identify slot candidates and automatically identify domain-relevant slots by using clustering algorithms. |
| Outcome: | The proposed method significantly improves an end-to-end dialogue response generation model compared to using no slot annotation at all. |
Enhancing the generalization for Intent Classification and Out-of-Domain Detection in SLU (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for intent classification are expensive to collect and train . evaluators have shown that the ability to detect out-of-domain utterances is limited . |
| Approach: | They propose to train a model with only IND data while supporting both intent classification and OOD detection. |
| Outcome: | The proposed model improves on existing models and strong baselines on four datasets. |
PROTAUGMENT: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning (2021.acl-long)
Copied to clipboard
| Challenge: | Recent research considers few-shot intent detection as a meta-learning problem because of labeled data scarcity and the number of classes involved. |
| Approach: | They propose a meta-learning algorithm for short texts classification that limits overfitting on the bias introduced by the few-shots classification objective at each episode. |
| Outcome: | The proposed algorithm limits overfitting on the bias introduced by the few-shots classification objective at each episode. |
Robustness Testing of Language Understanding in Task-Oriented Dialog (2021.acl-long)
Copied to clipboard
Jiexi Liu, Ryuichi Takanobu, Jiaxin Wen, Dazhen Wan, Hongguang Li, Weiran Nie, Cheng Li, Wei Peng, Minlie Huang
| Challenge: | a lack of systematic studies on the robustness of language understanding models in task-oriented dialog systems is limiting . authors propose a model-agnostic toolkit LAUG to approximate natural language perturbations . |
| Approach: | They propose a model-agnostic toolkit LAUG to approximate natural language perturbations for testing the robustness of language understanding models in task-oriented dialog systems. |
| Outcome: | The proposed toolkit reveals critical robustness issues in state-of-the-art models. |
Comprehensive Study: How the Context Information of Different Granularity Affects Dialogue State Tracking? (2021.acl-long)
Copied to clipboard
| Challenge: | Dialogue state tracking (DST) plays a key role in task-oriented dialogue systems to monitor the user’s goal. |
| Approach: | They propose to use scratch-based and previous-based strategies to track dialogue state . they explore how different granularities affect dialogue state tracking . |
| Outcome: | The scratch-based strategy obtains each slot value by inquiring all the dialogue history, while the previous-based method is not very useful for long-dependency dialogue state tracking. |
OTTers: One-turn Topic Transitions for Open-Domain Dialogue (2021.acl-long)
Copied to clipboard
| Challenge: | a mixed-initiative dialogue system is often purely responsive, make abrupt transitions, or fail to take initiative. |
| Approach: | They propose a task to generate a "bridging" utterance connecting a new topic to the previous conversation turn. |
| Outcome: | The proposed task generates a "bridging" utterance connecting a new topic to the previous topic. |
Towards Robustness of Text-to-SQL Models against Synonym Substitution (2021.acl-long)
Copied to clipboard
Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R. Woodward, Jinxia Xie, Pengsheng Huang
| Challenge: | Existing text-to-SQL models rely on lexical matching between words in NL questions and tokens in table schemas, which may break the schema linking mechanism. |
| Approach: | They propose a human-curated dataset for text-to-SQL translation . they replace schema-related words with manually selected synonyms . |
| Outcome: | The proposed model outperforms its counterparts without the defense. |
KACE: Generating Knowledge Aware Contrastive Explanations for Natural Language Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches in NLP focus on “WHY A” rather than contrastive “WHA NOT B” Experimental results show that contrastive explanations are beneficial to fit the scenarios by clarifying the difference between the predicted answer and other possible wrong ones. |
| Approach: | They propose to generate contrastive explanations with counterfactual examples in NLI by identifying key phrases from input sentences and using them as key perturbations to generate counterfacts. |
| Outcome: | The proposed framework improves on SNLI and ETPA models by 91.9%. |
Self-Guided Contrastive Learning for BERT Sentence Representations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to derive sentence embeddings from pre-trained Transformers are unclear . a self-guided training method is used to fine-tune BERT in a supervised fashion . |
| Approach: | They propose a contrastive learning method that utilizes self-guidance to improve BERT sentence representations. |
| Outcome: | The proposed method is more effective than baselines on diverse sentence-related tasks and robust to domain shifts. |
LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to encode text-to-SQL data are node-centric and ignore semantics embedded in the topological structure of edges. |
| Approach: | They propose a Line Graph Enhanced Text-to-SQL model to mine relational features without constructing meta-paths. |
| Outcome: | The proposed model achieves state-of-the-art on the cross-domain text-to-SQL benchmark Spider at the time of writing. |
Multi-stage Pre-training over Simplified Multimodal Pre-training Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal pre-training models require large amounts of training data and have huge model sizes, making them impossible to apply in low-resource situations. |
| Approach: | They propose a multi-stage pre-training method which uses information at different granularities from word, phrase to sentence in both texts and images to pre-train a model in stages. |
| Outcome: | The proposed method outperforms the original model in Image-Text Retrieval task and outperformed the original LXMERT model in downstream tasks. |
Beyond Sentence-Level End-to-End Speech Translation: Context Helps (2021.acl-long)
Copied to clipboard
| Challenge: | Document-level contextual information has shown benefits to text-based machine translation, but whether and how it helps end-to-end speech translation is still under-studied. |
| Approach: | They propose a concatenation-based ST model with adaptive feature selection for computational efficiency. |
| Outcome: | The proposed model improves translation quality and robustness to (artificial) audio segmentation errors. |
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding (2021.acl-long)
Copied to clipboard
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou
| Challenge: | Existing pre-training tasks for text and layout are effective in visually-rich document understanding tasks. |
| Approach: | They propose to combine pre-training tasks with a multi-modal model to model interaction between text, layout and image in a single multi-module framework. |
| Outcome: | The proposed model outperforms LayoutLM by a large margin on visual-rich document understanding tasks. |
UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing pre-training methods focus on single-modal tasks or multi-modal ones . large-scale pre- training has drawn much attention in both the community of Compute Vision (CV) and Natural Language Processing (NLP). |
| Approach: | They propose a UNIfied-MOdal pre-training architecture which can adapt to both single-modal and multi-modal understanding and generation tasks. |
| Outcome: | The proposed model can learn more generalizable representations with rich non-paired single-modal data. |
Missing Modality Imagination Network for Emotion Recognition with Uncertain Missing Modalities (2021.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal fusion models trained on full-modality samples fail when partial modalities are missing. |
| Approach: | They propose a model to deal with the uncertain missing modality problem by learning robust joint multimodal representations that can predict the representation of any missing modal given available modalities under different missing-modality conditions. |
| Outcome: | The proposed model significantly improves performance under uncertain missing-modality testing conditions and full-modalities ideal testing conditions. |
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders (2021.acl-long)
Copied to clipboard
| Challenge: | End-to-end Speech Translation (E2E ST) encoders lack global context representation, whereas MT encoder lacks it. |
| Approach: | They propose a Stacked Acoustic-and-Textual Encoding method for speech translation . they propose an adaptor module to alleviate representation inconsistency . |
| Outcome: | The proposed method achieves state-of-the-art BLEU scores of 18.3 and 25.2 on two ST tasks. |
N-ary Constituent Tree Parsing with Recursive Semi-Markov Model (2021.acl-long)
Copied to clipboard
| Challenge: | Existing graph-based constituent parsing methods generate hidden nodes with the dummy label inside the n-ary nodes to transform the tree into a binary tree for prediction. |
| Approach: | They propose a graph-based constituent parsing framework that uses a 1-order semi-Markov model to predict the immediate children sequence of a constituent candidate. |
| Outcome: | The proposed framework obtains the F1 of 95.92% and 92.50% on the datasets of PTB and CTB 5.1 respectively. |
Automated Concatenation of Embeddings for Structured Prediction (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work shows that better word representations can be obtained by concatenating different types of embeddings. |
| Approach: | They propose to automate the process of finding better concatenated embeddings for structured prediction tasks by concatending different types of embeddables. |
| Outcome: | The proposed approach outperforms baselines and achieves state-of-the-art with fine-tuned embeddings on 6 tasks and 21 datasets. |
Multi-View Cross-Lingual Structured Prediction with Minimum Supervision (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on cross-lingual transfer learning focuses on transferring knowledge from high-resource languages to low-resourced ones. |
| Approach: | They propose a multi-view framework that integrates multiple source models into an aggregated source view and transfers it to a target view based on a task-specific model. |
| Outcome: | The proposed framework improves on three structured prediction tasks on 16 datasets. |
The Limitations of Limited Context for Constituency Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | a language model that is syntax-aware can produce better samples, authors say . a recent study shows that neural approaches to syntax can perform unsupervised syntactic parsing . |
| Approach: | They propose to incorporate syntax into neural approaches in NLP to produce better samples . they find that the first time neural approaches were able to perform unsupervised syntactic parsing . |
| Outcome: | The proposed models can perform unsupervised syntactic parsing, but they are lagging behind . the proposed models are based on a sandbox of probabilistic context-free-grammars . |
Neural Bi-Lexicalized PCFG Induction (2021.acl-long)
Copied to clipboard
| Challenge: | Neural lexicalized PCFGs make strong independence assumption on the generation of the child word and thus bilexical dependencies are ignored. |
| Approach: | They propose an approach to parameterize L-PCFGs without making implausible independence assumptions. |
| Outcome: | The proposed approach improves both running speed and unsupervised parsing performance on the English WSJ dataset. |
Ruddit: Norms of Offensiveness for English Reddit Comments (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect offensive language have been limited by categorical labels . however, there are several challenges in the detection of such content . |
| Approach: | They analyze Reddit comments with fine-grained, real-valued offensiveness scores . they evaluate the ability of widely-used neural models to predict offensiveness . |
| Outcome: | The proposed method produces highly reliable offensiveness scores and can predict scores on reddit comments. |
Towards Quantifiable Dialogue Coherence Evaluation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing automatic dialogue coherence evaluation metrics are expensive and high-latency, which cannot meet the requirements of a dialogue system. |
| Approach: | They propose a framework to train a quantifiable dialogue coherence metric that can reflect actual human rating standards. |
| Outcome: | Experimental results show that the model trained by QuantiDCE presents stronger correlations with human judgements than the other state-of-the-art metrics. |
Assessing the Representations of Idiomaticity in Vector Models with a Noun Compound Dataset Labeled at Type and Token Levels (2021.acl-long)
Copied to clipboard
| Challenge: | Existing resources for idiomaticity annotation only include ratings at type level . idioms such as noun compounds have been considered a challenge for NLP . |
| Approach: | They present a dataset with human annotations for 280 noun compounds in English and 180 in Portuguese at both type and token levels. |
| Outcome: | The proposed dataset shows that human annotations are not capturing idiomaticity as human annotation models. |
Factoring Statutory Reasoning as Language Understanding Challenges (2021.acl-long)
Copied to clipboard
| Challenge: | Statutory reasoning is the task of determining whether a legal statute applies to a text description of a case. |
| Approach: | They propose to decompose statutory reasoning into four types of language-understanding challenge problems using Prolog programming. |
| Outcome: | The proposed framework improves on existing baselines and finer-grained models. |
Evaluating Evaluation Measures for Ordinal Classification and Ordinal Quantification (2021.acl-long)
Copied to clipboard
| Challenge: | Ordinal Classification (OC) tasks require ordinal classes, not nominal ones, to be evaluated. |
| Approach: | They use data from the SemEval and NTCIR communities to clarify evaluation measures for Ordinal Classification and Ordinal Quantification tasks. |
| Outcome: | The evaluation measures for Ordinal Classification (OC) and Ordinal Quantification (OQ) tasks are ordinal, not nominal. |
Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making (2021.acl-long)
Copied to clipboard
| Challenge: | Entity Matching (EM) aims at recognizing entity records that denote the same real-world object. |
| Approach: | They propose a novel EM framework that consists of Heterogeneous Information Fusion and Key Attribute Tree Induction to decouple feature representation from matching decision. |
| Outcome: | The proposed framework outperforms SOTA EM models on 6 public datasets and 3 industrial datasets. |
Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a well-studied task in natural language processing. |
| Approach: | They propose a method that generates span proposals and labels them with categories . they use boundary information of entities and partially matched spans to locate them . |
| Outcome: | The proposed method outperforms state-of-the-art models on nested NER datasets. |
Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to extract event records from text decompose complex structure prediction task into multiple subtasks. |
| Approach: | They propose a sequence-to-structure generation paradigm that can extract events from text . they propose unified event extraction, constrained decoding algorithm and curriculum learning algorithm . |
| Outcome: | The proposed method can achieve competitive performance using record-level annotations in both supervised learning and transfer learning settings. |
A Large-Scale Chinese Multimodal NER Dataset with Speech Clues (2021.acl-long)
Copied to clipboard
| Challenge: | Using a large-scale dataset, we explore Chinese named entity recognition (NER) with both textual and acoustic contents. |
| Approach: | They propose a Chinese multimodal named entity recognition dataset . their corpus contains 42,987 annotated sentences and 71 hours of speech data . |
| Outcome: | The proposed model yields state-of-the-art (SoTA) results on Chinese multimodal named entity recognition (NER) based on 42,987 annotated sentences and 71 hours of speech data. |
A Neural Transition-based Joint Model for Disease Named Entity Recognition and Normalization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for disease recognition and normalization ignore text surface form of each candidate concept, causing boundary inconsistency. |
| Approach: | They propose a neural transition-based joint model to normalize disease entities from biomedical text. |
| Outcome: | The proposed model improves on two publicly available datasets. |
OntoED: Low-resource Event Detection with Ontology Embedding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to ED rely on training instances and ignore correlation of event types. |
| Approach: | They propose a process of event ontology population linking event instances to pre-defined event types in event ontoology and ontological embedding to address these problems. |
| Outcome: | The proposed framework can be applied to new unseen event types by establishing linkages to existing ones. |
Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that enhancing the learning on uncertain monolingual sentences improves the translation quality of high-uncertainty sentences and also benefits the prediction of low-frequency words at the target side. |
| Approach: | They propose to use monolingual data to augment model training with synthetic parallel data by selecting the most informative monolingual sentences to complement the parallel data. |
| Outcome: | The proposed approach improves the performance of natural language models by selecting the most informative monolingual sentences. |
Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training (2021.acl-long)
Copied to clipboard
| Challenge: | Context-aware neural machine translation (NMT) remains challenging due to the lack of large-scale document-level parallel corpora. |
| Approach: | They propose to use large-scale parallel datasets and source-side monolingual documents to improve context-aware neural machine translation. |
| Outcome: | The proposed model can be used to translate both sentences and documents on four translation tasks. |
Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation models are usually based on attention-based encoder-decoder frameworks. |
| Approach: | They introduce a seer decoder into the encoder-decoder framework during training . they force the conventional decoded decodes to simulate the behavior of the seer . |
| Outcome: | The proposed method outperforms baselines on Chinese, English and German translation tasks. |
Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference? (2021.acl-long)
Copied to clipboard
Luisa Bentivogli, Mauro Cettolo, Marco Gaido, Alina Karakanta, Alberto Martinelli, Matteo Negri, Marco Turchi
| Challenge: | a gap between direct approaches to speech translation (ST) and traditional cascade solutions has gradually decreased . a recent study found that the subtle differences observed in their behavior are not sufficient for humans neither to distinguish them nor to prefer one over the other. |
| Approach: | They compare state-of-the-art systems representative of the two paradigms . they find subtle differences observed in their behavior are not sufficient . |
| Outcome: | The proposed system is compared with state-of-the-art systems representative of the two paradigms. |
Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Unsupervised machine translation suffers from data-scarce domains, authors report . a meta-learning algorithm trains the model to adapt to another domain by utilizing only a small amount of training data. |
| Approach: | They propose a meta-learning algorithm that trains the model to adapt to another domain . their model surpasses a transfer learning-based approach by up to 2-3 BLEU scores . |
| Outcome: | The proposed algorithm outperforms a transfer learning-based approach by 2-3 BLEU scores . the proposed model outperformed previous models in the domain of unsupervised machine translation . |
Lightweight Cross-Lingual Sentence Representation Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for learning fixed-dimensional cross-lingual sentence representations are impractical due to memory limitations. |
| Approach: | They propose a lightweight dual-transformer architecture with just 2 layers for generating memory-efficient cross-lingual sentence representations. |
| Outcome: | The proposed model improves performance on training tasks and improves memory efficiency. |
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for document-level language pretraining are not suitable for long documents due to their quadratically increasing memory and time consumption. |
| Approach: | They propose a document-level language pretraining model based on Recurrence Transformers. |
| Outcome: | The proposed model outperforms existing models on language understanding tasks. |
Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) has shown great success in BERT compression. |
| Approach: | They propose a knowledge distillation paradigm that extracts the teacher's hidden state knowledge and then compresses it into three dimensions. |
| Outcome: | The proposed paradigm gives rise to training speedup of 2.7x 3.4x for two kinds of student models and computing devices. |
Rational LAMOL: A Rationale-based Lifelong Learning Framework (2021.acl-long)
Copied to clipboard
Kasidis Kanwatchara, Thanapapas Horsuwan, Piyawat Lertvittayakumjorn, Boonserm Kijsirikul, Peerapon Vateekul
| Challenge: | Existing paradigms for machine learning suffer from catastrophic forgetting when a model completely forgets what it just learned in previous tasks. |
| Approach: | They propose to exploit unsupervised rationale generation to improve the performance of a lifelong language learning model by applying critical freezing guided by human rationales. |
| Outcome: | The proposed framework outperforms vanilla LAMOL on most permutations and unsupervised rationale generation consistently improves the overall performance. |
EnsLM: Ensemble Language Model for Data Diversity by Semantic Clustering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that data diversity affects the performance of LMs if we train a single LM over the entire dataset. |
| Approach: | They propose an autoencoding topic model with a mixture prior to perform clustering for the data. |
| Outcome: | The proposed model can learn knowledge from different samples while extracting cluster-specific features. |
LeeBERT: Learned Early Exit for BERT with cross-level optimization (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models are resource exhaustive and computationally expensive for industrial scenarios. |
| Approach: | They propose a learning scheme to learn from each other to speed up inference . they ask each exit to learn the weights of different loss terms, instead of learning only from the last layer . |
| Outcome: | The proposed scheme improves state-of-the-art (SOTA) early exit methods for pre-trained models on the GLUE benchmark. |
Unsupervised Extractive Summarization-Based Representations for Accurate and Explainable Collaborative Filtering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing extractive summarization-based collaborative filtering models learn accurate representations of users and items based on user-given numeric ratings, but employing them is an oversimplification of user preferences and item characteristics. |
| Approach: | They propose to use BERT, K-Means embedding clustering, and multilayer perceptron to learn sentence embeddations, representation-explanations, and user-item interactions to create extractive summaries. |
| Outcome: | The proposed model improves rating prediction accuracy and user/item explainability. |
PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction (2021.acl-long)
Copied to clipboard
| Challenge: | Chinese spelling correction (CSC) is a task to detect and correct spelling errors in texts. |
| Approach: | They propose a Pre-trained masked Language model with Misspelled knowledgE (PLOME) which jointly learns how to understand language and correct spelling errors. |
| Outcome: | The proposed model outperforms state-of-the-art methods on widely used benchmarks and achieves superior performance against existing models. |
Competence-based Multimodal Curriculum Learning for Medical Report Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Medical report generation is more challenging for data-driven neural models due to data bias and limited medical data. |
| Approach: | They propose a Competence-based Multimodal Curriculum Learning framework to alleviate the data bias by efficiently utilizing the limited medical data for medical report generation. |
| Outcome: | The proposed framework can be incorporated into existing models to improve their performance on the IU-Xray and MIMIC-CXR datasets. |
Learning Syntactic Dense Embedding with Correlation Graph for Automatic Readability Assessment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing deep learning models for automatic readability assessment discard linguistic features traditionally used for the task. |
| Approach: | They propose to incorporate linguistic features into machine learning models by learning syntactic dense embeddings based on linguistic feature extraction. |
| Outcome: | Experiments with six data sets of two proficiency levels show that the proposed model can perform better than existing models. |
Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have been successful in NLP tasks, but their large size and long inference time limit their deployment in real-time applications. |
| Approach: | They propose a meta-teacher model that captures transferable knowledge across domains and passes it to students. |
| Outcome: | The proposed model can distill large teacher models into small student models with guidance from the meta-teacher. |
A Semantic-based Method for Unsupervised Commonsense Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to score candidates without labeled task data are difficult to use . e.g., pre-trained language models can be easily affected by irrelevant factors . |
| Approach: | They propose a method that generates plausible answers with generative models and uses them to select the correct answer. |
| Outcome: | The proposed method achieves the best results in unsupervised situations. |
Explanations for CommonsenseQA: New Dataset and Models (2021.acl-long)
Copied to clipboard
Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, Dinesh Garg
| Challenge: | a dataset called CommonsenseQA (CQA) was recently released to advance the research on common-sense question answering (QA) |
| Approach: | They propose to retrieve and generate explanations for a given question, correct answer choice, incorrect answer choices tuple from a dataset called CommonsenseQA. |
| Outcome: | The proposed model beats baseline model by 100% in F1 score and similarity score of 61.9 . |
Few-Shot Question Answering by Pretraining Span Selection (2021.acl-long)
Copied to clipboard
| Challenge: | Pretraining models with recurring span selection are effective, but perform poorly in a few-shot setting. |
| Approach: | They propose recurring span selection scheme that asks model to select correct span in passage with multiple sets of recurring recurrings. |
| Outcome: | The proposed model achieves 72.7 F1 on multiple benchmarks while maintaining competitive performance in the high-resource setting. |
UnitedQA: A Hybrid Approach for Open Domain Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work on open-domain question answering focuses on either extractive or generative readers exclusively. |
| Approach: | They propose a hybrid approach to extractive and generative readers that leverages both models. |
| Outcome: | The proposed approach outperforms state-of-the-art models on NaturalQuestions and TriviaQA respectively. |
Database reasoning over text (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models cannot handle database queries such as “List/Count all female athletes who were born in 20th century”. |
| Approach: | They propose a modular architecture to answer database-style queries over multiple spans from text and aggregate them at scale. |
| Outcome: | The proposed architecture scales to databases containing thousands of facts whereas current models are limited by how many facts can be encoded. |
Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effort (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate multiple systems are expensive and require human evaluators. |
| Approach: | They propose a novel online learning approach that dynamically converges to the top-3 ranked systems for the language pairs considered by taking advantage of human feedback. |
| Outcome: | The proposed approach converges to the top-3 ranked systems for the language pairs considered despite the lack of human feedback for many translations. |
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Using pretraining data, we find that a designated monolingual tokenizer plays an equally important role in the downstream performance of the model. |
| Approach: | They propose to compare pretrained multilingual models with their monolingual counterparts on a set of five diverse monolingual downstream tasks. |
| Outcome: | The proposed models offer previously unmatched performance in all NLP tasks. |
Evaluating morphological typology in zero-shot cross-lingual transfer (2021.acl-long)
Copied to clipboard
| Challenge: | morphological typology has been used to improve cross-lingual transfer . however, some language families and typologies consistently perform worse . |
| Approach: | They examine effects of morphological typology on zero-shot cross-lingual transfer . they perform part-of-speech tagging and sentiment analysis on 19 languages . |
| Outcome: | The proposed model improves on fusional and introflexive languages, but some language families and typologies perform worse. |
From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text (2021.acl-long)
Copied to clipboard
| Challenge: | a computational model for code-switching text is lacking in the corpus of real text. |
| Approach: | They propose a neural machine translation model to generate Hindi-English code-switched sentences using monolingual Hindi sentences. |
| Outcome: | The proposed model reduces perplexity on a language modeling task and improves on linguistic inference tasks. |
Fast and Accurate Neural Machine Translation with Translation Memory (2021.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge demonstrates the superiority of TM-based neural machine translation only on TM specialized tasks . |
| Approach: | They propose a translation memory-based approach to machine translation using a single bilingual sentence as its TM. |
| Outcome: | The proposed approach surpasses baselines on two general tasks and improves on the TM-specialized translation tasks. |
Annotating Online Misogyny (2021.acl-long)
Copied to clipboard
| Challenge: | Online misogyny is a category of online abusive language with serious and harmful social consequences. |
| Approach: | They propose an iterative annotation process and a taxonomy of labels for annotating misogyny in natural written language and cite a high-quality dataset of annotated posts from social media posts. |
| Outcome: | The proposed method aims to identify misogynistic language in natural written language and annotate it in social media posts using a high-quality dataset. |
Few-NERD: A Few-shot Named Entity Recognition Dataset (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to few-shot named entity recognition (NER) focus on coarse-grained entities with few examples, while most unseen entities are fine-grounded. |
| Approach: | They present a human-annotated few-shot named entity recognition dataset . they construct benchmark tasks to assess the generalization capability of models . |
| Outcome: | The proposed model is the first few-shot NER dataset and the largest human-crafted NER data set. |
MultiMET: A Multimodal Dataset for Metaphor Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Metaphor is a linguistic phenomenon and a cognitive phenomenon structuring human thought, authors say . previous studies focused on texts, partly due to the unavailability of ground truth labels of multimodal metaphor . |
| Approach: | They propose a multimodal metaphor dataset that integrates multimodal text and image . it contains 10,437 text-image pairs with multimodal annotations of occurrences . |
| Outcome: | The proposed dataset examines multimodal cues and their interplay. |
Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on generating hate speech/counter narratives have failed to reach high-quality datasets. |
| Approach: | They propose a human-in-the-loop data collection methodology that refines a generative language model iteratively by using its own data from previous loops to generate new training samples. |
| Outcome: | The proposed method is the only expert-based multi-target HS/CN dataset available to the community. |
Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA? (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work is limited in using small benchmarks with high test-train overlaps. |
| Approach: | They construct a dataset of closed-book QA using SQuAD and investigate the performance of BART. |
| Outcome: | Experiments show that pre-trained language models can achieve high performance on closed-book QA tasks. |
Joint Models for Answer Verification in Question Answering Systems (2021.acl-long)
Copied to clipboard
| Challenge: | Using a joint approach, we found that the model is more efficient than those developed in machine reading (MR) work. |
| Approach: | They propose a joint model for selecting correct answer sentences among the top k provided by answer sentence selection modules. |
| Outcome: | The proposed model improves on WikiQA, TREC-QA, and a real-world dataset. |
Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction (2021.acl-long)
Copied to clipboard
Yifan Gao, Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
| Challenge: | Open-domain question answering is a task to answer questions using passages with diverse topics. |
| Approach: | They propose a model that aggregates evidence from multiple passages to adaptively predict a single answer or a set of question-answer pairs for ambiguous questions. |
| Outcome: | The proposed model achieves state-of-the-art performance on AmbigQA dataset and shows competitive performance on NQ-Open and TriviaQA. |
TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance (2021.acl-long)
Copied to clipboard
Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, Tat-Seng Chua
| Challenge: | Existing QA systems focus on unstructured text, structured knowledge base, or semi-structured tables. |
| Approach: | They propose a large-scale question answering model based on financial reports . numerical reasoning is usually required to infer the answer . |
| Outcome: | The proposed model achieves 58.0% inF1, an 11.1% increase over the baseline model, but still lags behind the best human model. |
Modeling Transitions of Focal Entities for Conversational Knowledge Base Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | a new method for conversational Knowledge Base Question Answering (KBQA) uses implied entities from the conversation history to answer questions. |
| Approach: | They propose to model the implied entities of conversational KBQA by applying a graph neural network to derive a probability distribution of focal entities for each question. |
| Outcome: | The proposed model captures transitions of focal entities and performs answer ranking on two datasets. |
Evidence-based Factual Error Correction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to correct factual errors are limited to labeled claims . a recent task of fact verification has attracted significant attention . |
| Approach: | They propose a task of factual error correction that performs edits to a claim so that the generated rewrite is better supported by evidence. |
| Outcome: | The proposed method produces accurate factual error corrections for 5x more instances in human evaluation and a .125 increase in SARI score. |
Probabilistic, Structure-Aware Algorithms for Improved Variety, Accuracy, and Coverage of AMR Alignments (2021.acl-long)
Copied to clipboard
| Challenge: | Current algorithms for AMR parsing suffer from limited coverage and less-than-ideal accuracy . a new algorithm for AML uses unsupervised learning and heuristics to align components of AMR graphs to spans in English sentences . |
| Approach: | They propose algorithms for aligning components of Abstract Meaning Representation graphs to spans in English sentences. |
| Outcome: | The proposed approach covers a wider variety of AMR substructures than previously considered . it achieves higher coverage of nodes and edges, and does so with higher accuracy. |
Meta-Learning to Compositionally Generalize (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that neural networks struggle with compositional generalization . prior work asserts that there are fundamental differences between cognitive and connectionist architectures that make compositional globalization unlikely. |
| Approach: | They propose a meta-learning augmented version of supervised learning that optimizes for out-of-distribution generalization. |
| Outcome: | The proposed model improves generalization performance on COGS and SCAN datasets. |
Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to train pre-trained models require domain-specific data and computational resources. |
| Approach: | They propose a domain-aware N-gram Adaptor to incorporate unseen and domain-specific words into a generic pretrained model. |
| Outcome: | The proposed model can improve on eight low-resource tasks using limited data with lower computational costs. |
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)
Copied to clipboard
Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou
| Challenge: | Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering. |
| Approach: | They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text. |
| Outcome: | The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings. |
Position Bias Mitigation: A Knowledge-Aware Graph Model for Emotion Cause Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for ECE tend to explore relative position information and suffer from the dataset bias. |
| Approach: | They propose to generate adversarial examples where relative position is no longer indicative feature of cause clauses to address the dataset bias. |
| Outcome: | The proposed method performs on par with existing state-of-the-art methods on the original ECE dataset and is more robust against adversarial attacks compared to existing models. |
Every Bite Is an Experience: Key Point Analysis of Business Reviews (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for summarizing reviews focus on measuring sentiment toward aspects of the review . recent work shows that KPA improves performance without domain-specific annotation . |
| Approach: | They propose a framework that provides both textual and quantitative summary of the main points in review data. |
| Outcome: | The proposed framework significantly improves on existing methods without annotations and human supervision. |
Structured Sentiment Analysis as Dependency Graph Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | Structured sentiment analysis attempts to extract full opinion tuples from a text, but has been subdivided into smaller and smaller sub-tasks, e.g., target extraction or targeted polarity classification. |
| Approach: | They propose a framework which jointly predicts all elements of an opinion tuple and their relations by using dependency graph parsing. |
| Outcome: | The proposed framework improves on five datasets in English, Norwegian, Basque, and Catalan and refining the sentiment graphs with syntactic dependency information further improves results. |
Consistency Regularization for Cross-Lingual Fine-Tuning (2021.acl-long)
Copied to clipboard
Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei
| Challenge: | Experimental results show that consistency regularization improves cross-lingual fine-tuning . pre-trained cross-linguistic models can transfer task-specific supervision from one language to the other . |
| Approach: | They propose to improve cross-lingual fine-tuning with consistency regularization . they use example consistency regularized to penalize prediction sensitivity to four types of data augmentations . |
| Outcome: | The proposed method improves cross-lingual fine-tuning across tasks . it can be generalized to other target languages without additional training . |
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that denoising word alignment improves cross-lingual transferability . most applications and resources are still English-centric, making non-English users hard to access. |
| Approach: | They propose to denoise word alignment as a cross-lingual pre-training task . they first self-label word alignments for parallel sentences and then mask tokens . |
| Outcome: | The proposed model improves cross-lingual transferability on token-level tasks, especially on question answering, and structured prediction. |
Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) is commonly used to construct synthetic data for training non-autoregressive translation models. |
| Approach: | They propose to use knowledge distillation to generate training data for non-autoregressive translation models by leveraging pretraining. |
| Outcome: | The proposed approach achieves 28.2 and 33.9 BLEU points on the WMT14 English-German and WMT16 Romanian-English datasets. |
G-Transformer for Document-Level Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work extends translation unit from single sentence to multiple sentences. |
| Approach: | They propose to introduce locality assumption as an inductive bias into Transformer and reduce the hypothesis space of attention from target to source. |
| Outcome: | The proposed model achieves state-of-the-art BLEU scores on three benchmark datasets. |
Prevent the Language Model from being Overconfident in Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models are based on partial translation and a language model that predicts the next token based only on partial. |
| Approach: | They propose a Margin-based Token-level Objective and a Sentence-level Goal to maximize the Margin . they propose to model the next token based on partial translation . |
| Outcome: | The proposed approach improves translation adequacy and fluency on English-to-German, Chinese-to English and French translation tasks. |
Towards Emotional Support Dialog Systems (2021.acl-long)
Copied to clipboard
| Challenge: | Emotional support is a crucial ability for many conversation scenarios, including social interactions, mental health support, and customer service chats. |
| Approach: | They propose an Emotional Support Conversation task and an ESC Framework to train emotional support into dialog systems. |
| Outcome: | The proposed framework provides an example of an Emotional Support Conversation task and shows that it is more effective than existing models. |
Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System (2021.acl-long)
Copied to clipboard
| Challenge: | Existing slot filling models can only recognize pre-defined in-domain slot types from a limited slot set. |
| Approach: | They introduce a task, Novel Slot Detection, in the task-oriented dialogue system. |
| Outcome: | The proposed task is based on two public NSD datasets and proposes strong baselines . it aims to identify a sequence of tokens and extract semantic constituents from user queries . |
GTM: A Generative Triple-wise Model for Conversational Question Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that opendomain conversational question generation improves the quality of questions in terms of fluency, coherence and diversity over competitive baselines. |
| Approach: | They propose a triple-wise model with hierarchical variations for open-domain conversational question generation using a post-question-answer triple and one-to-many semantic mappings. |
| Outcome: | The proposed model significantly improves the quality of questions in terms of fluency, coherence and diversity over baselines. |
Diversifying Dialog Generation via Adaptive Label Smoothing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to dialog generation suffer from the over-confidence issue, which leads to poor generation diversity. |
| Approach: | They propose an Adaptive Label Smoothing approach that can adaptively estimate a target label distribution at each time step for different dialog contexts. |
| Outcome: | The proposed approach outperforms competing models on two benchmark datasets in producing diverse responses. |
Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for out-of-scope intent detection rely on strong assumptions on data distribution and confidence threshold selection. |
| Approach: | They propose a method to train an out-of-scope intent classifier in a fully end-to-end manner by simulating the test scenario in training. |
| Outcome: | The proposed method improves on four benchmark dialogue datasets and improves over state-of-the-art methods. |
Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for document-level event extraction are not effective due to two challenges . existing methods fail to extract events whose arguments spread in multiple sentences . |
| Approach: | They propose a document-level event extraction model with a tracker to capture interdependency among the extracted events. |
| Outcome: | The proposed model outperforms existing models on a large-scale dataset by 2.8 F1 . it extracts multiple correlated events and event arguments that scatter across the document . |
Nested Named Entity Recognition via Explicitly Excluding the Influence of the Best Path (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition assume entities are not nested within other entities, so-called flat NER. |
| Approach: | They propose a layered method for nested named entity recognition . they use a set of hidden states to exclude the influence of the best path . |
| Outcome: | The proposed method performs better on ACE2004, ACE2005, and GENIA datasets. |
LearnDA: Learnable Knowledge-Guided Data Augmentation for Event Causality Identification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for event causality identification (ECI) rely on annotated training data. |
| Approach: | They propose a method to augment training data for event causality identification by iteratively generating new examples and classifying event causalities in a dual learning framework. |
| Outcome: | The proposed method outperforms existing methods on EventStoryLine and Causal-TimeBank. |
Revisiting the Negative Data of Distantly Supervised Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for relation extraction with distant supervision generate plenty of training samples but noisy labels and imbalanced training data cause problems. |
| Approach: | They propose a method that automatically labels a sentence with relational triples from a knowledge base. |
| Outcome: | The proposed method outperforms existing methods even with false positive samples. |
Knowing the No-match: Entity Alignment with Dangling Cases (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to find entities that cannot find alignment across knowledge graphs (KGs) despite their importance, knowledge graph is expensive and suffers from incompleteness. |
| Approach: | They propose a framework for entity alignment and dangling entity detection that can be used to abstain from predicting alignment for detected dangle entities. |
| Outcome: | The proposed framework can abstain from predicting alignment for detected dangling entities. |
Superbizarre Is Not Superb: Derivational Morphology Improves BERT’s Interpretation of Complex Words (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) are based on fixed-size vocabularies of words and subwords that are generated by compression algorithms such as bytepair encoding. |
| Approach: | They propose to use BERT as an example PLM to study its semantic representations of English derivatives to test their hypothesis. |
| Outcome: | The proposed model outperforms BERT on a series of semantic probing tasks. |
BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies? (2021.acl-long)
Copied to clipboard
| Challenge: | Analogies play a central role in human commonsense reasoning. |
| Approach: | They analyze the capabilities of transformer-based language models on an unsupervised task . they find off-the-shelf language models can identify analogies to a certain extent . |
| Outcome: | The proposed language models outperform word embedding models on an unsupervised task . the best results were obtained with GPT-2 and RoBERTa . |
Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models that represent different senses of words in context are not accurate for polysemous words. |
| Approach: | They propose a multilingual dataset that evaluates the ability of models to accurately represent different lexical-semantic relations such as homonymy and synonymy. |
| Outcome: | The proposed models can disambiguate homonyms in context, but fail to represent words with different senses when occurring in similar sentences. |
Measuring Fine-Grained Domain Relevance of Terms: A Hierarchical Core-Fringe Approach (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to measure fine-grained domain relevance are needed for downstream tasks in natural language processing. |
| Approach: | They propose to measure fine-grained domain relevance, defined as the degree that a term is relevant to a given domain. |
| Outcome: | The proposed method outperforms baselines and surpasses professional human performance. |
HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on detecting user disengagement requires hand-labeling many dialog samples . Currently, self-reported user ratings are constrained to a static corpus . |
| Approach: | They propose an efficient annotation framework that denoises dialog samples instead of manually labeling them. |
| Outcome: | The proposed framework improves annotation efficiency significantly and detects user disengagement in two dialog corpora. |
Value-Agnostic Conversational Semantic Parsing (2021.acl-long)
Copied to clipboard
Emmanouil Antonios Platanios, Adam Pauls, Subhro Roy, Yuchen Zhang, Alexander Kyte, Alan Guo, Sam Thomson, Jayant Krishnamurthy, Jason Wolfe, Jacob Andreas, Dan Klein
| Challenge: | Existing models rely on rich representations of dialogue history that include all previously generated components of the output. |
| Approach: | They propose a model that abstracts over values to focus prediction on type- and function-level context. |
| Outcome: | The proposed model outperforms baseline models by 7.3% and 10.6% on SMCalFlow and TreeDST datasets. |
MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for multi-party conversation represent interlocutors and utterances individually . existing methods ignore complicated structure of MPC which may provide crucial interlocutor and tertiary semantics. |
| Approach: | They propose a pre-trained model for multi-party conversation that considers learning who says what to whom in a unified model with elaborated self-supervised tasks. |
| Outcome: | The proposed model outperforms existing models on three downstream tasks at two benchmarks. |
Best of Both Worlds: Making High Accuracy Non-incremental Transformer-based Disfluency Detection Incremental (2021.acl-long)
Copied to clipboard
| Challenge: | Currently, Transformer-based text classifiers are not suitable for live incremental processing, operating only on the level of complete sentence inputs. |
| Approach: | They propose to introduce a method for word-by-word left-to-right incremental processing to Transformers such as BERT, models without an intrinsic sense of linear order. |
| Outcome: | The proposed method maintains high non-incremental performance while operating strictly incrementally. |
NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation (2021.acl-long)
Copied to clipboard
| Challenge: | NeuralWOZ generates dialogues from user’s goal instructions and system’s API call results. |
| Approach: | They propose a framework that uses model-based dialogue simulation to generate dialogues from user’s goal instructions and system’s API call results. |
| Outcome: | The proposed framework achieves 4.4% point joint goal accuracy on average across domains and 5.7% point of zero-shot coverage against the MultiWOZ 2.1 dataset. |
CDRNN: Discovering Complex Dynamics in Human Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Behavioral and fMRI experiments reveal detailed and plausible estimates of human language processing dynamics . central questions in psycholinguistics concern the mental processes involved in incremental human language understanding . |
| Approach: | They propose a continuous-time deconvolutional regressive neural network that captures time-varying, non-linear, and delayed influences of predictors on the response. |
| Outcome: | The proposed neural network captures time-varying, non-linear, and delayed influences on the response . Behavioral and fMRI experiments show it generalizes better than baselines . |
Structural Guidance for Transformer Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained Transformer language models have proven remarkably successful in learning generic transferable linguistic representations without resorting to data intensive pre-training. |
| Approach: | They propose to combine a generative parsing and a structural scaffolding idea to guide the model's representation via additional structure loss that separates the incremental constituency parse. |
| Outcome: | The proposed models achieve impressive perplexity results on language modelling datasets, perform well on grammatical judgments, and provide useful linguistic representations that benefit a wide range of downstream tasks. |
Surprisal Estimators for Human Reading Times Need Character Models (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that character models can be applied to a structural parser-based processing model to calculate word generation probabilities. |
| Approach: | They propose to use a character model to calculate word generation probabilities from a structural parser-based processing model. |
| Outcome: | The proposed model performs better on self-paced reading, eye-tracking, and fMRI data than large-scale language models trained on much more data. |
CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies integrate word embeddings with cognitive features into neural models of natural language processing (NLP) but there are some issues in the use of cognitive features in NLP. |
| Approach: | They propose a cog-align approach that aligns textual and cognitive inputs to capture differences and commonalities. |
| Outcome: | The proposed model improves on three NLP tasks with multiple cognitive features over state-of-the-art models. |
Self-Attention Networks Can Process Bounded Hierarchical Languages (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models that can process formal languages with hierarchical structure are limited in their performance. |
| Approach: | They propose to use a subset of Dyck-k with depth bounded by D to train self-attention networks. |
| Outcome: | The proposed model can process Dyck-(k, D) with depth bounded by D, which better captures the hierarchical structure of natural language. |
TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text style transfer require style-labeled training data, but use only labeled data at inference time. |
| Approach: | They propose a method that uses readily-available unlabeled text to train style transfer . they use a style vector to condition a decoder to perform style transfer using unlabelled text . |
| Outcome: | The proposed method is competitive on sentiment transfer, even compared to models trained fully on labeled data. |
H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences (2021.acl-long)
Copied to clipboard
| Challenge: | Existing hierarchical methods to compute attention are superior to sub-quadratic ones . a particular type of attention, called multi-head scaled dot-product attention, is one of the main components of the Transformer architecture . |
| Approach: | They propose a hierarchical method to compute attention in the Transformer architecture . they perform extensive experiments to show that it captures hierarchic structure in sequences . |
| Outcome: | The proposed method outperforms sub-quadratic models on the Long Range Arena benchmark by over +6 points on average. |
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |
A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger’s Adversarial Attacks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing adversarial examples can fool ML models by generating a fixed phrase that can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class. |
| Approach: | They propose a honeypot-based defense framework that greedily searches and injects multiple trapdoors into an NN model to “bait and catch” potential attacks. |
| Outcome: | The proposed algorithm detects attacks with 99% TPR and less than 2% FPR while maintaining prediction accuracy within 1% margin. |
Towards Propagation Uncertainty: Edge-enhanced Bayesian Graph Convolutional Networks for Rumor Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on rumor detection focus on text content and propagation structure . however, the uncertainty caused by unreliable relations in propagation structures is common . |
| Approach: | They propose a Bayesian-based model that captures propagation uncertainty for rumor detection. |
| Outcome: | The proposed model achieves better performance than baseline methods on rumor detection and early rumour detection tasks. |
Label-Specific Dual Graph Neural Network for Multi-Label Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies for multi-label text classification do not explore label-specific semantic components from documents. |
| Approach: | They propose a label-specific dual graph neural network that incorporates category information to learn label-related components from documents. |
| Outcome: | The proposed model outperforms state-of-the-art models on three benchmark datasets and achieves better performance with respect to tail labels. |
TAN-NTM: Topic Attention Networks for Neural Topic Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Topic models have been widely used to learn text representations and gain insight into document corpora. |
| Approach: | They propose a framework which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. |
| Outcome: | The proposed model improves on two downstream tasks: document classification and topic guided keyphrase generation. |
Cross-language Sentence Selection via Data Augmentation and Rationale Training (2021.acl-long)
Copied to clipboard
| Challenge: | a new approach to cross-language sentence selection is proposed for low-resource contexts . a cross-lingual embedding-based model is proposed that avoids translation entirely . |
| Approach: | They propose a cross-lingual embedding-based query relevance model that uses data augmentation and negative sampling techniques to directly learn a query-sentence pair. |
| Outcome: | The proposed approach performs better than state-of-the-art models on noisy parallel data . consistent improvements are seen across three language pairs over state- of-the art models . |
A Neural Model for Joint Document and Snippet Ranking in Question Answering for Large Document Collections (2021.acl-long)
Copied to clipboard
| Challenge: | Question answering systems typically use pipelines that retrieve documents at finer text granularities. |
| Approach: | They propose an architecture for document and snippet ranking that leverages intuition . they modified a natural questions dataset to test their model . |
| Outcome: | The proposed model outperforms pipelines in document retrieval on biomedical data . the proposed model is competitive with the existing model, despite fewer parameters . |
W-RST: Towards a Weighted RST-style Discourse Framework (2021.acl-long)
Copied to clipboard
| Challenge: | We show that weighted discourse trees from auxiliary tasks can benefit downstream applications . linguistic theories play a less and less critical role in the field of discourse . |
| Approach: | They propose a weighted-RST framework that assigns a binary assessment of importance between text segments by a relation attribute. |
| Outcome: | The proposed framework can be replaced by real-valued scores, the authors show . they show that weighted discourse trees can benefit key NLP downstream applications . |
ABCD: A Graph Framework to Convert Complex Sentences to a Covering Set of Simple Sentences (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work relies on rule-based methods dependent on parsing to identify atomic sentences. |
| Approach: | They propose a task to decompose complex sentences into simple ones . they propose atomic clauses as atomic sentences, and a graph edit task to predict edits . |
| Outcome: | The proposed model performs better than baselines on MinWiki and DeSSE. |
Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing Question-Answering (QA) datasets contain unanswerable questions . however, their treatment in QA systems remains primitive . |
| Approach: | They propose a framework that provides answers based on presupposition failure over oracle behavior of existing QA systems. |
| Outcome: | The proposed system provides responses based on presupposition failure over oracle behavior of existing QA systems. |
Adversarial Learning for Discourse Rhetorical Structure Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing top-down discourse rhetorical structure parsers make local decisions and ignore global parsing. |
| Approach: | They propose a method to transform gold standard and predicted constituency trees into tree diagrams with two color channels. |
| Outcome: | The proposed method improves performance on RST-DT and CDTB corpora and can leverage global context. |
Exploring Discourse Structures for Argument Impact Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that discourse structures influence the persuasiveness of arguments. |
| Approach: | They propose to fuse sentence-level structural discourse information with contextualized features derived from large-scale language models to investigate how discourse relations influence argument impact. |
| Outcome: | The proposed model improves its backbone RoBERTa around 1.67%, compared with other models, but side effects are brought by other models. |
Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models have been criticized for lack of integration of bilingual lexical information into the neural architecture. |
| Approach: | They propose a neural architecture to incorporate bilingual dictionaries into Neural Machine Translation models by introducing three new components: Pointer, Disambiguator, and Copier. |
| Outcome: | The proposed method achieves the following merits inherently compared with previous efforts: (1) Pointer leverages the semantic information from bilingual dictionaries, for the first time, to better locate source words whose translation in dictionary can potentially be used; (2) Disambiguator synthesizes contextual information from source view and target view, both of which contribute to distinguishing translation of a specific source word from multiple candidates in dicaries; (3) Copier systematically connects Pointer and Disambiguators based on a hierarchical |
VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work in multilingual pretraining relies on the shared vocabulary and bilingual contexts to encourage the correlation across languages. |
| Approach: | They propose to plug a cross-attention module into a Transformer encoder to explicitly build the interdependence between languages. |
| Outcome: | The proposed model outperforms existing models on XTREME and English-to-French translation datasets. |
A unified approach to sentence segmentation of punctuated text in many languages (2021.acl-long)
Copied to clipboard
| Challenge: | Existing tools for segmenting punctuated text in many languages are limited in their language coverage and evaluation is ad hoc. |
| Approach: | They propose a new context-based modeling approach that can be trained on noisily-annotated data. |
| Outcome: | The proposed model exceeds baselines set by existing methods on English corpora and performs well on average on new multilingual evaluation set. |
Towards User-Driven Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | a good translation should implicitly mirror user traits rather than translate the original content semantically. |
| Approach: | They propose a framework that captures user traits from historical inputs . they propose 'user-driven' NMT to model user behavior under a zero-shot learning fashion . |
| Outcome: | The proposed framework can capture user traits from historical inputs under zero-shot learning fashion. |
End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to enforce word forms in translations struggle to make them agree with the rest of the output. |
| Approach: | They propose to train neural machine translation models with lemmatized constraints to infer correct word inflection. |
| Outcome: | The proposed model reduces errors in translation of constrained terms in automatic and manual evaluations on English-Czech language pairs. |
Handling Extreme Class Imbalance in Technical Logbook Datasets (2021.acl-long)
Copied to clipboard
| Challenge: | Technical logbooks are a challenging and under-explored text type in automated event identification. |
| Approach: | They propose a feedback strategy that resamples the training data based on its error in the prediction process. |
| Outcome: | The proposed approach provides the best results for four different neural network models trained across a suite of technical logbook datasets from distinct technical domains. |
ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation (2021.acl-long)
Copied to clipboard
Vijit Malik, Rishabh Sanjay, Shubham Kumar Nigam, Kripabandhu Ghosh, Shouvik Kumar Guha, Arnab Bhattacharya, Ashutosh Modi
| Challenge: | a system that could assist a judge in predicting the outcome of a case should be explainable. |
| Approach: | They propose to use a corpus of 35k Indian Supreme Court cases annotated with original court decisions to promote research in this area. |
| Outcome: | The proposed system has an accuracy of 78% versus 94% for human legal experts. |
Supporting Cognitive and Emotional Empathic Writing of Students (2021.acl-long)
Copied to clipboard
| Challenge: | Empathy skills are an elementary skill in society for daily interaction and professional communication and are therefore elementary for educational curricula. |
| Approach: | They propose an annotation approach to capture emotional and cognitive empathy in student-written peer reviews on business models in germany. |
| Outcome: | The proposed annotation scheme guides annotators to a substantial to moderate agreement with the model and shows that it is effective. |
Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing generative models for open-domain question answering focus on generating direct answers from unstructured textual information, but a large amount of knowledge is stored in structured databases, and need to be accessed using query languages such as SQL. |
| Approach: | They propose a hybrid framework that takes both textual and tabular evidences as input and generates either direct answers or SQL queries depending on which form could better answer the question. |
| Outcome: | The proposed framework outperforms baseline models on OpenSQuAD datasets and can generate SQL queries on the associated databases to obtain the final answers. |
Generation-Augmented Retrieval for Open-Domain Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to answer open-domain questions use sparse representations and sparsity. |
| Approach: | They propose a method which augments a query by generating relevant contexts from heuristically discovered contexts without external supervision. |
| Outcome: | The proposed approach outperforms state-of-the-art dense retrieval methods on natural questions and triviaQA datasets. |
Check It Again:Progressive Visual Question Answering via Visual Entailment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to Visual Question Answering (VQA) only address superficial correlations between image and answer. |
| Approach: | They propose a select-and-rerank progressive framework based on Visual Entailment to address this problem. |
| Outcome: | The proposed framework improves on the Visual Question Answering (VQA) task with 7.55% accuracy. |
A Mutual Information Maximization Approach for the Spurious Solution Problem in Weakly Supervised Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Weakly supervised question answering usually has only final answers as supervision signals while correct solutions are not provided. |
| Approach: | They propose to explicitly exploit the semantic correlations between question-answer pairs and predicted answers by maximizing mutual information between question and answer pairs. |
| Outcome: | The proposed method significantly outperforms previous learning methods in terms of task performance and is more effective in training models to produce correct solutions. |
Breaking Down Walls of Text: How Can NLP Benefit Consumer Privacy? (2021.acl-long)
Copied to clipboard
| Challenge: | Privacy policies are long and complex documents that are difficult for users to read and comprehend. |
| Approach: | They propose language technologies to help users reclaim control over their privacy . they highlight many remaining opportunities to develop more precise or nuanced language technologies . |
| Outcome: | The proposed language technologies can address the privacy information gap . they can be more precise or nuanced in the way they use the text of privacy policies. |
Supporting Land Reuse of Former Open Pit Mining Sites using Text Classification and Active Learning (2021.acl-long)
Copied to clipboard
Christopher Schröder, Kim Bürgl, Yves Annanias, Andreas Niekler, Lydia Müller, Daniel Wiegreffe, Christian Bender, Christoph Mengs, Gerik Scheuermann, Gerhard Heyer
| Challenge: | open pit mines left many regions worldwide inhospitable or uninhabitable . aforementioned information has to be acquired to ensure safety and validity of land reuse . |
| Approach: | They propose a workflow for supporting the post-mining management of former open pit mines in the eastern part of Germany . they use active learning to perform multi-label sentence classification for two categories of restrictions and seven categories of topics . |
| Outcome: | The proposed system supports the post-mining management of former lignite open pit mines in the eastern part of Germany. |
Reliability Testing for Natural Language Processing Systems (2021.acl-long)
Copied to clipboard
| Challenge: | a lack of rigorous testing and ML implicit assumption of identical training and testing distributions may result in systems that discriminate against minorities. |
| Approach: | They argue that reliability testing is needed to address the issue of demographics . they argue that adversarial attacks can be reframed for this goal . |
| Outcome: | The proposed framework will enable rigorous and targeted testing and aid in the enactment and enforcement of industry standards. |
Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data (2021.acl-long)
Copied to clipboard
Paul Pu Liang, Terrance Liu, Anna Cai, Michal Muszynski, Ryo Ishii, Nick Allen, Randy Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency
| Challenge: | Mental health conditions remain underdiagnosed in many countries despite access to advanced medical care . a new approach to learn mood markers from mobile data is needed to improve accuracy and improve learning from typed text. |
| Approach: | They propose to use mobile data to learn mood markers without identifying users through personal or protected attributes. |
| Outcome: | The proposed model obfuscates user identities while remaining predictive . future directions include better models and pre-learning from typed text . |
Anonymisation Models for Text Data: State of the art, Challenges and Future Directions (2021.acl-long)
Copied to clipboard
| Challenge: | a paper examines the problem of automated text anonymisation . text anonymization is a prerequisite for secure sharing of documents containing sensitive information about individuals. |
| Approach: | They propose to incorporate explicit measures of disclosure risk into the text anonymisation process to reduce the risk of errors. |
| Outcome: | The proposed approach is based on a case study in which the authors outline the benefits and limitations of the proposed methods. |
End-to-End AMR Coreference Resolution (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on AMR focuses on individual sentences, but there is a need for multi-sentence AMRs. |
| Approach: | They propose to use an end-to-end AMR coreference resolution model to generate multi-sentence AMRs. |
| Outcome: | The proposed model reduces error propagation and is more robust for both in- and out-domain situations. |
How is BERT surprised? Layerwise detection of linguistic anomalies (2021.acl-long)
Copied to clipboard
| Challenge: | a number of studies have shown that transformer-based language models detect when a word is anomalous in context, but likelihood scores do not tell the cause of the anomaly. |
| Approach: | They propose to use Gaussian models for density estimation at intermediate layers of three language models to evaluate grammaticality. |
| Outcome: | The proposed method on BLiMP shows that language models employ different mechanisms to detect different types of linguistic anomalies. |
Psycholinguistic Tripartite Graph Network for Personality Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on personality detection from online posts adopts multifarious deep neural networks to represent the posts and builds predictive models in a data-driven manner without the exploitation of psycholinguistic knowledge. |
| Approach: | They propose a psycholinguistic knowledge-based tripartite graph network, TrigNet, which consists of a tripartitic graph network and a BERT-based graph initializer. |
| Outcome: | The proposed graph network outperforms the existing state-of-the-art model by 3.47 and 2.10 points in average F1 on two datasets. |
Verb Metaphor Detection via Contextual Relation Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work on verb metaphor detection focuses on analyzing restricted forms of linguistic context. |
| Approach: | They propose a model which explicitly models the relation between a verb and its various contexts. |
| Outcome: | The proposed model gets competitive results compared with state-of-the-art approaches on the VUA, MOH-X and TroFi datasets. |
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task (2021.acl-long)
Copied to clipboard
| Challenge: | Pretraining and multitask learning are widely used to improve the speech translation performance. |
| Approach: | They propose to train a speech translation model along with an auxiliary text translation task. |
| Outcome: | The proposed method improves translation quality by more than 2 BLEU over a strong baseline and achieves state-of-the-art results on the MuST-C English-German, English-French and English-Spanish language pairs. |
Probing Toxic Content in Large Pre-Trained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on pre-trained language models have shown that they carry harmful biases towards different social groups. |
| Approach: | They propose a method to probe English, French, and Arabic PTLMs and quantify the potentially harmful content they convey with respect to a set of templates. |
| Outcome: | The proposed method analyzes PTLMs to predict masked tokens at the end of sentences to assess their toxicity. |
Societal Biases in Language Generation: Progress and Challenges (2021.acl-long)
Copied to clipboard
| Challenge: | Language generation techniques can produce undesirable societal biases that can negatively impact marginalized populations. |
| Approach: | They propose to examine how decoding techniques contribute to biases in language generation . they also conduct experiments to quantify the effects of these techniques . |
| Outcome: | The proposed methods can reduce biases and improve user experience, the authors argue . they also show that the proposed techniques can reduce societal biase . |
Reservoir Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | Using random initialization, we show that some transformers obtain impressive performance even when some of the layers are frozen. |
| Approach: | They propose to freeze transformer layers and use them to improve performance . they find that the transformers obtain impressive performance even when some of the layers are randomly initialized and never updated. |
| Outcome: | The proposed model improves on translation and language modelling tasks even when some layers are frozen. |
Subsequence Based Deep Active Learning for Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Active Learning (AL) has been successfully applied to Deep Learning to drastically reduce the amount of data required to achieve high performance. |
| Approach: | They propose to query subsequences within sentences and propagate their labels to other sentences. |
| Outcome: | The proposed approach achieves high performance on OntoNotes 5.0 and CoNLL 2003 with only 13% of training data and 27% of the training data. |
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that convolutions have been successful in natural language learning. |
| Approach: | They propose a convolutional approach to construct relative position embeddings in self-attention layers and propose 'compact attention' they propose multiple ways to integrate convolutions into Transformer self- attention. |
| Outcome: | The proposed composite attention improves performance on multiple downstream tasks, replacing absolute position embeddings, and is more expressive than convolutions in NLP. |
BinaryBERT: Pushing the Limit of BERT Quantization (2021.acl-long)
Copied to clipboard
| Challenge: | Recent pre-trained language models have achieved remarkable performance improvement in various tasks, but the improvement generally comes at the cost of increasing model size and computation. |
| Approach: | They propose a binary quantization technique which initializes binaryBERT by splitting from a ternary network. |
| Outcome: | The proposed model achieves state-of-the-art performance on the GLUE and SQUAD benchmarks while being 24x smaller. |
Are Pretrained Convolutions Better than Pretrained Transformers? (2021.acl-long)
Copied to clipboard
| Challenge: | Recent research has shown promise in entirely convolutional, or CNN, architectures, but they have not been explored using the pre-train-fine-tune paradigm. |
| Approach: | They propose to use the pre-train-fine-tune paradigm to study convolutional models. |
| Outcome: | The proposed architectures outperform Transformers in certain scenarios, but with caveats. |
PairRE: Knowledge Graph Embeddings via Paired Relation Vectors (2021.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods fail to solve two major problems at the same time, leading to unsatisfactory results. |
| Approach: | They propose a model with paired vectors for each relation representation that can be adaptively adjusted to fit for different complex relations. |
| Outcome: | Experiments on two knowledge graph datasets show the proposed model can handle complex relations and encode relation patterns. |
Hierarchy-aware Label Semantics Matching Network for Hierarchical Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods ignore the semantic relationship between text and labels, so they cannot make full use of hierarchical information. |
| Approach: | They propose a hierarchy-aware label semantics matching network to model the semantic relationship between text and labels in a semantic matching problem. |
| Outcome: | The proposed model captures the text-label semantics matching relationship among coarse-grained labels and fine-grain labels in a hierarchy-aware manner. |
HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizability (2021.acl-long)
Copied to clipboard
| Challenge: | Using data augmentation to fine-tune pre-trained models with task-specific data has been shown to be ineffective and redundant during fine-timing. |
| Approach: | They propose a data augmentation technique to regularize pre-trained models and encourage them to learn more generalizable features by dropping contiguous spans during training. |
| Outcome: | The proposed method outperforms state-of-the-art methods on the GLUE benchmark and consistently exhibits superior generalization performances on out-of distribution and challenging counterexamples. |
Neural Stylistic Response Generation with Disentangled Latent Variables (2021.acl-long)
Copied to clipboard
| Challenge: | Existing parallel datasets for creating stylistic responses are not stylistically consistent. |
| Approach: | They propose to disentangle the content and style in latent space by diluting sentence-level information in style representations. |
| Outcome: | The proposed approach achieves a higher BERT-based style intensity score and comparable BLEU scores, compared with baselines. |
Intent Classification and Slot Filling for Privacy Policies (2021.acl-long)
Copied to clipboard
| Challenge: | Sentences written in privacy policies explain privacy practices and the constituent text spans convey further specific information. |
| Approach: | They propose an English corpus of 5,250 intent and 11,788 slot annotations . they propose two alternative neural approaches to model the corpus as a sequence-to-sequence learning task. |
| Outcome: | The proposed corpus predicts intent classification and slot filling, while the sequence tagging method outperforms slot filler by a large margin. |
RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems (2021.acl-long)
Copied to clipboard
| Challenge: | Existing task-oriented dialog systems are less than satisfactory in robustness evaluation . existing systems are weak in robustity evaluation based on pre-training and fine-tuning . |
| Approach: | They propose to use a set of training examples to evaluate model generalization ability . they propose to include tasks with limited training data to favor models with strong generalization abilities . |
| Outcome: | The proposed model generalizes well with limited training data and is robust to user input across domains. |
Semantic Representation for Dialogue Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for dialogue modeling lack ability to represent core semantics, such as ignoring important entities. |
| Approach: | They develop an algorithm to construct dialogue-level AMR graphs from sentence-level data and explore two ways to incorporate AMRs into dialogue modeling. |
| Outcome: | The proposed model is superior to existing models on dialogue understanding and response generation tasks. |
A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to train retrieval-based dialogue systems rely on crowd-sourced data . however, it is difficult to collect large-scale dialogues that are grounded on background knowledge . |
| Approach: | They propose to decompose training of knowledge-grounded response selection into three tasks . they propose to combine query-passage matching task with query-dialogue history matching task . |
| Outcome: | Experimental results show that the proposed model can perform comparable to existing methods . the retrieval-based system can leverage background knowledge when conversing with humans . |
Dependency-driven Relation Extraction with Attentive Graph Convolutional Networks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies suffer from noise in dependency trees, which can cause confusions in relation extraction. |
| Approach: | They propose a dependency-driven approach for relation extraction with attentive graph convolutional networks . they apply an attention mechanism upon graph convolutional networks to different word dependencies . |
| Outcome: | The proposed approach outperforms previous studies on two English datasets and achieves state-of-the-art performance. |
Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that retrievers underperform on rarer entities that share a name . open-domain tasks require a knowledge source to perform reasoning and produce an answer . |
| Approach: | They propose an evaluation benchmark for retrieving entities that share a name . they define Ambiguous Entity Retrieval sets as a collection of entities that have a common name - and query about those entities. |
| Outcome: | The proposed sets underperform on rarer entities that share a name . the retrievers exhibit popularity bias, and are twice as likely to retrieve erroneous documents . |
Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards? (2021.acl-long)
Copied to clipboard
| Challenge: | Rather than replacing leaderboards, we advocate a re-imagining of the model to highlight if and where progress is made. |
| Approach: | They propose a Bayesian leaderboard model where latent subject skill and latent item difficulty predict correct responses. |
| Outcome: | The proposed model can guide what to annotate, identify annotation errors, detect overfitting, and identify informative examples. |
Claim Matching Beyond English to Scale Global Fact-Checking (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to fact-check content are not scaled well in non-English contexts. |
| Approach: | They propose to use a WhatsApp tipline and public group message dataset to find pairs of textual messages containing claims that can be served with one fact-check. |
| Outcome: | The proposed model outperforms existing models in English, Hindi, and Tamil in all settings. |
SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Using pre-training methods for NMT models is difficult because of the cross-attention module . cross-linguistic embeddings are not used to pretrain a decoder . |
| Approach: | They propose a semantic interface between pre-trained encoder and pre-train decoder to improve NMT performance. |
| Outcome: | The proposed method improves on translation and unsupervised translation tasks. |
Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models (2021.acl-long)
Copied to clipboard
Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit Iyyer, Andrew McCallum
| Challenge: | Autoregressive neural machine translation (NMT) uses a tractable likelihood computation and efficient sampling. |
| Approach: | They propose to use an energy-based model to mimic the behavior of the task measure and use it to train an energy based re-ranking algorithm. |
| Outcome: | The proposed model improves on the samples drawn from the NMT with a higher BLEU score than the experimental model and the energy-based re-ranking algorithm. |
Syntax-augmented Multilingual BERT for Cross-lingual Transfer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained multilingual text encoders capture language syntax, helping cross-lingual transfer. |
| Approach: | They provide language syntax and train mBERT to encode universal dependency tree structure. |
| Outcome: | The proposed model improves cross-lingual transfer on PAWS-X and MLQA benchmarks by 1.4 and 1.6 points on average across all languages. |
How to Adapt Your Pretrained Multilingual Model to 1600 Languages (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained multilingual models perform best for languages seen during pretraining . methods exist to improve performance for unseen languages, but have been evaluated using amounts of raw text only available for a small fraction of the world’s languages. |
| Approach: | They evaluate the performance of existing methods to adapt pretrained multilingual models to new languages using a resource available for close to 1600 languages: the New Testament. |
| Outcome: | The proposed models perform best for languages seen during pretraining . the results show that the most efficient approach is simplest and the most accurate . |
Weakly Supervised Named Entity Tagging with Learnable Logical Rules (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for building entity tagging systems use weak supervision . previous methods focus on disambiguating entity types based on contexts and expert-provided rules . |
| Approach: | They propose a method that bootstraps high-quality logical rules to train a neural tagger in a fully automated manner. |
| Outcome: | The proposed method outperforms weakly supervised methods on three datasets . it rivals state-of-the-art supervised method with lexicon of over 2,000 terms . |
Prefix-Tuning: Optimizing Continuous Prompts for Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Fine-tuning is the prevalent paradigm for using large pretrained language models for downstream tasks, but it requires updating and storing all the parameters of the LM. |
| Approach: | They propose a lightweight alternative to fine-tuning for natural language generation tasks that optimizes a sequence of continuous vectors, which they call the prefix. |
| Outcome: | The proposed approach outperforms fine-tuning in the full data setting and extrapolates better to examples with topics that are unseen during training. |
One2Set: Generating Diverse Keyphrases as a Set (2021.acl-long)
Copied to clipboard
| Challenge: | Recent keyphrase generation models are wrongly imposing a predefined order on keyphrases . a new training paradigm is proposed to concatenate keyphrase sequences in parallel . |
| Approach: | They propose a training paradigm that concatenates keyphrases in a predefined order . they propose combining a fixed set of learned control codes with a bipartite matching mechanism . |
| Outcome: | The proposed model outperforms the state-of-the-art methods on multiple benchmarks. |
Continuous Language Generative Flow (2021.acl-long)
Copied to clipboard
| Challenge: | Recent years have witnessed various types of generative models for natural language generation (NLG), especially RNNs or transformers. |
| Approach: | They propose a flow-based language generation model that adapts flow-derived generative models to language generation via continuous input embeddings, adapted affine coupling structures, and a novel architecture for autoregressive text generation. |
| Outcome: | The proposed model improves on QG and NMT and improves performance over baselines on SQuAD and TVQA and NML16. |
TWAG: A Topic-Guided Wikipedia Abstract Generator (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models view Wikipedia abstract as plain text, ignoring that it is a description of a certain entity and can be decomposed into different topics. |
| Approach: | They propose a model that guides Wikipedia abstract generation with topical information. |
| Outcome: | The proposed model outperforms baselines and is capable of generating comprehensive abstracts. |
ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing automated forecasting studies rely on structured data to predict future events. |
| Approach: | They propose a question-answering task that limits access to unstructured text data . they use a crowdsourced dataset to form a restricted-domain, multiple-choice, question-announcement task . |
| Outcome: | The proposed model achieves 61.0% accuracy on the dataset, which still lags behind human performance by about 19%. |
Recursive Tree-Structured Self-Attention for Answer Sentence Selection (2021.acl-long)
Copied to clipboard
| Challenge: | Recent top-performing models in Answer Sentence Selection use self-attention and transfer learning, but not syntactic structure. |
| Approach: | They propose a recursive, tree-structured self-attention model that can represent all levels of syntactic parse trees with only one additional layer. |
| Outcome: | The proposed model can represent all levels of syntactic parse trees with only one additional layer without transfer learning. |
How Knowledge Graph and Attention Help? A Qualitative Analysis into Bag-level Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Knowledge Graph (KG) and attention mechanism have been demonstrated effective in introducing and selecting useful information for weakly supervised methods. |
| Approach: | They propose a paradigm to quantitatively evaluate the effect of attention and KG on bag-level relation extraction (RE) they propose to incorporate entity prior to KG-enhanced attention to improve RE performance . |
| Outcome: | The proposed model achieves significant improvements on two real-world datasets compared with three state-of-the-art baselines. |
Trigger is Not Sufficient: Exploiting Frame-aware Knowledge for Implicit Event Argument Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to extract event arguments focus on learning pair-wise information between arguments and the given trigger. |
| Approach: | They propose a framework to extract event-related arguments from a given event frame-level scope. |
| Outcome: | The proposed method achieves state-of-the-art on the RAMS dataset. |
Element Intervention for Open Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Current OpenRE models are often trained on the datasets generated from distant supervision, which often results in instability and makes the model easily collapsed. |
| Approach: | They propose to use a causal model to identify relation instances referring to the same relation . they propose to perform Element Interventions on context and entities respectively . |
| Outcome: | The proposed method outperforms existing methods and is robust across datasets. |
AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to extract product attribute values are limited by knowledge sharing across different attributes. |
| Approach: | They propose to use adaptive decoding to handle extraction of product attribute values by parameterizing the decoder with pretrained attribute embeddings, through a hypernetwork and a Mixture-of-Experts module. |
| Outcome: | The proposed model is able to handle multiple attributes without sharing the entire network parameters across all attributes. |
CoRI: Collective Relation Integration with Data Augmentation for Open Information Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to integrate extracted knowledge from the Web to knowledge graphs (KGs) however, the predictions are made independently, which can be mutually inconsistent. |
| Approach: | They propose a relation integration model that aligns free-text relations to relations in a target KG . they propose combining two stages to make independent predictions and a collective model that accesses all candidate predictions. |
| Outcome: | The proposed model outperforms baseline models on two datasets and improves AUC from .677 to .748 and from 1.716 to 1.780. |
Benchmarking Scalable Methods for Streaming Cross Document Entity Coreference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to disambiguate mentions of named entities are limited . existing approaches omit details needed to ensure fair comparisons . |
| Approach: | They propose to use streaming CDC to disambiguate mentions of named entities . they compare a set of existing and new datasets to evaluate their models . |
| Outcome: | The proposed system is well-suited for processing streams of data where new entities are frequently introduced. |
Search from History and Reason for Future: Two-stage Reasoning on Temporal Knowledge Graphs (2021.acl-long)
Copied to clipboard
| Challenge: | Temporal Knowledge Graphs (TKGs) are used in many different areas of research. |
| Approach: | They propose to use a beam search policy to induce multiple clues from historical facts . they propose to adopt a graph convolution network based sequence method to deduce answers from clues . |
| Outcome: | The proposed model can predict future facts in two stages, Clue Searching and Temporal Reasoning. |
Employing Argumentation Knowledge Graphs for Neural Argument Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating arguments use end-to-end knowledge graphs or are controlled with respect to the argument's topic, aspects, or stance. |
| Approach: | They construct and populate three knowledge graphs and encode them into debate portals and relevant paragraphs from Wikipedia. |
| Outcome: | The proposed model produces arguments with superior quality than those generated without knowledge. |
Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Recent models perform the triplet extraction in an end-to-end manner but heavily rely on the interactions between each word and opinion word. |
| Approach: | They propose a span-level approach which explicitly considers the interaction between whole spans of targets and opinions when predicting their sentiment relation. |
| Outcome: | The proposed approach improves on triplets with multi-word targets and opinions . it explicitly considers the interaction between whole spans of targets and opinion words . |
On Compositional Generalization of Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Modern neural machine translation models have shown competitive performance in benchmarks such as WMT, but there are significant issues such as robustness, domain generalization, etc. |
| Approach: | They propose a benchmark dataset for NMT models from the perspective of compositional generalization and quantitatively analyze the results. |
| Outcome: | The proposed model performs well under traditional metrics, but is low in out-of-domain and low-resource conditions. |
Mask-Align: Self-Supervised Neural Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Word alignment is an important task in many natural language processing tasks. |
| Approach: | They propose a self-supervised word alignment model that takes advantage of the full context on the target side. |
| Outcome: | The proposed model outperforms previous unsupervised models and obtains state-of-the-art results on four language pairs. |
GWLAN: General Word-Level AutocompletioN for Computer-Aided Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Computer-aided translation (CAT) is a form of software that assists a human translator in the translation process. |
| Approach: | They propose to use computer-aided translation (CAT) to assist a human translator in the translation process. |
| Outcome: | The proposed method can give significantly more accurate predictions than baseline methods on CAT datasets. |
De-biasing Distantly Supervised Named Entity Recognition via Causal Intervention (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Named entity recognition (NER) rely on labeled data, which is labor-intensive. |
| Approach: | They propose a method to de-biase DS-NER models by a structural Causal Model . they propose to use a causal invariance regularizer to make them more robust . |
| Outcome: | The proposed method significantly improves DS-NER models on four datasets and three DS NER models. |
A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for named entity recognition (NER) focus on overlapped or discontinuous entities. |
| Approach: | They propose a span-based named entity recognition model that can recognize both overlapped and discontinuous entities jointly. |
| Outcome: | The proposed model can recognize overlapped and discontinuous entities jointly. |
MLBiNet: A Cross-Sentence Collective Event Detection Network (2021.acl-long)
Copied to clipboard
| Challenge: | Detecting multiple events from natural language text is a challenge because of the following problems: a) Sentence-level contextual representation and document-level information aggregation are not enough to detect event triggers. |
| Approach: | They propose a multi-layer bidirectional network to capture document-level association of events and semantic information simultaneously. |
| Outcome: | The proposed approach improves performance over the current state-of-the-art approach. |
Exploiting Document Structures and Cluster Consistencies for Event Coreference Resolution (2021.acl-long)
Copied to clipboard
| Challenge: | Existing deep learning models for event coreference resolution are limited in that they cannot exploit important interactions between relevant objects for ECR. |
| Approach: | They propose a deep learning model that groups coreferent event mentions into the same clusters . they use document structures to capture relevant objects for ECR . |
| Outcome: | The proposed model achieves state-of-the-art on two benchmark datasets. |
StereoRel: Relational Triple Extraction from a Stereoscopic Perspective (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for relational triple extraction still face challenges, including information loss and error propagation. |
| Approach: | They propose a model which maps relational triples to a three-dimensional space and leverages three decoders to extract them. |
| Outcome: | The proposed model outperforms the baselines on five public datasets. |
Knowledge-Enriched Event Causality Identification via Latent Structure Induction Networks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for identifying causal relations of events are limited . Existing approaches cannot handle well the problem, especially in the condition of lacking training data. |
| Approach: | They propose a Latent Structure Induction Network to integrate external structural knowledge into a causality reasoning task. |
| Outcome: | The proposed approach outperforms existing state-of-the-art methods on two widely used datasets. |
Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that neural natural language processing models are vulnerable to backdoor attacks. |
| Approach: | They propose to inject neural models with backdoors activated by word substitution . their results raise a serious alarm to the security of NLP models, they argue . |
| Outcome: | The proposed backdoors are activated by a learnable combination of word substitution and exhibit higher invisibility than previous methods. |
Parameter-Efficient Transfer Learning with Diff Pruning (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained networks are difficult to deploy for multiple tasks in storage-constrained settings. |
| Approach: | Diff pruning enables parameter-efficient transfer learning that scales well with new tasks. |
| Outcome: | Diff pruning can match the performance of finetuned baselines on the GLUE benchmark while only modifying 0.5% of the pretrained model’s parameters per task. |
R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models with stacked layers do not explicitly model hierarchical structure of language understanding. |
| Approach: | They propose a recursive Transformer model based on differentiable CKY style binary trees to emulate hierarchical composition process. |
| Outcome: | The proposed model can predict words given their left and right abstraction nodes. |
Risk Minimization for Zero-shot Sequence Labeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot sequence labeling are expensive and hard to obtain for lowresource languages/domains. |
| Approach: | They propose a framework for zero-shot sequence labeling with minimum risk training and a decomposable risk function that models the relations between predicted labels from the source models and the true labels. |
| Outcome: | The proposed framework outperforms state-of-the-art systems on 21 datasets. |
WARP: Word-level Adversarial ReProgramming (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to transfer learning from pretrained language models are frozen and a task-specific head is trained on top of them. |
| Approach: | They propose an alternative approach that trains one or more task-specific layers on top of the language model. |
| Outcome: | The proposed approach outperforms existing methods on the GLUE leaderboard with just 32 training samples. |
Lexicon Learning for Few Shot Sequence Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Past work has shown that many failures of systematic generalization arise from neural models’ inability to disentangle lexical phenomena from syntactic ones. |
| Approach: | They propose a lexical translation mechanism that generalizes existing copy mechanisms to incorporate learned, decontextualized, token-level translation rules. |
| Outcome: | The proposed model improves generalization on a diverse set of sequence modeling tasks drawn from cognitive science, formal semantics, and machine translation. |
Personalized Transformer for Explainable Recommendation (2021.acl-long)
Copied to clipboard
| Challenge: | Recent years have witnessed the successful application of natural language generation. |
| Approach: | They propose a model that uses user and item IDs to predict the words in the target explanation to make personalized Transformer. |
| Outcome: | The proposed model outperforms BERT on the explainable recommendation task in terms of effectiveness and efficiency. |
Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques (2021.acl-long)
Copied to clipboard
| Challenge: | Creating digital SOAP notes is burdensome and contributes to physician burnout . authors propose a pipeline to generate these notes based on transcripts of clinical conversations . |
| Approach: | They propose a pipeline to leverage deep summarization models based on conversations between physicians and patients . they propose an algorithm that extracts important utterances relevant to each section and generates one summary sentence per cluster . |
| Outcome: | The proposed algorithm outperforms its abstract counterpart by 8 ROUGE-1 points and produces more factual sentences as assessed by human evaluators. |
Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results demonstrate the effectiveness of Tail-to-Tail (TtT) non-autoregressive sequence prediction for Chinese Grammatical Error Correction (CGEC) |
| Approach: | They propose a framework for Chinese Grammatical Error Correction (CGEC) that uses a BERT-initialized Transformer Encoder to model the error positions. |
| Outcome: | The proposed framework solves the problem of Chinese Grammatical Error Correction (CGEC) by modeling the token dependencies. |
Early Detection of Sexual Predators in Chats (2021.acl-long)
Copied to clipboard
| Challenge: | Prior work has attempted to identify grooming chats but only after an incidence has already happened in the context of legal prosecution. |
| Approach: | They propose to analyze a running chat and predict grooming attempts as early as possible . they propose to use a new dataset to evaluate the problem from the point of view of prevention . |
| Outcome: | The proposed model is based on existing datasets and their limitations . it can be used to predict grooming attempts as early as possible . |
Writing by Memorizing: Hierarchical Retrieval-based Medical Report Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for medical image analysis use predefined template databases or ignore hierarchical nature of medical report generation. |
| Approach: | They propose a hierarchical retrieval mechanism to extract both report and sentence-level templates for clinically accurate report generation. |
| Outcome: | The proposed model extracts both report and sentence-level templates for clinically accurate report generation. |
Concept-Based Label Embedding via Dynamic Routing for Hierarchical Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for hierarchical text classification focus on modeling the text, but the concept of sharing among classes has been ignored in previous work. |
| Approach: | They propose a concept-based method that explicitly represents the concept and model the sharing mechanism among classes for the hierarchical text classification. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two widely used datasets. |
VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words (2021.acl-long)
Copied to clipboard
| Challenge: | Existing text-to-image retrieval models lack good representations for textual and visual modalities. |
| Approach: | They propose a novel text-to-image retrieval model that uses a transformer to match images with textual queries. |
| Outcome: | The proposed model outperforms state-of-the-art models in MSCOCO and Flickr30K . it achieves substantial speed advantages for a 1 million image index . |
Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision (2021.acl-long)
Copied to clipboard
Si Sun, Yingzhuo Qian, Zhenghao Liu, Chenyan Xiong, Kaitao Zhang, Jie Bao, Zhiyuan Liu, Paul Bennett
| Challenge: | Neural information retrieval models have shown advanced results in many ranking scenarios where massive relevance labels or clickthrough data are available. |
| Approach: | They propose a domain adaptive learning method that generalizes Neu-IR models from label-rich source domains to few-shot target domains. |
| Outcome: | The proposed method improves the few-shot ranking accuracy of Neu-IR models on three TREC benchmarks in the web, news, and biomedical domains. |
Semi-Supervised Text Classification with Balanced Deep Representation Distributions (2021.acl-long)
Copied to clipboard
| Challenge: | Semi-Supervised Text Classification (SSTC) is a type of self-training that uses labeled and unlabeled data to perform certain applications. |
| Approach: | They propose a method to initialize a deep classifier by training over labeled texts . they then alternatively predict unlabeled texts as their pseudo-labels and train them over the mixture . |
| Outcome: | Empirical results show that the proposed method is more accurate when labeled texts are scarce. |
Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval (2021.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval models based on dense representations show better performance than sparse representations. |
| Approach: | They propose a method to mimic the queries to each of the documents by an iterative clustering process and represent the documents using multiple pseudo queries. |
| Outcome: | The proposed model achieves state-of-the-art results on a large dataset while remaining high efficiency. |
ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing BERT-based pre-trained language models achieve high performance on many downstream tasks, but native derived sentence representations are collapsed and thus poor performance on semantic textual similarity (STS) tasks. |
| Approach: | They propose a framework for self-supervised Sentence Representation Transfer that adopts contrastive learning to fine-tune BERT in an unsupervised way. |
| Outcome: | The proposed framework improves on the BERT-derived representations by 8% on STS datasets and shows robustness in data scarcity scenarios. |
Exploring Dynamic Selection of Branch Expansion Orders for Code Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing code generation models model abstract syntax tree (AST) but not suitable for all multi-branch nodes. |
| Approach: | They propose to equip a Seq2Tree model with a branch selector to determine optimal expansion orders for multi-branch nodes. |
| Outcome: | The proposed model can determine optimal expansion orders of branches for multi-branch nodes. |
COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion (2021.acl-long)
Copied to clipboard
| Challenge: | Existing inference models are opaque, but they can be made more interpretable by explicitly generating interim inference rules and using them to guide the generation of task-specific textual outputs. |
| Approach: | They propose a recursive inference framework that iteratively reads context sentences and dynamically generates contextualized inference rules, encodes them, and uses them to guide output generation. |
| Outcome: | The proposed framework generates better story sentences than baseline models, and is more interpretable than existing models. |
Reasoning over Entity-Action-Location Graph for Procedural Text Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Procedural text understanding aims at tracking the states and locations of entities mentioned in a paragraph. |
| Approach: | They propose a framework to model entities-entity, action, and location relations using a graph neural network. |
| Outcome: | The proposed approach outperforms strong baselines on two datasets, ProPara and Recipes. |
From Paraphrasing to Semantic Parsing: Unsupervised Semantic Parsing via Synchronous Semantic Decoding (2021.acl-long)
Copied to clipboard
Shan Wu, Bo Chen, Chunlei Xin, Xianpei Han, Le Sun, Weipeng Zhang, Jiansong Chen, Fan Yang, Xunliang Cai
| Challenge: | Experimental results show that Synchronous Semantic Decoding (SSD) can achieve state-of-the-art unsupervised semantic parsing performance on multiple datasets. |
| Approach: | They propose an unsupervised method which solves the semantic gap and the structure gap by leveraging paraphrasing and grammar-constrained decoding. |
| Outcome: | The proposed method can solve the semantic gap and structure gap on multiple datasets. |
Pre-training Universal Language Representation (2021.acl-long)
Copied to clipboard
| Challenge: | Despite the cutting-edge representation learning, most language models focus on specific levels of linguistic units. |
| Approach: | They propose a training objective MiSAD that utilizes meaningful n-grams extracted from large unlabeled corpus by an algorithm for pre-trained language models. |
| Outcome: | The proposed model achieves highest accuracy on analogy tasks in different language levels and significantly improves performance on downstream tasks. |
Structural Pre-training for Dialogue Comprehension (2021.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large-scale pre-training language models (PrLMs) have achieved remarkable successes in a variety of natural language processing tasks. |
| Approach: | They propose to use SPIDER to capture dialogue exclusive features from dialogue texts. |
| Outcome: | The proposed model performs well on widely used dialogue benchmarks. |
AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have achieved great success in natural language processing. |
| Approach: | They propose a method that automatically searches architecture hyper-parameters in BERT . they use one-shot learning and the search space to provide an adaptive development way . |
| Outcome: | The proposed method outperforms both the baseline and distillation-based methods on GLUE and SQUAD benchmarks. |
Data Augmentation with Adversarial Training for Cross-Lingual NLI (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to train cross-lingual models with labeled data are subpar, resulting in subpar results. |
| Approach: | They propose a data augmentation strategy that enriches data to reflect more diversity in a semantically faithful way and leverages adversarial training regimens to achieve greater robustness. |
| Outcome: | The proposed approach improves cross-lingual inference by leveraging the data to reflect more diversity in a semantically faithful way. |
Bootstrapped Unsupervised Sentence Representation Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to learn sentence representations rely on quality labeled data. |
| Approach: | They propose a Siamese Network which maximizes similarity between two augmented views of each sentence. |
| Outcome: | The proposed method outperforms state-of-the-art methods on STS and classification tasks. |
Learning Event Graph Knowledge for Abductive Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for abductive reasoning based on formal logic lack commonsense knowledge and effective reasoning mechanism. |
| Approach: | They propose a narrative text-based abductive reasoning task NLI with a latent variable to capture commonsense knowledge from event graph for guiding the abductive reasoning task. |
| Outcome: | The proposed model outperforms baseline methods on the abductive reasoning task. |
A Cognitive Regularizer for Language Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | a uniform information density hypothesis is used to explain certain linguistic phenomena . a regularizer that encodes the UID hypothesis can be used for language training . |
| Approach: | They propose to augment the canonical MLE objective with a regularizer that encodes UID . they find that regularization consistently improves perplexity in language models . |
| Outcome: | The proposed hypothesis can be operationalized as an inductive bias for language modeling. |
Lower Perplexity is Not Always Human-Like (2021.acl-long)
Copied to clipboard
| Challenge: | Existing efforts to build human-like computational models have focused on English . a cross-lingual evaluation is needed to build such models, but current research has focused on Japanese . |
| Approach: | They re-examine an established generalization that lower perplexity is not always human-like in Japanese . they propose a cross-lingual evaluation to build human-type computational models . |
| Outcome: | The proposed model lacks universality and lower perplexity is not always human-like . the results suggest a cross-lingual evaluation will be necessary to build human-type models . |
Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives (2021.acl-long)
Copied to clipboard
| Challenge: | Recent Word Sense Disambiguation systems have approached the upper bound of the task on standard evaluation benchmarks. |
| Approach: | They propose to convert the nearly isolated decisions into interrelated ones by exposing senses in context when learning sense embeddings in a similarity-based Sense Aware Context Exploitation architecture. |
| Outcome: | The proposed approach surpasses state-of-the-art on English and multilingual datasets by large margins. |
A Knowledge-Guided Framework for Frame Identification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing frameworks for frame identification are limited to only a few types of frame knowledge. |
| Approach: | They propose a Knowledge-Guided Frame Identification framework that integrates frame knowledge to learn better frame representation. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on two benchmark datasets. |
Obtaining Better Static Word Embeddings Using Contextual Embedding Models (2021.acl-long)
Copied to clipboard
| Challenge: | Recent contextual word embeddings have prohibitively high computational cost in many use-cases and are hard to interpret. |
| Approach: | They propose a distillation method which is an extension of CBOW-based training and improves computational efficiency of NLP applications. |
| Outcome: | The proposed method outperforms existing models and existing models in terms of quality and performance. |
Meta-Learning with Variational Semantic Memory for Word Sense Disambiguation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for word sense disambiguation (WSD) lack large annotated datasets with sufficient coverage of words . performance of such methods lags behind fully-supervised methods . a meta-learning model is proposed to solve this problem . |
| Approach: | They propose a model of semantic memory for supervised word sense disambiguation using meta-learning. |
| Outcome: | The proposed model improves performance in few-shot WSD and produces meaning prototypes that capture similar senses of distinct words. |
LexFit: Lexical Fine-Tuning of Pretrained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Transformer-based language models implicitly store a wealth of lexical semantic knowledge, but it is non-trivial to extract that knowledge effectively from their parameters. |
| Approach: | They propose to expose and enrich lexical knowledge from transformer-based language models to serve as effective decontextualized word encoders even when fed input words "in isolation" |
| Outcome: | The proposed model outperforms standard static WEs and vanilla LMs in lexical tasks over four established tasks in 8 languages. |
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for synthesising fluent, natural-sounding spoken audio captions do not require natural language text as an intermediate representation or source of supervision. |
| Approach: | They propose a model for directly synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or source of supervision. |
| Outcome: | The proposed model captures diverse visual semantics of images and can replace text with a set of discrete, sub-word speech units. |
CTFN: Hierarchical Learning for Multimodal Sentiment Analysis Using Coupled-Translation Fusion Network (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multimodal sentiment analysis require all modalities as input, thus are sensitive to missing modality at predicting time. |
| Approach: | They propose to model bi-direction interplay via couple learning and exploit multiple bi-directional translations to exploit multimodal fusion embeddings. |
| Outcome: | The proposed framework achieves state-of-the-art or often competitive performance on two multimodal benchmarks with extensive ablation studies. |
Positional Artefacts Propagate Through Masked Language Model Embeddings (2021.acl-long)
Copied to clipboard
| Challenge: | Existing word embedding models have a uniform pitfall in assigning a static vector to a word type. |
| Approach: | They propose a neuron-level analysis method to investigate the source of this information by comparing outlier neurons within BERT and RoBERTa’s hidden state vectors. |
| Outcome: | The proposed method pre-trains the RoBERTa-based models and shows that the outliers disappear without positional embeddings. |
Language Model Evaluation Beyond Perplexity (2021.acl-long)
Copied to clipboard
| Challenge: | a nascent literature on probing language models has focused on studying linguistic phenomena. |
| Approach: | They propose a framework for evaluating the fit of language models to natural language tendencies. |
| Outcome: | The proposed framework evaluates language models to the tendencies of natural language . it shows that the models learn only a subset of the tendancies considered . |
Learning to Explain: Generating Stable Explanations Fast (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for explaining outcome of machine learning models produce explanations, or rationales, which identify the attributions of features in an input example. |
| Approach: | They propose a Learning to Explain approach that learns the behaviour of an underlying explanation algorithm simultaneously from all training examples. |
| Outcome: | The proposed approach is 5 to 7.5104 times faster than existing models and has comparable faithfulness to the black-box model. |
StereoSet: Measuring stereotypical bias in pretrained language models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing literature on stereotypical biases in language models is limited . current evaluations focus on measuring bias without considering language modeling ability . |
| Approach: | They propose to measure stereotypical biases in four domains: gender, profession, race, and religion . they compare stereotypical and language modeling ability of popular models like BERT, GPT-2, RoBERTa and XLnet . |
| Outcome: | The proposed model shows strong stereotypical biases in gender, profession, race, and religion domains. |
Alignment Rationale for Natural Language Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing explanation methods pick prominent features, but alignments between words or phrases are more enlightening clues to explain the model. |
| Approach: | They propose a method to generate alignment rationale explanations for co-attention based models in NLI by feature selection. |
| Outcome: | The proposed method is more faithful and human-readable compared with existing methods. |
Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for pre-trained language models (PLMs) use parameter reduction techniques. |
| Approach: | They propose a pre-trained language model compression approach based on the matrix product operator from quantum many-body physics. |
| Outcome: | The proposed approach can decompose an original matrix into central tensors and auxiliary tenses . it can be applied to the original or compressed PLMs in a general way, with a lighter network . |
On Sample Based Explanation Methods for NLP: Faithfulness, Efficiency and Semantic Evaluation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for explaining "black-box" models such as Influence Functions are becoming more popular. |
| Approach: | They propose a semantic-based evaluation metric that can better align with humans’ judgment of explanations than the widely adopted diagnostic or re-training measures. |
| Outcome: | The proposed method can better align with humans’ judgment of explanations than diagnostic or re-training measures. |
Syntax-Enhanced Pre-trained Model (2021.acl-long)
Copied to clipboard
Zenan Xu, Daya Guo, Duyu Tang, Qinliang Su, Linjun Shou, Ming Gong, Wanjun Zhong, Xiaojun Quan, Daxin Jiang, Nan Duan
| Challenge: | Existing methods that use syntax of text in pre-training and fine-tuning suffer from discrepancy between the two stages. |
| Approach: | They propose a model that utilizes the syntactic structure of text in pre-training and fine-tuning stages. |
| Outcome: | The proposed model achieves state-of-the-art on six public benchmark datasets. |
Matching Distributions between Model and Data: Cross-domain Knowledge Distillation for Unsupervised Domain Adaptation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods require to learn to adapt the target model by exploiting the source data and sharing the network architecture across domains. |
| Approach: | They propose a framework that allows to transfer the knowledge of source domain to the unlabeled target domain without using source data. |
| Outcome: | The proposed framework matches distributions between a trained source model and a set of target data and achieves superior performance on cross-domain text classification. |
Counterfactual Inference for Text Classification Debiasing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to capture unintended dataset biases are expensive and require elaborate balancing strategies. |
| Approach: | They propose a model-agnostic text classification debiasing framework which can effectively avoid employing data manipulations or designing balancing mechanisms. |
| Outcome: | The proposed framework can effectively avoid data manipulations or designing balancing mechanisms to capture unintended dataset biases. |
HieRec: Hierarchical User Interest Modeling for Personalized News Recommendation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing news recommendation methods learn a single user embedding for each user from their previous behaviors to represent their overall interest. Existing methods only learn 'one' embeddable representation vectors to model user interest. |
| Approach: | They propose a news recommendation method with hierarchical user interest modeling that captures user interest in news rather than a single user embedding. |
| Outcome: | The proposed method can better capture multi-grained user interest in news. |
PP-Rec: News Recommendation with Personalized User Interest and Time-aware News Popularity (2021.acl-long)
Copied to clipboard
| Challenge: | Existing personalized news recommendation methods have difficulties in making accurate recommendations to cold-start users. |
| Approach: | They propose to incorporate news popularity information to improve cold-start recommendations . they propose to use a popularity-aware user encoder to eliminate popularity bias . |
| Outcome: | The proposed method improves accuracy and diversity of personalized news recommendation on two real-world datasets. |
Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect false claims ignore the characteristics of FC-articles . claims are often quoted to describe checked events, providing lexical information . sentence templates to introduce or debunk claims are common across articles, providing pattern information. |
| Approach: | They propose a model to rerank FC-articles using key sentences and pattern information. |
| Outcome: | The proposed model outperforms existing methods on two real-world datasets showing that key sentences can be used to predict if an article fact-checks the given claim. |
Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show vulnerability of deep neural networks to adversarial examples that intentionally fool the networks. |
| Approach: | They propose a method for training a robust model to defense synonym substitution-based attacks by sampling embedding vectors for each word in an input sentence and augmenting them with the training data. |
| Outcome: | The proposed method outperforms other proposed defense methods by a significant margin across different network architectures and multiple data sets. |
Shortformer: Better Language Modeling using Shorter Inputs (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods require computationally expensive relative position embeddings. |
| Approach: | They propose two methods that decrease input length to improve perplexity and perplexability. |
| Outcome: | The proposed methods speed up training by a factor of 1.65 and reduce memory usage. |
BanditMTL: Bandit-based Multi-task Learning for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to regularize task variance are unexplored in multi-task text classification. |
| Approach: | They propose a multi-task learning method based on adversarial multi-armed bandit to regularize the task variance by means of a mirror gradient ascent-descent algorithm. |
| Outcome: | The proposed method achieves state-of-the-art in multi-task text classification. |
Unified Interpretation of Softmax Cross-Entropy and Negative Sampling: With Case Study for Knowledge Graph Embedding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods do not provide a fairly accurate comparison of the two loss functions. |
| Approach: | They propose to use the Bregman divergence to provide a unified interpretation of the softmax cross-entropy and negative sampling loss functions. |
| Outcome: | The proposed model can be used to predict missing relational links between entities using a scoring method. |
De-Confounded Variational Encoder-Decoder for Logical Table-to-Text Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Logical table-to-text generation is challenging where deep learning models capture surface-level spurious correlations rather than the causal relationships between the table x and the sentence y. |
| Approach: | They propose to use variational inference to estimate the confounders in the latent space and cooperate with the causal intervention based on Pearl’s do-calculus to alleviate the spurious correlations. |
| Outcome: | The proposed model outperforms baselines and achieves new state-of-the-art performance on two logical table-to-text datasets in terms of logical fidelity. |
Rethinking Stealthiness of Backdoor Attack against NLP Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing backdoor attacks are not stealthy to system deployers or users. |
| Approach: | They propose a novel backdoor attack method based on negative data augmentation and modifying word embeddings that is much stealthier while maintaining pretty good attacking performance. |
| Outcome: | The proposed method is much stealthier while maintaining pretty good attacking performance. |
Crowdsourcing Learning as Domain Adaptation: A Case Study on Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that crowdsourced annotations are highly effective under supervised conditions. |
| Approach: | They propose an annotator-aware representation learning model that is inspired by domain adaptation methods which attempt to capture effective domain-alike features. |
| Outcome: | The proposed model is highly effective on a benchmark dataset and achieves state-of-the-art performance with only a very small scale of expert annotations. |
Exploring Distantly-Labeled Rationales in Neural Network Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus on distantly-labeled rationales, ignoring the potential important non-rationale words and not distinguishing the importance of different rationale words. |
| Approach: | They propose two novel auxiliary loss functions to make better use of distantly-labeled rationales, which encourage models to maintain their focus on important words beyond labeled rationals (PINs) and alleviate redundant training on non-helpful rationale (NoIRs). |
| Outcome: | The proposed methods outperform existing methods on two representative classification tasks while maintaining the ability to spread focus to other unlabeled important words. |
Learning to Perturb Word Embeddings for Out-of-distribution QA (2021.acl-long)
Copied to clipboard
| Challenge: | QA models that are pretraining with unlabeled data can overfit and may not generalize well to unseen data that falls outside the training distribution. |
| Approach: | They propose a method which perturbs word embedding without changing their semantics. |
| Outcome: | The proposed method outperforms baseline methods on five target domains on a single source dataset on five different target domain domains. |
Maria: A Visual Experience Powered Conversational Agent (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on grounding conversational agents on text-only corpora, but they lack the perception ability to our physical world. |
| Approach: | They propose to ground conversational agents on images retrieved from large-scale image indexes . they propose to use visual knowledge to generate informative responses based on the extracted knowledge . |
| Outcome: | The proposed agent outperforms state-of-the-art methods on automatic metrics and human evaluation. |
A Human-machine Collaborative Framework for Evaluating Malevolence in Dialogues (2021.acl-long)
Copied to clipboard
| Challenge: | Automatic evaluation of dialogues often shows insufficient correlation with human judgements due to the diversity of responses. |
| Approach: | They propose a framework that allows automatic evaluation of dialogues with human judgements. |
| Outcome: | The proposed framework achieves 99% evaluation accuracy with half the human effort spared. |
Generating Relevant and Coherent Dialogue Responses using Self-Separated Conditional Variational AutoEncoders (2021.acl-long)
Copied to clipboard
| Challenge: | Conditional Variational AutoEncoders (CVAE) can enhance the diversity and informativeness of responses in open-domain dialogue generation tasks. |
| Approach: | They propose a Conditional Variational AutoEncoder (CVAE) that regularizes latent variables and introduces group information to regularize them. |
| Outcome: | Empirical results show that the proposed model can significantly boost responses in well-established open-domain dialogue datasets. |
Learning to Ask Conversational Questions by Optimizing Levenshtein Distance (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for estimating maximum likelihood are limited by easily learned tokens . Existing systems that generate questions based on dialogue context are limited in their ability to learn tokens. |
| Approach: | They propose a framework that optimizes the minimum Levenshtein distance through explicit editing actions. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two benchmark datasets and generalizes well on unseen data. |
DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue (2021.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks do not have enough annotations to analyze video-grounded dialogue systems and understand their capabilities and limitations in isolation. |
| Approach: | They present a Diagnostic Dataset for Video-grounded dialogue with minimal biases and detailed annotations for the different types of reasoning over the spatio-temporal space of video. |
| Outcome: | The proposed system is based on 11k CATER synthetic videos and contains 10 instances of 10-round dialogues for each video. |
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation (2021.acl-long)
Copied to clipboard
| Challenge: | Emotion recognition in conversation is a crucial component in affective dialogue systems, which helps the system understand users’ emotions and generate empathetic responses. |
| Approach: | They propose a multimodal fused graph convolutional network model which leverages multimodal dependencies and speaker information to model inter-speaker and intra-speech dependency. |
| Outcome: | The proposed model outperforms other SOTA methods on two public benchmark datasets, IEMOCAP and MELD. |
DynaEval: Unifying Turn and Dialogue Level Evaluation (2021.acl-long)
Copied to clipboard
Chen Zhang, Yiming Chen, Luis Fernando D’Haro, Yan Zhang, Thomas Friedrichs, Grandee Lee, Haizhou Li
| Challenge: | Existing evaluation metrics focus on the turn-level quality of a dialogue . a unified framework that holistically considers the quality of the entire dialogue is needed . |
| Approach: | They propose a unified automatic evaluation framework which holistically considers the quality of the entire dialogue. |
| Outcome: | The proposed framework outperforms the state-of-the-art dialogue coherence model and correlates strongly with human judgements across multiple evaluation aspects at both turn and dialogue level. |
CoSQA: 20,000+ Web Queries for Code Search and Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Using deep neural networks to find codes is difficult . we present a dataset that includes 20,604 labels for natural language queries and codes . |
| Approach: | They introduce a contrastive learning method to enhance text-code matching . they find that CoSQA improves the accuracy of code question answering by 5.1% . |
| Outcome: | The proposed method improves the accuracy of code question answering by 5.1% and improves by 10.5% on a CodeBERT model. |
Rewriter-Evaluator Architecture for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve neural machine translation models with multiple decoding passes lack proper policies to terminate multi-pass processes. |
| Approach: | They propose a novel architecture of Rewriter-Evaluator to terminate multi-pass decoding . they propose prioritized gradient descent to jointly and efficiently train rewriter and evaluator . |
| Outcome: | The proposed architecture significantly outperforms existing methods on three translation tasks and reduces performance gaps to oracle policies. |
Modeling Bilingual Conversational Characteristics for Neural Chat Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural chat translation aims to translate bilingual conversational text due to its inherent characteristics such as role preference, dialogue coherence, and translation consistency. |
| Approach: | They propose to model the translation quality of conversational text by learning distributions of bilingual conversational characteristics. |
| Outcome: | The proposed approach outperforms baseline models and is widely available. |
Importance-based Neuron Allocation for Multilingual Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multilingual neural machine translation tend to preserve general knowledge, but ignore language-specific knowledge. |
| Approach: | They propose to divide model neurons into general and language-specific parts based on their importance across languages. |
| Outcome: | The proposed model can preserve general knowledge but ignore language-specific knowledge on several languages, and is universal and cost-effective. |
Transfer Learning for Sequence Generation: from Single-source to Multi-source (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that pretrained models are effective for low-resource downstream tasks. |
| Approach: | They propose a two-stage finetuning method to transfer pretrained models to MSG tasks by concatenating multiple sources into a single long sequence. |
| Outcome: | The proposed model outperforms baselines on the WMT17 APE task and multi-source translation task using the WTM14 test set. |
A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters (2021.acl-long)
Copied to clipboard
| Challenge: | Few-shot crosslingual transfer outperforms zero-shot with pretrained encoders like multilingual BERT. |
| Approach: | They conduct an experimental study on 40 sets of sampled few shots for six diverse NLP tasks across up to 40 languages. |
| Outcome: | The proposed model outperforms state-of-the-art approaches on lexical features and a full model finetuning approach outperformed several state- of-the art approaches. |
Coreference Reasoning in Machine Reading Comprehension (2021.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for machine reading comprehension do not reflect the natural distribution and, consequently, the challenges of coreference reasoning. |
| Approach: | They propose to use existing coreference resolution datasets to train machine reading comprehension models to better reflect the natural distribution and, consequently, the challenges of coreference reasoning. |
| Outcome: | The proposed method improves the performance of state-of-the-art models on a set of coreference-related datasets. |
Adapting Unsupervised Syntactic Parsing Methodology for Discourse Dependency Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | Discourse dependency parsing is a task that requires a large amount of training data, but there is little research on it. |
| Approach: | They propose to adapt unsupervised syntactic dependency parsing methods for unsupervised discourse dependency parses using unlabeled training data. |
| Outcome: | The proposed methods outperform existing methods in semi-supervised and supervised settings and outperformed existing methods. |
A Conditional Splitting Framework for Efficient Constituency Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | Developing efficient and effective parsing solutions has always been a key focus in NLP. |
| Approach: | They propose a generic seq2seq parsing framework that casts constituency parsers into a series of conditional splitting decisions. |
| Outcome: | The proposed framework outperforms state-of-the-art (SoTA) methods in discourse parsing . it is based on a syntactic and discourse parsed model and is linear in number of nodes . |
A Unified Generative Framework for Various NER Subtasks (2021.acl-long)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is the task of identifying spans that represent entities in sentences. |
| Approach: | They propose to formulate NER subtasks as entity span sequence generation task . framework can be used to solve all three kinds of NER tasks without tagging schema . |
| Outcome: | The proposed framework achieves state-of-the-art (SoTA) or near SoTA performance on eight English NER datasets. |
An In-depth Study on Internal Structure of Chinese Words (2021.acl-long)
Copied to clipboard
Chen Gong, Saihao Huang, Houquan Zhou, Zhenghua Li, Min Zhang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan
| Challenge: | Unlike English letters, Chinese characters have rich and specific meanings. |
| Approach: | They propose to model Chinese words' internal structures as dependency trees with 11 labels for distinguishing syntactic relationships. |
| Outcome: | The proposed model of Chinese word-internal structures shows it can be used to parse sentences . it shows that the model can be applied to a sentence-level task with a competitive dependency parser. |
MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual NER are labeled sequence translation and instance-based transfer via machine translation (MT) Existing methods to cross NER include label projection and labeling, but they are expensive and time-consuming. |
| Approach: | They propose a simple but effective labeled sequence translation method to translate source-language training data to target languages and avoids word order change and entity span determination. |
| Outcome: | The proposed method avoids word order change and entity span determination and can be generalized with the language-specific features from the target-language synthetic data and the language independent features from multilingual synthetic data. |
Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Chinese sequence labelling only fuse lexicon features via a shallow and random initialized sequence layer and do not integrate them into the bottom layers of BERT. |
| Approach: | They propose a Lexicon Enhanced BERT model which integrates external lexicon knowledge into BERT layers directly by a lexiccon Adapter layer. |
| Outcome: | The proposed model integrates external lexicon knowledge into BERT layers directly by a Lexicon Adapter layer. |
Math Word Problem Solving with Explicit Numerical Values (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for solving math word problems ignore numerical values in solving problems. |
| Approach: | They propose a numerically-based approach that explicitly incorporates numerical values into a sequence-to-tree network and uses a mathematical properties prediction mechanism to capture category and comparison information of numerals. |
| Outcome: | The proposed model outperforms existing state-of-the-art models on the Math23K and APE datasets. |
Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing solutions for math word problems lack explicit integration of math symbolic constraints, leading to unexplainable and unreasonable predictions. |
| Approach: | They propose a novel mathematical model that explicitly incorporates symbolic constraints by auxiliary tasks to enforce different symbolic reasoning. |
| Outcome: | The proposed solver incorporates symbolic constraints by auxiliary tasks to enforce different symbolic reasoning. |
SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining (2021.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge-based PLMs are based on linked-entity information, but they only use linked-enemy information as auxiliary information. |
| Approach: | They propose to integrate semantic knowledge from neighbours of linked-entity into a medical PLM that integrates heterogeneous-entities into the homogeneously neighbouring entity structure. |
| Outcome: | Experiments show that SMedBERT outperforms baselines in knowledge-intensive Chinese medical tasks. |
What is Your Article Based On? Inferring Fine-grained Provenance (2021.acl-long)
Copied to clipboard
| Challenge: | a new study of claim provenance seeks to trace and explain the origins of claims . a critical reader must be able to assess where the information comes from and where it originates from . |
| Approach: | They propose a method to model and reason about the provenance of multiple interacting claims . they propose generating metadata for the source article based on context and search signals . |
| Outcome: | The proposed method improves on baselines by identifying key external information in the source article. |
Cross-modal Memory Networks for Radiology Report Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Medical imaging reports are essential in clinical practice, and generating the reports is beneficial to reduce the burden of radiologists. |
| Approach: | They propose to use a shared memory to enhance the encoder-decoder framework for radiology report generation. |
| Outcome: | The proposed model can generate more accurate reports on two widely used datasets. |
Controversy and Conformity: from Generalized to Personalized Aggressiveness Detection (2021.acl-long)
Copied to clipboard
Kamil Kanclerz, Alicja Figas, Marcin Gruza, Tomasz Kajdanowicz, Jan Kocon, Daria Puchalska, Przemyslaw Kazienko
| Challenge: | a new method to personalize documents that are perceived differently by users is needed . a recent study found that only a few annotations of controversial documents outperform classic methods . |
| Approach: | They propose to use some known, most controversial texts whose offensiveness is very ambiguous . they use user conformity-based measures or embeddings of their previous annotations to improve personalized reasoning . |
| Outcome: | The proposed methods outperform standard methods in document controversy and user nonconformity . the more controversial the content, the greater the gain, the authors say . |
Multi-perspective Coherent Reasoning for Helpfulness Prediction of Multimodal Reviews (2021.acl-long)
Copied to clipboard
| Challenge: | Existing review helpfulness prediction tasks rely on text and image modalities to analyze review helpfuliness. |
| Approach: | They propose a task to analyze review helpfulness from text and visual modalities and propose 'multi-perspective coherent reasoning' method to combine coherence between product and review is proposed. |
| Outcome: | The proposed method can lead to performance increase of 8.5% compared to the best performing text-only model. |
Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve online inference efficiency of the Transformer for instantaneous Grammatical Error Correction (GEC) are sequenceto-sequence (seq2sequ) and sequenceto sequence (saq2eq) |
| Approach: | They propose a novel approach to improve the online inference efficiency of the Transformer model for instantaneous Grammatical Error Correction (GEC) it aggressively decodes as many tokens as possible in parallel instead of always decoding only one token in each step to improve computational parallelism. |
| Outcome: | The proposed approach can achieve state-of-the-art results in English and Chinese benchmarks with 10x speedup over the Transformer-big model. |
Automatic ICD Coding via Interactive Shared Representation Networks with Self-distillation Mechanism (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for ICD coding ignore the long-tail of code frequency or noisy clinical notes. |
| Approach: | They propose to use an interactive shared representation network to model code co-occurrences while focusing on the clinical note's noteworthy part and extract valuable information through a self-distillation learning mechanism to solve the long-tail problem. |
| Outcome: | The proposed model reduces the long-tail of code frequency and noise in clinical notes and extracts valuable information through a self-distillation learning mechanism. |
PHMOSpell: Phonological and Morphological Knowledge Guided Chinese Spelling Check (2021.acl-long)
Copied to clipboard
| Challenge: | False gram and phonological errors make Chinese spelling check difficult . a novel end-to-end trainable model outperforms existing methods . |
| Approach: | They propose a trainable Chinese spelling check model that integrates phonological and visual information into a pre-trained language model. |
| Outcome: | The proposed model outperforms existing state-of-the-art models on three benchmarks. |
Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting (2021.acl-long)
Copied to clipboard
| Challenge: | Existing QG systems perform substantially worse in answering multi-hop questions than single-hop ones. |
| Approach: | They propose a framework that progressively increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. |
| Outcome: | The proposed framework increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. |
Improving Encoder by Auxiliary Supervision Tasks for Table-to-Text Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that our method not only has a good generalization but also outperforms previous methods on several metrics: BLEU, Content Selection, Content Ordering. |
| Approach: | They propose to build an entity graph from the input tables and introduce a reasoning module to perform reasoning on the graph. |
| Outcome: | The proposed method outperforms previous methods on several metrics: BLEU, Content Selection, Content Ordering. |
POS-Constrained Parallel Decoding for Non-autoregressive Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing non-autoregressive generation systems face multimodality problem due to conditionally independent decoding. |
| Approach: | They propose to incorporate linguistic structure into NAG inference instead of teacher AG . they propose a method that provides a specific POS sequence to constrain the NAG model . |
| Outcome: | The proposed method improves NAG models on four text generation tasks to a greater extent compared to knowledge distillation. |
Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to pretrain language models are limited by one-size-fits-all vocabulary . embeddings of mismatch tokens can be efficiently initialized in downstream tasks . |
| Approach: | They propose to extend pretrain-finetune pipeline with an embedding transfer step . plug-and-play embeddable generator is introduced to generate any input token . |
| Outcome: | The proposed approach allows for more efficient and better performed NLG models. |
TGEA: An Error-Annotated Dataset and Benchmark Tasks for TextGeneration from Pretrained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Using pretrained language models, we propose an error-annotated dataset for text generation . we use carefully selected prompt words to guide GPT-2 to generate candidate sentences . |
| Approach: | They propose an error-annotated dataset with multiple benchmark tasks for text generation from pretrained language models. |
| Outcome: | The proposed dataset covers 24 types of errors according to common sense and linguistics. |
Long-Span Summarization via Local Attention and Content Selection (2021.acl-long)
Copied to clipboard
| Challenge: | Transformer-based models are state-of-the-art for a wide range of natural language processing tasks, including document summarization. |
| Approach: | They exploit large pre-trained transformer-based models and address long-span dependencies in abstractive summarization using two methods: local self-attention; and explicit content selection. |
| Outcome: | The proposed models achieve state-of-the-art on Spotify Podcast, arXiv, and PubMed datasets. |
RepSum: Unsupervised Dialogue Summarization based on Replacement Strategy (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to learn vital information from dialogue context with limited data are limited due to limited words in utterances and huge gap between dialogue and its summary. |
| Approach: | They propose an unsupervised strategy to learn vital information from dialogue context . the proposed model uses a hypothetical foundation that a superior summary approximates a replacement of the original dialogue . |
| Outcome: | The proposed model outperforms existing models on a number of datasets. |
BASS: Boosting Abstractive Summarization with Unified Semantic Graph (2021.acl-long)
Copied to clipboard
| Challenge: | Abstractive summarization for long-document or multi-document remains challenging for Seq2Seq as it does not analyze long-distance relations in text. |
| Approach: | They propose a framework for Boosting Abstractive Summarization based on a unified Semantic graph which aggregates co-referent phrases distributing across a long range of context and conveys rich relations between phrases. |
| Outcome: | The proposed framework improves document representation and summary generation process by leveraging the graph structure. |
Capturing Relations between Scientific Papers: An Abstractive Model for Related Work Section Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing related work generation models are inflexible and extract sentences from multiple papers to form a related work discussion. |
| Approach: | They propose a Relation-aware Related work generator which generates an abstractive related work from the given multiple scientific papers in the same research area. |
| Outcome: | The proposed model improves over existing models and can be used to familiarize researchers with the state of the art in the field. |
Focus Attention: Promoting Faithfulness and Diversity in Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Currently, document summarization is challenging even for humans. |
| Approach: | They propose a focus attention mechanism which encourages decoders to generate tokens that are topically similar to the input document. |
| Outcome: | The proposed method outperforms top-k and nucleus sampling methods on the BBC extreme summarization task and is more accurate than focus attention-based models. |
Generating Query Focused Summaries from Query-Free Resources (2021.acl-long)
Copied to clipboard
| Challenge: | Existing datasets are small for data-hungry neural architectures and are limited to evaluation purposes. |
| Approach: | They propose to decompose QFS into query modeling and conditional language modeling . they propose a Masked ROUGE Regression framework for evidence estimation and ranking . |
| Outcome: | The proposed model achieves state-of-the-art performance despite weak supervision. |
Robustifying Multi-hop QA through Pseudo-Evidentiality Training (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to robustify multi-hop question answering models require expensive annotations. |
| Approach: | They propose a method to supervise answers with right reasoning chains without annotations . they compare answers confidence with and without evidence sentences to generate "pseudo-evidentiality" annotations. |
| Outcome: | The proposed model is accurate and robust in multi-hop reasoning. |
xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to find relevant passages using sparse keywords are not effective for open domain question answering. |
| Approach: | They propose a new contrastive learning method for learning a dual-encoder model for question-passage matching using a large pool of negative samples. |
| Outcome: | The proposed method maintains large pool of negative samples and optimizes question-to-passage and passage-to question matching tasks. |
Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches do not explicitly train QA models on how to resolve conversational dependency, and thus these models are limited in understanding human dialogues. |
| Approach: | They propose a framework that generates self-contained questions that can be understood without the conversation history and then trains a QA model with the pairs of original and self-constructed questions using a consistency-based regularizer. |
| Outcome: | The proposed framework improves the models’ performance by up to 1.2 F1 on QuAC, and 5.2 F1 for CANARD, while addressing the limitations of the existing approaches. |
PhotoChat: A Human-Human Dialogue Dataset With Photo Sharing Behavior For Joint Image-Text Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | PhotoChat contains 12k dialogues, each of which is paired with a user photo that is shared during the conversation. |
| Approach: | They propose to use PhotoChat to facilitate research on image-text modeling by combining a photo-sharing intent prediction task and a picture retrieval task to retrieve the most relevant photo according to the dialogue context. |
| Outcome: | The proposed tasks achieve 10.4% recall@1 and 58.1% F1 scores, indicating that the proposed dataset presents interesting yet challenging real-world problems. |
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies report improvements when equipping models with multimodal information, but it remains unclear whether such improvements actually come from the multimodal part. |
| Approach: | They propose to extend conventional text-only translation models with multimodal information by extending them with visual input. |
| Outcome: | The proposed models replicate similar gains as recently developed multimodal-integrated systems achieved, but learn to ignore multimodal information. |
Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Recent advances in natural language processing and computer vision have made significant progress in artificial intelligence (AI). |
| Approach: | They propose Motion-Appearance Synergistic Networks which embed cross-modal features grounded on motion and appearance information and selectively utilize them depending on the question’s intentions. |
| Outcome: | The proposed network achieves state-of-the-art on the TGIF-QA and MSVD-QA datasets and qualitatively analyzes the results. |
BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Existing NER models are supervised by a large number of training sequences, each pre-annotated with token-level labels. |
| Approach: | They propose a conditional hidden Markov model which can effectively infer true labels from multi-source noisy labels in an unsupervised way. |
| Outcome: | The proposed model outperforms state-of-the-art weakly supervised NER models on four benchmarks from various domains. |
CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reduce noise from DS generated training data are not effective for distantly supervised relation extraction (DSRE) |
| Approach: | They propose a multi-instance learning framework to reduce DS noise by dividing training instances into several bags and using them as new data units. |
| Outcome: | The proposed framework improves on NYT10, GDS and KBP with significant improvements over existing methods. |
SENT: Sentence-level Distant Relation Extraction via Negative Training (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for relation extraction use bag labels, which introduce noise, to train the model. |
| Approach: | They propose to use negative training to train a model using complementary labels to separate the noisy data from the training data. |
| Outcome: | The proposed method improves on previous methods on sentence-level evaluation and de-noise effect. |
An End-to-End Progressive Multi-Task Learning Framework for Medical Named Entity Recognition and Normalization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for medical named entity recognition and named entity normalization suffer from error propagation between the two tasks. |
| Approach: | They propose an end-to-end progressive multi-task learning model for jointly modeling medical named entity recognition and normalization in an effective way. |
| Outcome: | The proposed model reduces error propagation by exploiting the learnable features for both tasks to improve performance. |
PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction (2021.acl-long)
Copied to clipboard
Hengyi Zheng, Rui Wen, Xi Chen, Yifan Yang, Yunyan Zhang, Ziheng Zhang, Ningyu Zhang, Bin Qin, Xu Ming, Yefeng Zheng
| Challenge: | Recent methods for extracting entities and relations from unstructured texts suffer from limitations, such as redundancy of relation prediction and inefficiency. |
| Approach: | They propose a joint relational triple extraction framework based on Potential Relation and Global Correspondence (PRGC) they propose overlapping triples for relation prediction and relation-relational alignment . |
| Outcome: | The proposed framework achieves state-of-the-art performance on public benchmarks with higher efficiency and consistent performance gain on complex scenarios of overlapping triples. |
Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to classify named entity mentions with fewshots fail to differentiate rich semantics in other-class words, which will aggravate overfitting under few shot scenario. |
| Approach: | They propose a model that can automatically induce different unde- fined classes from the other class to improve few-shot Named Entity Recognition (NER) . |
| Outcome: | The proposed model outperforms five state-of-the-art models in 1- shot and 5-shots settings on four NER bench marks. |
Joint Biomedical Entity and Relation Extraction with Knowledge-Enhanced Collective Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for information extraction from biomedical texts do not utilize external knowledge . despite the exponential growth of biomedically published articles, many existing methods fall behind . |
| Approach: | They propose a framework that utilizes external knowledge for entity and relation extraction . KECI uses an initial span graph to construct a knowledge graph containing relevant background knowledge . |
| Outcome: | The proposed framework achieves state-of-the-art results in two biomedical datasets . it achieves 4.59% and 4.91% improvement in F1 scores over the state- of-the art methods . |
Fine-grained Information Extraction from Biomedical Literature based on Knowledge-enriched Abstract Meaning Representation (2021.acl-long)
Copied to clipboard
| Challenge: | Compared with general natural language texts, sentences from scientific papers usually possess wider contexts between knowledge elements. |
| Approach: | They propose a novel biomedical Information Extraction model to extract scientific entities and events from English research papers using Abstract Meaning Representation (AMR) they construct a sentence-level knowledge graph from an external knowledge base and encode it to improve the model's understanding of complex scientific concepts. |
| Outcome: | The proposed model can extract scientific entities and events from scientific literature and improve its understanding of complex scientific concepts. |
Unleash GPT-2 Power for Event Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Event Detection (ED) aims to recognize mentions of events and their types in text. |
| Approach: | They propose to exploit a pre-trained language model to generate training samples for ED. |
| Outcome: | The proposed model improves on multiple ED benchmark datasets and establishes state-of-the-art results. |
CLEVE: Contrastive Pre-training for Event Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing EE methods do not model event characteristics from large unsupervised data. |
| Approach: | They propose a contrastive pre-training framework for event extraction to better learn event knowledge from large unsupervised data and their semantic structures. |
| Outcome: | The proposed framework improves on ACE 2005 and MAVEN datasets on event extraction tasks. |
Document-level Event Extraction via Parallel Prediction Networks (2021.acl-long)
Copied to clipboard
| Challenge: | Document-level event extraction (DEE) is indispensable when events are described throughout a document. |
| Approach: | They propose a document-level event extraction model that can extract structured events from a text in parallel. |
| Outcome: | The proposed model outperforms current state-of-the-art methods on a document-level event extraction task. |
StructuralLM: Structural Pre-training for Form Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing pre-trained language models focus on text-only representation, neglecting cell-level layout information. |
| Approach: | They propose a pre-training approach to leverage cell and layout information from scanned documents. |
| Outcome: | The proposed model achieves state-of-the-art in various downstream tasks . it uses 2Dposition embeddings to model word-level layout information . |
Dual Graph Convolutional Networks for Aspect-based Sentiment Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to model relationships between aspects and opinion words are inefficient due to informal expressions and complexity of online reviews. |
| Approach: | They propose a dual graph convolutional networks model that considers complementarity of syntax structures and semantic correlations simultaneously. |
| Outcome: | The proposed model outperforms state-of-the-art methods on three public datasets and validates it. |
Multi-Label Few-Shot Learning for Aspect Category Detection (2021.acl-long)
Copied to clipboard
| Challenge: | Existing few-shot learning methods focus on single-label predictions, which can not work well for ACD since a sentence may contain multiple aspect categories. |
| Approach: | They propose a few-shot learning method that uses the prototypical network to learn aspects from a set of aspects. |
| Outcome: | The proposed method significantly outperforms baseline methods on three datasets. |
Argument Pair Extraction via Attention-guided Multi-Layer Multi-Cross Encoding (2021.acl-long)
Copied to clipboard
| Challenge: | Argument pair extraction (APE) is a research task for extracting arguments from two passages and identifying potential argument pairs. |
| Approach: | They propose a novel attention-guided multi-layer multi-cross encoding scheme that processes two passages with two individual sequence encoders and updates their representations using each other’s attention. |
| Outcome: | The proposed model significantly improves the performance over several alternatives. |
A Neural Transition-based Model for Argumentation Mining (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for identifying argumentation structures are inefficient and class imbalanced. |
| Approach: | They propose a neural transition-based model that incrementally builds an argumentation graph by generating a sequence of actions. |
| Outcome: | The proposed model can handle tree and non-tree structured argumentation without structural constraints. |
Keep It Simple: Unsupervised Simplification of Multi-Paragraph Text (2021.acl-long)
Copied to clipboard
| Challenge: | a novel approach to text simplification learns to balance a reward across three properties: fluency, salience and simplicity. |
| Approach: | They propose a novel algorithm to optimize the reward which proposes several candidate simplifications and a realistic text comprehension task as an evaluation method for text simplification. |
| Outcome: | The proposed model outperforms strong supervised baselines on the English news domain and can help people complete a comprehension task an average of 18% faster while retaining accuracy. |
Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence (2021.acl-long)
Copied to clipboard
| Challenge: | Existing generation models struggle to maintain a coherent event sequence throughout the generated text. |
| Approach: | They propose a long text generation model which can represent prefix sentences at sentence level and discourse level in the decoding process. |
| Outcome: | The proposed model can generate more coherent texts than state-of-the-art models. |
OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics (2021.acl-long)
Copied to clipboard
Jian Guan, Zhexin Zhang, Zhuoer Feng, Zitao Liu, Wenbiao Ding, Xiaoxi Mao, Changjie Fan, Minlie Huang
| Challenge: | Existing automatic metrics are observed to correlate poorly with human evaluation. |
| Approach: | They propose to use OpenMEVA to evaluate open-ended story generation metrics. |
| Outcome: | The proposed test suite assesses the capabilities of open-ended story generation metrics on annotated stories and auto-constructed test examples. |
DYPLOC: Dynamic Planning of Content Using Mixed Language Models for Text Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing neural generation models fall short of coherence, thus requiring efficient content planning. |
| Approach: | They propose a generation framework that conducts dynamic planning of content while generating the output based on a novel design of mixed language models. |
| Outcome: | The proposed model outperforms competing models on argument generation and writing articles using New York Times’ Opinion section. |
Controllable Open-ended Question Generation with A New Question Type Ontology (2021.acl-long)
Copied to clipboard
| Challenge: | Existing question types are limited to generating multiple-sense questions . we present a question type-aware question generation framework to generate open-ended questions based on multiple-phrase questions - a task that is less explored . |
| Approach: | They propose a question type-aware question generation framework which predicts question focuses and produces the question. |
| Outcome: | The proposed model improves question quality over competitive comparisons on large-scale datasets. |
BERTGen: Multi-task Generation through BERT (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work in unsupervised and self-supervised pre-training has revolutionised the field of natural language understanding (NLU). |
| Approach: | They propose to use multimodal and multilingual pre-trained models to extend BERT by fusing them together for language generation tasks. |
| Outcome: | The proposed model outperforms baseline models in image captioning, machine translation and multimodal machine translation tasks and is competitive with supervised counterparts. |
Selective Knowledge Distillation for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Neural Machine Translation models achieve state-of-the-art performance on many translation benchmarks. |
| Approach: | They propose a protocol that analyzes different impacts of samples by comparing various samples’ partitions. |
| Outcome: | The proposed methods yield up to +1.28 and +0.89 BLEU points improvements over the Transformer baseline, respectively. |
Measuring and Increasing Context Usage in Context-Aware Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work in neural machine translation has demonstrated the necessity and feasibility of using inter-sentential context, but it is often not clear how much they actually utilize it at translation time. |
| Approach: | They propose a conditional cross-mutual information metric to quantify usage of context by model architectures that can use it at translation time. |
| Outcome: | The proposed method increases context usage and improves translation quality according to BLEU and COMET metrics. |
Beyond Offline Mapping: Learning Cross-lingual Word Embeddings through Context Anchoring (2021.acl-long)
Copied to clipboard
| Challenge: | Recent research on cross-lingual word embeddings has been dominated by unsupervised mapping approaches that align monolingual embedders. |
| Approach: | They propose an unsupervised mapping approach that fixes fixed embeddings and learns new ones for the source language that are aligned with them. |
| Outcome: | The proposed method outperforms conventional mapping methods on bilingual lexicon induction and obtains competitive results in the downstream XNLI task. |
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web (2021.acl-long)
Copied to clipboard
| Challenge: | Using a curated common crawl corpus, we were able to mine 10.8 billion parallel sentences out of which only 2.9 billions are aligned with English. |
| Approach: | They use 32 snapshots of a curated common crawl corpus totaling 71 billion unique sentences to mine 10.8 billion parallel sentences out of which only 2.9 billions are aligned with English. |
| Outcome: | The proposed system outperforms the best single systems on the WMT’19 test set for English-German/Russian/Chinese and outperformed the best submission at the 2020 WAT workshop. |
Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models have achieved notable improvements in various NLP tasks. |
| Approach: | They propose a Length-Adaptive Transformer that can be used for various inference scenarios after one-shot training. |
| Outcome: | The proposed model can be used for various inference scenarios after one-shot training. |
GhostBERT: Generate More Features with Cheap Operations for BERT (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that some parameters in pre-trained language models can be pruned away without severe accuracy degradation. |
| Approach: | They propose a method which generates more features with very cheap operations from the remaining features and can be applied to unpruned BERT models to enhance their performance. |
| Outcome: | Empirical results on the GLUE benchmark on three backbone models (i.e., BERT, RoBERTa and ELECTRA) verify the efficacy of the proposed method. |
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization (2021.acl-long)
Copied to clipboard
Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao, Weizhu Chen
| Challenge: | 'lottery tickets' can be trained to match the performance of a full model . subnetwork training can also outperform random sampled subnetworks of the same size . |
| Approach: | They propose to train a subnetwork of 'lottery tickets' to match the full model's performance. |
| Outcome: | The proposed model outperforms subnetworks of the same size in a phase transition phenomenon . the proposed model improves single task fine-tuning by 0.9 points on BERT-base and 1.0 points on GLUE large . |
A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representations (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning disentangled representations of textual data are difficult to implement and suffer from the degeneracy of other losses in multi-class scenarios. |
| Approach: | They propose a variational upper bound to the mutual information between an attribute and the latent code of an encoder that controls the approximation error. |
| Outcome: | The proposed method is superior on fair classification and on textual style transfer tasks. |
Determinantal Beam Search (2021.acl-long)
Copied to clipboard
| Challenge: | a new beam search approach allows for a diverse subset selection process . standard beam search does not encode interactions between candidates . |
| Approach: | They propose a beam search reformulation that casts subset selection as the subdeterminant optimization problem. |
| Outcome: | The proposed method offers competitive performance against other diverse set generation strategies while providing a more general approach to optimizing for diversity. |
Multi-hop Graph Convolutional Network with High-order Chebyshev Approximation for Text Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing single-hop graph reasoning in Graph convolutional networks may miss some important non-consecutive dependencies. |
| Approach: | They propose a graph convolutional network with the high-order dynamic Chebyshev approximation which augments multi-hop graph reasoning by fusing messages aggregated from direct and long-term dependencies into one convolutionalist layer. |
| Outcome: | The proposed model improves on four transductive and inductive NLP tasks and the ablation of the existing model. |
Accelerating Text Communication via Abbreviated Sentence Input (2021.acl-long)
Copied to clipboard
| Challenge: | Skipping spaces or other characters may be able to speed input and reduce a user’s physical input effort. |
| Approach: | They designed a neural language model to expand noisy abbreviated input where users often omit spaces and mid-word vowels. |
| Outcome: | The proposed recognizer can expand noisy abbreviated input even if a third of characters is omitted. |
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates (2021.acl-long)
Copied to clipboard
| Challenge: | Using negative flips, we quantify, reduce and analyze regression errors in deep neural networks. |
| Approach: | They propose to quantify, reduce and analyze regression errors in NLP models by negative flips. |
| Outcome: | The proposed model update regression has a prevalent presence across tasks in the GLUE benchmark. |
Detecting Propaganda Techniques in Memes (2021.acl-long)
Copied to clipboard
Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, Giovanni Da San Martino
| Challenge: | Propaganda can be defined as a form of communication that aims to influence opinions or the actions of people towards a specific goal. |
| Approach: | They propose to detect the type of propaganda techniques used in memes by annotating them with 22 techniques. |
| Outcome: | The proposed model identifies 22 propaganda techniques in memes, which can appear in text, image or both . |
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that adversarial data collection (ADC) models perform better on other adversarially collected data but are liable under plausible domain shifts. |
| Approach: | They conduct a large-scale controlled study on question answering by assigning workers at random to compose questions either adversarially (with a model in the loop) or in the standard fashion (without a modeling). |
| Outcome: | The proposed model performs better on other adversarial datasets but worse on diverse collection of out-of-domain evaluation sets. |
Learning Dense Representations of Phrases at Scale (2021.acl-long)
Copied to clipboard
| Challenge: | Existing phrase retrieval models rely on sparse representations and still underperform retriever-reader approaches. |
| Approach: | They propose a method to learn phrase representations from reading comprehension tasks using negative sampling methods. |
| Outcome: | The proposed model improves over previous models by 15%-25% absolute accuracy and matches the performance of state-of-the-art retrieval models. |
End-to-End Training of Neural Retrievers for Open-Domain Question Answering (2021.acl-long)
Copied to clipboard
Devendra Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro
| Challenge: | Recent work on training neural retrievers for open-domain question answering (OpenQA) has employed both supervised and unsupervised methods. |
| Approach: | They propose an approach of unsupervised pre-training with the Inverse Cloze Task and masked salient spans followed by supervised finetuning using question-context pairs. |
| Outcome: | The proposed approach outperforms models like REALM and RAG in retrieval accuracy and answer extraction. |
Question Answering Over Temporal Knowledge Graphs (2021.acl-long)
Copied to clipboard
| Challenge: | Temporal Knowledge Graphs (Temporal KGs) provide temporal scopes (start and end times) on each edge in the Knowledge . Lack of broad coverage datasets has been limiting progress in this area . |
| Approach: | They propose a transformer-based solution that exploits recent advances in Temporal Knowledge Graph embeddings and achieves an increase of 120% in accuracy over the next best performing method. |
| Outcome: | The proposed solution improves on the only known dataset by 340x . it increases accuracy by 120% over the baseline solution . |
Language Model Augmented Relevance Score (2021.acl-long)
Copied to clipboard
| Challenge: | Existing metrics that compare the candidate with the human reference do not consider the context, resulting in poor correlation with human judgements. |
| Approach: | They propose a language model-aware metric that augments the human reference while considering the context to provide evaluation scores that correlate highly with human judgements. |
| Outcome: | The proposed metric achieves higher correlation with human reference judgements and differentiates well-formed candidates from adversarial samples to a larger degree. |
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts (2021.acl-long)
Copied to clipboard
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, Yejin Choi
| Challenge: | Decoding-time Experts is a decoding- time method for controlled text generation . it combines a pretrained language model with "expert" LMs and/or "anti-expert" experts . |
| Approach: | They propose a decoding-time method that combines a pretrained language model with "expert" LMs and/or "anti-expert" experts to generate controlled text. |
| Outcome: | The proposed method outperforms existing controllable generation methods on automatic and human evaluations. |
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing counterfactual generation methods rely on manual labor to create very few counterf actuals or only instantiate limited types of perturbations such as paraphrases or word substitutions. |
| Approach: | They propose a general-purpose counterfactual generator that allows for control over perturbation types and locations. |
| Outcome: | The proposed generator produces diverse sets of realistic counterfactuals that are useful in various applications. |
Metaphor Generation with Conceptual Mappings (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for metaphor generation lack conceptualization of meaning of the metaphors . recent neural models have led to advances in many areas of natural language generation . |
| Approach: | They propose to encode conceptual mappings between cognitive domains to generate metaphoric expressions by embedding verbs into a literal expression and deriving source/target pairs to train a controlled seq-to-seq generation model. |
| Outcome: | The proposed method outperforms existing models in automatic and human evaluations for basic metaphoricity and conceptual metaphor presence. |
Learning Latent Structures for Cross Action Phrase Relations in Wet Lab Protocols (2021.acl-long)
Copied to clipboard
| Challenge: | Lack of careful documentation has led to reproducibility crisis in biosciences . wet laboratory protocols are written in natural language and are unclear, composed of disconnected and distant parts and built upon implicit information that were referenced earlier or omitted entirely. |
| Approach: | They propose a model that extracts all action relationships across multiple sentences to generate a MSTG for a given protocol . they propose extending annotations in the WLP corpus for inter-sentence relations and implicit arguments to improve the model's temporal and causal relations. |
| Outcome: | The proposed model achieves an F1 score of 54.53% for temporal and causal relations in protocols from the corpus WLP-MSTG. |
Multimodal Multi-Speaker Merger & Acquisition Financial Modeling: A New Task, Dataset, and Neural Baselines (2021.acl-long)
Copied to clipboard
| Challenge: | Merger and acquisition (M&A) calls provide key insights into claims made by company executives about restructuring of financial firms. |
| Approach: | They propose a baseline architecture that leverages multimodal multi-speaker input to forecast financial risk associated with M&A calls. |
| Outcome: | The proposed model performs marginally better than existing models based on BERT inputs . the proposed model is expected to be validated by the end of the year . |
Mid-Air Hand Gestures for Post-Editing of Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | In a well-connected world, translation is of everincreasing importance. |
| Approach: | They propose to use mid-air hand gestures in combination with the keyboard for editing in machine translation and post-editing workflows to improve quality. |
| Outcome: | The proposed prototype supports mid-air hand gestures for cursor placement, text selection, deletion, and reordering. |
Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for solving geometric problems are either small in scale or not publicly available. |
| Approach: | They propose a large-scale benchmark for geometric problem solving using formal language and symbolic reasoning. |
| Outcome: | The proposed approach parses geometry problems into formal language and performs symbolic reasoning step by step. |
Joint Verification and Reranking for Open Fact Checking Over Tables (2021.acl-long)
Copied to clipboard
Michael Sejr Schlichtkrull, Vladimir Karpukhin, Barlas Oguz, Mike Lewis, Wen-tau Yih, Sebastian Riedel
| Challenge: | Existing research into structured data has focused on textual data and the closed-domain setting is not reflective of real-world fact checking tasks. |
| Approach: | They propose a joint reranking-and-verification model which fuses evidence documents in the verification component and a heuristic retrieval baseline. |
| Outcome: | The proposed model achieves comparable performance to the closed-domain state-of-the-art on the TabFact dataset and significantly improves over a heuristic retrieval baseline. |
Evaluation of Thematic Coherence in Microblogs (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work on grouping together views about tweets expressing opinions about the same entities has been criticized for their lack of thematic coherence. |
| Approach: | They propose to use a corpus of microblogs representing opinions about the same topics within the same time window to evaluate thematic coherence. |
| Outcome: | The proposed method outperforms surface level metrics, topic model coherence and text generation metrics (TGMs) but is not as reliable as TGMs due to being less sensitive to time windows. |
Neural semi-Markov CRF for Monolingual Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that our proposed model outperforms all previous approaches for monolingual word alignment. |
| Approach: | They propose a neural semi-Markov CRF alignment model which unifies word and phrase alignments through variable-length spans. |
| Outcome: | The proposed model outperforms existing models on in-domain and out-of-domain evaluations and a QA-based benchmark with human annotations. |
Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies (2021.acl-long)
Copied to clipboard
| Challenge: | Existing tools to interpret privacy policies have been used to understand them but there is a lack of large privacy policy corpora to simplify the process. |
| Approach: | They propose to use a corpus of 1,005,380 English language privacy policies collected from the web to create semi-supervised and unsupervised models to interpret and simplify privacy policies. |
| Outcome: | The proposed model outperforms all other publicly available privacy policy corpora and is ten times larger than the next largest public collection of privacy policies combined. |
The statistical advantage of automatic NLG metrics at the system level (2021.acl-long)
Copied to clipboard
| Challenge: | Statistically, humans are unbiased, high variance estimators, while metrics are biased, low variance estimator. |
| Approach: | They compare automatic metrics to humans and a derived, perfect segment-level annotator by applying a bias-variance-noise decomposition to adjust the error to a noise-free, infinite test set setting. |
| Outcome: | The proposed method outperforms humans and a derived, perfect segment-level annotator in two settings. |
Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion (2021.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for Knowledge Graph Completion (KGC) are unsatisfactory . |
| Approach: | They propose to use rule-guided train/test generation instead of conventional random split to ensure that each testing sample is predictable with supportive data in the training set. |
| Outcome: | The proposed model improves on existing benchmarks in inferential ability, assumptions, and patterns. |
ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining (2021.acl-long)
Copied to clipboard
| Challenge: | Abstractive text summarization has primarily focused on modeling news articles . lack of standardized datasets for summarizing online conversations is a major problem . |
| Approach: | They propose to crowdsource four new datasets for summarizing online conversations . they incorporate argument mining through graph construction to directly model issues, viewpoints, and assertions present in a conversation. |
| Outcome: | The proposed models are compared against widely-used conversation summarization datasets and show comparable or improved results. |
Improving Factual Consistency of Abstractive Summarization via Question Answering (2021.acl-long)
Copied to clipboard
Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, Bing Xiang
| Challenge: | Recent studies show that about 30% of summaries generated by neural text summarization suffer from fact fabrication. |
| Approach: | They propose an automatic evaluation metric to measure factual consistency and a learning algorithm that maximizes the metric during model training. |
| Outcome: | The proposed method improves factual consistency and overall quality of summarization models. |
EmailSum: Abstractive Email Thread Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Recent years have brought about interest in the task of summarizing conversation threads. |
| Approach: | They develop an email thread summarization dataset that contains human-annotated short and long email threads over a wide variety of topics. |
| Outcome: | The proposed dataset contains human-annotated short (30 words) and long (100 words) summaries of 2,549 email threads over a wide variety of topics. |
Cross-Lingual Abstractive Summarization with Limited Parallel Resources (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual summarization use limited available cross-linguistic resources. |
| Approach: | They propose a multi-task framework for cross-lingual abstractive summarization that uses a single decoder to generate monolingual and cross-linguistic summaries. |
| Outcome: | Experiments on two CLS datasets show that the proposed model outperforms baseline models in low-resource and full-dataset scenarios. |
Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution (2021.acl-long)
Copied to clipboard
| Challenge: | Abstractive summarization models have made great strides in recent years, but little is known about how they actually form summaries and how to understand where their decisions come from. |
| Approach: | They propose a two-step method to interpret summarization model decisions by categorizing each decoder decision into one of several generation modes. |
| Outcome: | The proposed method can identify phrases the summarization model has memorized and determine where in the training pipeline this memorization happened, and study complex generation phenomena on a per-instance basis. |
Learning Prototypical Functions for Physical Artifacts (2021.acl-long)
Copied to clipboard
| Challenge: | a new task is designed to learn the prototypical uses of human-made physical objects . human beings are creative, and they create things for a reason . humans often infer that the object will be used in the most prototypical way unless told otherwise . |
| Approach: | They propose a task to learn the prototypical uses for human-made physical objects . they use frames from FrameNet to represent a set of common functions for objects based on their prototypical function . |
| Outcome: | The proposed task uses masked patterns to model prototypical uses for objects . the proposed model predicts the prototypical functions of objects and can be used to make models . |
Verb Knowledge Injection for Multilingual Event Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that pretrainers implicitly extract a non-negligible amount of linguistic knowledge from text corpora in an unsupervised fashion. |
| Approach: | They propose to inject explicit verb knowledge into dedicated adapter modules to complement the linguistic knowledge obtained during LM-pretraining. |
| Outcome: | The proposed model improves in English event extraction tasks, while injecting verb knowledge improves other languages. |
Dynamic Contextualized Word Embeddings (2021.acl-long)
Copied to clipboard
| Challenge: | Static word embeddings that represent words by a single vector cannot capture word meaning in different linguistic and extralinguistic contexts. |
| Approach: | They propose dynamic contextualized word embeddings that represent words as a function of linguistic and extralinguistic contexts. |
| Outcome: | The proposed model models time and social space jointly, making them attractive for NLP tasks involving semantic variability. |
Lexical Semantic Change Discovery (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to Lexical Semantic Change Detection are limited. |
| Approach: | They propose a shift from change detection to change discovery by fine-tuning a type-based and a token-based approach on recently published German data. |
| Outcome: | The proposed models can be applied to discover new words undergoing meaning change from the full corpus vocabulary. |
The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity (2021.acl-long)
Copied to clipboard
| Challenge: | We analyze 2,500 phrasings related to the intent of “Are you a robot?” and 2,500 adversarially selected utterances to determine whether systems are non-human. |
| Approach: | They analyze 2,500 phrasings related to the intent of "Are you a robot?" and 2,500 adversarially selected utterances to determine whether systems are non-human. |
| Outcome: | The proposed model and two systems fail to confirm non-human intent, and the proposed model is complex. |
Using Meta-Knowledge Mined from Identifiers to Improve Intent Recognition in Conversational Systems (2021.acl-long)
Copied to clipboard
Claudio Pinhanez, Paulo Cavalin, Victor Henrique Alves Ribeiro, Ana Appel, Heloisa Candello, Julio Nogima, Mauro Pichiliani, Melina Guerra, Maira de Bayser, Gabriel Malfatti, Henrique Ferreira
| Challenge: | a recent study shows that meta-knowledge embedded in intent identifiers improves intent recognition in conversational systems . a meta-learning approach is used to classify sentences into discrete sets of classes . classification is a key part of professional conversational system implementations . |
| Approach: | They use meta-knowledge embedded in intent identifiers to improve intent recognition . authors found that meta-knowledge improved accuracy in conversational systems . |
| Outcome: | The meta-knowledge enabled improved intent recognition in conversational systems . the meta-learning improved the false acceptance rate in two thirds of the chatbots . |
Space Efficient Context Encoding for Non-Task-Oriented Dialogue Generation with Graph Attention Transformer (2021.acl-long)
Copied to clipboard
| Challenge: | Recent Transformer-based models aim to integrate fixed background context into non-task-oriented dialogue systems, but the context length is fixed in these architectures, which restricts how much background or dialogue context can be kept. |
| Approach: | They propose a more concise encoding for background context structured in the form of knowledge graphs by expressing the graph connections through restrictions on the attention weights. |
| Outcome: | The proposed architecture reduces space requirements without negative effects on the precision of reproduction of knowledge and perceived consistency. |
DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies on ERC lack the ability to extract and integrate emotional clues from the conversational context. |
| Approach: | They propose a new model that uses multi-turn reasoning modules to extract and integrate emotional clues from conversational context. |
| Outcome: | The proposed model outperforms existing models on three public benchmark datasets and is highly effective and superior to existing models. |
Cross-replication Reliability - An Empirical Approach to Interpreting Inter-rater Reliability (2021.acl-long)
Copied to clipboard
| Challenge: | Respectable journals typically require reporting quantitative evidence for inter-rater reliability (IRR) of the data. |
| Approach: | They propose to benchmark IRR against baseline measures in a replication dataset and use Cohen's (1960) kappa to measure inter-rater reliability. |
| Outcome: | The proposed framework can be used to measure the quality of crowdsourced datasets. |
TIMEDIAL: Temporal Commonsense Reasoning in Dialog (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on pre-trained language models for dialog reasoning fail to understand context correctly. |
| Approach: | They propose to use a crowd-sourced English task and a time-based task to test models' temporal reasoning abilities in dialogs. |
| Outcome: | The proposed task and crowd-sourced English challenge set show that even the best performing models struggle on this task compared to humans. |
RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English) (2021.acl-long)
Copied to clipboard
| Challenge: | lexical ambiguity is a problem for NLP, but few tasks evaluate its impact on human intuitions. |
| Approach: | They propose to use contextualized word embeddings to evaluate word meaning . they use a dataset of human relatedness judgments and human estimates of sense dominance . |
| Outcome: | The proposed model matches human intuitions with contextualized embeddings on 112 ambiguous words in context with 672 sentence pairs. |
ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs) are expensive and limited in inference time . a new benchmark for multi-dialectal Arabic language understanding evaluation is developed . |
| Approach: | They introduce two powerful deep bidirectional transformer-based models, ARBERT and MARBERT . they also introduce ARLUE, a new benchmark for multi-dialectal Arabic language understanding evaluation . |
| Outcome: | The proposed models outperform monolingual models with larger vocabulary and larger datasets in Arabic language understanding evaluation. |
Improving Paraphrase Detection with the Adversarial Paraphrasing Task (2021.acl-long)
Copied to clipboard
| Challenge: | a new adversarial method of paraphrase identification is being used to identify paraphrases based on word overlap and syntax . authors propose a dataset that generates semantically equivalent but lexically and syntactically disparate paraphrase pairs . |
| Approach: | They propose an adversarial method for paraphrase identification that uses word overlap and syntax to identify paraphrases. |
| Outcome: | The proposed method improves paraphrase detection accuracy and speed of generation of datasets. |
ADEPT: An Adjective-Dependent Plausibility Task (2021.acl-long)
Copied to clipboard
| Challenge: | ADEPT is a large-scale semantic plausibility task that requires a significant degree of world knowledge and common-sense reasoning. |
| Approach: | They propose a large-scale semantic plausibility task that pairs 16 thousand sentences with slightly modified versions obtained by adding an adjective to a noun. |
| Outcome: | The proposed task is easier for humans (85% accuracy), but more difficult for transformer-based models (71% accuracy). |
ReadOnce Transformers: Reusable Representations of Text for Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | ReadOnce Transformers is a task-independent, task-dependent, and compressed representation of text. |
| Approach: | They propose a transformer-based model that can build an information-capturing, task-independent, and compressed representation of text. |
| Outcome: | The proposed model can build an information-capturing, task-independent, and compressed representation of text. |
Conditional Generation of Temporally-ordered Event Sequences (2021.acl-long)
Copied to clipboard
| Challenge: | a new model of narrative schema knowledge does not capture the temporal relationships between events . a temporal order model is able to unscramble event sequences without access to labeled temporal training data . |
| Approach: | They propose a temporal order-based model that can be flexibly applied to different tasks . they use a BART-based conditional generation model that captures temporal co-occurrence . |
| Outcome: | The proposed model outperforms existing models on temporal ordering and event infilling tasks. |
Hate Speech Detection Based on Sentiment Knowledge Sharing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for hate speech detection are stereotyped and biased . et al., a paper examining the effectiveness of multitask learning in hate speech recognition tasks . |
| Approach: | They propose a hate speech detection framework based on sentiment knowledge sharing . they extract affective features of the target sentence and use sentiment features from external resources . |
| Outcome: | The proposed model can detect hate speech over two public datasets. |
Transition-based Bubble Parsing: Improvements on Coordination Structure Prediction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing bubble representations encoding coordination boundaries and internal relationships are difficult to detect and parse . |
| Approach: | They propose a bubble parser to perform coordination structure identification and dependency-based syntactic analysis simultaneously. |
| Outcome: | The proposed bubble parser beats state-of-the-art approaches on coordination structure prediction . the proposed system is based on a GENIA corpus and a Penn treebank . |
SpanNER: Named Entity Re-/Recognition as Span Prediction (2021.acl-long)
Copied to clipboard
| Challenge: | Recent years have seen the paradigm shift of Named Entity Recognition (NER) systems from sequence labeling to span prediction. |
| Approach: | They experimentally implement 154 named entity recognition models on 11 datasets and show that span prediction can serve as a system combiner to re-recognize named entities from different systems’ outputs. |
| Outcome: | The proposed model can be used to re-recognize named entities from different systems’ outputs. |
StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models that induce grammar structures from data focus on constituency or dependency structures alone. |
| Approach: | They propose a model that can induce dependency and constituency structure at the same time. |
| Outcome: | The proposed model can induce both constituency and dependency structures at the same time. |
Language Embeddings for Typology and Cross-lingual Transfer Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Recent efforts to leverage multilingual datasets highlight potential of multilingual models that can perform well across various languages. |
| Approach: | They propose to generate language representations that capture relationships among languages and evaluate them using WALS and two extrinsic tasks. |
| Outcome: | The proposed model can be leveraged in cross-lingual tasks without parallel data . the proposed model is based on the World Atlas of Language Structures (WALS) and two extrinsic tasks . |
Can Sequence-to-Sequence Models Crack Substitution Ciphers? (2021.acl-long)
Copied to clipboard
| Challenge: | Current methods for deciphering historical ciphers use beam search and a neural language model . but, this approach assumes that the target plaintext language is known . |
| Approach: | They propose an end-to-end multilingual decipherment model that can solve 1:1 substitution ciphers without explicit language identification. |
| Outcome: | The proposed model can decipher text without explicit language identification while still being robust to noise. |
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Prior work treats all types of mismatches between source and target as noise . Consequently, it remains unclear how noisy parallel training samples impact NMT training. |
| Approach: | They propose a divergent-aware NMT framework that uses factors to help NMT recover from the degradation caused by naturally occurring divergences. |
| Outcome: | The proposed framework improves translation quality and model calibration on EN-FR tasks. |
Discriminative Reranking for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | reranking models allow the integration of rich features to select a better output hypothesis within an n-best list or lattice. |
| Approach: | They use discriminative reranking to train a large transformer architecture to train an ranked list of hypotheses. |
| Outcome: | Experiments on four WMT directions show that discriminative reranking improves translation quality. |
Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Currently, language-equipped vision systems such as VizWiz, TapTapSee, BeMyEyes, and CamFind are actively being deployed across a broad spectrum of users. |
| Approach: | They propose to identify collective outliers in active learning methods that are hard and often impossible for models to learn . they also propose to use visual inputs to identify these outlier examples as examples assigned low model confidence and prediction variability during training. |
| Outcome: | The proposed methods outperform random selection on visual question answering tasks. |
All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text (2021.acl-long)
Copied to clipboard
| Challenge: | evaluators distinguish between human- and machine-authored text in three domains without training . evals' accuracy improved up to 55%, but it did not significantly improve across the three domain. |
| Approach: | They examine the role untrained human evaluations play in NLG evaluation and propose ways to improve their evaluations. |
| Outcome: | The evaluators distinguished between human- and machine-authored text at random chance level without training, but their accuracy did not improve across the three domains. |
Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers (2021.acl-long)
Copied to clipboard
| Challenge: | a meta-evaluation of machine translation (MT) has been conducted in 769 research papers . a recent study shows that evaluation practices have changed over the past decade . |
| Approach: | They propose a meta-evaluation method for machine translation that uses BLEU scores to evaluate MT performance. |
| Outcome: | The proposed meta-evaluation of machine translation shows that evaluation practices have changed over the past decade . the authors suggest that the evaluation process should be streamlined and standardized to ensure the validity of the evaluation method . |
Neural Machine Translation with Monolingual Translation Memory (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work has shown that Translation Memory (TM) can boost the performance of Neural Machine Translation (NMT) |
| Approach: | They propose a framework that uses monolingual memory and performs learnable memory retrieval in a cross-lingual manner. |
| Outcome: | The proposed framework outperforms strong TM-augmented NMT baselines using bilingual TM and outperformed existing models in low-resource and domain adaptation scenarios. |
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models can be fine-tuned to produce state-of-the-art results for a wide range of language understanding tasks. |
| Approach: | They propose to analyze fine-tuning through the lens of intrinsic dimension . they show that pre-trained models have a low intrinsic dimension reparameterization . |
| Outcome: | The proposed model can achieve 90% of the full parameter performance levels on MRPC with low data regime. |
UnNatural Language Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained NLU models understand human-like syntax . however, these models are word order invariant, causing them to assign gold labels to permutations . |
| Approach: | They propose to measure the severity of this issue by examining the properties of particular permutations that lead models to be word order invariant. |
| Outcome: | The proposed model is word order invariant, but it's not human-like syntax. |
Including Signed Languages in Natural Language Processing (2021.acl-long)
Copied to clipboard
| Challenge: | Existing research in Sign Language Processing (SLP) rarely explores signed languages . authors urge adoption of an efficient tokenization method and the collection of real-world signed language data . |
| Approach: | They propose to include signed languages as a research area with high social and scientific impact . they review the limitations of current SLP models and identify the open challenges . |
| Outcome: | The proposed model should include signed languages as a research area with high social and scientific impact. |
Vocabulary Learning via Optimal Transport for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Empirical results show that VOLT beats widely-used vocabularies in diverse scenarios, including WMT-14 English-German translation, TED bilingual translation, and TED multilingual translation. |
| Approach: | They propose a token dictionary solution that can be used without trial training to find the best dictionary with a proper size. |
| Outcome: | The proposed solution beats widely-used vocabularies in English-German translation, TED bilingual translation, and TED multilingual translation. |