Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics
Did the Cat Drink the Coffee? Challenging Transformers with Generalized Event Knowledge (2021.starsem-1)
Copied to clipboard
Paolo Pedinotti, Giulia Rambelli, Emmanuele Chersoni, Enrico Santus, Alessandro Lenci, Philippe Blache
| Challenge: | Prior work has explored the ability of computational models to predict word semantic fit with a given predicate. |
| Approach: | They compare Transformers Language Models to SDM to assess their performance . they found that TLMs do not capture important aspects of event knowledge . people can discriminate between typical and atypical events, they say . |
| Outcome: | The proposed models can achieve comparable performance to SDM, but they lack important aspects of event knowledge. |
Can Transformer Language Models Predict Psychometric Properties? (2021.starsem-1)
Copied to clipboard
| Challenge: | Transformer-based language models (LMs) are gaining popularity on many NLP benchmark tasks. |
| Approach: | They use human responses to calculate psychometric properties of test items . they find transformer-based LMs predict psychometric property consistently well . |
| Outcome: | The transformer-based language models are able to predict psychometric properties of test items . the models can predict psychometries well in certain categories but poorly in others . |
Semantic shift in social networks (2021.starsem-1)
Copied to clipboard
| Challenge: | lexical semantic change manifests differently across different communities, according to a new study . social network analysis is a tool of sociolinguists studying variation and change . |
| Approach: | They use distributional methods to quantify lexical semantic change and induce a social network on communities based on interactions between members. |
| Outcome: | The proposed method is based on interactions between members and the community. |
A Study on Using Semantic Word Associations to Predict the Success of a Novel (2021.starsem-1)
Copied to clipboard
Syeda Jannatus Saba, Biddut Sarker Bijoy, Henry Gorelick, Sabir Ismail, Md Saiful Islam, Mohammad Ruhul Amin
| Challenge: | Existing methods for book success prediction are not effective. |
| Approach: | They propose to represent a book as a spectrum of concepts based on the association score between its content embedding and a global embeddment for a set of semantically linked word clusters. |
| Outcome: | The proposed method outperforms the previous methods for book success prediction. |
Recovering Lexically and Semantically Reused Texts (2021.starsem-1)
Copied to clipboard
| Challenge: | Writers often repurpose material from existing texts when composing new documents. |
| Approach: | They propose to use local text reuse detection to detect localized regions of lexically or semantically similar text embedded in otherwise unrelated material. |
| Outcome: | The proposed methods perform better on three LTRD tasks, detecting plagiarism, modeling journalists’ use of press releases, and identifying scientists’ citation of earlier papers. |
Generating Hypothetical Events for Abductive Inference (2021.starsem-1)
Copied to clipboard
| Challenge: | Abductive reasoning is inference to the best explanation given an incomplete set of observations about everyday situations. |
| Approach: | They propose a model that generates what could happen next from a hypothetical scenario and then proposes the most plausible explanation from varying hypothetical scenarios. |
| Outcome: | The proposed model improves over previous vanilla pre-trained models fine-tuned on Abductive NLI. |
NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning (2021.starsem-1)
Copied to clipboard
| Challenge: | Currently, symbolic and deep learning approaches to NLI are receiving less attention. |
| Approach: | They propose a symbolic-based inference framework that integrates symbolic reasoning and semantic formalism to solve NLI tasks. |
| Outcome: | The proposed framework improves accuracy on the NLI task and on the SICK and MED datasets. |
Teach the Rules, Provide the Facts: Targeted Relational-knowledge Enhancement for Textual Inference (2021.starsem-1)
Copied to clipboard
| Challenge: | InferBERT is a method to enhance transformer-based inference models with relevant relational knowledge. |
| Approach: | They propose to enhance transformer-based inference models with relevant relational knowledge by injecting relevant facts at test time into the model. |
| Outcome: | The proposed method outperforms existing models on the challenge datasets while outperforming existing models. |
ParsFEVER: a Dataset for Farsi Fact Extraction and Verification (2021.starsem-1)
Copied to clipboard
Majid Zarharan, Mahsa Ghaderan, Amin Pourdabiri, Zahra Sayedi, Behrouz Minaei-Bidgoli, Sauleh Eetemadi, Mohammad Taher Pilehvar
| Challenge: | Existing methods for fact-checking and verification require large amounts of annotated data, but this is limited to low-resource languages. |
| Approach: | They present a first publicly available Farsi dataset for fact extraction and verification . they use the construction procedure of the standard English dataset for the task . |
| Outcome: | The proposed dataset improves on the standard English dataset and is available on github. |
BiQuAD: Towards QA based on deeper text understanding (2021.starsem-1)
Copied to clipboard
| Challenge: | Recent question answering and machine reading benchmarks require systems to pinpoint the span of the answer to a given text. |
| Approach: | They propose a dataset that requires deeper comprehension to answer questions extractively and deductively. |
| Outcome: | The proposed dataset outperforms existing benchmarks on extractive and deductive questions. |
Evaluating Universal Dependency Parser Recovery of Predicate Argument Structure via CompChain Analysis (2021.starsem-1)
Copied to clipboard
| Challenge: | Compchains are a categorization of the hierarchy of predicate dependency relations present within a UD parse. |
| Approach: | They introduce compchains, a categorization of the hierarchy of predicate dependency relations present within a UD parse. |
| Outcome: | The proposed model performs poorly on sentences with predicate-argument structure with more than one level of embedding. |
InFillmore: Frame-Guided Language Generation with Bidirectional Context (2021.starsem-1)
Copied to clipboard
| Challenge: | Existing methods for automatic story plan generation use coarse-to-fine representations of semantic content. |
| Approach: | They propose a structured extension to bidirectional-context conditional language generation, or "infilling" they propose evocative frame annotations and a method for frame-guided generation that leverages frame semantic lexical units. |
| Outcome: | The proposed method allows for explicit manipulation of intended infill semantics with minimal loss of distinguishability from human-generated text. |
Realistic Evaluation Principles for Cross-document Coreference Resolution (2021.starsem-1)
Copied to clipboard
| Challenge: | Using permissive evaluation protocols, cross-document coreference resolution models produce inflated results. |
| Approach: | They propose to decouple evaluation of mention detection from coreference linking . they argue that models should not exploit the synthetic topic structure of the standard ECB+ dataset . |
| Outcome: | The proposed evaluation principles yield lower results than previous lenient evaluation methods. |
Disentangling Online Chats with DAG-structured LSTMs (2021.starsem-1)
Copied to clipboard
| Challenge: | a number of messaging systems allow fast and synchronous textual communication but they often have a more complicated structure in which independent sub-conversations are interwoven with one another. |
| Approach: | They propose a model that can handle directed acyclic dependencies and integrates structured information into the conversation. |
| Outcome: | The proposed model achieves state-of-the-art status on the task of recovering reply-to relations and is competitive on other disentanglement metrics. |
Toward Diverse Precondition Generation (2021.starsem-1)
Copied to clipboard
| Challenge: | a typical goal for language understanding is to logically connect the events of a discourse, but connective events are not described due to their commonsense nature. |
| Approach: | They propose a system that generates unique and diverse preconditions by using an event sampler, candidate generator, and post-processor. |
| Outcome: | The proposed system can generate unique and diverse preconditions without training on diverse examples. |
One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets (2021.starsem-1)
Copied to clipboard
| Challenge: | Existing semantic parsing datasets lack a single standard for meaning representations . lack of a standard led to the creation of plethora of datasets requiring expert annotators . |
| Approach: | They propose to use multi-task learning to unify different datasets and train a single model for them. |
| Outcome: | The proposed architectures yield better parsing accuracies and composition generalization than single-task models. |
Multilingual Neural Semantic Parsing for Low-Resourced Languages (2021.starsem-1)
Copied to clipboard
| Challenge: | a large amount of training data is needed to understand multilingual semantic parsing models. |
| Approach: | They propose to use machine translation to bootstrap multilingual training data from English data. |
| Outcome: | The proposed model outperforms existing models on human-written sentences and the state-of-the-art models on the public NLMaps dataset. |
Script Parsing with Hierarchical Sequence Modelling (2021.starsem-1)
Copied to clipboard
| Challenge: | Script knowledge is a category of commonsense knowledge that describes how people conduct everyday activities sequentially. |
| Approach: | They propose a hierarchical sequence model and transfer learning to do script parsing with a sequence model that accurately tags script participants. |
| Outcome: | The proposed model improves state of the art of event parsing by over 16 points F-score and, for the first time, accurately tags script participants. |
Incorporating EDS Graph for AMR Parsing (2021.starsem-1)
Copied to clipboard
| Challenge: | AMR is abstract and conceptual, while EDS is low level, closer to the lexical structures of the given sentences. |
| Approach: | They propose to add EDS graphs as additional semantic features to AMR parsers by adding transition-based parser to add LSTM layer and GCN layer. |
| Outcome: | The proposed parser adds EDS graphs as additional semantic features to boost performance . Currently the parsing accuracies for AMR are in low 80s, while they can be improved by adding more information from EDS. |
Dependency Patterns of Complex Sentences and Semantic Disambiguation for Abstract Meaning Representation Parsing (2021.starsem-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a sentence-level meaning representation based on predicate argument structure. |
| Approach: | They propose to use a dictionary to capture the structure of complex sentences . they train models on data derived from AMR and Wikipedia corpus . |
| Outcome: | The proposed model will be made public and the proposed patterns will be validated. |
Neural Metaphor Detection with Visibility Embeddings (2021.starsem-1)
Copied to clipboard
| Challenge: | Using Visibility Embeddings, sequence metaphor labeling is improved . many metaphors involve noticeable differences between the abstractness of words constructing them . |
| Approach: | They propose to concatenate sequence metaphor labeling with BiLSTM inputs to obtain improvements . they use visibility embeddings to provide a good estimation of a word's concreteness . |
| Outcome: | The proposed method improves the problem of sequence metaphor labeling with BERT . it allows for consistent and significant improvements at almost no cost . |
Inducing Language-Agnostic Multilingual Representations (2021.starsem-1)
Copied to clipboard
| Challenge: | Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world, but they currently require large pretraining corpora or access to typologically similar languages. |
| Approach: | They propose to remove language identity signals from multilingual embeddings by re-aligning vector spaces of target languages to a pivot source language and removing language-specific means and variances. |
| Outcome: | The proposed approaches reduce cross-lingual transfer gap by 8.9 points (m-BERT) and 18.2 points (XLM-R) on average across all tasks and languages. |
Modeling Sense Structure in Word Usage Graphs with the Weighted Stochastic Block Model (2021.starsem-1)
Copied to clipboard
| Challenge: | Word Usage Graphs capture fine-grained semantic proximity distinctions between word uses. |
| Approach: | They propose to model word use Graphs using a Bayesian weighted stochastic block model and a probabilistic weightes-based model to capture fine-grained semantic proximity distinctions between word uses. |
| Outcome: | The proposed model is compared with existing models and is empirically most adequate. |
Compound or Term Features? Analyzing Salience in Predicting the Difficulty of German Noun Compounds across Domains (2021.starsem-1)
Copied to clipboard
| Challenge: | Using domain-specific vocabulary, it is important to analyse domain-related characteristics to improve the communication between lay people and experts. |
| Approach: | They focus on the interaction of compound-based lexical features (such as frequency and productivity) and terminology-based features (contrasting domain-specific and general language) across word representations and classifiers. |
| Outcome: | The proposed model shows that the interaction of compound-based lexical features and terminology-based features across word representations and classifiers is important for a broad binary distinction into ‘easy’ vs. ‘difficult’ general-language compound frequency is sufficient, but for . a more fine-grained four-class distinction it is crucial to include contrastive termhood features and compound and constituent features. |
Spurious Correlations in Cross-Topic Argument Mining (2021.starsem-1)
Copied to clipboard
| Challenge: | Recent work in cross-topic argument mining attempts to learn models that generalise across topics rather than relying on within-topic spurious correlations. |
| Approach: | They propose to use linear approximations of decision boundaries and manual feature grouping to learn models that generalise across topics rather than relying on within-topic spurious correlations. |
| Outcome: | The proposed model generalise across topics rather than relying on spurious correlations. |
Learning Embeddings for Rare Words Leveraging Internet Search Engine and Spatial Location Relationships (2021.starsem-1)
Copied to clipboard
| Challenge: | Existing word embedding techniques depend heavily on the frequencies of words in the corpus, and fail to provide reliable representations for rare words. |
| Approach: | They propose an algorithm to learn embeddings for rare words based on an Internet search engine and the spatial location relationships. |
| Outcome: | The proposed algorithm can learn more accurate representations for a wider range of vocabulary. |
Overcoming Poor Word Embeddings with Word Definitions (2021.starsem-1)
Copied to clipboard
| Challenge: | Modern natural language understanding models depend on pretrained word embeddings, but applications may need to reason about words that were never or rarely seen during pretraining. |
| Approach: | They propose a method to improve a model's ability to learn to use definitions in natural text to overcome this handicap. |
| Outcome: | The proposed model learns to use definitions in natural text to overcome this handicap. |
Denoising Word Embeddings by Averaging in a Shared Space (2021.starsem-1)
Copied to clipboard
| Challenge: | Continuous word embeddings have been introduced several years ago as a standard building block for NLP tasks. |
| Approach: | They propose a method of fusing word embeddings that were trained on the same corpus but with different initializations. |
| Outcome: | The proposed method improves word embeddings on a range of tasks. |
Evaluating a Joint Training Approach for Learning Cross-lingual Embeddings with Sub-word Information without Parallel Corpora on Lower-resource Languages (2021.starsem-1)
Copied to clipboard
| Challenge: | Cross-lingual word embeddings provide a way for information to be transferred between languages. |
| Approach: | They propose a joint training approach that incorporates sub-word information during training to learn cross-lingual embeddings. |
| Outcome: | The proposed method improves bilingual lexicon induction, especially for out-of-vocabulary words (OOVs) it is able to represent out- of-vocal words (OVs) and is more isomorphic than previous methods. |
Adversarial Training for Machine Reading Comprehension with Virtual Embeddings (2021.starsem-1)
Copied to clipboard
| Challenge: | Neural networks are vulnerable to adversarial examples that have been mixed with certain perturbations. |
| Approach: | They propose a novel adversarial training method that perturbs the embedding matrix instead of word vectors to differentiate the roles of passages and questions. |
| Outcome: | The proposed method is effective universally and further improves the performance of MRC tasks. |