Incorporating Contextual and Syntactic Structures Improves Semantic Similarity Modeling (D19-1)
Copied to clipboard
| Challenge: | Semantic similarity modeling is central to many NLP problems such as question answering. |
| Approach: | They propose a pairwise word interaction model with syntactic structure priors to explore their effectiveness. |
| Outcome: | Extensive evaluations on eight benchmark datasets show that incorporating structural information improves over strong baselines. |
Similar Papers
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach (2020.acl-main)
Copied to clipboard
| Challenge: | incorporating syntactic structure into language models has been a challenge since the 1990s. |
| Approach: | They propose to use syntactic information to integrate syntastic structure into neural language models by providing ground truth parse trees as additional training signals. |
| Outcome: | The proposed model achieves lower perplexity and better quality when ground truth parse trees are provided as training signals. |
Paths to Relation Extraction through Semantic Structure (2021.findings-acl)
Copied to clipboard
| Challenge: | Syntactic and semantic structure directly reflect relations expressed by the text at hand and are therefore very useful for relation extraction (RE) |
| Approach: | They propose two methods for integrating broad-coverage semantic structure into supervised RE models by encoding semantic DAGs. |
| Outcome: | The proposed methods overshadow the use of syntactic integrations in RE . they reduce UCCA into a bilexical structure and encode semantic DAG structures . |
Similarity Analysis of Contextual Word Representation Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing and novel similarity measures are used to analyze contextual word representations . different architectures have rather similar representations, but different individual neurons. |
| Approach: | They propose a method to analyze contextual word representation models using similarity analysis. |
| Outcome: | The proposed approach can be used to analyze model similarity without external annotations. |
Latent Structure Models for Natural Language Processing (P19-4)
Copied to clipboard
| Challenge: | Latent structure models are a powerful tool for compositional data modeling and pipelines. |
| Approach: | This tutorial will cover recent advances in discrete latent structure models . it will discuss their motivation, potential, and limitations . |
| Outcome: | This tutorial will cover recent advances in discrete latent structure models . it will discuss their motivation, potential, and limitations . |
Dissecting Contextual Word Embeddings: Architecture and Representation (D18-1)
Copied to clipboard
| Challenge: | Existing work on learning contextual representations has used LSTM-based biLMs, but there is no reason to believe this is effective. |
| Approach: | They propose to use pre-trained bidirectional language models to learn contextual word embeddings for four NLP tasks and to use them to study the effects of architecture on endtask accuracy. |
| Outcome: | The proposed models outperform word embeddings for four NLP tasks and all learn representations that vary with network depth. |
Improving Text Understanding via Deep Syntax-Semantics Communication (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that integrating syntactic tree models with sequential semantic models can bring improved task performance. |
| Approach: | They propose a deep neural communication model between syntax and semantics to improve the performance of text understanding. |
| Outcome: | The proposed model outperforms baseline models on syntax-dependent tasks by a large margin. |
Parsing All: Syntax and Semantics, Dependencies and Spans (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Syntactic and semantic structures are key linguistic contextual clues, but few studies have explored how they can be used to improve syntactical parsing. |
| Approach: | They propose a syntactic and semantic parsing model which integrates syntaktic information in the encoder of neural network and benefits from two representation formalisms in a uniform way. |
| Outcome: | The proposed model achieves state-of-the-art or competitive results on both span and dependency representations and on Penn Treebank. |
Towards Explainable Evaluation of Language Models on the Semantic Similarity of Visual Concepts (2022.coling-1)
Copied to clipboard
Maria Lymperaiou, George Manoliadis, Orfeas Menis Mastromichalakis, Edmund G. Dervakos, Giorgos Stamou
| Challenge: | Recent advances in NLP research have focused on robustness and explainability issues of their evaluation strategies. |
| Approach: | They propose to use pre-trained transformers to evaluate semantic similarity for visual vocabularies . they propose to provide explainable metrics for understanding the quality of retrieved instances . |
| Outcome: | The proposed metrics highlight inabilities of widely used evaluation methods and highlight weaknesses in learned linguistic representations. |
Context Dependent Semantic Parsing: A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Semantic parsing is the task of translating natural language utterances into machine-readable meaning representations. |
| Approach: | They propose to use contextual information to translate natural language utterances into machine-readable meaning representations. |
| Outcome: | The proposed methods do not utilize contextual information, which could boost the semantic parsing systems. |
Exploiting the Syntax-Model Consistency for Neural Relation Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing deep learning models for Relation Extraction (RE) have limited generalization beyond the syntactic structures of the input sentences. |
| Approach: | They propose a deep learning model that uses dependency trees to extract syntactic importance of words for Relation Extraction. |
| Outcome: | The proposed model outperforms existing models on three RE benchmark datasets. |