| Challenge: | Structured semantic sentence representations are useful in various NLP tasks, but their quality can vary greatly and jeopardize their usefulness. |
| Approach: | They propose to transfer the AMR graph to the domain of images and create a convolutional neural network that imitates a human judge tasked with rating graph quality. |
| Outcome: | The proposed model can rate quality more accurately than strong baselines, in several quality dimensions, and reduces energy consumption. |
Similar Papers
Evaluate AMR Graph Similarity via Self-supervised Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Current AMR metrics do not consider the entire structure of AMR graphs . |
| Approach: | They propose to learn automatic AMR graph similarity evaluation metric by encoding AMR to a pre-trained language model and using GNN adapters to capture structural information of AMR diagrams. |
| Outcome: | The proposed metric significantly improves the correlations with human semantic scores and remains robust under diverse challenges. |
DocAMR: Multi-Sentence AMR Representation and Evaluation (2022.naacl-main)
Copied to clipboard
Tahira Naseem, Austin Blodgett, Sadhana Kumaravel, Tim O’Gorman, Young-Suk Lee, Jeffrey Flanigan, Ramón Astudillo, Radu Florian, Salim Roukos, Nathan Schneider
| Challenge: | Abstract Meaning Representation (AMR) graphs are compared to gold graphs by the Smatch metric, but lack a well-defined representation and evaluation. |
| Approach: | They propose an algorithm for deriving a unified graph representation using a super-sentential annotation method. |
| Outcome: | The proposed algorithm avoids the pitfalls of over-merging and lacks coherence from under merging. |
Graph Pre-training for AMR Parsing and Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Abstract meaning representation (AMR) highlights the core semantic information of text in a graph structure. |
| Approach: | They propose two graph auto-encoding strategies for graph-to-graph pre-training and four tasks to integrate text and graph information during pre-tuning to improve structure awareness. |
| Outcome: | The proposed model is superior to pre-trained language models on AMR parsing and AMR-to-text generation tasks. |
Stacked AMR Parsing with Silver Data (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Lack of large-scale annotated data is one main challenge for abstract meaning representation (AMR) parsing. |
| Approach: | They propose to use silver data to train a pre-trained abstract meaning representation model. |
| Outcome: | The proposed model outperforms previous models on the AMR2.0 dataset and is faster than the SOTA model. |
Pushing the Limits of AMR Parsing with Self-Learning (2020.findings-emnlp)
Copied to clipboard
Young-Suk Lee, Ramón Fernandez Astudillo, Tahira Naseem, Revanth Gangi Reddy, Radu Florian, Salim Roukos
| Challenge: | Abstract Meaning Representation (AMR) parsing has experienced a notable growth in performance in the last two years due to the impact of transfer learning and the development of novel architectures specific to AMR. |
| Approach: | They propose to use AMR annotations to generate synthetic text and refine actions oracle without additional human annotations for AMR parsing. |
| Outcome: | The proposed models improve on AMR 1.0 and 2.0 without human annotations. |
deepQuest: A Framework for Neural-based Quality Estimation (C18-1)
Copied to clipboard
| Challenge: | Predicting Machine Translation (MT) quality has been limited to word and sentence-level prediction. |
| Approach: | They propose a framework that can generalize neural QE approaches to the level of documents. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on document-level quality estimates and is 40 times faster to train. |
Neural Text Generation from Rich Semantic Representations (N19-1)
Copied to clipboard
| Challenge: | 2 is a neural model that maps a linearization of Dependency MRS to text . 1 is based on a BLEU score of 66.11 when trained on gold data . |
| Approach: | They propose to use Minimal Recursion Semantics to generate high-quality text from structured representations. |
| Outcome: | The proposed model achieves a BLEU score of 77.17 on the full test set and 83.37 on the subset of test data most closely matching the silver data domain. |
PrefScore: Pairwise Preference Learning for Reference-free Summarization Quality Assessment (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on summarization evaluation without a human-written reference summary have shown high correlations with human ratings. |
| Approach: | They propose to judge summary quality by learning preference rank from corrupted summaries . they use Bradley-Terry power ranking model to learn preference rank . |
| Outcome: | Experiments on several datasets show that the proposed model can produce scores highly correlated with human ratings. |
SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features (2022.aacl-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (S3BERT) embeddings are composed of explainable sub-embeddings that emphasize various sentence meaning features. |
| Approach: | They propose to induce Semantically Structured Sentence BERT embeddings (S3BERT) that emphasize various sentence meaning features. |
| Outcome: | The proposed model shows high correlation to human similarity ratings, but lacks interpretability. |
Guided Neural Language Generation for Abstractive Summarization using Abstract Meaning Representation (D18-1)
Copied to clipboard
| Challenge: | Recent work on abstractive summarization has made progress with neural encoder-decoder architectures, but these models lack explicit semantic modeling of the source document and its summary. |
| Approach: | They extend previous work on abstractive summarization using Abstract Meaning Representation (AMR) with a neural language generation stage which they guide using the source document. |
| Outcome: | The proposed approach improves summarization performance by 7.4 and 10.5 points in ROUGE-2 using gold standard AMR parses and parses obtained from an off-the-shelf parser respectively. |