Improving Compositional Generalization with Self-Training for Data-to-Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). |
| Approach: | They propose a template-based input representation that greatly improves the model’s generalization capability. |
| Outcome: | The proposed model improves tree accuracy by 46%+ and reduces slot error rates by 73%+ over the strong baselines on SGD and Weather benchmarks. |
Similar Papers
Compositional Generalization for Data-to-Text Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Data-to-text generation models can be used to generate textual descriptions from structured data . despite advances, systems struggle when confronted with unseen combinations of predicates . |
| Approach: | They propose a data-to-text generation model that addresses compositional generalization by clustering predicates into groups. |
| Outcome: | The proposed model outperforms T5-baselines in all evaluation metrics. |
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning (2023.acl-long)
Copied to clipboard
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Yang Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
| Challenge: | Existing methods for data-to-text generation focus on specific types of structured data. |
| Approach: | They propose a method that provides a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. |
| Outcome: | The proposed method improves zero-shot and few-shot scenarios and can adapt to new structured data. |
Simple and effective data augmentation for compositional generalization (2024.naacl-long)
Copied to clipboard
| Challenge: | Compositional generalization is the ability of a system to correctly predict the meaning of complex sentences when trained on simpler sentences. |
| Approach: | They propose to use data augmentation methods to generate additional training data by sampling from an augmentation distribution to generalize to the out-of-distribution test data. |
| Outcome: | The proposed method outperforms existing methods that sampled from the training distribution and outperformed existing methods. |
Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research shows that automatic generation of synthetic utterance-program pairs can alleviate the first problem, but its potential for the second has thus far been under-explored. |
| Approach: | They propose to generate synthetic utterance-program pairs for improving compositional generalization in semantic parsing by using structurally-diverse examples. |
| Outcome: | The proposed approach leads to dramatic improvements in compositional generalization and moderate improvements in the traditional i.i.d setup. |
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to semantic parsing only evaluated on synthetic datasets that are not representative of natural language variation. |
| Approach: | They propose a semantic parsing approach that handles both natural language variation and compositional generalization. |
| Outcome: | The proposed model outperforms existing models across compositional generalization challenges on non-synthetic datasets while being competitive with the state-of-the-art on standard evaluations. |
Generating Demonstrations for In-Context Compositional Generalization in Grounded Language Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-Context-learning and few-shot prompting are viable methods for compositional output generation but they are sensitive to the choice of support examples. |
| Approach: | They propose a method which generates supports and targets current state of the world and then uses them in-context-learning to solve a query. |
| Outcome: | The proposed agent improves performance on a previously unsolved compositional generalization test without loss of performance in other areas. |
A synthetic data approach for domain generalization of NLI models (2024.acl-long)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) datasets are important benchmark tasks for LLMs . however, their realistic performance on out-of-distribution/domain data is less well-understood . a T5-small model trained with our data improves around 7% on average compared to the best alternative dataset . |
| Approach: | They propose a new approach for generating NLI data in diverse domains and lengths . they show that models trained on this data have the best generalization to completely new downstream test settings . |
| Outcome: | The proposed model can be trained on datasets with high-quality examples with meaningful premises and high accuracy. |
Few-Shot NLG with Pre-Trained Language Model (2020.acl-main)
Copied to clipboard
| Challenge: | Neural-based approaches to natural language generation are data-hungry and difficult to adopt in real-world applications. |
| Approach: | They propose a task of few-shot natural language generation from structured data or knowledge to generate coherent sentences from input data and language modeling to compose coherent sentences. |
| Outcome: | The proposed approach outperforms the strongest baseline approach by over 8.0 BLEU points improvement. |
Improving Compositional Generalization in Semantic Parsing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Generalization of models to out-of-distribution data has sparked substantial interest . compositional generalization is the ability to systematically generalize to test examples composed of components seen during training . |
| Approach: | They propose to extend compositional generalization in semantic parsing by using contextual representations and training attention to agree with pre-computed token alignments. |
| Outcome: | The proposed extensions improve compositional generalization on OOD compositions. |
Grammar-based Decoding for Improved Compositional Generalization in Semantic Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Sequence-to-sequence (seq2sequ) models have been successful in semantic parsing tasks but struggle on out-of-distribution data. |
| Approach: | They propose to use a large-scale dialogue dataset to evaluate compositional generalization of semantic parsing. |
| Outcome: | The proposed model outperforms BART- and T5-based models on the SMCalflow-CS dataset on the zero-shot learning task. |