| Challenge: | Existing supervised attention methods that use human knowledge to learn better alignments are costly or infeasible. |
| Approach: | They propose a generalized supervised attention method based on quasi alignments that are easier to obtain than ideal alignments. |
| Outcome: | The proposed framework improves generation performance and is robust against errors in attention supervision. |
Similar Papers
Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence (2020.acl-main)
Copied to clipboard
| Challenge: | Recent neural attention models conflate all steps into a single end-to-end system and simplify training process. |
| Approach: | They propose to explicitly segment target text into fragment units and align them with their data correspondences. |
| Outcome: | The proposed model outperforms neural attention models on E2E and WebNLG benchmarks. |
Grouped-Attention for Content-Selection and Content-Plan Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent neural data-to-text generation models explicitly learn content-plan given a set of attributes as input. |
| Approach: | They propose a neural content-planner that captures local and global contexts . they use a token-level attention constrained within each input attribute . |
| Outcome: | The proposed model outperforms competitors by 4.92%, 4.70%, and 16.56% on real-world datasets. |
KLAAD: Refining Attention Mechanisms to Reduce Societal Bias in Generative Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit societal biases in their outputs, prompting ethical and societal challenges. |
| Approach: | They propose an attention-based debiasing framework that implicitly aligns attention distributions between stereotypical and anti-stereotypical sentence pairs without directly modifying model weights. |
| Outcome: | The proposed framework improves on BBQ and BOLD benchmarks while maintaining fluency and coherence. |
Bootstrapping Generators from Noisy Data (N18-1)
Copied to clipboard
| Challenge: | Existing methods for data-to-text generation focus on learning correspondences between structured data and associated texts. |
| Approach: | They aim to bootstrap generators from large scale datasets where data and related texts are loosely aligned. |
| Outcome: | The proposed model improves on a vanilla encoder-decoder which relies on soft attention. |
Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention (2021.naacl-main)
Copied to clipboard
Pengcheng Yin, Hao Fang, Graham Neubig, Adam Pauls, Emmanouil Antonios Platanios, Yu Su, Sam Thomson, Jacob Andreas
| Challenge: | Existing approaches to compositional generalization in semantic parsers focus on word-level alignments, but they focus on spans. |
| Approach: | They propose a span-level supervised attention loss that improves compositional generalization in semantic parsers by focusing on spans. |
| Outcome: | The proposed method improves on three benchmarks of compositional generalization. |
Attribute Alignment: Controlling Text Generation from Pre-trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can generate text with sentiment polarity or specific topics without changing the original model parameters. |
| Approach: | They propose a method for controlling text generation by aligning disentangled attribute representations. |
| Outcome: | The proposed method shows large performance gains while maintaining diversity and fluency. |
Entropy- and Distance-Based Predictors From GPT-2 Attention Patterns Predict Reading Times Over and Above GPT-2 Surprisal (2022.emnlp-main)
Copied to clipboard
| Challenge: | Transformer-based large language models are trained to make predictions about the next word by aggregating representations of previous tokens through their self-attention mechanism. |
| Approach: | They propose an entropy-based predictor that quantifies the diffuseness of self-attention and a distance-based one that captures the incremental change in attention patterns across timesteps. |
| Outcome: | The proposed models perform better over a rigorous baseline including GPT-2 surprisal than previous models that used entropy-based predictors and distance-based ones. |
Text Generation with Exemplar-based Adaptive Decoding (N19-1)
Copied to clipboard
| Challenge: | Empirical results show that the proposed model achieves strong performance and outperforms comparable baselines. |
| Approach: | They propose a conditioned text generation model that uses a template-based approach to generate content from input text. |
| Outcome: | The proposed model outperforms baselines on abstractive text summarization and data-to-text generation. |
Data-to-text Generation with Macro Planning (2021.tacl-1)
Copied to clipboard
| Challenge: | Recent approaches to data-to-text generation adopt the encoder-decoder architecture . however, these models perform poorly at selecting appropriate content and ordering it coherently . |
| Approach: | They propose a neural model with a macro planning stage followed by a generation stage . they use data from databases of records, simulations of physical systems, accounting spreadsheets . |
| Outcome: | The proposed model outperforms baselines on two data-to-text benchmarks . it uses the encoderdecoder architecture and is compared with existing models . |
Building Joint Relationship Attention Network for Image-Text Generation (2022.coling-1)
Copied to clipboard
| Challenge: | et al., 2017) focus on visual features individually, while ignoring relationship information among image features that provides important guidance for generating sentences. |
| Approach: | They propose a joint relationship attention network that explores the relationships among image features. |
| Outcome: | The proposed method achieves state-of-the-art performance on large-scale datasets and on Flickr30k datasets. |