Papers with SWAG
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference (D18-1)
Copied to clipboard
| Challenge: | a new dataset presents a task of grounded commonsense inference, unifying natural language inference and commonsensical reasoning. |
| Approach: | They propose a procedure that constructs a de-biased dataset by iteratively training stylistic classifiers and using them to filter the data. |
| Outcome: | The proposed procedure oversamples a de-biased dataset using state-of-the-art language models . human models struggle on the proposed procedure, indicating significant opportunities for future research. |
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks (D19-60)
Copied to clipboard
| Challenge: | Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations. |
| Approach: | They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention. |
| Outcome: | The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets. |
XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understanding (2022.acl-short)
Copied to clipboard
| Challenge: | XDBERT (cross-modal distilled BERT) outperforms pretrained-BERT in general language understanding evaluation (GLUE), situations with adversarial generations (SWAG) benchmarks, and readability benchmarks. |
| Approach: | They propose to distill visual information from pretrained multimodal transformers to pretrained language encoders to cater to the language-heavy characteristics of NLU. |
| Outcome: | The proposed framework outperforms pretrained-BERT in general language understanding evaluation (GLUE), situations with adversarial generations (SWAG), and readability benchmarks. |
Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing language pretraining frameworks only take the language context as selfsupervision . current frameworks do not take grounding information from the external visual world . |
| Approach: | They propose a visually-supervised language model that extrapolates multimodal alignments to language-only data by contextually mapping language tokens to related images. |
| Outcome: | The proposed model improves on multiple pure-language tasks. |
Imagination-Augmented Natural Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for Natural Language Understanding focus on textual signals, which hinders models from learning efficiently from limited data samples. |
| Approach: | They propose an Imagination-Augmented Cross-modal Encoder to solve natural language understanding tasks from a novel learning perspective. |
| Outcome: | The proposed learning paradigm bridges the gap between human and agent language understanding in both linguistic and perceptual procedures. |
How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG (D19-1)
Copied to clipboard
| Challenge: | a recent study has improved the state-of-the-art on common-sense reasoning benchmarks . a san francisco-based approach to common-ense reasoning is challenging . |
| Approach: | They propose to use common-sense reasoning benchmarks to test machine learning's common-sentence inference task SWAG to test common-mind systems. |
| Outcome: | a new study shows that improved performance on common-sense reasoning benchmarks is genuine . the proposed task is more difficult than the current one, but it is more efficient than the previous one. |
Reasoning about Goals, Steps, and Temporal Ordering with WikiHow (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets focus on relation between procedural events, but little attention has been paid to relation between events. |
| Approach: | They propose a set of reasoning tasks targeting goal-step relations and step-step temporal relations based on wikiHow articles . their automatically-generated training set allows models to transfer to out-of-domain tasks requiring knowledge of procedural events . |
| Outcome: | The proposed dataset improves on SWAG, Snips, and Story Cloze Test in zero- and few-shot settings. |
SWAG: Storytelling With Action Guidance (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used for one-shot creation, but they can produce inconsistent but not necessarily engaging content. |
| Approach: | They propose a novel approach to storytelling with large language models that reduces story writing to a search problem through a two-model feedback loop. |
| Outcome: | The proposed approach outperforms existing methods when evaluated by GPT-4 and through human evaluation. |