Papers with convolutional
Towards efficient self-supervised representation learning in speech processing (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing models require several GPUs for days to pretrain, generating environmental concerns because of their high energy consumption. |
| Approach: | They propose an efficient self-supervised model that uses a single GPU during 24 to 48 hours of pretraining to address high computational costs. |
| Outcome: | The proposed model represents two orders of magnitude better than existing models. |
Generating Natural Language Adversarial Examples through Probability Weighted Word Saliency (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to attack text classification are limited due to the complexity of the problem. |
| Approach: | They propose a greedy algorithm to generate adversarial examples that maintain lexical correctness, grammatical correctiness and semantic similarity. |
| Outcome: | The proposed algorithm maintains lexical correctness, grammatical correctity and semantic similarity well and is hard for humans to perceive. |
Testing the limits of logical reasoning in neural and hybrid models (2024.findings-naacl)
Copied to clipboard
| Challenge: | despite the successes of deep learning models, we still need to know more about how and what they learn. |
| Approach: | They create tests to analyze logical reasoning patterns in neural and hybrid models . they find that models can generalize logical thinking only to a limited degree . |
| Outcome: | The proposed models can capture elementary aspects of meaning but only to limited extent . authors say they need to understand how and what they learn . |
Training a Broad-Coverage German Sentiment Classification Model for Dialog Systems (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing sentiment data sets are not available for sentiment analysis. |
| Approach: | They propose to combine a German sentiment corpus with existing resources to train a general-purpose German sentiment classification model. |
| Outcome: | The proposed model trains a general-purpose German sentiment classification model . the data set contains 5.4 million labelled samples . |
An Exploration of Arbitrary-Order Sequence Labeling via Energy-Based Inference Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that conditional random fields (CRFs) perform well in sequence labeling tasks. |
| Approach: | They propose several high-order energy terms to capture dependencies among labels in sequence labeling . they use convolutional, recurrent, and self-attention networks to construct these energy terms . |
| Outcome: | The proposed approach improves on four sequence labeling tasks while having the same decoding speed as simple classifiers. |
Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures (D18-1)
Copied to clipboard
| Challenge: | Recent studies show that non-recurrent architectures outperform RNNs in neural machine translation. |
| Approach: | They hypothesize that CNNs and self-attentional networks could extract semantic features from source text. |
| Outcome: | The proposed architectures outperform RNNs on two tasks: subject-verb agreement and word sense disambiguation. |