Papers with NNs
Injecting Relational Structural Representation in Neural Networks for Question Similarity (P18-2)
Copied to clipboard
| Challenge: | Recent years have seen exponential growth and use of web forums, where users can exchange and find information just asking questions in natural language. |
| Approach: | They propose to use Tree Kernels to learn a model on relatively few pairs of questions as gold standard (GS) predicting labels on a very large corpus of question pairs is also a useful approach, they propose . |
| Outcome: | The proposed model can learn more accurate models after fine tuning on GS. |
Approximating Two-Layer Feedforward Networks for Efficient Transformers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work uses sparse Mixtures of Experts (MoEs) to build resource-efficient large language models. |
| Approach: | They propose a general framework that unifies various methods to build two-layer NNs . they propose methods to improve both MoEs and PKMs based on their results . |
| Outcome: | The proposed framework improves both MoEs and product-key memories (PKMs) it shows that MoE's are competitive with dense Transformer-XL on two different scales while being much more resource efficient. |
Pivot Based Language Modeling for Improved Neural Domain Adaptation (N18-1)
Copied to clipboard
| Challenge: | Existing work on domain adaptation does not exploit the structure of the input text . PBLM can naturally feed structure aware text classifiers such as LSTM and CNN . |
| Approach: | They propose a model that integrates pivot-based and NN modeling in a structure aware manner. |
| Outcome: | The proposed model can naturally feed structure aware text classifiers such as LSTM and CNN. |
CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural nets fail to generalize systematically due to superficial differences in training data. |
| Approach: | They propose a new diagnostic dataset based on compositions of unary symbolic functions that tests systematicity of NNs. |
| Outcome: | The proposed dataset shows that recent CTL-solving Transformer variants fail on CTL++. |