Papers with SPE
Lightweight Text Classifier using Sinusoidal Positional Encoding (2020.aacl-main)
Copied to clipboard
| Challenge: | Large and complex models require many parameters and time to solve various problems in natural language processing. |
| Approach: | They propose to use the sinusoidal positional encoding (SPE) to construct a convolutional neural network using the SPE in text classification. |
| Outcome: | The proposed model reduces parameter size and training time while maintaining similar performance to the current model on multiple benchmark datasets. |
Serial Position Effects of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Serial position effects (SPE) are well-documented cognitive biases in human behavior. |
| Approach: | They propose to use binary choices instead of multiple choices where feasible . they also suggest limiting prompt length and placing crucial information at the beginning of prompts . |
| Outcome: | The proposed framework shows that the effects are widespread across LLMs and the proposed mitigation methods are effective. |
SPE Attention: Making Attention Equivariant to Semantic-Preserving Permutation for Code Processing (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train code processing models to capture symmetry of code . semantic-preserving permutations are not found in natural languages . |
| Approach: | They propose a mechanism that captures a unique symmetry of code, called the SPE attention . they propose symmetry graphs that are then combined to create a symmetry mask . |
| Outcome: | The proposed model can be used to analyze code summarization and error detection tasks. |
SPE: Symmetrical Prompt Enhancement for Fact Probing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work probes PLMs for the extent of factual knowledge through prompts . however, these methods do not consider symmetry of the task: object and subject prediction. |
| Approach: | They propose a continuous prompt-based method that leverages symmetry of the task by constructing symmetrical prompts for subject and object prediction. |
| Outcome: | The proposed method improves on a popular factual probing dataset on lAMA. |