Papers by Lukas Kinder
Positional Overload: Positional Debiasing and Context Window Extension for Large Language Models using Set Encoding (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models typically track the order of tokens using positional encoding, which causes two significant limitations: 1. Positional Bias: When processing long text sequences, the number of token can exceed the range the model was trained on. |
| Approach: | They propose a method that allows multiple pieces of text to be encoded in the same position, eliminating positional bias entirely. |
| Outcome: | The proposed method eliminates positional bias entirely and increases the size of the input an LLM can handle. |