Papers by Tijmen Blankevoort
Understanding and Overcoming the Challenges of Efficient Transformer Quantization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in transformer quantization have shown remarkable improvement in many Natural Language Processing tasks and beyond. |
| Approach: | They propose a novel quantization scheme for transformers that can be quantized to ultra-low bit-widths, leading to significant memory savings with a minimum accuracy loss. |
| Outcome: | The proposed methods achieve state-of-the-art results on the GLUE benchmark using BERT, while preserving memory and accuracy. |
Bitune: Leveraging Bidirectional Attention to Improve Decoder-Only LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Decoder-only large language models rely on masked causal attention, which limits expressiveness by restricting information flow to one direction. |
| Approach: | They propose a method that incorporates bidirectional attention into prompt processing to enhance pretrained decoder-only LLMs. |
| Outcome: | The proposed method shows significant improvements in commonsense reasoning, arithmetic, and language understanding tasks. |