Papers by Peter Belcak
Text Compression for Efficient Language Generation (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing models rely on sub-word tokens for text generation, but there is no evidence for a more efficient way to generate text. |
| Approach: | They propose a hierarchical transformer language model capable of text generation by compressing text into sentence embeddings and employing a sentence attention mechanism. |
| Outcome: | The proposed model achieves an up to an order of magnitude improvement in FLOPs efficiency and a threefold increase in runtime speed compared to equally-sized models in the low-size regime. |
PROBE: PROcess-Based BEnchmark for Hallucination Detection (2026.findings-acl)
Copied to clipboard
Yu Zhang, Peter Belcak, Shizhe Diao, Yonggan Fu, Shaona Ghosh, Morteza Mardani, Eileen Margaret Peters Long, Bei Yu, Pavlo Molchanov
| Challenge: | Existing agentic applications rely on LLMs to self-assess the factuality of outputs . but current LLM systems fail to detect hallucinations . |
| Approach: | They propose a benchmark that breaks down hallucination detection into four critical steps . they show that when halluciation detection is treated as a multi-step process, all models achieve considerably better performance. |
| Outcome: | The proposed benchmark breaks down hallucination detection into four critical steps . it shows that when halluciation detection is treated as a multi-step process, all models achieve considerably better performance. |
UltraSparseBERT: 99% Conditionally Sparse Language Modelling (2024.acl-short)
Copied to clipboard
| Challenge: | UltraSparseBERT uses 0.3% of its neurons during inference, compared to similar BERT models. |
| Approach: | They propose a BERT variant that selectively engages just 12 out of 4095 neurons for each layer inference. |
| Outcome: | The proposed model employs 0.3% of its neurons during inference while performing on par with similar models. |