Papers by Georgios Kollias
NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Models (2024.findings-acl)
Copied to clipboard
Amit Dhurandhar, Tejaswini Pedapati, Ronny Luss, Soham Dan, Aurelie Lozano, Payel Das, Georgios Kollias
| Challenge: | Transformer-based Language Models have become ubiquitous in natural language processing due to impressive performance on various tasks. |
| Approach: | They explore how sparsity affects network topology by exploiting mechanisms seen in biological networks . they show that model-agnostic sparsities are performant across diverse NLP tasks . |
| Outcome: | The proposed model-agnostic sparsity approaches are performant and efficient across NLP tasks. |
Multi-Sense Embeddings for Language Models and Knowledge Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | Transformer-based large language models generate different representations for the same token depending on context . however, words and tokens typically have only a limited number of senses . a knowledge distillation method can be used to learn a smaller student model . |
| Approach: | They propose a multi-sense embedding method that uses a clustering algorithm to generate a sense embeddable dictionary. |
| Outcome: | The proposed method offers significant space and inference time savings while maintaining competitive performance. |
ImReasoner: Improving Memory-based Language Models for Reasoning-in-a-Haystack Tasks (2026.acl-long)
Copied to clipboard
Ching-Yun Ko, Payel Das, Sihui Dai, Georgios Kollias, Subhajit Chaudhury, Aurelie C. Lozano, Pin-Yu Chen
| Challenge: | despite advances, large language models exhibit brittleness on tasks that require multi-step reasoning over long contexts. |
| Approach: | They propose to explicitly encode contexts as ordered memory and perform iterative retrieval to construct reasoning chains. |
| Outcome: | The proposed frameworks fail to show emergent reasoning generalization in a weakly supervised scenario . the proposed framework is based on a synthetic benchmark to stress-test the models . |
EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts (2025.acl-long)
Copied to clipboard
Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer
| Challenge: | Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks, but efficient processing of long contexts remains a significant challenge. |
| Approach: | They propose a method for processing long contexts in an episodic memory module while holistically attending to semantically-relevant context chunks. |
| Outcome: | The proposed method outperforms baseline decoders on multiple long-context recall and question-answering benchmarks on 16k to 256k tokens. |