Papers by Leonid Karlinsky
NumeroLogic: Number Encoding for Enhanced LLMs’ Numerical Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language models struggle with numerical and arithmetical tasks, such as multiplying 3-digit numbers. |
| Approach: | They propose a method to include the count of digits before each number instead of “42”. |
| Outcome: | The proposed format improves the reasoning process before generating the actual number. |
Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs (2023.emnlp-main)
Copied to clipboard
Roei Herzig, Alon Mendelson, Leonid Karlinsky, Assaf Arbelle, Rogerio Feris, Trevor Darrell, Amir Globerson
| Challenge: | Vision and language models (VLMs) have demonstrated remarkable zero-shot (ZS) performance in a variety of tasks. |
| Approach: | They propose to integrate structured annotations into visual and textual representations to improve VLMs' understanding of compositional scenes. |
| Outcome: | The proposed method improves VLMs on multiple VL datasets with only a mild degradation in ZS capabilities. |
Self-Specialization: Uncovering Latent Expertise within Large Language Models (2024.findings-acl)
Copied to clipboard
Junmo Kang, Hongyin Luo, Yada Zhu, Jacob Hansen, James Glass, David Cox, Alan Ritter, Rogerio Feris, Leonid Karlinsky
| Challenge: | Recent studies have demonstrated the effectiveness of self-alignment in which a large language model is aligned to follow general instructions using instructional data generated from the model itself. |
| Approach: | They propose to use human-written seeds to align large language models to follow general instructions to achieve cross-task generalization. |
| Outcome: | The proposed model outperforms base models and models that are generally instruction-tuned or have been adapted to the target domain by a large margin. |
Activation Reward Models for Few-Shot Model Alignment (2026.findings-acl)
Copied to clipboard
Tianning Chai, Chancharik Mitra, Brandon Huang, Gautam Rajendrakumar Gare, Zhiqiu Lin, Assaf Arbelle, Leonid Karlinsky, Rogerio Feris, Trevor Darrell, Deva Ramanan, Roei Herzig
| Challenge: | A common approach is to use reward models that enable reinforcement-learning post-training. |
| Approach: | They propose a method that steers LLM activations to align with few-shot preference data without finetuning. |
| Outcome: | The proposed method surpasses zero-shot, few-shot and voting-based benchmarks on reward hacking and noise signals. |
REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark (2025.acl-long)
Copied to clipboard
Navve Wasserman, Roi Pony, Oshri Naparstek, Adi Raz Goldfarb, Eli Schwartz, Udi Barzelay, Leonid Karlinsky
| Challenge: | Existing benchmarks do not fully capture real-world retrieval challenges . existing benchmarks lack a complete understanding of how models perform in realistic setups . |
| Approach: | They propose an automatic benchmark to address four key properties essential for real-world retrieval: (i) multi-modal documents, (ii) enhanced difficulty, ( (iv) Realistic-RAG queries and (v) accurate labeling. |
| Outcome: | The proposed model reveals significant model weaknesses, particularly in handling table-heavy documents and robustness to query rephrasing. |