Papers by Daniil Orel
Droid: A Resource Suite for AI-Generated Code Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing detectors fail to generalise to diverse coding domains and programming languages outside of their narrow training data. |
| Approach: | They propose to use DroidCollection to train machine-generated code detectors that can be trained on a multi-task objective. |
| Outcome: | The proposed detectors fail to generalise to diverse coding domains and programming languages outside of their narrow training data. |
Qorǵau: Evaluating Safety in Kazakh-Russian Bilingual Contexts (2025.findings-acl)
Copied to clipboard
Maiya Goloburda, Nurkhan Laiyk, Diana Turmakhan, Yuxia Wang, Mukhammed Togmanov, Jonibek Mansurov, Askhat Sametov, Nurdaulet Mukhituly, Minghan Wang, Daniil Orel, Zain Muhammad Mujahid, Fajri Koto, Timothy Baldwin, Preslav Nakov
| Challenge: | Large language models (LLMs) have the potential to generate harmful content, posing risks to users. |
| Approach: | They propose a dataset specifically designed for safety evaluation in Kazakh and Russian . they use a bilingual context in Kazakhstan where both Kazakh (a low-resource language) and Russian (a high-resourced language) |
| Outcome: | The proposed dataset is designed for safety evaluation in Kazakh and Russian . it shows that both multilingual and language-specific LLMs perform better than others . |
FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning (2026.acl-long)
Copied to clipboard
Zhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan, Hachem Madmoun, Fan Zhang, Debopriyo Banerjee, Georgi Nenkov Georgiev, Xueqing Peng, Lingfei Qian, Jimin Huang, Jinyan Su, Aaryamonvikram Singh, Rui Xing, Rania Elbadry, Chen Xu, Haonan Li, Fajri Koto, Ivan Koychev, Tanmoy Chakraborty, Yuxia Wang, Salem Lahlou, Veselin Stoyanov, Sophia Ananiadou, Preslav Nakov
| Challenge: | Existing benchmarks emphasize final numerical answers while neglecting intermediate reasoning steps. |
| Approach: | They propose a symbolic benchmark for verifiable Chain-of-Thought evaluation in finance . FINCHAIN spans 58 topics across 12 financial domains and three difficulty levels . |
| Outcome: | The proposed benchmark aims to bridge symbolic reasoning and factual verification. |
AICD Bench: A Challenging Benchmark for AI-Generated Code Detection (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for detecting AI-generated code are limited to binary human–machine classification under in-distribution settings. |
| Approach: | They propose to use AICD Bench to build a robust binary classification framework for large language models. |
| Outcome: | The proposed benchmark spans 2M examples, 77 models across 11 families, and 9 programming languages. |
Stereotype Bias in a Bilingual Setting: A Culturally Grounded Evaluation in Kazakhstan (2026.acl-long)
Copied to clipboard
Nurkhan Laiyk, Daniil Orel, Ayana Mussabayeva, Maiya Goloburda, Kamila Kuishibekova, Liya Goloburda, Diana Turmakhan, Preslav Nakov, Yuxia Wang, Fajri Koto
| Challenge: | Stereotype bias in language models is largely understudied in English . language models perform strongly on downstream NLP tasks, but they are pre-trained on large text corpora . |
| Approach: | They use a dataset to assess stereotype bias in language models in Kazakhstan . they find that stereotype bias is most pronounced in code-mixed inputs . |
| Outcome: | The proposed dataset shows that stereotype bias is most pronounced in code-mixed inputs. |
CaMMT: Benchmarking Culturally Aware Multimodal Machine Translation (2025.findings-emnlp)
Copied to clipboard
Emilio Villa-Cueva, Sholpan Bolatzhanova, Diana Turmakhan, Kareem Elzeky, Henok Biadglign Ademtew, Alham Fikri Aji, Vladimir Araujo, Israel Abebe Azime, Jinheon Baek, Frederico Belcavello, Fermin Cristobal, Jan Christian Blaise Cruz, Mary Dabre, Raj Dabre, Toqeer Ehsan, Naome A Etori, Fauzan Farooqui, Jiahui Geng, Guido Ivetta, Thanmay Jayakumar, Soyeong Jeong, Zheng Wei Lim, Aishik Mandal, Sofía Martinelli, Mihail Minkov Mihaylov, Daniil Orel, Aniket Pramanick, Sukannya Purkayastha, Israfel Salazar, Haiyue Song, Tiago Timponi Torrent, Debela Desalegn Yadeta, Injy Hamed, Atnafu Lambebo Tonja, Thamar Solorio
| Challenge: | a human-curated benchmark of over 5,800 triples of images is used to evaluate multimodal translation systems. |
| Approach: | They introduce a human-curated benchmark of over 5,800 triples of images along with parallel captions in English and regional languages. |
| Outcome: | The results show that visual context improves translation quality in culturally-specific items . |
A Multi-View Media Profiling Suite: Resources, Evaluation, and Analysis (2026.findings-acl)
Copied to clipboard
Muhammad Arslan Manzoor, Dilshod Azizov, Daniil Orel, Umer Siddique, Zain Muhammad Mujahid, Yufang Hou, Preslav Nakov
| Challenge: | a large-scale label set for media outlets from Media Bias/Fact Check (MBFC) is lacking in the field. |
| Approach: | They propose to use a large-scale label set to analyze outlets' representations . they also propose to evaluate embedding views and fusion strategies . |
| Outcome: | The proposed method achieves state-of-the-art results on ACL-2020 and establishes strong benchmarks on MBFC-2025. |
CoDet-M4: Detecting Machine-Generated Code in Multi-Lingual, Multi-Generator and Multi-Domain Settings (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized code generation but have significant consequences for programming skills, ethics, and assessment integrity. |
| Approach: | They propose a framework capable of distinguishing between human-written and LLM-generated program code across multiple programming languages, code generators, and domains. |
| Outcome: | The proposed framework distinguishes between human-written and LLM-generated program code across multiple programming languages, code generators, and domains. |
Instruction Tuning on Public Government and Cultural Data for Low-Resource Language: a Case Study in Kazakh (2025.acl-long)
Copied to clipboard
| Challenge: | Instruction tuning in low-resource languages remains underexplored due to limited text data, particularly in government and cultural domains. |
| Approach: | They propose to open-source a large-scale instruction-following dataset covering key institutional and cultural knowledge relevant to Kazakhstan. |
| Outcome: | The proposed dataset improves LLMs’ understanding of procedural, legal, and structural governance topics. |
Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling (2026.findings-acl)
Copied to clipboard
Ivan Rodkin, Daniil Orel, Konstantin Smirnov, Arman Bolatov, Bilal Elbouardi, Besher Hassan, Yuri Kuratov, Aydar Bulatov, Preslav Nakov, Timothy Baldwin, Artem Shelmanov, Mikhail Burtsev
| Challenge: | Reasoning is a core capability of large language models, yet how multi-step reasoning is learned and executed remains unclear. |
| Approach: | They evaluate how large language models learn multi-step reasoning without memorization . they find that most neural architectures trained from scratch can learn rule inference . |
| Outcome: | The proposed framework fails to solve a natural-language proxy task with high accuracy. |