Papers by Christian Herold
Is Encoder-Decoder Redundant for Neural Machine Translation? (2022.aacl-main)
Copied to clipboard
| Challenge: | Encoder-decoder architecture is widely adopted for sequence-to-sequence modeling tasks. |
| Approach: | They propose to combine bilingual and multilingual translations to train a language model to do translation. |
| Outcome: | The proposed approach performs on par with the baseline encoder-decoder Transformer . the proposed approach is compared with the translation model in the target language . |
Revisiting Checkpoint Averaging for Neural Machine Translation (2022.findings-aacl)
Copied to clipboard
| Challenge: | Checkpoint averaging is a simple and effective method to boost the performance of converged neural machine translation models. |
| Approach: | They propose to use checkpoint averaging to increase model performance . they also propose to calculate weighted average instead of simple mean . |
| Outcome: | The proposed method is widely adopted in neural machine translation research. |
Detecting Various Types of Noise for Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | a recent study investigated the impact of noise on the performance of machine translation systems. |
| Approach: | They propose to combine recent research on data filtering with original analysis . they find that most of the suggested noise types can be detected with 90% accuracy . |
| Outcome: | The proposed filtering systems can detect noise types with 90% accuracy in high resource settings. |
Improving Language Model Integration for Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to integrate external language models into machine translation systems have been based on the assumption that the external model learns an implicit target-side language model at decoding time. |
| Approach: | They transfer this concept to the task of machine translation and compare it with the most prominent way of including additional monolingual data - namely back-translation. |
| Outcome: | The proposed approach outperforms the most prominent way of including additional monolingual data, namely back-translation. |
Towards a Better Understanding of Label Smoothing in Neural Machine Translation (2020.aacl-main)
Copied to clipboard
| Challenge: | In recent years, Neural Network (NN) models bring steady and concrete improvements on the task of Machine Translation (MT). |
| Approach: | They propose to penalize over-confident outputs and regularize the model so that its outputs do not diverge too much from some prior distribution. |
| Outcome: | The proposed method is well-motivated and can improve the performance of strong neural machine translation systems. |
On Search Strategies for Document-Level Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Document-level neural machine translation models produce a more consistent output across a document . however, the exact decoding strategy is often not described and not mentioned at all. |
| Approach: | They propose to use standard automatic metrics and specific linguistic phenomena to compare different decoding schemes. |
| Outcome: | The proposed decoding strategies perform similar to each other on three standard document-level translation benchmarks. |
Adapting Vision-Language Models for E-commerce Understanding at Scale (2026.eacl-industry)
Copied to clipboard
Matteo Nulli, Vladimir Orshulevich, Tala Bazazo, Christian Herold, Michael Kozielski, Marcin Mazur, Szymon Tuzel, Cees G. M. Snoek, Seyyed Hadi Hashemi, Omar Javed, Yannick Versley, Shahram Khadivi
| Challenge: | Existing approaches to adapt VLMs to attribute-centric, multi-image, and noisy data are limited. |
| Approach: | They propose a novel evaluation suite that incorporates deep product understanding, strict instruction following, and dynamic attribute extraction. |
| Outcome: | The proposed model improves e-commerce performance while preserving broad multimodal capabilities. |
Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on public benchmarks and an in-house e-commerce dataset demonstrate Unilogit’s superior performance in balancing forget and retain objectives, outperforming state-of-the-art methods such as NPO and UnDIAL. |
| Approach: | They propose a self-distillation method that dynamically adjusts target logits to achieve a uniform probability for the target token. |
| Outcome: | Extensive experiments on public benchmarks and an in-house e-commerce dataset demonstrate Unilogit’s superior performance in balancing forget and retain objectives. |
Domain Adaptation of Foundation LLMs for e-Commerce (2025.acl-industry)
Copied to clipboard
Christian Herold, Michael Kozielski, Tala Bazazo, Pavel Petrushkov, Yannick Versley, Seyyed Hadi Hashemi, Patrycja Cieplicka, Dominika Basaj, Shahram Khadivi
| Challenge: | Large Language Models (LLMs) have greatly improved the performance on most natural language tasks, and often show surprisingly good zero-shot generalization to new domains. |
| Approach: | They propose to continuously pretrain the Llama 3.1 base models on 1 trillion tokens of e-commerce data to introduce domain specific knowledge into the model while at the same time keeping the general capabilities intact. |
| Outcome: | The proposed model can be adapted to the new domain without sacrificing performance on general domain tasks. |
ApiQ: Finetuning of 2-Bit Quantized Large Language Model (2024.emnlp-main)
Copied to clipboard
| Challenge: | Memory-efficient finetuning of large language models (LLMs) has attracted huge attention with the increasing size of LLMs due to the constraints posed by GPU memory limitations and the effectiveness of these methods compared to full finetune. |
| Approach: | They propose a memory-efficient finetuning framework called ApiQ to restore lost information from quantization by initializing LoRA components and quantizing weights of LLMs. |
| Outcome: | The proposed framework maintains the original LLM’s activation precision while mitigating error propagation from shallower into deeper layers. |
Data Filtering using Cross-Lingual Word Embeddings (2021.naacl-main)
Copied to clipboard
| Challenge: | varying task definitions and data conditions make it difficult to draw a meaningful comparison. |
| Approach: | They propose to use language identification to perform data filtering on MT data based on cross-lingual word embeddings to identify weaknesses in language identification tool. |
| Outcome: | The proposed methods perform well on three real-life, high resource MT tasks while performing weakly within more realistic task conditions. |
CONGRAD: Conflicting Gradient Filtering for Multilingual Preference Alignment (2026.eacl-long)
Copied to clipboard
Jiangnan Li, Thuy-Trang Vu, Christian Herold, Amirhossein Tebbifakhr, Shahram Khadivi, Gholamreza Haffari
| Challenge: | Naive joint training of large language models can suffer from negative interference. |
| Approach: | They propose a filtering method that aggregates cross-lingually beneficial gradients and filters for those with high cross-linguistic affinity. |
| Outcome: | The proposed method outperforms baselines in both seen and unseen languages with minimal alignment tax. |
Does Joint Training Really Help Cascaded Speech Translation? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Currently, in speech translation, the straightforward approach delivers state-of-the-art results, but fundamental challenges such as error propagation remain. |
| Approach: | They propose to combine a cascaded recognition system with a machine translation system to improve cascade speech translation. |
| Outcome: | The proposed methods can improve cascaded speech translation and suggest alternative training methods. |
ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Weight-only quantization reduces model size but suffers from performance degradation at lower bit widths. |
| Approach: | They propose a weight-only quantization paradigm that clusters weight matrices into codebooks and finetunes them block-by-block. |
| Outcome: | The proposed paradigm outperforms quantization methods and fine tunes LLMs to 1-bit compression and fine tuning. |