Papers by Pinzhen Chen
Parallel Sentence Mining by Constrained Decoding (2020.acl-main)
Copied to clipboard
| Challenge: | Using a neural machine translation system to extract parallel sentences from two monolingual corpora is challenging. |
| Approach: | They propose a method to extract parallel sentences from two monolingual corpora by constraining the decoding by a prefix tree built on the other corpus. |
| Outcome: | The proposed method achieves comparable results to other submissions on the BUCC shared task. |
Exploring Data Augmentation for Code Generation Tasks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have impacted how models are trained for programming language tasks. |
| Approach: | They propose to use augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively. |
| Outcome: | The proposed methods improve translation and summarization by 6.9% and 7.5% respectively. |
ParaCrawl: Web-Scale Acquisition of Parallel Corpora (2020.acl-main)
Copied to clipboard
Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà-Gomis, Mikel L. Forcada, Amir Kamran, Faheem Kirefu, Philipp Koehn, Sergio Ortiz Rojas, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Elsa Sarrías, Marek Strelec, Brian Thompson, William Waites, Dion Wiggins, Jaume Zaragoza
| Challenge: | We describe methods to create the largest publicly available parallel corpora by crawling the web . parallel corpus is essential for building highquality machine translation systems . |
| Approach: | They describe methods to create largest publicly available parallel corpora by crawling web sites . they empirically compare alternative methods and publish benchmark data sets . |
| Outcome: | The proposed methods improve state-of-the-art results on common benchmarks, the authors show . the pipeline has been tested on Russian, Sinhala, Nepali, Tagalog, Swahili, and Somali . |
XL-Suite: Cross-Lingual Synthetic Training and Evaluation Data for Open-Ended Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Cross-lingual open-ended generation is an important yet understudied problem. |
| Approach: | They propose XL-Instruct, a novel technique for generating high-quality synthetic data, and introduce Xl-AlpacaEval, evaluating cross-lingual generation capabilities of large language models. |
| Outcome: | The proposed technique improves model performance by fine tuning with just 8K instructions generated using XL-Instruct, and also by improving on several fine-grained quality metrics. |
UniArk: Improving Generalisation and Consistency for Factual Knowledge Extraction through Debiasing (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have investigated the potential of language models as knowledge bases and the existence of severe biases when extracting factual knowledge. |
| Approach: | They propose an adapter-based framework for generalised factual knowledge extraction using simple methods without introducing extra parameters. |
| Outcome: | The proposed framework improves the model’s out-of-domain generalisation and consistency under various prompts. |
Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing practices of fine-tuning and evaluating multilingual large language models may not align with this objective due to a heavy reliance on translation. |
| Approach: | They propose to use translated or native instruction data to fine-tune multilingual large language models. |
| Outcome: | The proposed model can be fine tuned and evaluated in multilingual large language models . the results show that native or translated data can be used to compare model performance . |
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT) (2025.acl-long)
Copied to clipboard
Laurie Burchell, Ona De Gibert Bonet, Nikolay Arefyev, Mikko Aulamo, Marta Bañón, Pinzhen Chen, Mariia Fedorova, Liane Guillou, Barry Haddow, Jan Hajič, Jindřich Helcl, Erik Henriksson, Mateusz Klimaszewski, Ville Komulainen, Andrey Kutuzov, Joona Kytöniemi, Veronika Laippala, Petter Mæhlum, Bhavitvya Malik, Farrokh Mehryary, Vladislav Mikhailov, Nikita Moghe, Amanda Myntti, Dayyán O’Brien, Stephan Oepen, Proyag Pal, Jousia Piha, Sampo Pyysalo, Gema Ramírez-Sánchez, David Samuel, Pavel Stepachev, Jörg Tiedemann, Dušan Variš, Tereza Vojtěchová, Jaume Zaragoza-Bernabeu
| Challenge: | a large number of textual data is needed to train state-of-the-art large language models. |
| Approach: | They propose a collection of monolingual and parallel corpora from the Internet Archive . they document the entire data pipeline and release the code to reproduce it . |
| Outcome: | The proposed collection of monolingual and parallel corpora is based on the HPLT v2 dataset . it includes 8T tokens covering 193 languages and 380M sentence pairs covering 51 languages . |
Fine-Tuning Large Language Models with Sequential Instructions (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing instruction-tuned models struggle to adhere to a query with multiple intentions, which impairs their performance when the completion of several tasks is demanded by a single command. |
| Approach: | They develop an automatic process that turns existing data into diverse and complex task chains and a new benchmark to evaluate a model’s ability to follow all the instructions in a sequence. |
| Outcome: | The proposed model can follow instructions better and deliver higher results in coding, maths, and open-ended generation. |
Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca (2024.findings-eacl)
Copied to clipboard
| Challenge: | Foundational large language models (LLMs) can be instruction-tuned to perform open-domain question answering, facilitating applications like chat assistants. |
| Approach: | They employ a dataset and machine translations of it to form multilingual data and use it to tune LLMs. |
| Outcome: | The proposed model is on par or better than a model for each language, and multilingual tuning with downsampled data is as powerful and robust. |
EEE-QA: Exploring Effective and Efficient Question-Answer Representations (2024.lrec-main)
Copied to clipboard
| Challenge: | Current approaches to question answering rely on pre-trained language models like RoBERTa. |
| Approach: | They propose a pooling approach that embeds all answer candidates with the question . they also propose enabling cross-reference between answer choices . |
| Outcome: | The proposed methods improve throughput and memory efficiency with little sacrifice in performance. |
Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Traditionally, success in multilingual machine translation depends on large volume, diverse directions, and high quality of training data. |
| Approach: | They revisit the importance of large language models for translation by fine-tuning on 32 parallel sentences. |
| Outcome: | The proposed model can be fine-tuned on as few as 32 parallel sentences . however, the choice of direction is critical to avoid misinterpretation, the authors say . |
How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM (2025.coling-main)
Copied to clipboard
| Challenge: | Many large language models (LLMs) support many languages, while others only support a few, e.g. the Llama series. |
| Approach: | They present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. |
| Outcome: | The proposed model can be used to perform multilingual tasks on 1 to 52 languages. |
When Flores Bloomz Wrong: Cross-Direction Contamination in Machine Translation Evaluation (2026.eacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be benchmark-contaminated, resulting in inflated scores that mask memorization as generalization. |
| Approach: | They use the FLORES-200 translation benchmark as a diagnostic to investigate cross-direction data contamination. |
| Outcome: | The proposed model can be cross-directional, boosting performance in unseen translation directions due to target-side memorization. |
A Unified Model for Reverse Dictionary and Definition Modelling (2022.aacl-short)
Copied to clipboard
| Challenge: | Using neural networks, we argue that both tasks can be learned and dealt with concurrently, based on the intuition that a word and its definition share the same meaning. |
| Approach: | They build a dual-way neural dictionary to retrieve words given definitions and produce definitions for queried words. |
| Outcome: | The proposed model achieves high scores on previous benchmarks without extra resources. |
PMIndiaSum: Multilingual and Cross-lingual Headline Summarization for Languages in India (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets for Indian languages are limited in terms of coverage and size. |
| Approach: | They propose a multilingual and massively parallel summarization corpus focused on languages in India that provides a training and testing ground for four language families, 14 languages, and the largest to date with 196 language pairs. |
| Outcome: | The proposed dataset provides a training and testing ground for four language families, 14 languages, and the largest to date with 196 language pairs. |