Papers by Arul Menezes
Leveraging GPT-4 for Automatic Translation Post-Editing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models still require translation post-editing to rectify errors and enhance quality under critical settings. |
| Approach: | They use GPT-4 to automatically post-edit NMT outputs across several language pairs . they show that GPT4 is adept at translation post- editing, producing meaningful edits . |
| Outcome: | The proposed translation post-editor improves on state-of-the-art language models on English-Chinese, English-German, Chinese-English and German-English language pairs. |
TRIP: Accelerating Document-level Multilingual Pre-training via Triangular Document-level Pre-training on Parallel Data Triplets (2023.findings-emnlp)
Copied to clipboard
Hongyuan Lu, Haoyang Huang, Shuming Ma, Dongdong Zhang, Wai Lam, Zhaochuan Gao, Anthony Aue, Arul Menezes, Furu Wei
| Challenge: | Existing approaches to multilingual sequence-to-sequence pre-training rely on monolingual corpora and sometimes synthetic document-level bilingual corporata. |
| Approach: | They propose to leverage document-level trilingual parallel corpora to improve sequence-to-sequence multilingual pre-training by using a novel method called Grafting. |
| Outcome: | The proposed method achieves strong state-of-the-art (SOTA) scores on three multilingual document-level machine translation benchmarks and one cross-lingual abstractive summarization benchmark. |
SALTED: A Framework for SAlient Long-tail Translation Error Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional machine translation metrics are insensitive to the long tail of behavioral problems. |
| Approach: | They propose a specification-based framework for behavioral testing of NMT models . they use high-precision detectors that flag errors between a source sentence and output . |
| Outcome: | The proposed framework provides a reliable view of problems that were previously invisible. |
Dissecting In-Context Learning of Translations in GPT-3 (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on fewshot prompting in Large Language Models has focused on selecting the few-shot samples for prompting. |
| Approach: | They propose a method to add demonstration attributes to prompting in machine translations by perturbations of high-quality in-domain demonstrations. |
| Outcome: | The proposed method improves upon the zero-shot translation performance of GPT-3, even making it competitive with few-shot prompted translations. |
The Curious Case of Hallucinations in Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) suffers from well known pathologies such as coverage, mistranslation of named entities, etc. |
| Approach: | They propose a theory that explains hallucinations under source perturbation and a method that generates hallucines under corpus-level noise without any source perturbations. |
| Outcome: | The proposed hypothesis is validated by a corpus-level noise analysis and is validateable in other datasets. |
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Memorization presents a challenge for constrained Natural Language Generation tasks . previous studies focused on counterfactual memorization, linking it to hallucinations . |
| Approach: | They propose an algorithm for extractive memorization in constrained sequence generation tasks . they propose to elicit non-memorized translations of memorized samples from the same model . |
| Outcome: | The proposed algorithm could be leveraged to mitigate memorization in the model through finetuning. |
Do GPTs Produce Less Literal Translations? (2023.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are general-purpose language models capable of many natural language generation or understanding tasks. |
| Approach: | They investigate how LLMs differ qualitatively from standard Neural Machine Translation models by measuring literalness and monotonicity. |
| Outcome: | The proposed models achieve close to state-of-the-art translation performance under few-shot prompting . the results are backed up by human evaluations and a newer MT quality metrics . |
MUTANT: A Recipe for Multilingual Tokenizer Design (2026.acl-long)
Copied to clipboard
| Challenge: | Subword tokenization schemes such as Byte Pair Encoding (BPE) are widely adopted, but their effectiveness in multilingual settings remains understudied. |
| Approach: | They propose a multilingual tokenizer that produces linguistically coherent tokens for multilingual LLMs. |
| Outcome: | The proposed tokenizer improves fertility score by 39.5% over LLaMA4 and 18% over Sutra. |