Papers by Mark Fishel
Cross-lingual Similarity of Multilingual Representations Revisited (2022.aacl-main)
Copied to clipboard
| Challenge: | Similarity indexes like CKA and CCA are not suitable for cross-lingual learning analysis. |
| Approach: | They propose an alternative that is exempt from the difficulties of CKA/CCA and is good specifically in a cross-lingual context. |
| Outcome: | The proposed method is exempt from the difficulties of CKA/CCA and is good specifically in a cross-lingual context. |
To Err Is Human, but Llamas Can Learn It Too (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Specifically, we fine-tune Llama 2 LMs for error generation and find that this approach yields synthetic errors akin to human errors. |
| Approach: | They propose to fine-tune Llama 2 LMs for error generation and train GEC Llma models using these artificial errors. |
| Outcome: | The proposed approach outperforms state-of-the-art models with gains ranging between 0.8 and 6 F0.5 points across all languages tested. |
Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pretrained Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Extending existing vocabulary is a widely used step in adapting pre-trained language models to new domains or languages. |
| Approach: | They propose to extend a pre-trained tokenizer by continuing the BPE merge learning process on new data. |
| Outcome: | The proposed method improves tokenization efficiency and improves model utilization. |
Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to adapt pretrained Large Language Models to new lower-resource languages are limited to English. |
| Approach: | They propose to combine cross-lingual instruction-tuning with additional monolingual pretraining to adapt LLMs to new lower-resource languages. |
| Outcome: | The proposed model is the first open-source instruction-following LLM for Estonian . the proposed model improves commonsense reasoning and multi-turn conversation capabilities . |
No Error Left Behind: Multilingual Grammatical Error Correction with Pre-trained Translation Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Grammatical Error Correction (GEC) research has primarily focused on English with little coverage for other languages. |
| Approach: | They propose a multilingual machine translation model that can be fine-tuned to improve error correction out-of-the-box. |
| Outcome: | The proposed model outperforms similar-sized MT5 models and competes favourably with larger models. |
Machine Translation for Livonian: Catering to 20 Speakers (2022.acl-short)
Copied to clipboard
| Challenge: | Livonian is one of the most endangered languages in Europe with just a tiny handful of speakers and virtually no publicly available corpora. |
| Approach: | They aim to develop machine translation between Livonian and English using a linguistic similarity test and a dataset of parallel and monolingual data. |
| Outcome: | The proposed systems and the collected data, including a manually translated and verified translation benchmark, are publicly released via OPUS and Huggingface repositories. |
Unsupervised Quality Estimation for Neural Machine Translation (2020.tacl-1)
Copied to clipboard
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, Lucia Specia
| Challenge: | Existing approaches require large amounts of expert annotated data, computation, and time for training. |
| Approach: | They propose an unsupervised approach to QE where no training is required . they use a dataset that enables work on both black-box and glass-box approaches . |
| Outcome: | The proposed approach rivals state-of-the-art supervised QE models in terms of correlation with human judgments of quality. |
LLMs for Extremely Low-Resource Finno-Ugric Languages (2025.findings-naacl)
Copied to clipboard
| Challenge: | Low-resource languages such as those in the Finno-Ugric family are underrepresented in large language models. |
| Approach: | They propose to develop large language models for extremely low-resource languages . they focus on Vro, Livonian, and Komi, which are underrepresented . |
| Outcome: | The proposed models cover almost the entire cycle of creation, from data collection to instruction tuning and evaluation. |
Multilinguality or Back-translation? A Case Study with Estonian (2024.lrec-main)
Copied to clipboard
| Challenge: | a limited amount of parallel data is available for machine translation, and synthetic data is often used to improve translation quality. |
| Approach: | They propose a large-scale synthetic corpus of Estonian translations that contains over 1 billion parallel sentences. |
| Outcome: | The proposed model improves the baseline model while maintaining multilinguality . the proposed model is 6 times larger than the Estonian corpus and twice the size of the Estonial part of the CulturaX corpus. |
True Detective: A Deep Abductive Reasoning Benchmark Undoable for GPT-3 and Challenging for GPT-4 (2023.starsem-1)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated solid zero-shot reasoning capabilities, which is reflected in their performance on the current test tasks. |
| Approach: | They propose a benchmark consisting of 191 long-form mystery narratives constructed as detective puzzles. |
| Outcome: | The proposed benchmark outperforms random models on the current test tasks while state-of-the-art models only solve 38% of puzzles. |