Papers by Eric Malmi
Teaching Small Language Models to Reason (2023.acl-short)
Copied to clipboard
| Challenge: | Chain of thought prompting improves reasoning capabilities of large language models but only emerges in models with tens of billions of parameters. |
| Approach: | They propose to fine tune a student model on chain of thought outputs generated by a larger teacher model. |
| Outcome: | The proposed method improves task performance across arithmetic, commonsense and symbolic reasoning datasets. |
EdiT5: Semi-Autoregressive Text Editing with T5 Warm-Start (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained seq2seq models have established strong baselines for text-to-text transduction tasks. |
| Approach: | They propose a semi-autoregressive text-editing approach that combines the strengths of non-auto-regressively text- editing and autoregressive decoding. |
| Outcome: | The proposed model is faster at inference times than conventional models while being capable of modeling flexible input-output transformations. |
Encode, Tag, Realize: High-Precision Text Editing (D19-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models provide a powerful framework for learning to translate source texts into target texts. |
| Approach: | They propose a sequence tagging approach that casts text generation as a text editing task. |
| Outcome: | The proposed model outperforms strong seq2seq models on sentence fusion, sentence splitting, abstractive summarization, and grammar correction tasks and achieves state-of-the-art performance. |
Text Generation with Text-Editing Models (2022.naacl-tutorials)
Copied to clipboard
Eric Malmi, Yue Dong, Jonathan Mallinson, Aleksandr Chuklin, Jakub Adamek, Daniil Mirylenka, Felix Stahlberg, Sebastian Krause, Shankar Kumar, Aliaksei Severyn
| Challenge: | Text-editing models are a popular alternative to seq2seq for monolingual text generation tasks such as text summarization and style transfer. |
| Approach: | They propose to use text-editing models to predict edit operations applied to the source sequence and to generate outputs word-by-word from scratch. |
| Outcome: | This paper provides an overview of the text-edit based models and their current state-of-the-art approaches. |
DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion (N19-1)
Copied to clipboard
| Challenge: | Existing datasets for sentence fusion are small and insufficient for training modern neural models. |
| Approach: | They propose a method for automatically-generating fusion examples from raw text . they apply their method to Wikipedia and Sports articles to generate fusion models . |
| Outcome: | The proposed method improves performance on WebSplit when viewed as a sentence fusion task. |
Small Language Models Improve Giants by Rewriting Their Outputs (2024.eacl-long)
Copied to clipboard
| Challenge: | despite impressive performance of large language models, they lag behind specialized models in various tasks. |
| Approach: | They propose a training model that can be integrated with different LLMs at inference to improve their performance without task-specific training. |
| Outcome: | The proposed model outperforms standard models on four natural language generation tasks. |
Unsupervised Text Style Transfer with Padded Masked Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for style transfer are difficult to obtain and require substantial amounts of parallel training examples to work well. |
| Approach: | They propose an unsupervised method for style transfer that uses masked language models to find the text spans where the two models disagree the most in terms of likelihood. |
| Outcome: | The proposed method performs competitively in a fully unsupervised setting and improves accuracy in low-resource settings by over 10 percentage points when pre-training on silver training data generated by Masker. |
Automatic Prediction of Discourse Connectives (L18-1)
Copied to clipboard
| Challenge: | Discourse connectives are used to bind together and explicate the relation between pieces of text. |
| Approach: | They propose to use a dataset of 2.9M sentence pairs separated by discourse connectives to test their accuracy. |
| Outcome: | The proposed model outperforms the human model in the prediction task . the proposed model has a higher F1 under specific conditions . |
FELIX: Flexible Text Editing Through Tagging and Insertion (2020.findings-emnlp)
Copied to clipboard
| Challenge: | FELIX is efficient in low-resource settings and fast at inference time, while being capable of modeling flexible input-output transformations. |
| Approach: | They propose a flexible text-editing approach that decomposes a text-generating task into two sub-tasks: tagging and insertion. |
| Outcome: | The proposed model is efficient in low-resource settings and fast at inference time while being capable of modeling flexible input-output transformations. |
Semantically Driven Sentence Fusion: Modeling and Evaluation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Sentence fusion is the task of joining related sentences into coherent text. |
| Approach: | They propose a method where ground-truth solutions are automatically expanded into multiple references via curated equivalence classes of connective phrases. |
| Outcome: | The proposed approach improves on state-of-the-art models by expanding ground-truth solutions into multiple references. |
A Simple Recipe for Multilingual Grammatical Error Correction (2021.acl-short)
Copied to clipboard
| Challenge: | Modern approaches view the task of Grammatical Error Correction (GEC) as monolingual text-to-text rewriting and employ encoderdecoder neural architectures. |
| Approach: | They propose a language-agnostic method to generate a large number of synthetic examples and use large-scale multilingual language models to train state-of-the-art GEC models. |
| Outcome: | The proposed method surpasses state-of-the-art results on GEC benchmarks in English, Czech, German and Russian. |