Papers by Craig Stewart
MT-Telescope: An interactive platform for contrastive evaluation of MT systems (2021.acl-demo)
Copied to clipboard
| Challenge: | MT-Telescope is an open source, written in Python, and is built around a user friendly and dynamic web interface. |
| Approach: | They propose a platform to facilitate comparative analysis of the output quality of two Machine Translation (MT) systems. |
| Outcome: | The proposed platform supports fine-grained segment-level analysis and interactive visualisations that expose the fundamental differences in the performance of the compared systems. |
Lost in Interpretation: Predicting Untranslated Terminology in Simultaneous Interpretation (N19-1)
Copied to clipboard
| Challenge: | Experimental results on a newly-annotated version of the NAIST Simultaneous Translation Corpus indicate the promise of our proposed method. |
| Approach: | They propose a task of predicting which terminology simultaneous interpreters will leave untranslated using supervised sequence taggers. |
| Outcome: | The proposed method predicts which terminology interpreters leave untranslated . it is based on an annotated version of the NAIST Simultaneous Translation Corpus . |
COMET: A Neural Framework for MT Evaluation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Historically, metrics for evaluating the quality of machine translation (MT) have relied on basic, lexical-level features such as counting the number of matching n-grams between the MT hypothesis and the reference translation. |
| Approach: | They propose a neural framework for training multilingual machine translation evaluation models which exploits human judgements to obtain new state-of-the-art levels of correlation with MT quality. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the WMT 2019 Metrics shared task and demonstrate robustness to high-performing systems. |
Rapidly Piloting Real-time Linguistic Assistance for Simultaneous Interpreters with Untrained Bilingual Surrogates (2024.lrec-main)
Copied to clipboard
Alvin C. Grissom II, Jo Shoemaker, Benjamin Goldman, Ruikang Shi, Craig Stewart, C. Anton Rytting, Leah Findlater, Jordan Boyd-Graber
| Challenge: | Simultaneous interpretation is a cognitively taxing task, and even seasoned professionals benefit from real-time assistance. |
| Approach: | They propose a simultaneous interpretation task that mimics the cognitive load of interpretation with crowdworker surrogates. |
| Outcome: | The proposed task mimics the cognitive load of interpretation with crowdworker surrogates . the evaluation setup provides consistent results between expert and proxy participants . |
Automatic Estimation of Simultaneous Interpreter Performance (P18-2)
Copied to clipboard
| Challenge: | Existing methods to predict interpreter confidence and the adequacy of the interpreted message are lacking. |
| Approach: | They propose to extend a QE pipeline to estimate interpreter performance by using five settings in three language pairs. |
| Outcome: | The proposed method can predict interpreter confidence and adequacy over five settings in three language pairs and improves interpretation strategy and evaluation measures. |
Improving Robustness of Machine Translation with Synthetic Noise (N19-1)
Copied to clipboard
| Challenge: | Recent work on MT robustness has demonstrated the need to build or adapt systems that are resilient to such noise. |
| Approach: | They propose to synthesize natural noise in social media data to enhance robustness of MT systems by leveraging natural noise. |
| Outcome: | The proposed method can make a vanilla MT system more resilient to noise, partially mitigating loss in accuracy resulting therefrom. |